Merge lp:~free.ekanayaka/landscape-client/drop-configobjd-and-amp into lp:~landscape/landscape-client/trunk

[Free Ekanayaka]

This branch:

- drops landscape.lib.configobj and landscape.lib.persist.ConfigObjectBackend,
  as they are not used

- drops landscape.lib.twisted_amp, which was needed only for Dapper

- improves the module docstring of landscape.broker.store, describing more
  in detail the logic behind "sequence" and "pending offset" when dealing
  with message exchanges

- improves docstrings in landscape.lib.persist

This branch is quite large but only have trivial changes:

- drops landscape.lib.configobj and landscape.lib.persist.ConfigObjectBackend, as they are not used
- drops landscape.lib.twisted_amp, which was needed only for Dapper
- improves the module docstring of landscape.broker.store, describing more in detail the logic behind "sequence" and "pending offset" when dealing with message exchanges
- improves docstrings in landscape.lib.persist

[Alberto Donato (ack)]

+1, nice cleanup

Thanks for adding documentation!

review: Approve

[Björn Tillenius (bjornt)]

+1!

review: Approve

=== modified file 'landscape/broker/registration.py'
--- landscape/broker/registration.py	2013-04-03 14:40:57 +0000
+++ landscape/broker/registration.py	2013-04-05 16:00:29 +0000
@@ -110,14 +110,14 @@
 
     def should_register(self):
         id = self._identity
-        # boolean logic is hard, I'm gonna use an if
+        if id.secure_id:
+            # We already have a secure ID, no need to register
+            return False
         if self._config.cloud:
-            return bool(not id.secure_id
-                        and self._message_store.accepts("register-cloud-vm"))
+            return self._message_store.accepts("register-cloud-vm")
         elif self._config.provisioning_otp:
-            return (not id.secure_id) and\
-                self._message_store.accepts("register-provisioned-machine")
-        return bool(not id.secure_id and id.computer_title and id.account_name
+            return self._message_store.accepts("register-provisioned-machine")
+        return bool(id.computer_title and id.account_name
                     and self._message_store.accepts("register"))
 
     def register(self):

=== modified file 'landscape/broker/store.py'
--- landscape/broker/store.py	2013-03-31 10:09:42 +0000
+++ landscape/broker/store.py	2013-04-05 16:00:29 +0000
@@ -1,4 +1,96 @@
-"""Message storage."""
+"""Message storage.
+
+The sequencing system we use in the message store may be quite confusing
+if you haven't looked at it in the last 10 minutes. For that reason, let's
+review the mechanics here.
+
+Our goal is to implement a reasonably robust system for delivering messages
+from us to our peer. The system should be smart enough to recover if the peer
+happens to lose messages that we have already sent, provided that these
+messages are not too old (we'll see below what 'too old' means).
+
+Messages added to the store are identified by increasing natural numbers, the
+first message added is identified by 0, the second by 1, and so on. We call
+"sequence" the number identifying the next message that we want to send. For
+example, if the store has been added ten messages (that we represent with
+uppercase letters) and we want start sending the first of them, our store
+would like like::
+
+    sequence: 0
+    messages: A, B, C, D, E, F, G, H, I, J
+              ^
+
+The "^" marker is what we call "pending offset" and is the displacement of the
+message we want to send next from the first message we have in the store.
+
+Let's say we now send to our peer a batch of 3 sequential messages. In the
+payload we include the body of the messages being sent and the sequence, which
+identifies the first message of the batch. In this case the payload would look
+like (pseudo-code)::
+
+    (sequence: 0, messages: A, B, C)
+
+If everything works fine on the other end, our peer replies with a payload that
+would like::
+
+    (next-expected-sequence: 4)
+
+meaning that the peer as received all the three messages that we sent, an so
+the next message it expects to receive is the one identified by the number 4.
+At this point we update both our pending offset and our sequence values, and
+the store now looks like::
+
+    sequence: 4
+    messages: A, B, C, D, E, F, G, H, I, J
+                       ^
+
+Great, now let's pretend that we send another batch, this time with five
+messages::
+
+    (sequence: 4, messages: D, E, F, G, H)
+
+Our peer receives them fine responding with a payload looking like::
+
+    (next-expected-sequence: 9)
+
+meaning that it received all the eight messages we sent so far and it's waiting
+for the ninth. This is the second successful batch that we send in a row, so we
+can be reasonably confident that at least the messages in the first batch are
+not really needed anymore. We delete them and we update our sequence and
+pending offset accordingly::
+
+    sequence: 9
+    messages: D, E, F, G, H, I, J
+                             ^
+
+Note that we still want to keep around the messages we sent in the very last
+batch, just in case. Indeed we now try to send a third batch with the last two
+messages that we have, but our peer surprisingly replies us with this payload::
+
+    (next-expected-sequence: 6)
+
+Ouch! This means that something bad happened and our peer has somehow lost not
+only the two messages that we sent in the last batch, but also the last three
+messages of the former batch :(
+
+Luckly we've kept enough old messages around that we can try to send them
+again, we update our sequence and pending offset and the store looks like::
+
+    sequence: 6
+    messages: D, E, F, G, H, I, J
+                    ^
+
+We can now start again sending messages using the same strategy.
+
+Note however that in the worst case scenario we could receive from our peer
+a next-expected-sequence value which is so old to be outside our buffer
+of already-sent messages. In that case there is now way we can recover the
+lost messages, and we'll just send the oldest one that we have.
+
+See L{MessageStore} for details about how messages are stored on the file
+system and L{landscape.lib.message.got_next_expected} to check how the
+strategy for updating the pending offset and the sequence is implemented.
+"""
 
 import time
 import itertools
@@ -16,24 +108,15 @@
 class MessageStore(object):
     """A message store which stores its messages in a file system hierarchy.
 
-    The sequencing system we use in the message store may be quite
-    confusing if you haven't looked at it in the last 10 minutes.  For
-    that reason, let's review the terminology here.
-
-    Assume we have 10 messages in the store, which we label by
-    the following uppercase letters::
-
-        A, B, C, D, E, F, G, H, I, J
-                 ^
-
-    Let's say that the next message we should send to the server is D.
-    What we call "pending offset" is the displacement from the first
-    message, which in our example above would be 3.  What we call
-    "sequence" is the number that the server expects us to label message
-    D as.  It could be pretty much any natural number, depending on the
-    history of our exchanges with the server.  What we call "server
-    sequence", is the next message number expected by the *client* itself,
-    and is entirely unrelated to the stored messages.
+    Beside the "sequence" and the "pending offset" values described in the
+    module docstring above, the L{MessageStore} also stores what we call
+    "server sequence", which is the next message number expected by the
+    *client* itself (because we are in turn the peer of a specular message
+    system running in the server, which tries to deliver messages to us).
+
+    The server sequence is entirely unrelated to the stored messages, but is
+    incremented when successfully receiving messages from the server, in the
+    very same way described above but with the roles inverted.
 
     @param persist: a L{Persist} used to save state parameters like the
         accepted message types, sequence, server uuid etc.

=== modified file 'landscape/lib/amp.py'
--- landscape/lib/amp.py	2013-03-27 22:58:20 +0000
+++ landscape/lib/amp.py	2013-04-05 16:00:29 +0000
@@ -4,12 +4,8 @@
 from twisted.internet.protocol import ReconnectingClientFactory
 from twisted.python.failure import Failure
 
-try:
-    from twisted.protocols.amp import (
-        Argument, String, Integer, Command, AMP, MAX_VALUE_LENGTH)
-except ImportError:
-    from landscape.lib.twisted_amp import (
-        Argument, String, Integer, Command, AMP, MAX_VALUE_LENGTH)
+from twisted.protocols.amp import (
+    Argument, String, Integer, Command, AMP, MAX_VALUE_LENGTH)
 
 from landscape.lib.bpickle import loads, dumps, dumps_table
 

=== removed file 'landscape/lib/configobj.py'
--- landscape/lib/configobj.py	2010-04-28 13:07:53 +0000
+++ landscape/lib/configobj.py	1970-01-01 00:00:00 +0000
@@ -1,3728 +0,0 @@
-# configobj.py
-# A config file reader/writer that supports nested sections in config files.
-# Copyright (C) 2005-2006 Michael Foord, Nicola Larosa
-# E-mail: fuzzyman AT voidspace DOT org DOT uk
-#         nico AT tekNico DOT net
-
-# ConfigObj 4
-# http://www.voidspace.org.uk/python/configobj.html
-
-# Released subject to the BSD License
-# Please see http://www.voidspace.org.uk/python/license.shtml
-
-# Scripts maintained at http://www.voidspace.org.uk/python/index.shtml
-# For information about bugfixes, updates and support, please join the
-# ConfigObj mailing list:
-# http://lists.sourceforge.net/lists/listinfo/configobj-develop
-# Comments, suggestions and bug reports welcome.
-
-from __future__ import generators
-
-"""
-    >>> z = ConfigObj()
-    >>> z['a'] = 'a'
-    >>> z['sect'] = {
-    ...    'subsect': {
-    ...         'a': 'fish',
-    ...         'b': 'wobble',
-    ...     },
-    ...     'member': 'value',
-    ... }
-    >>> x = ConfigObj(z.write())
-    >>> z == x
-    1
-"""
-
-import sys
-INTP_VER = sys.version_info[:2]
-if INTP_VER < (2, 2):
-    raise RuntimeError("Python v.2.2 or later needed")
-
-import os
-import re
-import compiler
-from types import StringTypes
-from warnings import warn
-from codecs import BOM_UTF8, BOM_UTF16, BOM_UTF16_BE, BOM_UTF16_LE
-
-# A dictionary mapping BOM to
-# the encoding to decode with, and what to set the
-# encoding attribute to.
-BOMS = {
-    BOM_UTF8: ('utf_8', None),
-    BOM_UTF16_BE: ('utf16_be', 'utf_16'),
-    BOM_UTF16_LE: ('utf16_le', 'utf_16'),
-    BOM_UTF16: ('utf_16', 'utf_16'),
-    }
-# All legal variants of the BOM codecs.
-# TODO: the list of aliases is not meant to be exhaustive, is there a
-#   better way ?
-BOM_LIST = {
-    'utf_16': 'utf_16',
-    'u16': 'utf_16',
-    'utf16': 'utf_16',
-    'utf-16': 'utf_16',
-    'utf16_be': 'utf16_be',
-    'utf_16_be': 'utf16_be',
-    'utf-16be': 'utf16_be',
-    'utf16_le': 'utf16_le',
-    'utf_16_le': 'utf16_le',
-    'utf-16le': 'utf16_le',
-    'utf_8': 'utf_8',
-    'u8': 'utf_8',
-    'utf': 'utf_8',
-    'utf8': 'utf_8',
-    'utf-8': 'utf_8',
-    }
-
-# Map of encodings to the BOM to write.
-BOM_SET = {
-    'utf_8': BOM_UTF8,
-    'utf_16': BOM_UTF16,
-    'utf16_be': BOM_UTF16_BE,
-    'utf16_le': BOM_UTF16_LE,
-    None: BOM_UTF8}
-
-try:
-    from validate import VdtMissingValue
-except ImportError:
-    VdtMissingValue = None
-
-try:
-    enumerate
-except NameError:
-
-    def enumerate(obj):
-        """enumerate for Python 2.2."""
-        i = -1
-        for item in obj:
-            i += 1
-            yield i, item
-
-try:
-    True, False
-except NameError:
-    True, False = 1, 0
-
-
-__version__ = '4.3.0alpha2'
-
-__revision__ = '$Id: configobj.py 156 2006-01-31 14:57:08Z fuzzyman $'
-
-__docformat__ = "restructuredtext en"
-
-# NOTE: Does it make sense to have the following in __all__ ?
-# NOTE: DEFAULT_INDENT_TYPE, NUM_INDENT_SPACES, MAX_INTERPOL_DEPTH
-# NOTE: If used via ``from configobj import...``
-# NOTE: They are effectively read only
-__all__ = (
-    '__version__',
-    'DEFAULT_INDENT_TYPE',
-    'NUM_INDENT_SPACES',
-    'MAX_INTERPOL_DEPTH',
-    'ConfigObjError',
-    'NestingError',
-    'ParseError',
-    'DuplicateError',
-    'ConfigspecError',
-    'ConfigObj',
-    'SimpleVal',
-    'InterpolationError',
-    'InterpolationDepthError',
-    'MissingInterpolationOption',
-    'RepeatSectionError',
-    '__docformat__',
-    'flatten_errors',
-)
-
-DEFAULT_INDENT_TYPE = ' '
-NUM_INDENT_SPACES = 4
-MAX_INTERPOL_DEPTH = 10
-
-OPTION_DEFAULTS = {
-    'interpolation': True,
-    'raise_errors': False,
-    'list_values': True,
-    'create_empty': False,
-    'file_error': False,
-    'configspec': None,
-    'stringify': True,
-    # option may be set to one of ('', ' ', '\t')
-    'indent_type': None,
-    'encoding': None,
-    'default_encoding': None,
-    'unrepr': False,
-    'write_empty_values': False,
-}
-
-
-def getObj(s):
-    s = "a=" + s
-    p = compiler.parse(s)
-    return p.getChildren()[1].getChildren()[0].getChildren()[1]
-
-
-class UnknownType(Exception):
-    pass
-
-
-class Builder:
-
-    def build(self, o):
-        m = getattr(self, 'build_' + o.__class__.__name__, None)
-        if m is None:
-            raise UnknownType(o.__class__.__name__)
-        return m(o)
-
-    def build_List(self, o):
-        return map(self.build, o.getChildren())
-
-    def build_Const(self, o):
-        return o.value
-
-    def build_Dict(self, o):
-        d = {}
-        i = iter(map(self.build, o.getChildren()))
-        for el in i:
-            d[el] = i.next()
-        return d
-
-    def build_Tuple(self, o):
-        return tuple(self.build_List(o))
-
-    def build_Name(self, o):
-        if o.name == 'None':
-            return None
-        if o.name == 'True':
-            return True
-        if o.name == 'False':
-            return False
-
-        # An undefinted Name
-        raise UnknownType('Undefined Name')
-
-    def build_Add(self, o):
-        real, imag = map(self.build_Const, o.getChildren())
-        try:
-            real = float(real)
-        except TypeError:
-            raise UnknownType('Add')
-        if not isinstance(imag, complex) or imag.real != 0.0:
-            raise UnknownType('Add')
-        return real + imag
-
-    def build_Getattr(self, o):
-        parent = self.build(o.expr)
-        return getattr(parent, o.attrname)
-
-
-def unrepr(s):
-    if not s:
-        return s
-    return Builder().build(getObj(s))
-
-
-class ConfigObjError(SyntaxError):
-    """
-    This is the base class for all errors that ConfigObj raises.
-    It is a subclass of SyntaxError.
-
-    >>> raise ConfigObjError
-    Traceback (most recent call last):
-    ConfigObjError
-    """
-
-    def __init__(self, message='', line_number=None, line=''):
-        self.line = line
-        self.line_number = line_number
-        self.message = message
-        SyntaxError.__init__(self, message)
-
-
-class NestingError(ConfigObjError):
-    """
-    This error indicates a level of nesting that doesn't match.
-
-    >>> raise NestingError
-    Traceback (most recent call last):
-    NestingError
-    """
-
-
-class ParseError(ConfigObjError):
-    """
-    This error indicates that a line is badly written.
-    It is neither a valid ``key = value`` line,
-    nor a valid section marker line.
-
-    >>> raise ParseError
-    Traceback (most recent call last):
-    ParseError
-    """
-
-
-class DuplicateError(ConfigObjError):
-    """
-    The keyword or section specified already exists.
-
-    >>> raise DuplicateError
-    Traceback (most recent call last):
-    DuplicateError
-    """
-
-
-class ConfigspecError(ConfigObjError):
-    """
-    An error occured whilst parsing a configspec.
-
-    >>> raise ConfigspecError
-    Traceback (most recent call last):
-    ConfigspecError
-    """
-
-
-class InterpolationError(ConfigObjError):
-    """Base class for the two interpolation errors."""
-
-
-class InterpolationDepthError(InterpolationError):
-    """Maximum interpolation depth exceeded in string interpolation."""
-
-    def __init__(self, option):
-        """
-        >>> raise InterpolationDepthError('yoda')
-        Traceback (most recent call last):
-        InterpolationDepthError: max interpolation depth exceeded in value "yoda".
-        """
-        InterpolationError.__init__(
-            self,
-            'max interpolation depth exceeded in value "%s".' % option)
-
-
-class RepeatSectionError(ConfigObjError):
-    """
-    This error indicates additional sections in a section with a
-    ``__many__`` (repeated) section.
-
-    >>> raise RepeatSectionError
-    Traceback (most recent call last):
-    RepeatSectionError
-    """
-
-
-class MissingInterpolationOption(InterpolationError):
-    """A value specified for interpolation was missing."""
-
-    def __init__(self, option):
-        """
-        >>> raise MissingInterpolationOption('yoda')
-        Traceback (most recent call last):
-        MissingInterpolationOption: missing option "yoda" in interpolation.
-        """
-        InterpolationError.__init__(
-            self,
-            'missing option "%s" in interpolation.' % option)
-
-
-class Section(dict):
-    """
-    A dictionary-like object that represents a section in a config file.
-
-    It does string interpolation if the 'interpolate' attribute
-    of the 'main' object is set to True.
-
-    Interpolation is tried first from the 'DEFAULT' section of this object,
-    next from the 'DEFAULT' section of the parent, lastly the main object.
-
-    A Section will behave like an ordered dictionary - following the
-    order of the ``scalars`` and ``sections`` attributes.
-    You can use this to change the order of members.
-
-    Iteration follows the order: scalars, then sections.
-    """
-
-    _KEYCRE = re.compile(r"%\(([^)]*)\)s|.")
-
-    def __init__(self, parent, depth, main, indict=None, name=None):
-        """
-        * parent is the section above
-        * depth is the depth level of this section
-        * main is the main ConfigObj
-        * indict is a dictionary to initialise the section with
-        """
-        if indict is None:
-            indict = {}
-        dict.__init__(self)
-        # used for nesting level *and* interpolation
-        self.parent = parent
-        # used for the interpolation attribute
-        self.main = main
-        # level of nesting depth of this Section
-        self.depth = depth
-        # the sequence of scalar values in this Section
-        self.scalars = []
-        # the sequence of sections in this Section
-        self.sections = []
-        # purely for information
-        self.name = name
-        # for comments :-)
-        self.comments = {}
-        self.inline_comments = {}
-        # for the configspec
-        self.configspec = {}
-        self._order = []
-        self._configspec_comments = {}
-        self._configspec_inline_comments = {}
-        self._cs_section_comments = {}
-        self._cs_section_inline_comments = {}
-        # for defaults
-        self.defaults = []
-        #
-        # we do this explicitly so that __setitem__ is used properly
-        # (rather than just passing to ``dict.__init__``)
-        for entry in indict:
-            self[entry] = indict[entry]
-
-    def _interpolate(self, value):
-        """Nicked from ConfigParser."""
-        depth = MAX_INTERPOL_DEPTH
-        # loop through this until it's done
-        while depth:
-            depth -= 1
-            if value.find("%(") != -1:
-                value = self._KEYCRE.sub(self._interpolation_replace, value)
-            else:
-                break
-        else:
-            raise InterpolationDepthError(value)
-        return value
-
-    def _interpolation_replace(self, match):
-        """ """
-        s = match.group(1)
-        if s is None:
-            return match.group()
-        else:
-            # switch off interpolation before we try and fetch anything !
-            self.main.interpolation = False
-            # try the 'DEFAULT' member of *this section* first
-            val = self.get('DEFAULT', {}).get(s)
-            # try the 'DEFAULT' member of the *parent section* next
-            if val is None:
-                val = self.parent.get('DEFAULT', {}).get(s)
-            # last, try the 'DEFAULT' member of the *main section*
-            if val is None:
-                val = self.main.get('DEFAULT', {}).get(s)
-            self.main.interpolation = True
-            if val is None:
-                raise MissingInterpolationOption(s)
-            return val
-
-    def __getitem__(self, key):
-        """Fetch the item and do string interpolation."""
-        val = dict.__getitem__(self, key)
-        if self.main.interpolation and isinstance(val, StringTypes):
-            return self._interpolate(val)
-        return val
-
-    def __setitem__(self, key, value, unrepr=False):
-        """
-        Correctly set a value.
-
-        Making dictionary values Section instances.
-        (We have to special case 'Section' instances - which are also dicts)
-
-        Keys must be strings.
-        Values need only be strings (or lists of strings) if
-        ``main.stringify`` is set.
-
-        `unrepr`` must be set when setting a value to a dictionary, without
-        creating a new sub-section.
-        """
-        if not isinstance(key, StringTypes):
-            raise ValueError, 'The key "%s" is not a string.' % key
-        # add the comment
-        if not key in self.comments:
-            self.comments[key] = []
-            self.inline_comments[key] = ''
-        # remove the entry from defaults
-        if key in self.defaults:
-            self.defaults.remove(key)
-        #
-        if isinstance(value, Section):
-            if not self.has_key(key):
-                self.sections.append(key)
-            dict.__setitem__(self, key, value)
-        elif isinstance(value, dict)and not unrepr:
-            # First create the new depth level,
-            # then create the section
-            if not self.has_key(key):
-                self.sections.append(key)
-            new_depth = self.depth + 1
-            dict.__setitem__(
-                self,
-                key,
-                Section(
-                    self,
-                    new_depth,
-                    self.main,
-                    indict=value,
-                    name=key))
-        else:
-            if not self.has_key(key):
-                self.scalars.append(key)
-            if not self.main.stringify:
-                if isinstance(value, StringTypes):
-                    pass
-                elif isinstance(value, (list, tuple)):
-                    for entry in value:
-                        if not isinstance(entry, StringTypes):
-                            raise TypeError, (
-                                'Value is not a string "%s".' % entry)
-                else:
-                    raise TypeError, 'Value is not a string "%s".' % value
-            dict.__setitem__(self, key, value)
-
-    def __delitem__(self, key):
-        """Remove items from the sequence when deleting."""
-        dict. __delitem__(self, key)
-        if key in self.scalars:
-            self.scalars.remove(key)
-        else:
-            self.sections.remove(key)
-        del self.comments[key]
-        del self.inline_comments[key]
-
-    def get(self, key, default=None):
-        """A version of ``get`` that doesn't bypass string interpolation."""
-        try:
-            return self[key]
-        except KeyError:
-            return default
-
-    def update(self, indict):
-        """
-        A version of update that uses our ``__setitem__``.
-        """
-        for entry in indict:
-            self[entry] = indict[entry]
-
-    def pop(self, key, *args):
-        """ """
-        val = dict.pop(self, key, *args)
-        if key in self.scalars:
-            del self.comments[key]
-            del self.inline_comments[key]
-            self.scalars.remove(key)
-        elif key in self.sections:
-            del self.comments[key]
-            del self.inline_comments[key]
-            self.sections.remove(key)
-        if self.main.interpolation and isinstance(val, StringTypes):
-            return self._interpolate(val)
-        return val
-
-    def popitem(self):
-        """Pops the first (key,val)"""
-        sequence = (self.scalars + self.sections)
-        if not sequence:
-            raise KeyError, ": 'popitem(): dictionary is empty'"
-        key = sequence[0]
-        val = self[key]
-        del self[key]
-        return key, val
-
-    def clear(self):
-        """
-        A version of clear that also affects scalars/sections
-        Also clears comments and configspec.
-
-        Leaves other attributes alone :
-            depth/main/parent are not affected
-        """
-        dict.clear(self)
-        self.scalars = []
-        self.sections = []
-        self.comments = {}
-        self.inline_comments = {}
-        self.configspec = {}
-
-    def setdefault(self, key, default=None):
-        """A version of setdefault that sets sequence if appropriate."""
-        try:
-            return self[key]
-        except KeyError:
-            self[key] = default
-            return self[key]
-
-    def items(self):
-        """ """
-        return zip((self.scalars + self.sections), self.values())
-
-    def keys(self):
-        """ """
-        return (self.scalars + self.sections)
-
-    def values(self):
-        """ """
-        return [self[key] for key in (self.scalars + self.sections)]
-
-    def iteritems(self):
-        """ """
-        return iter(self.items())
-
-    def iterkeys(self):
-        """ """
-        return iter((self.scalars + self.sections))
-
-    __iter__ = iterkeys
-
-    def itervalues(self):
-        """ """
-        return iter(self.values())
-
-    def __repr__(self):
-        return '{%s}' % ', '.join([('%s: %s' % (repr(key), repr(self[key])))
-            for key in (self.scalars + self.sections)])
-
-    __str__ = __repr__
-
-    # Extra methods - not in a normal dictionary
-
-    def dict(self):
-        """
-        Return a deepcopy of self as a dictionary.
-
-        All members that are ``Section`` instances are recursively turned to
-        ordinary dictionaries - by calling their ``dict`` method.
-
-        >>> n = a.dict()
-        >>> n == a
-        1
-        >>> n is a
-        0
-        """
-        newdict = {}
-        for entry in self:
-            this_entry = self[entry]
-            if isinstance(this_entry, Section):
-                this_entry = this_entry.dict()
-            # XXX Modified to return tuples as tuples. -- niemeyer, 2006-04-24
-            elif isinstance(this_entry, list):
-                this_entry = list(this_entry)
-            elif isinstance(this_entry, tuple):
-                this_entry = tuple(this_entry)
-            newdict[entry] = this_entry
-        return newdict
-
-    def merge(self, indict):
-        """
-        A recursive update - useful for merging config files.
-
-        >>> a = '''[section1]
-        ...     option1 = True
-        ...     [[subsection]]
-        ...     more_options = False
-        ...     # end of file'''.splitlines()
-        >>> b = '''# File is user.ini
-        ...     [section1]
-        ...     option1 = False
-        ...     # end of file'''.splitlines()
-        >>> c1 = ConfigObj(b)
-        >>> c2 = ConfigObj(a)
-        >>> c2.merge(c1)
-        >>> c2
-        {'section1': {'option1': 'False', 'subsection': {'more_options': 'False'}}}
-        """
-        for key, val in indict.items():
-            if (key in self and isinstance(self[key], dict) and
-                                isinstance(val, dict)):
-                self[key].merge(val)
-            else:
-                self[key] = val
-
-    def rename(self, oldkey, newkey):
-        """
-        Change a keyname to another, without changing position in sequence.
-
-        Implemented so that transformations can be made on keys,
-        as well as on values. (used by encode and decode)
-
-        Also renames comments.
-        """
-        if oldkey in self.scalars:
-            the_list = self.scalars
-        elif oldkey in self.sections:
-            the_list = self.sections
-        else:
-            raise KeyError, 'Key "%s" not found.' % oldkey
-        pos = the_list.index(oldkey)
-        #
-        val = self[oldkey]
-        dict.__delitem__(self, oldkey)
-        dict.__setitem__(self, newkey, val)
-        the_list.remove(oldkey)
-        the_list.insert(pos, newkey)
-        comm = self.comments[oldkey]
-        inline_comment = self.inline_comments[oldkey]
-        del self.comments[oldkey]
-        del self.inline_comments[oldkey]
-        self.comments[newkey] = comm
-        self.inline_comments[newkey] = inline_comment
-
-    def walk(self, function, raise_errors=True,
-            call_on_sections=False, **keywargs):
-        """
-        Walk every member and call a function on the keyword and value.
-
-        Return a dictionary of the return values
-
-        If the function raises an exception, raise the errror
-        unless ``raise_errors=False``, in which case set the return value to
-        ``False``.
-
-        Any unrecognised keyword arguments you pass to walk, will be pased on
-        to the function you pass in.
-
-        Note: if ``call_on_sections`` is ``True`` then - on encountering a
-        subsection, *first* the function is called for the *whole* subsection,
-        and then recurses into it's members. This means your function must be
-        able to handle strings, dictionaries and lists. This allows you
-        to change the key of subsections as well as for ordinary members. The
-        return value when called on the whole subsection has to be discarded.
-
-        See  the encode and decode methods for examples, including functions.
-
-        .. caution::
-
-            You can use ``walk`` to transform the names of members of a section
-            but you mustn't add or delete members.
-
-        >>> config = '''[XXXXsection]
-        ... XXXXkey = XXXXvalue'''.splitlines()
-        >>> cfg = ConfigObj(config)
-        >>> cfg
-        {'XXXXsection': {'XXXXkey': 'XXXXvalue'}}
-        >>> def transform(section, key):
-        ...     val = section[key]
-        ...     newkey = key.replace('XXXX', 'CLIENT1')
-        ...     section.rename(key, newkey)
-        ...     if isinstance(val, (tuple, list, dict)):
-        ...         pass
-        ...     else:
-        ...         val = val.replace('XXXX', 'CLIENT1')
-        ...         section[newkey] = val
-        >>> cfg.walk(transform, call_on_sections=True)
-        {'CLIENT1section': {'CLIENT1key': None}}
-        >>> cfg
-        {'CLIENT1section': {'CLIENT1key': 'CLIENT1value'}}
-        """
-        out = {}
-        # scalars first
-        for i in range(len(self.scalars)):
-            entry = self.scalars[i]
-            try:
-                val = function(self, entry, **keywargs)
-                # bound again in case name has changed
-                entry = self.scalars[i]
-                out[entry] = val
-            except Exception:
-                if raise_errors:
-                    raise
-                else:
-                    entry = self.scalars[i]
-                    out[entry] = False
-        # then sections
-        for i in range(len(self.sections)):
-            entry = self.sections[i]
-            if call_on_sections:
-                try:
-                    function(self, entry, **keywargs)
-                except Exception:
-                    if raise_errors:
-                        raise
-                    else:
-                        entry = self.sections[i]
-                        out[entry] = False
-                # bound again in case name has changed
-                entry = self.sections[i]
-            # previous result is discarded
-            out[entry] = self[entry].walk(
-                function,
-                raise_errors=raise_errors,
-                call_on_sections=call_on_sections,
-                **keywargs)
-        return out
-
-    def decode(self, encoding):
-        """
-        Decode all strings and values to unicode, using the specified encoding.
-
-        Works with subsections and list values.
-
-        Uses the ``walk`` method.
-
-        Testing ``encode`` and ``decode``.
-        >>> m = ConfigObj(a)
-        >>> m.decode('ascii')
-        >>> def testuni(val):
-        ...     for entry in val:
-        ...         if not isinstance(entry, unicode):
-        ...             print >> sys.stderr, type(entry)
-        ...             raise AssertionError, 'decode failed.'
-        ...         if isinstance(val[entry], dict):
-        ...             testuni(val[entry])
-        ...         elif not isinstance(val[entry], unicode):
-        ...             raise AssertionError, 'decode failed.'
-        >>> testuni(m)
-        >>> m.encode('ascii')
-        >>> a == m
-        1
-        """
-        warn('use of ``decode`` is deprecated.', DeprecationWarning)
-
-        def decode(section, key, encoding=encoding, warn=True):
-            """ """
-            val = section[key]
-            if isinstance(val, (list, tuple)):
-                newval = []
-                for entry in val:
-                    newval.append(entry.decode(encoding))
-            elif isinstance(val, dict):
-                newval = val
-            else:
-                newval = val.decode(encoding)
-            newkey = key.decode(encoding)
-            section.rename(key, newkey)
-            section[newkey] = newval
-        # using ``call_on_sections`` allows us to modify section names
-        self.walk(decode, call_on_sections=True)
-
-    def encode(self, encoding):
-        """
-        Encode all strings and values from unicode,
-        using the specified encoding.
-
-        Works with subsections and list values.
-        Uses the ``walk`` method.
-        """
-        warn('use of ``encode`` is deprecated.', DeprecationWarning)
-        def encode(section, key, encoding=encoding):
-            """ """
-            val = section[key]
-            if isinstance(val, (list, tuple)):
-                newval = []
-                for entry in val:
-                    newval.append(entry.encode(encoding))
-            elif isinstance(val, dict):
-                newval = val
-            else:
-                newval = val.encode(encoding)
-            newkey = key.encode(encoding)
-            section.rename(key, newkey)
-            section[newkey] = newval
-        self.walk(encode, call_on_sections=True)
-
-    def istrue(self, key):
-        """A deprecated version of ``as_bool``."""
-        warn('use of ``istrue`` is deprecated. Use ``as_bool`` method '
-                'instead.', DeprecationWarning)
-        return self.as_bool(key)
-
-    def as_bool(self, key):
-        """
-        Accepts a key as input. The corresponding value must be a string or
-        the objects (``True`` or 1) or (``False`` or 0). We allow 0 and 1 to
-        retain compatibility with Python 2.2.
-
-        If the string is one of  ``True``, ``On``, ``Yes``, or ``1`` it returns
-        ``True``.
-
-        If the string is one of  ``False``, ``Off``, ``No``, or ``0`` it returns
-        ``False``.
-
-        ``as_bool`` is not case sensitive.
-
-        Any other input will raise a ``ValueError``.
-
-        >>> a = ConfigObj()
-        >>> a['a'] = 'fish'
-        >>> a.as_bool('a')
-        Traceback (most recent call last):
-        ValueError: Value "fish" is neither True nor False
-        >>> a['b'] = 'True'
-        >>> a.as_bool('b')
-        1
-        >>> a['b'] = 'off'
-        >>> a.as_bool('b')
-        0
-        """
-        val = self[key]
-        if val == True:
-            return True
-        elif val == False:
-            return False
-        else:
-            try:
-                if not isinstance(val, StringTypes):
-                    raise KeyError
-                else:
-                    return self.main._bools[val.lower()]
-            except KeyError:
-                raise ValueError('Value "%s" is neither True nor False' % val)
-
-    def as_int(self, key):
-        """
-        A convenience method which coerces the specified value to an integer.
-
-        If the value is an invalid literal for ``int``, a ``ValueError`` will
-        be raised.
-
-        >>> a = ConfigObj()
-        >>> a['a'] = 'fish'
-        >>> a.as_int('a')
-        Traceback (most recent call last):
-        ValueError: invalid literal for int(): fish
-        >>> a['b'] = '1'
-        >>> a.as_int('b')
-        1
-        >>> a['b'] = '3.2'
-        >>> a.as_int('b')
-        Traceback (most recent call last):
-        ValueError: invalid literal for int(): 3.2
-        """
-        return int(self[key])
-
-    def as_float(self, key):
-        """
-        A convenience method which coerces the specified value to a float.
-
-        If the value is an invalid literal for ``float``, a ``ValueError`` will
-        be raised.
-
-        >>> a = ConfigObj()
-        >>> a['a'] = 'fish'
-        >>> a.as_float('a')
-        Traceback (most recent call last):
-        ValueError: invalid literal for float(): fish
-        >>> a['b'] = '1'
-        >>> a.as_float('b')
-        1.0
-        >>> a['b'] = '3.2'
-        >>> a.as_float('b')
-        3.2000000000000002
-        """
-        return float(self[key])
-
-
-class ConfigObj(Section):
-    """
-    An object to read, create, and write config files.
-
-    Testing with duplicate keys and sections.
-
-    >>> c = '''
-    ... [hello]
-    ... member = value
-    ... [hello again]
-    ... member = value
-    ... [ "hello" ]
-    ... member = value
-    ... '''
-    >>> ConfigObj(c.split('\\n'), raise_errors = True)
-    Traceback (most recent call last):
-    DuplicateError: Duplicate section name at line 5.
-
-    >>> d = '''
-    ... [hello]
-    ... member = value
-    ... [hello again]
-    ... member1 = value
-    ... member2 = value
-    ... 'member1' = value
-    ... [ "and again" ]
-    ... member = value
-    ... '''
-    >>> ConfigObj(d.split('\\n'), raise_errors = True)
-    Traceback (most recent call last):
-    DuplicateError: Duplicate keyword name at line 6.
-    """
-
-    _keyword = re.compile(r'''^ # line start
-        (\s*)                   # indentation
-        (                       # keyword
-            (?:".*?")|          # double quotes
-            (?:'.*?')|          # single quotes
-            (?:[^'"=].*?)       # no quotes
-        )
-        \s*=\s*                 # divider
-        (.*)                    # value (including list values and comments)
-        $   # line end
-        ''',
-        re.VERBOSE)
-
-    _sectionmarker = re.compile(r'''^
-        (\s*)                     # 1: indentation
-        ((?:\[\s*)+)              # 2: section marker open
-        (                         # 3: section name open
-            (?:"\s*\S.*?\s*")|    # at least one non-space with double quotes
-            (?:'\s*\S.*?\s*')|    # at least one non-space with single quotes
-            (?:[^'"\s].*?)        # at least one non-space unquoted
-        )                         # section name close
-        ((?:\s*\])+)              # 4: section marker close
-        \s*(\#.*)?                # 5: optional comment
-        $''',
-        re.VERBOSE)
-
-    # this regexp pulls list values out as a single string
-    # or single values and comments
-    # FIXME: this regex adds a '' to the end of comma terminated lists
-    #   workaround in ``_handle_value``
-    _valueexp = re.compile(r'''^
-        (?:
-            (?:
-                (
-                    (?:
-                        (?:
-                            (?:".*?")|              # double quotes
-                            (?:'.*?')|              # single quotes
-                            (?:[^'",\#][^,\#]*?)    # unquoted
-                        )
-                        \s*,\s*                     # comma
-                    )*      # match all list items ending in a comma (if any)
-                )
-                (
-                    (?:".*?")|                      # double quotes
-                    (?:'.*?')|                      # single quotes
-                    (?:[^'",\#\s][^,]*?)|           # unquoted
-                    (?:(?<!,))                      # Empty value
-                )?          # last item in a list - or string value
-            )|
-            (,)             # alternatively a single comma - empty list
-        )
-        \s*(\#.*)?          # optional comment
-        $''',
-        re.VERBOSE)
-
-    # use findall to get the members of a list value
-    _listvalueexp = re.compile(r'''
-        (
-            (?:".*?")|          # double quotes
-            (?:'.*?')|          # single quotes
-            (?:[^'",\#].*?)       # unquoted
-        )
-        \s*,\s*                 # comma
-        ''',
-        re.VERBOSE)
-
-    # this regexp is used for the value
-    # when lists are switched off
-    _nolistvalue = re.compile(r'''^
-        (
-            (?:".*?")|          # double quotes
-            (?:'.*?')|          # single quotes
-            (?:[^'"\#].*?)|     # unquoted
-            (?:)                # Empty value
-        )
-        \s*(\#.*)?              # optional comment
-        $''',
-        re.VERBOSE)
-
-    # regexes for finding triple quoted values on one line
-    _single_line_single = re.compile(r"^'''(.*?)'''\s*(#.*)?$")
-    _single_line_double = re.compile(r'^"""(.*?)"""\s*(#.*)?$')
-    _multi_line_single = re.compile(r"^(.*?)'''\s*(#.*)?$")
-    _multi_line_double = re.compile(r'^(.*?)"""\s*(#.*)?$')
-
-    _triple_quote = {
-        "'''": (_single_line_single, _multi_line_single),
-        '"""': (_single_line_double, _multi_line_double),
-    }
-
-    # Used by the ``istrue`` Section method
-    _bools = {
-        'yes': True, 'no': False,
-        'on': True, 'off': False,
-        '1': True, '0': False,
-        'true': True, 'false': False,
-        }
-
-    def __init__(self, infile=None, options=None, **kwargs):
-        """
-        Parse or create a config file object.
-
-        ``ConfigObj(infile=None, options=None, **kwargs)``
-        """
-        if infile is None:
-            infile = []
-        if options is None:
-            options = {}
-        # keyword arguments take precedence over an options dictionary
-        options.update(kwargs)
-        # init the superclass
-        Section.__init__(self, self, 0, self)
-        #
-        defaults = OPTION_DEFAULTS.copy()
-        for entry in options.keys():
-            if entry not in defaults.keys():
-                raise TypeError, 'Unrecognised option "%s".' % entry
-        # TODO: check the values too.
-        #
-        # Add any explicit options to the defaults
-        defaults.update(options)
-        #
-        # initialise a few variables
-        self.filename = None
-        self._errors = []
-        self.raise_errors = defaults['raise_errors']
-        self.interpolation = defaults['interpolation']
-        self.list_values = defaults['list_values']
-        self.create_empty = defaults['create_empty']
-        self.file_error = defaults['file_error']
-        self.stringify = defaults['stringify']
-        self.indent_type = defaults['indent_type']
-        self.encoding = defaults['encoding']
-        self.default_encoding = defaults['default_encoding']
-        self.BOM = False
-        self.newlines = None
-        self.write_empty_values = defaults['write_empty_values']
-        self.unrepr = defaults['unrepr']
-        #
-        self.initial_comment = []
-        self.final_comment = []
-        #
-        if isinstance(infile, StringTypes):
-            self.filename = infile
-            if os.path.isfile(infile):
-                infile = open(infile).read() or []
-            elif self.file_error:
-                # raise an error if the file doesn't exist
-                raise IOError, 'Config file not found: "%s".' % self.filename
-            else:
-                # file doesn't already exist
-                if self.create_empty:
-                    # this is a good test that the filename specified
-                    # isn't impossible - like on a non existent device
-                    h = open(infile, 'w')
-                    h.write('')
-                    h.close()
-                infile = []
-        elif isinstance(infile, (list, tuple)):
-            infile = list(infile)
-        elif isinstance(infile, dict):
-            # initialise self
-            # the Section class handles creating subsections
-            if isinstance(infile, ConfigObj):
-                # get a copy of our ConfigObj
-                infile = infile.dict()
-            for entry in infile:
-                self[entry] = infile[entry]
-            del self._errors
-            if defaults['configspec'] is not None:
-                self._handle_configspec(defaults['configspec'])
-            else:
-                self.configspec = None
-            return
-        elif hasattr(infile, 'read'):
-            # This supports file like objects
-            infile = infile.read() or []
-            # needs splitting into lines - but needs doing *after* decoding
-            # in case it's not an 8 bit encoding
-        else:
-            raise TypeError, ('infile must be a filename,'
-                ' file like object, or list of lines.')
-        #
-        if infile:
-            # don't do it for the empty ConfigObj
-            infile = self._handle_bom(infile)
-            # infile is now *always* a list
-            #
-            # Set the newlines attribute (first line ending it finds)
-            # and strip trailing '\n' or '\r' from lines
-            for line in infile:
-                if (not line) or (line[-1] not in '\r\n'):
-                    continue
-                for end in ('\r\n', '\n', '\r'):
-                    if line.endswith(end):
-                        self.newlines = end
-                        break
-                break
-            infile = [line.rstrip('\r\n') for line in infile]
-        #
-        self._parse(infile)
-        # if we had any errors, now is the time to raise them
-        if self._errors:
-            error = ConfigObjError("Parsing failed.")
-            # set the errors attribute; it's a list of tuples:
-            # (error_type, message, line_number)
-            error.errors = self._errors
-            # set the config attribute
-            error.config = self
-            raise error
-        # delete private attributes
-        del self._errors
-        #
-        if defaults['configspec'] is None:
-            self.configspec = None
-        else:
-            self._handle_configspec(defaults['configspec'])
-
-    def _handle_bom(self, infile):
-        """
-        Handle any BOM, and decode if necessary.
-
-        If an encoding is specified, that *must* be used - but the BOM should
-        still be removed (and the BOM attribute set).
-
-        (If the encoding is wrongly specified, then a BOM for an alternative
-        encoding won't be discovered or removed.)
-
-        If an encoding is not specified, UTF8 or UTF16 BOM will be detected and
-        removed. The BOM attribute will be set. UTF16 will be decoded to
-        unicode.
-
-        NOTE: This method must not be called with an empty ``infile``.
-
-        Specifying the *wrong* encoding is likely to cause a
-        ``UnicodeDecodeError``.
-
-        ``infile`` must always be returned as a list of lines, but may be
-        passed in as a single string.
-        """
-        if ((self.encoding is not None) and
-            (self.encoding.lower() not in BOM_LIST)):
-            # No need to check for a BOM
-            # encoding specified doesn't have one
-            # just decode
-            return self._decode(infile, self.encoding)
-        #
-        if isinstance(infile, (list, tuple)):
-            line = infile[0]
-        else:
-            line = infile
-        if self.encoding is not None:
-            # encoding explicitly supplied
-            # And it could have an associated BOM
-            # TODO: if encoding is just UTF16 - we ought to check for both
-            # TODO: big endian and little endian versions.
-            enc = BOM_LIST[self.encoding.lower()]
-            if enc == 'utf_16':
-                # For UTF16 we try big endian and little endian
-                for BOM, (encoding, final_encoding) in BOMS.items():
-                    if not final_encoding:
-                        # skip UTF8
-                        continue
-                    if infile.startswith(BOM):
-                        ### BOM discovered
-                        ##self.BOM = True
-                        # Don't need to remove BOM
-                        return self._decode(infile, encoding)
-                #
-                # If we get this far, will *probably* raise a DecodeError
-                # As it doesn't appear to start with a BOM
-                return self._decode(infile, self.encoding)
-            #
-            # Must be UTF8
-            BOM = BOM_SET[enc]
-            if not line.startswith(BOM):
-                return self._decode(infile, self.encoding)
-            #
-            newline = line[len(BOM):]
-            #
-            # BOM removed
-            if isinstance(infile, (list, tuple)):
-                infile[0] = newline
-            else:
-                infile = newline
-            self.BOM = True
-            return self._decode(infile, self.encoding)
-        #
-        # No encoding specified - so we need to check for UTF8/UTF16
-        for BOM, (encoding, final_encoding) in BOMS.items():
-            if not line.startswith(BOM):
-                continue
-            else:
-                # BOM discovered
-                self.encoding = final_encoding
-                if not final_encoding:
-                    self.BOM = True
-                    # UTF8
-                    # remove BOM
-                    newline = line[len(BOM):]
-                    if isinstance(infile, (list, tuple)):
-                        infile[0] = newline
-                    else:
-                        infile = newline
-                    # UTF8 - don't decode
-                    if isinstance(infile, StringTypes):
-                        return infile.splitlines(True)
-                    else:
-                        return infile
-                # UTF16 - have to decode
-                return self._decode(infile, encoding)
-        #
-        # No BOM discovered and no encoding specified, just return
-        if isinstance(infile, StringTypes):
-            # infile read from a file will be a single string
-            return infile.splitlines(True)
-        else:
-            return infile
-
-    def _a_to_u(self, string):
-        """Decode ascii strings to unicode if a self.encoding is specified."""
-        if not self.encoding:
-            return string
-        else:
-            return string.decode('ascii')
-
-    def _decode(self, infile, encoding):
-        """
-        Decode infile to unicode. Using the specified encoding.
-
-        if is a string, it also needs converting to a list.
-        """
-        if isinstance(infile, StringTypes):
-            # can't be unicode
-            # NOTE: Could raise a ``UnicodeDecodeError``
-            return infile.decode(encoding).splitlines(True)
-        for i, line in enumerate(infile):
-            if not isinstance(line, unicode):
-                # NOTE: The isinstance test here handles mixed lists of unicode/string
-                # NOTE: But the decode will break on any non-string values
-                # NOTE: Or could raise a ``UnicodeDecodeError``
-                infile[i] = line.decode(encoding)
-        return infile
-
-    def _decode_element(self, line):
-        """Decode element to unicode if necessary."""
-        if not self.encoding:
-            return line
-        if isinstance(line, str) and self.default_encoding:
-            return line.decode(self.default_encoding)
-        return line
-
-    def _str(self, value):
-        """
-        Used by ``stringify`` within validate, to turn non-string values
-        into strings.
-        """
-        if not isinstance(value, StringTypes):
-            return str(value)
-        else:
-            return value
-
-    def _parse(self, infile):
-        """
-        Actually parse the config file
-
-        Testing Interpolation
-
-        >>> c = ConfigObj()
-        >>> c['DEFAULT'] = {
-        ...     'b': 'goodbye',
-        ...     'userdir': 'c:\\\\home',
-        ...     'c': '%(d)s',
-        ...     'd': '%(c)s'
-        ... }
-        >>> c['section'] = {
-        ...     'a': '%(datadir)s\\\\some path\\\\file.py',
-        ...     'b': '%(userdir)s\\\\some path\\\\file.py',
-        ...     'c': 'Yo %(a)s',
-        ...     'd': '%(not_here)s',
-        ...     'e': '%(c)s',
-        ... }
-        >>> c['section']['DEFAULT'] = {
-        ...     'datadir': 'c:\\\\silly_test',
-        ...     'a': 'hello - %(b)s',
-        ... }
-        >>> c['section']['a'] == 'c:\\\\silly_test\\\\some path\\\\file.py'
-        1
-        >>> c['section']['b'] == 'c:\\\\home\\\\some path\\\\file.py'
-        1
-        >>> c['section']['c'] == 'Yo hello - goodbye'
-        1
-
-        Switching Interpolation Off
-
-        >>> c.interpolation = False
-        >>> c['section']['a'] == '%(datadir)s\\\\some path\\\\file.py'
-        1
-        >>> c['section']['b'] == '%(userdir)s\\\\some path\\\\file.py'
-        1
-        >>> c['section']['c'] == 'Yo %(a)s'
-        1
-
-        Testing the interpolation errors.
-
-        >>> c.interpolation = True
-        >>> c['section']['d']
-        Traceback (most recent call last):
-        MissingInterpolationOption: missing option "not_here" in interpolation.
-        >>> c['section']['e']
-        Traceback (most recent call last):
-        InterpolationDepthError: max interpolation depth exceeded in value "%(c)s".
-
-        Testing our quoting.
-
-        >>> i._quote('\"""\'\'\'')
-        Traceback (most recent call last):
-        SyntaxError: EOF while scanning triple-quoted string
-        >>> try:
-        ...     i._quote('\\n', multiline=False)
-        ... except ConfigObjError, e:
-        ...    e.msg
-        'Value "\\n" cannot be safely quoted.'
-        >>> k._quote(' "\' ', multiline=False)
-        Traceback (most recent call last):
-        SyntaxError: EOL while scanning single-quoted string
-
-        Testing with "stringify" off.
-        >>> c.stringify = False
-        >>> c['test'] = 1
-        Traceback (most recent call last):
-        TypeError: Value is not a string "1".
-
-        Testing Empty values.
-        >>> cfg_with_empty = '''
-        ... k =
-        ... k2 =# comment test
-        ... val = test
-        ... val2 = ,
-        ... val3 = 1,
-        ... val4 = 1, 2
-        ... val5 = 1, 2, '''.splitlines()
-        >>> cwe = ConfigObj(cfg_with_empty)
-        >>> cwe == {'k': '', 'k2': '', 'val': 'test', 'val2': [],
-        ...  'val3': ['1'], 'val4': ['1', '2'], 'val5': ['1', '2']}
-        1
-        >>> cwe = ConfigObj(cfg_with_empty, list_values=False)
-        >>> cwe == {'k': '', 'k2': '', 'val': 'test', 'val2': ',',
-        ...  'val3': '1,', 'val4': '1, 2', 'val5': '1, 2,'}
-        1
-        """
-        temp_list_values = self.list_values
-        if self.unrepr:
-            self.list_values = False
-        comment_list = []
-        done_start = False
-        this_section = self
-        maxline = len(infile) - 1
-        cur_index = -1
-        reset_comment = False
-        while cur_index < maxline:
-            if reset_comment:
-                comment_list = []
-            cur_index += 1
-            line = infile[cur_index]
-            sline = line.strip()
-            # do we have anything on the line ?
-            if not sline or sline.startswith('#'):
-                reset_comment = False
-                comment_list.append(line)
-                continue
-            if not done_start:
-                # preserve initial comment
-                self.initial_comment = comment_list
-                comment_list = []
-                done_start = True
-            reset_comment = True
-            # first we check if it's a section marker
-            mat = self._sectionmarker.match(line)
-##            print >> sys.stderr, sline, mat
-            if mat is not None:
-                # is a section line
-                (indent, sect_open, sect_name, sect_close, comment) = (
-                    mat.groups())
-                if indent and (self.indent_type is None):
-                    self.indent_type = indent[0]
-                cur_depth = sect_open.count('[')
-                if cur_depth != sect_close.count(']'):
-                    self._handle_error(
-                        "Cannot compute the section depth at line %s.",
-                        NestingError, infile, cur_index)
-                    continue
-                if cur_depth < this_section.depth:
-                    # the new section is dropping back to a previous level
-                    try:
-                        parent = self._match_depth(
-                            this_section,
-                            cur_depth).parent
-                    except SyntaxError:
-                        self._handle_error(
-                            "Cannot compute nesting level at line %s.",
-                            NestingError, infile, cur_index)
-                        continue
-                elif cur_depth == this_section.depth:
-                    # the new section is a sibling of the current section
-                    parent = this_section.parent
-                elif cur_depth == this_section.depth + 1:
-                    # the new section is a child the current section
-                    parent = this_section
-                else:
-                    self._handle_error(
-                        "Section too nested at line %s.",
-                        NestingError, infile, cur_index)
-                #
-                sect_name = self._unquote(sect_name)
-                if parent.has_key(sect_name):
-##                    print >> sys.stderr, sect_name
-                    self._handle_error(
-                        'Duplicate section name at line %s.',
-                        DuplicateError, infile, cur_index)
-                    continue
-                # create the new section
-                this_section = Section(
-                    parent,
-                    cur_depth,
-                    self,
-                    name=sect_name)
-                parent[sect_name] = this_section
-                parent.inline_comments[sect_name] = comment
-                parent.comments[sect_name] = comment_list
-##                print >> sys.stderr, parent[sect_name] is this_section
-                continue
-            #
-            # it's not a section marker,
-            # so it should be a valid ``key = value`` line
-            mat = self._keyword.match(line)
-##            print >> sys.stderr, sline, mat
-            if mat is not None:
-                # is a keyword value
-                # value will include any inline comment
-                (indent, key, value) = mat.groups()
-                if indent and (self.indent_type is None):
-                    self.indent_type = indent[0]
-                # check for a multiline value
-                if value[:3] in ['"""', "'''"]:
-                    try:
-                        (value, comment, cur_index) = self._multiline(
-                            value, infile, cur_index, maxline)
-                    except SyntaxError:
-                        self._handle_error(
-                            'Parse error in value at line %s.',
-                            ParseError, infile, cur_index)
-                        continue
-                    else:
-                        if self.unrepr:
-                            value = unrepr(value)
-                else:
-                    # extract comment and lists
-                    try:
-                        (value, comment) = self._handle_value(value)
-                    except SyntaxError:
-                        self._handle_error(
-                            'Parse error in value at line %s.',
-                            ParseError, infile, cur_index)
-                        continue
-                #
-##                print >> sys.stderr, sline
-                key = self._unquote(key)
-                if this_section.has_key(key):
-                    self._handle_error(
-                        'Duplicate keyword name at line %s.',
-                        DuplicateError, infile, cur_index)
-                    continue
-                # add the key
-##                print >> sys.stderr, this_section.name
-                # we set unrepr because if we have got this far we will never
-                # be creating a new section
-                this_section.__setitem__(key, value, unrepr=True)
-                this_section.inline_comments[key] = comment
-                this_section.comments[key] = comment_list
-##                print >> sys.stderr, key, this_section[key]
-##                if this_section.name is not None:
-##                    print >> sys.stderr, this_section
-##                    print >> sys.stderr, this_section.parent
-##                    print >> sys.stderr, this_section.parent[this_section.name]
-                continue
-            #
-            # it neither matched as a keyword
-            # or a section marker
-            self._handle_error(
-                'Invalid line at line "%s".',
-                ParseError, infile, cur_index)
-        if self.indent_type is None:
-            # no indentation used, set the type accordingly
-            self.indent_type = ''
-        # preserve the final comment
-        if not self and not self.initial_comment:
-            self.initial_comment = comment_list
-        elif not reset_comment:
-            self.final_comment = comment_list
-        self.list_values = temp_list_values
-
-    def _match_depth(self, sect, depth):
-        """
-        Given a section and a depth level, walk back through the sections
-        parents to see if the depth level matches a previous section.
-
-        Return a reference to the right section,
-        or raise a SyntaxError.
-        """
-        while depth < sect.depth:
-            if sect is sect.parent:
-                # we've reached the top level already
-                raise SyntaxError
-            sect = sect.parent
-        if sect.depth == depth:
-            return sect
-        # shouldn't get here
-        raise SyntaxError
-
-    def _handle_error(self, text, ErrorClass, infile, cur_index):
-        """
-        Handle an error according to the error settings.
-
-        Either raise the error or store it.
-        The error will have occured at ``cur_index``
-        """
-        line = infile[cur_index]
-        message = text % cur_index
-        error = ErrorClass(message, cur_index, line)
-        if self.raise_errors:
-            # raise the error - parsing stops here
-            raise error
-        # store the error
-        # reraise when parsing has finished
-        self._errors.append(error)
-
-    def _unquote(self, value):
-        """Return an unquoted version of a value"""
-        if (value[0] == value[-1]) and (value[0] in ('"', "'")):
-            value = value[1:-1]
-        return value
-
-    def _quote(self, value, multiline=True):
-        """
-        Return a safely quoted version of a value.
-
-        Raise a ConfigObjError if the value cannot be safely quoted.
-        If multiline is ``True`` (default) then use triple quotes
-        if necessary.
-
-        Don't quote values that don't need it.
-        Recursively quote members of a list and return a comma joined list.
-        Multiline is ``False`` for lists.
-        Obey list syntax for empty and single member lists.
-
-        If ``list_values=False`` then the value is only quoted if it contains
-        a ``\n`` (is multiline).
-
-        If ``write_empty_values`` is set, and the value is an empty string, it
-        won't be quoted.
-        """
-        if multiline and self.write_empty_values and value == '':
-            # Only if multiline is set, so that it is used for values not
-            # keys, and not values that are part of a list
-            return ''
-        if multiline and isinstance(value, (list, tuple)):
-            if not value:
-                return ','
-            elif len(value) == 1:
-                return self._quote(value[0], multiline=False) + ','
-            return ', '.join([self._quote(val, multiline=False)
-                for val in value])
-        if not isinstance(value, StringTypes):
-            if self.stringify:
-                value = str(value)
-            else:
-                raise TypeError, 'Value "%s" is not a string.' % value
-        squot = "'%s'"
-        dquot = '"%s"'
-        noquot = "%s"
-        wspace_plus = ' \r\t\n\v\t\'"'
-        tsquot = '"""%s"""'
-        tdquot = "'''%s'''"
-        if not value:
-            return '""'
-        if (not self.list_values and '\n' not in value) or not (multiline and
-                ((("'" in value) and ('"' in value)) or ('\n' in value))):
-            if not self.list_values:
-                # we don't quote if ``list_values=False``
-                quot = noquot
-            # for normal values either single or double quotes will do
-            elif '\n' in value:
-                # will only happen if multiline is off - e.g. '\n' in key
-                raise ConfigObjError, ('Value "%s" cannot be safely quoted.' %
-                    value)
-            elif ((value[0] not in wspace_plus) and
-                    (value[-1] not in wspace_plus) and
-                    (',' not in value)):
-                quot = noquot
-            else:
-                if ("'" in value) and ('"' in value):
-                    raise ConfigObjError, (
-                        'Value "%s" cannot be safely quoted.' % value)
-                elif '"' in value:
-                    quot = squot
-                else:
-                    quot = dquot
-        else:
-            # if value has '\n' or "'" *and* '"', it will need triple quotes
-            if (value.find('"""') != -1) and (value.find("'''") != -1):
-                raise ConfigObjError, (
-                    'Value "%s" cannot be safely quoted.' % value)
-            if value.find('"""') == -1:
-                quot = tdquot
-            else:
-                quot = tsquot
-        return quot % value
-
-    def _handle_value(self, value):
-        """
-        Given a value string, unquote, remove comment,
-        handle lists. (including empty and single member lists)
-
-        Testing list values.
-
-        >>> testconfig3 = '''
-        ... a = ,
-        ... b = test,
-        ... c = test1, test2   , test3
-        ... d = test1, test2, test3,
-        ... '''
-        >>> d = ConfigObj(testconfig3.split('\\n'), raise_errors=True)
-        >>> d['a'] == []
-        1
-        >>> d['b'] == ['test']
-        1
-        >>> d['c'] == ['test1', 'test2', 'test3']
-        1
-        >>> d['d'] == ['test1', 'test2', 'test3']
-        1
-
-        Testing with list values off.
-
-        >>> e = ConfigObj(
-        ...     testconfig3.split('\\n'),
-        ...     raise_errors=True,
-        ...     list_values=False)
-        >>> e['a'] == ','
-        1
-        >>> e['b'] == 'test,'
-        1
-        >>> e['c'] == 'test1, test2   , test3'
-        1
-        >>> e['d'] == 'test1, test2, test3,'
-        1
-
-        Testing creating from a dictionary.
-
-        >>> f = {
-        ...     'key1': 'val1',
-        ...     'key2': 'val2',
-        ...     'section 1': {
-        ...         'key1': 'val1',
-        ...         'key2': 'val2',
-        ...         'section 1b': {
-        ...             'key1': 'val1',
-        ...             'key2': 'val2',
-        ...         },
-        ...     },
-        ...     'section 2': {
-        ...         'key1': 'val1',
-        ...         'key2': 'val2',
-        ...         'section 2b': {
-        ...             'key1': 'val1',
-        ...             'key2': 'val2',
-        ...         },
-        ...     },
-        ...      'key3': 'val3',
-        ... }
-        >>> g = ConfigObj(f)
-        >>> f == g
-        1
-
-        Testing we correctly detect badly built list values (4 of them).
-
-        >>> testconfig4 = '''
-        ... config = 3,4,,
-        ... test = 3,,4
-        ... fish = ,,
-        ... dummy = ,,hello, goodbye
-        ... '''
-        >>> try:
-        ...     ConfigObj(testconfig4.split('\\n'))
-        ... except ConfigObjError, e:
-        ...     len(e.errors)
-        4
-
-        Testing we correctly detect badly quoted values (4 of them).
-
-        >>> testconfig5 = '''
-        ... config = "hello   # comment
-        ... test = 'goodbye
-        ... fish = 'goodbye   # comment
-        ... dummy = "hello again
-        ... '''
-        >>> try:
-        ...     ConfigObj(testconfig5.split('\\n'))
-        ... except ConfigObjError, e:
-        ...     len(e.errors)
-        4
-        """
-        # do we look for lists in values ?
-        if not self.list_values:
-            mat = self._nolistvalue.match(value)
-            if mat is None:
-                raise SyntaxError
-            (value, comment) = mat.groups()
-            if not self.unrepr:
-                # NOTE: we don't unquote here
-                return (value, comment)
-            else:
-                return (unrepr(value), comment)
-        mat = self._valueexp.match(value)
-        if mat is None:
-            # the value is badly constructed, probably badly quoted,
-            # or an invalid list
-            raise SyntaxError
-        (list_values, single, empty_list, comment) = mat.groups()
-        if (list_values == '') and (single is None):
-            # change this if you want to accept empty values
-            raise SyntaxError
-        # NOTE: note there is no error handling from here if the regex
-        # is wrong: then incorrect values will slip through
-        if empty_list is not None:
-            # the single comma - meaning an empty list
-            return ([], comment)
-        if single is not None:
-            # handle empty values
-            if list_values and not single:
-                # FIXME: the '' is a workaround because our regex now matches
-                #   '' at the end of a list if it has a trailing comma
-                single = None
-            else:
-                single = single or '""'
-                single = self._unquote(single)
-        if list_values == '':
-            # not a list value
-            return (single, comment)
-        the_list = self._listvalueexp.findall(list_values)
-        the_list = [self._unquote(val) for val in the_list]
-        if single is not None:
-            the_list += [single]
-        return (the_list, comment)
-
-    def _multiline(self, value, infile, cur_index, maxline):
-        """
-        Extract the value, where we are in a multiline situation
-
-        Testing multiline values.
-
-        >>> i == {
-        ...     'name4': ' another single line value ',
-        ...     'multi section': {
-        ...         'name4': '\\n        Well, this is a\\n        multiline '
-        ...             'value\\n        ',
-        ...         'name2': '\\n        Well, this is a\\n        multiline '
-        ...             'value\\n        ',
-        ...         'name3': '\\n        Well, this is a\\n        multiline '
-        ...             'value\\n        ',
-        ...         'name1': '\\n        Well, this is a\\n        multiline '
-        ...             'value\\n        ',
-        ...     },
-        ...     'name2': ' another single line value ',
-        ...     'name3': ' a single line value ',
-        ...     'name1': ' a single line value ',
-        ... }
-        1
-        """
-        quot = value[:3]
-        newvalue = value[3:]
-        single_line = self._triple_quote[quot][0]
-        multi_line = self._triple_quote[quot][1]
-        mat = single_line.match(value)
-        if mat is not None:
-            retval = list(mat.groups())
-            retval.append(cur_index)
-            return retval
-        elif newvalue.find(quot) != -1:
-            # somehow the triple quote is missing
-            raise SyntaxError
-        #
-        while cur_index < maxline:
-            cur_index += 1
-            newvalue += '\n'
-            line = infile[cur_index]
-            if line.find(quot) == -1:
-                newvalue += line
-            else:
-                # end of multiline, process it
-                break
-        else:
-            # we've got to the end of the config, oops...
-            raise SyntaxError
-        mat = multi_line.match(line)
-        if mat is None:
-            # a badly formed line
-            raise SyntaxError
-        (value, comment) = mat.groups()
-        return (newvalue + value, comment, cur_index)
-
-    def _handle_configspec(self, configspec):
-        """Parse the configspec."""
-        # FIXME: Should we check that the configspec was created with the
-        #   correct settings ? (i.e. ``list_values=False``)
-        if not isinstance(configspec, ConfigObj):
-            try:
-                configspec = ConfigObj(
-                    configspec,
-                    raise_errors=True,
-                    file_error=True,
-                    list_values=False)
-            except ConfigObjError, e:
-                # FIXME: Should these errors have a reference
-                # to the already parsed ConfigObj ?
-                raise ConfigspecError('Parsing configspec failed: %s' % e)
-            except IOError, e:
-                raise IOError('Reading configspec failed: %s' % e)
-        self._set_configspec_value(configspec, self)
-
-    def _set_configspec_value(self, configspec, section):
-        """Used to recursively set configspec values."""
-        if '__many__' in configspec.sections:
-            section.configspec['__many__'] = configspec['__many__']
-            if len(configspec.sections) > 1:
-                # FIXME: can we supply any useful information here ?
-                raise RepeatSectionError
-        if hasattr(configspec, 'initial_comment'):
-            section._configspec_initial_comment = configspec.initial_comment
-            section._configspec_final_comment = configspec.final_comment
-            section._configspec_encoding = configspec.encoding
-            section._configspec_BOM = configspec.BOM
-            section._configspec_newlines = configspec.newlines
-            section._configspec_indent_type = configspec.indent_type
-        for entry in configspec.scalars:
-            section._configspec_comments[entry] = configspec.comments[entry]
-            section._configspec_inline_comments[entry] = (
-                configspec.inline_comments[entry])
-            section.configspec[entry] = configspec[entry]
-            section._order.append(entry)
-        for entry in configspec.sections:
-            if entry == '__many__':
-                continue
-            section._cs_section_comments[entry] = configspec.comments[entry]
-            section._cs_section_inline_comments[entry] = (
-                configspec.inline_comments[entry])
-            if not section.has_key(entry):
-                section[entry] = {}
-            self._set_configspec_value(configspec[entry], section[entry])
-
-    def _handle_repeat(self, section, configspec):
-        """Dynamically assign configspec for repeated section."""
-        try:
-            section_keys = configspec.sections
-            scalar_keys = configspec.scalars
-        except AttributeError:
-            section_keys = [entry for entry in configspec
-                                if isinstance(configspec[entry], dict)]
-            scalar_keys = [entry for entry in configspec
-                                if not isinstance(configspec[entry], dict)]
-        if '__many__' in section_keys and len(section_keys) > 1:
-            # FIXME: can we supply any useful information here ?
-            raise RepeatSectionError
-        scalars = {}
-        sections = {}
-        for entry in scalar_keys:
-            val = configspec[entry]
-            scalars[entry] = val
-        for entry in section_keys:
-            val = configspec[entry]
-            if entry == '__many__':
-                scalars[entry] = val
-                continue
-            sections[entry] = val
-        #
-        section.configspec = scalars
-        for entry in sections:
-            if not section.has_key(entry):
-                section[entry] = {}
-            self._handle_repeat(section[entry], sections[entry])
-
-    def _write_line(self, indent_string, entry, this_entry, comment):
-        """Write an individual line, for the write method"""
-        # NOTE: the calls to self._quote here handles non-StringType values.
-        if not self.unrepr:
-            val = self._decode_element(self._quote(this_entry))
-        else:
-            val = repr(this_entry)
-        return '%s%s%s%s%s' % (
-            indent_string,
-            self._decode_element(self._quote(entry, multiline=False)),
-            self._a_to_u(' = '),
-            val,
-            self._decode_element(comment))
-
-    def _write_marker(self, indent_string, depth, entry, comment):
-        """Write a section marker line"""
-        return '%s%s%s%s%s' % (
-            indent_string,
-            self._a_to_u('[' * depth),
-            self._quote(self._decode_element(entry), multiline=False),
-            self._a_to_u(']' * depth),
-            self._decode_element(comment))
-
-    def _handle_comment(self, comment):
-        """
-        Deal with a comment.
-
-        >>> filename = a.filename
-        >>> a.filename = None
-        >>> values = a.write()
-        >>> index = 0
-        >>> while index < 23:
-        ...     index += 1
-        ...     line = values[index-1]
-        ...     assert line.endswith('# comment ' + str(index))
-        >>> a.filename = filename
-
-        >>> start_comment = ['# Initial Comment', '', '#']
-        >>> end_comment = ['', '#', '# Final Comment']
-        >>> newconfig = start_comment + testconfig1.split('\\n') + end_comment
-        >>> nc = ConfigObj(newconfig)
-        >>> nc.initial_comment
-        ['# Initial Comment', '', '#']
-        >>> nc.final_comment
-        ['', '#', '# Final Comment']
-        >>> nc.initial_comment == start_comment
-        1
-        >>> nc.final_comment == end_comment
-        1
-        """
-        if not comment:
-            return ''
-        if self.indent_type == '\t':
-            start = self._a_to_u('\t')
-        else:
-            start = self._a_to_u(' ' * NUM_INDENT_SPACES)
-        if not comment.startswith('#'):
-            start += self._a_to_u('# ')
-        return (start + comment)
-
-    def _compute_indent_string(self, depth):
-        """
-        Compute the indent string, according to current indent_type and depth
-        """
-        if self.indent_type == '':
-            # no indentation at all
-            return ''
-        if self.indent_type == '\t':
-            return '\t' * depth
-        if self.indent_type == ' ':
-            return ' ' * NUM_INDENT_SPACES * depth
-        raise SyntaxError
-
-    # Public methods
-
-    def write(self, outfile=None, section=None):
-        """
-        Write the current ConfigObj as a file
-
-        tekNico: FIXME: use StringIO instead of real files
-
-        >>> filename = a.filename
-        >>> a.filename = 'test.ini'
-        >>> a.write()
-        >>> a.filename = filename
-        >>> a == ConfigObj('test.ini', raise_errors=True)
-        1
-        >>> os.remove('test.ini')
-        >>> b.filename = 'test.ini'
-        >>> b.write()
-        >>> b == ConfigObj('test.ini', raise_errors=True)
-        1
-        >>> os.remove('test.ini')
-        >>> i.filename = 'test.ini'
-        >>> i.write()
-        >>> i == ConfigObj('test.ini', raise_errors=True)
-        1
-        >>> os.remove('test.ini')
-        >>> a = ConfigObj()
-        >>> a['DEFAULT'] = {'a' : 'fish'}
-        >>> a['a'] = '%(a)s'
-        >>> a.write()
-        ['a = %(a)s', '[DEFAULT]', 'a = fish']
-        """
-        if self.indent_type is None:
-            # this can be true if initialised from a dictionary
-            self.indent_type = DEFAULT_INDENT_TYPE
-        #
-        out = []
-        cs = self._a_to_u('#')
-        csp = self._a_to_u('# ')
-        if section is None:
-            int_val = self.interpolation
-            self.interpolation = False
-            section = self
-            for line in self.initial_comment:
-                line = self._decode_element(line)
-                stripped_line = line.strip()
-                if stripped_line and not stripped_line.startswith(cs):
-                    line = csp + line
-                out.append(line)
-        #
-        indent_string = self._a_to_u(
-            self._compute_indent_string(section.depth))
-        for entry in (section.scalars + section.sections):
-            if entry in section.defaults:
-                # don't write out default values
-                continue
-            for comment_line in section.comments[entry]:
-                comment_line = self._decode_element(comment_line.lstrip())
-                if comment_line and not comment_line.startswith(cs):
-                    comment_line = csp + comment_line
-                out.append(indent_string + comment_line)
-            this_entry = section[entry]
-            comment = self._handle_comment(section.inline_comments[entry])
-            #
-            if isinstance(this_entry, dict):
-                # a section
-                out.append(self._write_marker(
-                    indent_string,
-                    this_entry.depth,
-                    entry,
-                    comment))
-                out.extend(self.write(section=this_entry))
-            else:
-                out.append(self._write_line(
-                    indent_string,
-                    entry,
-                    this_entry,
-                    comment))
-        #
-        if section is self:
-            for line in self.final_comment:
-                line = self._decode_element(line)
-                stripped_line = line.strip()
-                if stripped_line and not stripped_line.startswith(cs):
-                    line = csp + line
-                out.append(line)
-            self.interpolation = int_val
-        #
-        if section is not self:
-            return out
-        #
-        if (self.filename is None) and (outfile is None):
-            # output a list of lines
-            # might need to encode
-            # NOTE: This will *screw* UTF16, each line will start with the BOM
-            if self.encoding:
-                out = [l.encode(self.encoding) for l in out]
-            if (self.BOM and ((self.encoding is None) or
-                (BOM_LIST.get(self.encoding.lower()) == 'utf_8'))):
-                # Add the UTF8 BOM
-                if not out:
-                    out.append('')
-                out[0] = BOM_UTF8 + out[0]
-            return out
-        #
-        # Turn the list to a string, joined with correct newlines
-        output = (self._a_to_u(self.newlines or os.linesep)).join(out)
-        if self.encoding:
-            output = output.encode(self.encoding)
-        if (self.BOM and ((self.encoding is None) or
-            (BOM_LIST.get(self.encoding.lower()) == 'utf_8'))):
-            # Add the UTF8 BOM
-            output = BOM_UTF8 + output
-        if outfile is not None:
-            outfile.write(output)
-        else:
-            h = open(self.filename, 'wb')
-            h.write(output)
-            h.close()
-
-    def validate(self, validator, preserve_errors=False, copy=False,
-        section=None):
-        """
-        Test the ConfigObj against a configspec.
-
-        It uses the ``validator`` object from *validate.py*.
-
-        To run ``validate`` on the current ConfigObj, call: ::
-
-            test = config.validate(validator)
-
-        (Normally having previously passed in the configspec when the ConfigObj
-        was created - you can dynamically assign a dictionary of checks to the
-        ``configspec`` attribute of a section though).
-
-        It returns ``True`` if everything passes, or a dictionary of
-        pass/fails (True/False). If every member of a subsection passes, it
-        will just have the value ``True``. (It also returns ``False`` if all
-        members fail).
-
-        In addition, it converts the values from strings to their native
-        types if their checks pass (and ``stringify`` is set).
-
-        If ``preserve_errors`` is ``True`` (``False`` is default) then instead
-        of a marking a fail with a ``False``, it will preserve the actual
-        exception object. This can contain info about the reason for failure.
-        For example the ``VdtValueTooSmallError`` indeicates that the value
-        supplied was too small. If a value (or section) is missing it will
-        still be marked as ``False``.
-
-        You must have the validate module to use ``preserve_errors=True``.
-
-        You can then use the ``flatten_errors`` function to turn your nested
-        results dictionary into a flattened list of failures - useful for
-        displaying meaningful error messages.
-
-        >>> try:
-        ...     from validate import Validator
-        ... except ImportError:
-        ...     print >> sys.stderr, 'Cannot import the Validator object, skipping the related tests'
-        ... else:
-        ...     config = '''
-        ...     test1=40
-        ...     test2=hello
-        ...     test3=3
-        ...     test4=5.0
-        ...     [section]
-        ...         test1=40
-        ...         test2=hello
-        ...         test3=3
-        ...         test4=5.0
-        ...         [[sub section]]
-        ...             test1=40
-        ...             test2=hello
-        ...             test3=3
-        ...             test4=5.0
-        ... '''.split('\\n')
-        ...     configspec = '''
-        ...     test1= integer(30,50)
-        ...     test2= string
-        ...     test3=integer
-        ...     test4=float(6.0)
-        ...     [section ]
-        ...         test1=integer(30,50)
-        ...         test2=string
-        ...         test3=integer
-        ...         test4=float(6.0)
-        ...         [[sub section]]
-        ...             test1=integer(30,50)
-        ...             test2=string
-        ...             test3=integer
-        ...             test4=float(6.0)
-        ...     '''.split('\\n')
-        ...     val = Validator()
-        ...     c1 = ConfigObj(config, configspec=configspec)
-        ...     test = c1.validate(val)
-        ...     test == {
-        ...         'test1': True,
-        ...         'test2': True,
-        ...         'test3': True,
-        ...         'test4': False,
-        ...         'section': {
-        ...             'test1': True,
-        ...             'test2': True,
-        ...             'test3': True,
-        ...             'test4': False,
-        ...             'sub section': {
-        ...                 'test1': True,
-        ...                 'test2': True,
-        ...                 'test3': True,
-        ...                 'test4': False,
-        ...             },
-        ...         },
-        ...     }
-        1
-        >>> val.check(c1.configspec['test4'], c1['test4'])
-        Traceback (most recent call last):
-        VdtValueTooSmallError: the value "5.0" is too small.
-
-        >>> val_test_config = '''
-        ...     key = 0
-        ...     key2 = 1.1
-        ...     [section]
-        ...     key = some text
-        ...     key2 = 1.1, 3.0, 17, 6.8
-        ...         [[sub-section]]
-        ...         key = option1
-        ...         key2 = True'''.split('\\n')
-        >>> val_test_configspec = '''
-        ...     key = integer
-        ...     key2 = float
-        ...     [section]
-        ...     key = string
-        ...     key2 = float_list(4)
-        ...        [[sub-section]]
-        ...        key = option(option1, option2)
-        ...        key2 = boolean'''.split('\\n')
-        >>> val_test = ConfigObj(val_test_config, configspec=val_test_configspec)
-        >>> val_test.validate(val)
-        1
-        >>> val_test['key'] = 'text not a digit'
-        >>> val_res = val_test.validate(val)
-        >>> val_res == {'key2': True, 'section': True, 'key': False}
-        1
-        >>> configspec = '''
-        ...     test1=integer(30,50, default=40)
-        ...     test2=string(default="hello")
-        ...     test3=integer(default=3)
-        ...     test4=float(6.0, default=6.0)
-        ...     [section ]
-        ...         test1=integer(30,50, default=40)
-        ...         test2=string(default="hello")
-        ...         test3=integer(default=3)
-        ...         test4=float(6.0, default=6.0)
-        ...         [[sub section]]
-        ...             test1=integer(30,50, default=40)
-        ...             test2=string(default="hello")
-        ...             test3=integer(default=3)
-        ...             test4=float(6.0, default=6.0)
-        ...     '''.split('\\n')
-        >>> default_test = ConfigObj(['test1=30'], configspec=configspec)
-        >>> default_test
-        {'test1': '30', 'section': {'sub section': {}}}
-        >>> default_test.validate(val)
-        1
-        >>> default_test == {
-        ...     'test1': 30,
-        ...     'test2': 'hello',
-        ...     'test3': 3,
-        ...     'test4': 6.0,
-        ...     'section': {
-        ...         'test1': 40,
-        ...         'test2': 'hello',
-        ...         'test3': 3,
-        ...         'test4': 6.0,
-        ...         'sub section': {
-        ...             'test1': 40,
-        ...             'test3': 3,
-        ...             'test2': 'hello',
-        ...             'test4': 6.0,
-        ...         },
-        ...     },
-        ... }
-        1
-
-        Now testing with repeated sections : BIG TEST
-
-        >>> repeated_1 = '''
-        ... [dogs]
-        ...     [[__many__]] # spec for a dog
-        ...         fleas = boolean(default=True)
-        ...         tail = option(long, short, default=long)
-        ...         name = string(default=rover)
-        ...         [[[__many__]]]  # spec for a puppy
-        ...             name = string(default="son of rover")
-        ...             age = float(default=0.0)
-        ... [cats]
-        ...     [[__many__]] # spec for a cat
-        ...         fleas = boolean(default=True)
-        ...         tail = option(long, short, default=short)
-        ...         name = string(default=pussy)
-        ...         [[[__many__]]] # spec for a kitten
-        ...             name = string(default="son of pussy")
-        ...             age = float(default=0.0)
-        ...         '''.split('\\n')
-        >>> repeated_2 = '''
-        ... [dogs]
-        ...
-        ...     # blank dogs with puppies
-        ...     # should be filled in by the configspec
-        ...     [[dog1]]
-        ...         [[[puppy1]]]
-        ...         [[[puppy2]]]
-        ...         [[[puppy3]]]
-        ...     [[dog2]]
-        ...         [[[puppy1]]]
-        ...         [[[puppy2]]]
-        ...         [[[puppy3]]]
-        ...     [[dog3]]
-        ...         [[[puppy1]]]
-        ...         [[[puppy2]]]
-        ...         [[[puppy3]]]
-        ... [cats]
-        ...
-        ...     # blank cats with kittens
-        ...     # should be filled in by the configspec
-        ...     [[cat1]]
-        ...         [[[kitten1]]]
-        ...         [[[kitten2]]]
-        ...         [[[kitten3]]]
-        ...     [[cat2]]
-        ...         [[[kitten1]]]
-        ...         [[[kitten2]]]
-        ...         [[[kitten3]]]
-        ...     [[cat3]]
-        ...         [[[kitten1]]]
-        ...         [[[kitten2]]]
-        ...         [[[kitten3]]]
-        ... '''.split('\\n')
-        >>> repeated_3 = '''
-        ... [dogs]
-        ...
-        ...     [[dog1]]
-        ...     [[dog2]]
-        ...     [[dog3]]
-        ... [cats]
-        ...
-        ...     [[cat1]]
-        ...     [[cat2]]
-        ...     [[cat3]]
-        ... '''.split('\\n')
-        >>> repeated_4 = '''
-        ... [__many__]
-        ...
-        ...     name = string(default=Michael)
-        ...     age = float(default=0.0)
-        ...     sex = option(m, f, default=m)
-        ... '''.split('\\n')
-        >>> repeated_5 = '''
-        ... [cats]
-        ... [[__many__]]
-        ...     fleas = boolean(default=True)
-        ...     tail = option(long, short, default=short)
-        ...     name = string(default=pussy)
-        ...     [[[description]]]
-        ...         height = float(default=3.3)
-        ...         weight = float(default=6)
-        ...         [[[[coat]]]]
-        ...             fur = option(black, grey, brown, "tortoise shell", default=black)
-        ...             condition = integer(0,10, default=5)
-        ... '''.split('\\n')
-        >>> from validate import Validator
-        >>> val= Validator()
-        >>> repeater = ConfigObj(repeated_2, configspec=repeated_1)
-        >>> repeater.validate(val)
-        1
-        >>> repeater == {
-        ...     'dogs': {
-        ...         'dog1': {
-        ...             'fleas': True,
-        ...             'tail': 'long',
-        ...             'name': 'rover',
-        ...             'puppy1': {'name': 'son of rover', 'age': 0.0},
-        ...             'puppy2': {'name': 'son of rover', 'age': 0.0},
-        ...             'puppy3': {'name': 'son of rover', 'age': 0.0},
-        ...         },
-        ...         'dog2': {
-        ...             'fleas': True,
-        ...             'tail': 'long',
-        ...             'name': 'rover',
-        ...             'puppy1': {'name': 'son of rover', 'age': 0.0},
-        ...             'puppy2': {'name': 'son of rover', 'age': 0.0},
-        ...             'puppy3': {'name': 'son of rover', 'age': 0.0},
-        ...         },
-        ...         'dog3': {
-        ...             'fleas': True,
-        ...             'tail': 'long',
-        ...             'name': 'rover',
-        ...             'puppy1': {'name': 'son of rover', 'age': 0.0},
-        ...             'puppy2': {'name': 'son of rover', 'age': 0.0},
-        ...             'puppy3': {'name': 'son of rover', 'age': 0.0},
-        ...         },
-        ...     },
-        ...     'cats': {
-        ...         'cat1': {
-        ...             'fleas': True,
-        ...             'tail': 'short',
-        ...             'name': 'pussy',
-        ...             'kitten1': {'name': 'son of pussy', 'age': 0.0},
-        ...             'kitten2': {'name': 'son of pussy', 'age': 0.0},
-        ...             'kitten3': {'name': 'son of pussy', 'age': 0.0},
-        ...         },
-        ...         'cat2': {
-        ...             'fleas': True,
-        ...             'tail': 'short',
-        ...             'name': 'pussy',
-        ...             'kitten1': {'name': 'son of pussy', 'age': 0.0},
-        ...             'kitten2': {'name': 'son of pussy', 'age': 0.0},
-        ...             'kitten3': {'name': 'son of pussy', 'age': 0.0},
-        ...         },
-        ...         'cat3': {
-        ...             'fleas': True,
-        ...             'tail': 'short',
-        ...             'name': 'pussy',
-        ...             'kitten1': {'name': 'son of pussy', 'age': 0.0},
-        ...             'kitten2': {'name': 'son of pussy', 'age': 0.0},
-        ...             'kitten3': {'name': 'son of pussy', 'age': 0.0},
-        ...         },
-        ...     },
-        ... }
-        1
-        >>> repeater = ConfigObj(repeated_3, configspec=repeated_1)
-        >>> repeater.validate(val)
-        1
-        >>> repeater == {
-        ...     'cats': {
-        ...         'cat1': {'fleas': True, 'tail': 'short', 'name': 'pussy'},
-        ...         'cat2': {'fleas': True, 'tail': 'short', 'name': 'pussy'},
-        ...         'cat3': {'fleas': True, 'tail': 'short', 'name': 'pussy'},
-        ...     },
-        ...     'dogs': {
-        ...         'dog1': {'fleas': True, 'tail': 'long', 'name': 'rover'},
-        ...         'dog2': {'fleas': True, 'tail': 'long', 'name': 'rover'},
-        ...         'dog3': {'fleas': True, 'tail': 'long', 'name': 'rover'},
-        ...     },
-        ... }
-        1
-        >>> repeater = ConfigObj(configspec=repeated_4)
-        >>> repeater['Michael'] = {}
-        >>> repeater.validate(val)
-        1
-        >>> repeater == {
-        ...     'Michael': {'age': 0.0, 'name': 'Michael', 'sex': 'm'},
-        ... }
-        1
-        >>> repeater = ConfigObj(repeated_3, configspec=repeated_5)
-        >>> repeater == {
-        ...     'dogs': {'dog1': {}, 'dog2': {}, 'dog3': {}},
-        ...     'cats': {'cat1': {}, 'cat2': {}, 'cat3': {}},
-        ... }
-        1
-        >>> repeater.validate(val)
-        1
-        >>> repeater == {
-        ...     'dogs': {'dog1': {}, 'dog2': {}, 'dog3': {}},
-        ...     'cats': {
-        ...         'cat1': {
-        ...             'fleas': True,
-        ...             'tail': 'short',
-        ...             'name': 'pussy',
-        ...             'description': {
-        ...                 'weight': 6.0,
-        ...                 'height': 3.2999999999999998,
-        ...                 'coat': {'fur': 'black', 'condition': 5},
-        ...             },
-        ...         },
-        ...         'cat2': {
-        ...             'fleas': True,
-        ...             'tail': 'short',
-        ...             'name': 'pussy',
-        ...             'description': {
-        ...                 'weight': 6.0,
-        ...                 'height': 3.2999999999999998,
-        ...                 'coat': {'fur': 'black', 'condition': 5},
-        ...             },
-        ...         },
-        ...         'cat3': {
-        ...             'fleas': True,
-        ...             'tail': 'short',
-        ...             'name': 'pussy',
-        ...             'description': {
-        ...                 'weight': 6.0,
-        ...                 'height': 3.2999999999999998,
-        ...                 'coat': {'fur': 'black', 'condition': 5},
-        ...             },
-        ...         },
-        ...     },
-        ... }
-        1
-
-        Test that interpolation is preserved for validated string values.
-        Also check that interpolation works in configspecs.
-        >>> t = ConfigObj()
-        >>> t['DEFAULT'] = {}
-        >>> t['DEFAULT']['test'] = 'a'
-        >>> t['test'] = '%(test)s'
-        >>> t['test']
-        'a'
-        >>> v = Validator()
-        >>> t.configspec = {'test': 'string'}
-        >>> t.validate(v)
-        1
-        >>> t.interpolation = False
-        >>> t
-        {'test': '%(test)s', 'DEFAULT': {'test': 'a'}}
-        >>> specs = [
-        ...    'interpolated string  = string(default="fuzzy-%(man)s")',
-        ...    '[DEFAULT]',
-        ...    'man = wuzzy',
-        ...    ]
-        >>> c = ConfigObj(configspec=specs)
-        >>> c.validate(v)
-        1
-        >>> c['interpolated string']
-        'fuzzy-wuzzy'
-
-        FIXME: Above tests will fail if we couldn't import Validator (the ones
-        that don't raise errors will produce different output and still fail as
-        tests)
-
-        Test
-        """
-        if section is None:
-            if self.configspec is None:
-                raise ValueError, 'No configspec supplied.'
-            if preserve_errors:
-                if VdtMissingValue is None:
-                    raise ImportError('Missing validate module.')
-            section = self
-        #
-        spec_section = section.configspec
-        if copy and hasattr(section, '_configspec_initial_comment'):
-            section.initial_comment = section._configspec_initial_comment
-            section.final_comment = section._configspec_final_comment
-            section.encoding = section._configspec_encoding
-            section.BOM = section._configspec_BOM
-            section.newlines = section._configspec_newlines
-            section.indent_type = section._configspec_indent_type
-        if '__many__' in section.configspec:
-            many = spec_section['__many__']
-            # dynamically assign the configspecs
-            # for the sections below
-            for entry in section.sections:
-                self._handle_repeat(section[entry], many)
-        #
-        out = {}
-        ret_true = True
-        ret_false = True
-        order = [k for k in section._order if k in spec_section]
-        order += [k for k in spec_section if k not in order]
-        for entry in order:
-            if entry == '__many__':
-                continue
-            if (not entry in section.scalars) or (entry in section.defaults):
-                # missing entries
-                # or entries from defaults
-                missing = True
-                val = None
-                if copy and not entry in section.scalars:
-                    # copy comments
-                    section.comments[entry] = (
-                        section._configspec_comments.get(entry, []))
-                    section.inline_comments[entry] = (
-                        section._configspec_inline_comments.get(entry, ''))
-                #
-            else:
-                missing = False
-                val = section[entry]
-            try:
-                check = validator.check(spec_section[entry],
-                                        val,
-                                        missing=missing
-                                        )
-            except validator.baseErrorClass, e:
-                if not preserve_errors or isinstance(e, VdtMissingValue):
-                    out[entry] = False
-                else:
-                    # preserve the error
-                    out[entry] = e
-                    ret_false = False
-                ret_true = False
-            else:
-                ret_false = False
-                out[entry] = True
-                if self.stringify or missing:
-                    # if we are doing type conversion
-                    # or the value is a supplied default
-                    if not self.stringify:
-                        if isinstance(check, (list, tuple)):
-                            # preserve lists
-                            check = [self._str(item) for item in check]
-                        elif missing and check is None:
-                            # convert the None from a default to a ''
-                            check = ''
-                        else:
-                            check = self._str(check)
-                    if (check != val) or missing:
-                        section[entry] = check
-                if not copy and missing and entry not in section.defaults:
-                    section.defaults.append(entry)
-        #
-        # Missing sections will have been created as empty ones when the
-        # configspec was read.
-        for entry in section.sections:
-            # FIXME: this means DEFAULT is not copied in copy mode
-            if section is self and entry == 'DEFAULT':
-                continue
-            if copy:
-                section.comments[entry] = section._cs_section_comments[entry]
-                section.inline_comments[entry] = (
-                    section._cs_section_inline_comments[entry])
-            check = self.validate(validator, preserve_errors=preserve_errors,
-                copy=copy, section=section[entry])
-            out[entry] = check
-            if check == False:
-                ret_true = False
-            elif check == True:
-                ret_false = False
-            else:
-                ret_true = False
-                ret_false = False
-        #
-        if ret_true:
-            return True
-        elif ret_false:
-            return False
-        else:
-            return out
-
-
-class SimpleVal(object):
-    """
-    A simple validator.
-    Can be used to check that all members expected are present.
-
-    To use it, provide a configspec with all your members in (the value given
-    will be ignored). Pass an instance of ``SimpleVal`` to the ``validate``
-    method of your ``ConfigObj``. ``validate`` will return ``True`` if all
-    members are present, or a dictionary with True/False meaning
-    present/missing. (Whole missing sections will be replaced with ``False``)
-
-    >>> val = SimpleVal()
-    >>> config = '''
-    ... test1=40
-    ... test2=hello
-    ... test3=3
-    ... test4=5.0
-    ... [section]
-    ... test1=40
-    ... test2=hello
-    ... test3=3
-    ... test4=5.0
-    ...     [[sub section]]
-    ...     test1=40
-    ...     test2=hello
-    ...     test3=3
-    ...     test4=5.0
-    ... '''.split('\\n')
-    >>> configspec = '''
-    ... test1=''
-    ... test2=''
-    ... test3=''
-    ... test4=''
-    ... [section]
-    ... test1=''
-    ... test2=''
-    ... test3=''
-    ... test4=''
-    ...     [[sub section]]
-    ...     test1=''
-    ...     test2=''
-    ...     test3=''
-    ...     test4=''
-    ... '''.split('\\n')
-    >>> o = ConfigObj(config, configspec=configspec)
-    >>> o.validate(val)
-    1
-    >>> o = ConfigObj(configspec=configspec)
-    >>> o.validate(val)
-    0
-    """
-
-    def __init__(self):
-        self.baseErrorClass = ConfigObjError
-
-    def check(self, check, member, missing=False):
-        """A dummy check method, always returns the value unchanged."""
-        if missing:
-            raise self.baseErrorClass
-        return member
-
-# Check / processing functions for options
-def flatten_errors(cfg, res, levels=None, results=None):
-    """
-    An example function that will turn a nested dictionary of results
-    (as returned by ``ConfigObj.validate``) into a flat list.
-
-    ``cfg`` is the ConfigObj instance being checked, ``res`` is the results
-    dictionary returned by ``validate``.
-
-    (This is a recursive function, so you shouldn't use the ``levels`` or
-    ``results`` arguments - they are used by the function.
-
-    Returns a list of keys that failed. Each member of the list is a tuple :
-    ::
-
-        ([list of sections...], key, result)
-
-    If ``validate`` was called with ``preserve_errors=False`` (the default)
-    then ``result`` will always be ``False``.
-
-    *list of sections* is a flattened list of sections that the key was found
-    in.
-
-    If the section was missing then key will be ``None``.
-
-    If the value (or section) was missing then ``result`` will be ``False``.
-
-    If ``validate`` was called with ``preserve_errors=True`` and a value
-    was present, but failed the check, then ``result`` will be the exception
-    object returned. You can use this as a string that describes the failure.
-
-    For example *The value "3" is of the wrong type*.
-
-    # FIXME: is the ordering of the output arbitrary ?
-    >>> import validate
-    >>> vtor = validate.Validator()
-    >>> my_ini = '''
-    ...     option1 = True
-    ...     [section1]
-    ...     option1 = True
-    ...     [section2]
-    ...     another_option = Probably
-    ...     [section3]
-    ...     another_option = True
-    ...     [[section3b]]
-    ...     value = 3
-    ...     value2 = a
-    ...     value3 = 11
-    ...     '''
-    >>> my_cfg = '''
-    ...     option1 = boolean()
-    ...     option2 = boolean()
-    ...     option3 = boolean(default=Bad_value)
-    ...     [section1]
-    ...     option1 = boolean()
-    ...     option2 = boolean()
-    ...     option3 = boolean(default=Bad_value)
-    ...     [section2]
-    ...     another_option = boolean()
-    ...     [section3]
-    ...     another_option = boolean()
-    ...     [[section3b]]
-    ...     value = integer
-    ...     value2 = integer
-    ...     value3 = integer(0, 10)
-    ...         [[[section3b-sub]]]
-    ...         value = string
-    ...     [section4]
-    ...     another_option = boolean()
-    ...     '''
-    >>> cs = my_cfg.split('\\n')
-    >>> ini = my_ini.split('\\n')
-    >>> cfg = ConfigObj(ini, configspec=cs)
-    >>> res = cfg.validate(vtor, preserve_errors=True)
-    >>> errors = []
-    >>> for entry in flatten_errors(cfg, res):
-    ...     section_list, key, error = entry
-    ...     section_list.insert(0, '[root]')
-    ...     if key is not None:
-    ...        section_list.append(key)
-    ...     else:
-    ...         section_list.append('[missing]')
-    ...     section_string = ', '.join(section_list)
-    ...     errors.append((section_string, ' = ', error))
-    >>> errors.sort()
-    >>> for entry in errors:
-    ...     print entry[0], entry[1], (entry[2] or 0)
-    [root], option2  =  0
-    [root], option3  =  the value "Bad_value" is of the wrong type.
-    [root], section1, option2  =  0
-    [root], section1, option3  =  the value "Bad_value" is of the wrong type.
-    [root], section2, another_option  =  the value "Probably" is of the wrong type.
-    [root], section3, section3b, section3b-sub, [missing]  =  0
-    [root], section3, section3b, value2  =  the value "a" is of the wrong type.
-    [root], section3, section3b, value3  =  the value "11" is too big.
-    [root], section4, [missing]  =  0
-    """
-    if levels is None:
-        # first time called
-        levels = []
-        results = []
-    if res is True:
-        return results
-    if res is False:
-        results.append((levels[:], None, False))
-        if levels:
-            levels.pop()
-        return results
-    for (key, val) in res.items():
-        if val == True:
-            continue
-        if isinstance(cfg.get(key), dict):
-            # Go down one level
-            levels.append(key)
-            flatten_errors(cfg[key], val, levels, results)
-            continue
-        results.append((levels[:], key, val))
-    #
-    # Go up one level
-    if levels:
-        levels.pop()
-    #
-    return results
-
-
-# FIXME: test error code for badly built multiline values
-# FIXME: test handling of StringIO
-# FIXME: test interpolation with writing
-
-def _doctest():
-    """
-    Dummy function to hold some of the doctests.
-
-    >>> a.depth
-    0
-    >>> a == {
-    ...     'key2': 'val',
-    ...     'key1': 'val',
-    ...     'lev1c': {
-    ...         'lev2c': {
-    ...             'lev3c': {
-    ...                 'key1': 'val',
-    ...             },
-    ...         },
-    ...     },
-    ...     'lev1b': {
-    ...         'key2': 'val',
-    ...         'key1': 'val',
-    ...         'lev2ba': {
-    ...             'key1': 'val',
-    ...         },
-    ...         'lev2bb': {
-    ...             'key1': 'val',
-    ...         },
-    ...     },
-    ...     'lev1a': {
-    ...         'key2': 'val',
-    ...         'key1': 'val',
-    ...     },
-    ... }
-    1
-    >>> b.depth
-    0
-    >>> b == {
-    ...     'key3': 'val3',
-    ...     'key2': 'val2',
-    ...     'key1': 'val1',
-    ...     'section 1': {
-    ...         'keys11': 'val1',
-    ...         'keys13': 'val3',
-    ...         'keys12': 'val2',
-    ...     },
-    ...     'section 2': {
-    ...         'section 2 sub 1': {
-    ...             'fish': '3',
-    ...     },
-    ...     'keys21': 'val1',
-    ...     'keys22': 'val2',
-    ...     'keys23': 'val3',
-    ...     },
-    ... }
-    1
-    >>> t = '''
-    ... 'a' = b # !"$%^&*(),::;'@~#= 33
-    ... "b" = b #= 6, 33
-    ... ''' .split('\\n')
-    >>> t2 = ConfigObj(t)
-    >>> assert t2 == {'a': 'b', 'b': 'b'}
-    >>> t2.inline_comments['b'] = ''
-    >>> del t2['a']
-    >>> assert t2.write() == ['','b = b', '']
-
-    # Test ``list_values=False`` stuff
-    >>> c = '''
-    ...     key1 = no quotes
-    ...     key2 = 'single quotes'
-    ...     key3 = "double quotes"
-    ...     key4 = "list", 'with', several, "quotes"
-    ...     '''
-    >>> cfg = ConfigObj(c.splitlines(), list_values=False)
-    >>> cfg == {'key1': 'no quotes', 'key2': "'single quotes'",
-    ... 'key3': '"double quotes"',
-    ... 'key4': '"list", \\'with\\', several, "quotes"'
-    ... }
-    1
-    >>> cfg = ConfigObj(list_values=False)
-    >>> cfg['key1'] = 'Multiline\\nValue'
-    >>> cfg['key2'] = '''"Value" with 'quotes' !'''
-    >>> cfg.write()
-    ["key1 = '''Multiline\\nValue'''", 'key2 = "Value" with \\'quotes\\' !']
-    >>> cfg.list_values = True
-    >>> cfg.write() == ["key1 = '''Multiline\\nValue'''",
-    ... 'key2 = \\'\\'\\'"Value" with \\'quotes\\' !\\'\\'\\'']
-    1
-
-    Test flatten_errors:
-
-    >>> from validate import Validator, VdtValueTooSmallError
-    >>> config = '''
-    ...     test1=40
-    ...     test2=hello
-    ...     test3=3
-    ...     test4=5.0
-    ...     [section]
-    ...         test1=40
-    ...         test2=hello
-    ...         test3=3
-    ...         test4=5.0
-    ...         [[sub section]]
-    ...             test1=40
-    ...             test2=hello
-    ...             test3=3
-    ...             test4=5.0
-    ... '''.split('\\n')
-    >>> configspec = '''
-    ...     test1= integer(30,50)
-    ...     test2= string
-    ...     test3=integer
-    ...     test4=float(6.0)
-    ...     [section ]
-    ...         test1=integer(30,50)
-    ...         test2=string
-    ...         test3=integer
-    ...         test4=float(6.0)
-    ...         [[sub section]]
-    ...             test1=integer(30,50)
-    ...             test2=string
-    ...             test3=integer
-    ...             test4=float(6.0)
-    ...     '''.split('\\n')
-    >>> val = Validator()
-    >>> c1 = ConfigObj(config, configspec=configspec)
-    >>> res = c1.validate(val)
-    >>> flatten_errors(c1, res) == [([], 'test4', False), (['section',
-    ...     'sub section'], 'test4', False), (['section'], 'test4', False)]
-    True
-    >>> res = c1.validate(val, preserve_errors=True)
-    >>> check = flatten_errors(c1, res)
-    >>> check[0][:2]
-    ([], 'test4')
-    >>> check[1][:2]
-    (['section', 'sub section'], 'test4')
-    >>> check[2][:2]
-    (['section'], 'test4')
-    >>> for entry in check:
-    ...     isinstance(entry[2], VdtValueTooSmallError)
-    ...     print str(entry[2])
-    True
-    the value "5.0" is too small.
-    True
-    the value "5.0" is too small.
-    True
-    the value "5.0" is too small.
-
-    Test unicode handling, BOM, write witha file like object and line endings :
-    >>> u_base = '''
-    ... # initial comment
-    ...     # inital comment 2
-    ...
-    ... test1 = some value
-    ... # comment
-    ... test2 = another value    # inline comment
-    ... # section comment
-    ... [section]    # inline comment
-    ...     test = test    # another inline comment
-    ...     test2 = test2
-    ...
-    ... # final comment
-    ... # final comment2
-    ... '''
-    >>> u = u_base.encode('utf_8').splitlines(True)
-    >>> u[0] = BOM_UTF8 + u[0]
-    >>> uc = ConfigObj(u)
-    >>> uc.encoding = None
-    >>> uc.BOM == True
-    1
-    >>> uc == {'test1': 'some value', 'test2': 'another value',
-    ... 'section': {'test': 'test', 'test2': 'test2'}}
-    1
-    >>> uc = ConfigObj(u, encoding='utf_8', default_encoding='latin-1')
-    >>> uc.BOM
-    1
-    >>> isinstance(uc['test1'], unicode)
-    1
-    >>> uc.encoding
-    'utf_8'
-    >>> uc.newlines
-    '\\n'
-    >>> uc['latin1'] = "This costs lot's of "
-    >>> a_list = uc.write()
-    >>> len(a_list)
-    15
-    >>> isinstance(a_list[0], str)
-    1
-    >>> a_list[0].startswith(BOM_UTF8)
-    1
-    >>> u = u_base.replace('\\n', '\\r\\n').encode('utf_8').splitlines(True)
-    >>> uc = ConfigObj(u)
-    >>> uc.newlines
-    '\\r\\n'
-    >>> uc.newlines = '\\r'
-    >>> from cStringIO import StringIO
-    >>> file_like = StringIO()
-    >>> uc.write(file_like)
-    >>> file_like.seek(0)
-    >>> uc2 = ConfigObj(file_like)
-    >>> uc2 == uc
-    1
-    >>> uc2.filename == None
-    1
-    >>> uc2.newlines == '\\r'
-    1
-
-    Test validate in copy mode
-    >>> a = '''
-    ... # Initial Comment
-    ...
-    ... key1 = string(default=Hello)    # comment 1
-    ...
-    ... # section comment
-    ... [section] # inline comment
-    ... # key1 comment
-    ... key1 = integer(default=6) # an integer value
-    ... # key2 comment
-    ... key2 = boolean(default=True) # a boolean
-    ...
-    ...     # subsection comment
-    ...     [[sub-section]] # inline comment
-    ...     # another key1 comment
-    ...     key1 = float(default=3.0) # a float'''.splitlines()
-    >>> b = ConfigObj(configspec=a)
-    >>> b.validate(val, copy=True)
-    1
-    >>> b.write() == ['',
-    ... '# Initial Comment',
-    ... '',
-    ... 'key1 = Hello    # comment 1',
-    ... '',
-    ... '# section comment',
-    ... '[section]    # inline comment',
-    ... '    # key1 comment',
-    ... '    key1 = 6    # an integer value',
-    ... '    # key2 comment',
-    ... '    key2 = True    # a boolean',
-    ... '    ',
-    ... '    # subsection comment',
-    ... '    [[sub-section]]    # inline comment',
-    ... '        # another key1 comment',
-    ... '        key1 = 3.0    # a float']
-    1
-
-    Test Writing Empty Values
-    >>> a = '''
-    ...     key1 =
-    ...     key2 =# a comment'''
-    >>> b = ConfigObj(a.splitlines())
-    >>> b.write()
-    ['', 'key1 = ""', 'key2 = ""    # a comment']
-    >>> b.write_empty_values = True
-    >>> b.write()
-    ['', 'key1 = ', 'key2 =     # a comment']
-
-    Test unrepr when reading
-    >>> a = '''
-    ...     key1 = (1, 2, 3)    # comment
-    ...     key2 = True
-    ...     key3 = 'a string'
-    ...     key4 = [1, 2, 3, 'a mixed list']
-    ... '''.splitlines()
-    >>> b = ConfigObj(a, unrepr=True)
-    >>> b == {'key1': (1, 2, 3),
-    ... 'key2': True,
-    ... 'key3': 'a string',
-    ... 'key4': [1, 2, 3, 'a mixed list']}
-    1
-
-    Test unrepr when writing
-    >>> c = ConfigObj(b.write(), unrepr=True)
-    >>> c == b
-    1
-
-    Test unrepr with multiline values
-    >>> a = '''k = \"""{
-    ... 'k1': 3,
-    ... 'k2': 6.0}\"""
-    ... '''.splitlines()
-    >>> c = ConfigObj(a, unrepr=True)
-    >>> c == {'k': {'k1': 3, 'k2': 6.0}}
-    1
-
-    Test unrepr with a dictionary
-    >>> a = 'k = {"a": 1}'.splitlines()
-    >>> c = ConfigObj(a, unrepr=True)
-    >>> type(c['k']) == dict
-    1
-    """
-
-if __name__ == '__main__':
-    # run the code tests in doctest format
-    #
-    testconfig1 = """\
-    key1= val    # comment 1
-    key2= val    # comment 2
-    # comment 3
-    [lev1a]     # comment 4
-    key1= val    # comment 5
-    key2= val    # comment 6
-    # comment 7
-    [lev1b]    # comment 8
-    key1= val    # comment 9
-    key2= val    # comment 10
-    # comment 11
-        [[lev2ba]]    # comment 12
-        key1= val    # comment 13
-        # comment 14
-        [[lev2bb]]    # comment 15
-        key1= val    # comment 16
-    # comment 17
-    [lev1c]    # comment 18
-    # comment 19
-        [[lev2c]]    # comment 20
-        # comment 21
-            [[[lev3c]]]    # comment 22
-            key1 = val    # comment 23"""
-    #
-    testconfig2 = """\
-                        key1 = 'val1'
-                        key2 =   "val2"
-                        key3 = val3
-                        ["section 1"] # comment
-                        keys11 = val1
-                        keys12 = val2
-                        keys13 = val3
-                        [section 2]
-                        keys21 = val1
-                        keys22 = val2
-                        keys23 = val3
-
-                            [['section 2 sub 1']]
-                            fish = 3
-    """
-    #
-    testconfig6 = '''
-    name1 = """ a single line value """ # comment
-    name2 = \''' another single line value \''' # comment
-    name3 = """ a single line value """
-    name4 = \''' another single line value \'''
-        [ "multi section" ]
-        name1 = """
-        Well, this is a
-        multiline value
-        """
-        name2 = \'''
-        Well, this is a
-        multiline value
-        \'''
-        name3 = """
-        Well, this is a
-        multiline value
-        """     # a comment
-        name4 = \'''
-        Well, this is a
-        multiline value
-        \'''  # I guess this is a comment too
-    '''
-    #
-    import doctest
-    m = sys.modules.get('__main__')
-    globs = m.__dict__.copy()
-    a = ConfigObj(testconfig1.split('\n'), raise_errors=True)
-    b = ConfigObj(testconfig2.split('\n'), raise_errors=True)
-    i = ConfigObj(testconfig6.split('\n'), raise_errors=True)
-    globs.update({
-        'INTP_VER': INTP_VER,
-        'a': a,
-        'b': b,
-        'i': i,
-    })
-    doctest.testmod(m, globs=globs)
-
-"""
-    BUGS
-    ====
-
-    None known.
-
-    TODO
-    ====
-
-    Better support for configuration from multiple files, including tracking
-    *where* the original file came from and writing changes to the correct
-    file.
-
-    Make ``newline`` an option (as well as an attribute) ?
-
-    ``UTF16`` encoded files, when returned as a list of lines, will have the
-    BOM at the start of every line. Should this be removed from all but the
-    first line ?
-
-    Option to set warning type for unicode decode ? (Defaults to strict).
-
-    A method to optionally remove uniform indentation from multiline values.
-    (do as an example of using ``walk`` - along with string-escape)
-
-    Should the results dictionary from validate be an ordered dictionary if
-    `odict <http://www.voidspace.org.uk/python/odict.html>`_ is available ?
-
-    Implement a better ``__repr__`` ? (``ConfigObj({})``)
-
-    Implement some of the sequence methods (which include slicing) from the
-    newer ``odict`` ?
-
-    ISSUES
-    ======
-
-    There is currently no way to specify the encoding of a configspec.
-
-    When using ``copy`` mode for validation, it won't copy ``DEFAULT``
-    sections. This is so that you *can* use interpolation in configspec
-    files.
-
-    ``validate`` doesn't report *extra* values or sections.
-
-    You can't have a keyword with the same name as a section (in the same
-    section). They are both dictionary keys - so they would overlap.
-
-    ConfigObj doesn't quote and unquote values if ``list_values=False``.
-    This means that leading or trailing whitespace in values will be lost when
-    writing. (Unless you manually quote).
-
-    Interpolation checks first the 'DEFAULT' subsection of the current
-    section, next it checks the 'DEFAULT' section of the parent section,
-    last it checks the 'DEFAULT' section of the main section.
-
-    Logically a 'DEFAULT' section should apply to all subsections of the *same
-    parent* - this means that checking the 'DEFAULT' subsection in the
-    *current section* is not necessarily logical ?
-
-    Does it matter that we don't support the ':' divider, which is supported
-    by ``ConfigParser`` ?
-
-    String interpolation and validation don't play well together. When
-    validation changes type it sets the value. This will correctly fetch the
-    value using interpolation - but then overwrite the interpolation reference.
-    If the value is unchanged by validation (it's a string) - but other types
-    will be.
-
-
-    List Value Syntax
-    =================
-
-    List values allow you to specify multiple values for a keyword. This
-    maps to a list as the resulting Python object when parsed.
-
-    The syntax for lists is easy. A list is a comma separated set of values.
-    If these values contain quotes, the hash mark, or commas, then the values
-    can be surrounded by quotes. e.g. : ::
-
-        keyword = value1, 'value 2', "value 3"
-
-    If a value needs to be a list, but only has one member, then you indicate
-    this with a trailing comma. e.g. : ::
-
-        keyword = "single value",
-
-    If a value needs to be a list, but it has no members, then you indicate
-    this with a single comma. e.g. : ::
-
-        keyword = ,     # an empty list
-
-    Using triple quotes it will be possible for single values to contain
-    newlines and *both* single quotes and double quotes. Triple quotes aren't
-    allowed in list values. This means that the members of list values can't
-    contain carriage returns (or line feeds :-) or both quote values.
-
-    CHANGELOG
-    =========
-
-    2006/03/20
-    ----------
-
-    Empty values are now valid syntax. They are read as an empty string ``''``.
-    (``key =``, or ``key = # comment``.)
-
-    ``validate`` now honours the order of the configspec.
-
-    Added the ``copy`` mode to validate.
-
-    Fixed bug where files written on windows could be given '\r\r\n' line
-    terminators.
-
-    Fixed bug where last occuring comment line could be interpreted as the
-    final comment if the last line isn't terminated.
-
-    Fixed bug where nested list values would be flattened when ``write`` is
-    called. Now sub-lists have a string representation written instead.
-
-    Deprecated ``encode`` and ``decode`` methods instead.
-
-    You can now pass in a COnfigObj instance as a configspec (remember to read
-    the file using ``list_values=False``).
-
-    2006/02/04
-    ----------
-
-    Removed ``BOM_UTF8`` from ``__all__``.
-
-    The ``BOM`` attribute has become a boolean. (Defaults to ``False``.) It can
-    be ``True`` for the ``UTF16/UTF8`` encodings.
-
-    File like objects no longer need a ``seek`` attribute.
-
-    ConfigObj no longer keeps a reference to file like objects. Instead the
-    ``write`` method takes a file like object as an optional argument. (Which
-    will be used in preference of the ``filename`` attribute if htat exists as
-    well.)
-
-    Full unicode support added. New options/attributes ``encoding``,
-    ``default_encoding``.
-
-    utf16 files decoded to unicode.
-
-    If ``BOM`` is ``True``, but no encoding specified, then the utf8 BOM is
-    written out at the start of the file. (It will normally only be ``True`` if
-    the utf8 BOM was found when the file was read.)
-
-    File paths are *not* converted to absolute paths, relative paths will
-    remain relative as the ``filename`` attribute.
-
-    Fixed bug where ``final_comment`` wasn't returned if ``write`` is returning
-    a list of lines.
-
-    2006/01/31
-    ----------
-
-    Added ``True``, ``False``, and ``enumerate`` if they are not defined.
-    (``True`` and ``False`` are needed for *early* versions of Python 2.2,
-    ``enumerate`` is needed for all versions ofPython 2.2)
-
-    Deprecated ``istrue``, replaced it with ``as_bool``.
-
-    Added ``as_int`` and ``as_float``.
-
-    utf8 and utf16 BOM handled in an endian agnostic way.
-
-    2005/12/14
-    ----------
-
-    Validation no longer done on the 'DEFAULT' section (only in the root
-    level). This allows interpolation in configspecs.
-
-    Change in validation syntax implemented in validate 0.2.1
-
-    4.1.0
-
-    2005/12/10
-    ----------
-
-    Added ``merge``, a recursive update.
-
-    Added ``preserve_errors`` to ``validate`` and the ``flatten_errors``
-    example function.
-
-    Thanks to Matthew Brett for suggestions and helping me iron out bugs.
-
-    Fixed bug where a config file is *all* comment, the comment will now be
-    ``initial_comment`` rather than ``final_comment``.
-
-    2005/12/02
-    ----------
-
-    Fixed bug in ``create_empty``. Thanks to Paul Jimenez for the report.
-
-    2005/11/04
-    ----------
-
-    Fixed bug in ``Section.walk`` when transforming names as well as values.
-
-    Added the ``istrue`` method. (Fetches the boolean equivalent of a string
-    value).
-
-    Fixed ``list_values=False`` - they are now only quoted/unquoted if they
-    are multiline values.
-
-    List values are written as ``item, item`` rather than ``item,item``.
-
-    4.0.1
-
-    2005/10/09
-    ----------
-
-    Fixed typo in ``write`` method. (Testing for the wrong value when resetting
-    ``interpolation``).
-
-    4.0.0 Final
-
-    2005/09/16
-    ----------
-
-    Fixed bug in ``setdefault`` - creating a new section *wouldn't* return
-    a reference to the new section.
-
-    2005/09/09
-    ----------
-
-    Removed ``PositionError``.
-
-    Allowed quotes around keys as documented.
-
-    Fixed bug with commas in comments. (matched as a list value)
-
-    Beta 5
-
-    2005/09/07
-    ----------
-
-    Fixed bug in initialising ConfigObj from a ConfigObj.
-
-    Changed the mailing list address.
-
-    Beta 4
-
-    2005/09/03
-    ----------
-
-    Fixed bug in ``Section.__delitem__`` oops.
-
-    2005/08/28
-    ----------
-
-    Interpolation is switched off before writing out files.
-
-    Fixed bug in handling ``StringIO`` instances. (Thanks to report from
-    "Gustavo Niemeyer" <[e-mail]>)
-
-    Moved the doctests from the ``__init__`` method to a separate function.
-    (For the sake of IDE calltips).
-
-    Beta 3
-
-    2005/08/26
-    ----------
-
-    String values unchanged by validation *aren't* reset. This preserves
-    interpolation in string values.
-
-    2005/08/18
-    ----------
-
-    None from a default is turned to '' if stringify is off - because setting
-    a value to None raises an error.
-
-    Version 4.0.0-beta2
-
-    2005/08/16
-    ----------
-
-    By Nicola Larosa
-
-    Actually added the RepeatSectionError class ;-)
-
-    2005/08/15
-    ----------
-
-    If ``stringify`` is off - list values are preserved by the ``validate``
-    method. (Bugfix)
-
-    2005/08/14
-    ----------
-
-    By Michael Foord
-
-    Fixed ``simpleVal``.
-
-    Added ``RepeatSectionError`` error if you have additional sections in a
-    section with a ``__many__`` (repeated) section.
-
-    By Nicola Larosa
-
-    Reworked the ConfigObj._parse, _handle_error and _multiline methods:
-    mutated the self._infile, self._index and self._maxline attributes into
-    local variables and method parameters
-
-    Reshaped the ConfigObj._multiline method to better reflect its semantics
-
-    Changed the "default_test" test in ConfigObj.validate to check the fix for
-    the bug in validate.Validator.check
-
-    2005/08/13
-    ----------
-
-    By Nicola Larosa
-
-    Updated comments at top
-
-    2005/08/11
-    ----------
-
-    By Michael Foord
-
-    Implemented repeated sections.
-
-    By Nicola Larosa
-
-    Added test for interpreter version: raises RuntimeError if earlier than
-    2.2
-
-    2005/08/10
-    ----------
-
-    By Michael Foord
-
-    Implemented default values in configspecs.
-
-    By Nicola Larosa
-
-    Fixed naked except: clause in validate that was silencing the fact
-    that Python2.2 does not have dict.pop
-
-    2005/08/08
-    ----------
-
-    By Michael Foord
-
-    Bug fix causing error if file didn't exist.
-
-    2005/08/07
-    ----------
-
-    By Nicola Larosa
-
-    Adjusted doctests for Python 2.2.3 compatibility
-
-    2005/08/04
-    ----------
-
-    By Michael Foord
-
-    Added the inline_comments attribute
-
-    We now preserve and rewrite all comments in the config file
-
-    configspec is now a section attribute
-
-    The validate method changes values in place
-
-    Added InterpolationError
-
-    The errors now have line number, line, and message attributes. This
-    simplifies error handling
-
-    Added __docformat__
-
-    2005/08/03
-    ----------
-
-    By Michael Foord
-
-    Fixed bug in Section.pop (now doesn't raise KeyError if a default value
-    is specified)
-
-    Replaced ``basestring`` with ``types.StringTypes``
-
-    Removed the ``writein`` method
-
-    Added __version__
-
-    2005/07/29
-    ----------
-
-    By Nicola Larosa
-
-    Indentation in config file is not significant anymore, subsections are
-    designated by repeating square brackets
-
-    Adapted all tests and docs to the new format
-
-    2005/07/28
-    ----------
-
-    By Nicola Larosa
-
-    Added more tests
-
-    2005/07/23
-    ----------
-
-    By Nicola Larosa
-
-    Reformatted final docstring in ReST format, indented it for easier folding
-
-    Code tests converted to doctest format, and scattered them around
-    in various docstrings
-
-    Walk method rewritten using scalars and sections attributes
-
-    2005/07/22
-    ----------
-
-    By Nicola Larosa
-
-    Changed Validator and SimpleVal "test" methods to "check"
-
-    More code cleanup
-
-    2005/07/21
-    ----------
-
-    Changed Section.sequence to Section.scalars and Section.sections
-
-    Added Section.configspec
-
-    Sections in the root section now have no extra indentation
-
-    Comments now better supported in Section and preserved by ConfigObj
-
-    Comments also written out
-
-    Implemented initial_comment and final_comment
-
-    A scalar value after a section will now raise an error
-
-    2005/07/20
-    ----------
-
-    Fixed a couple of bugs
-
-    Can now pass a tuple instead of a list
-
-    Simplified dict and walk methods
-
-    Added __str__ to Section
-
-    2005/07/10
-    ----------
-
-    By Nicola Larosa
-
-    More code cleanup
-
-    2005/07/08
-    ----------
-
-    The stringify option implemented. On by default.
-
-    2005/07/07
-    ----------
-
-    Renamed private attributes with a single underscore prefix.
-
-    Changes to interpolation - exceeding recursion depth, or specifying a
-    missing value, now raise errors.
-
-    Changes for Python 2.2 compatibility. (changed boolean tests - removed
-    ``is True`` and ``is False``)
-
-    Added test for duplicate section and member (and fixed bug)
-
-    2005/07/06
-    ----------
-
-    By Nicola Larosa
-
-    Code cleanup
-
-    2005/07/02
-    ----------
-
-    Version 0.1.0
-
-    Now properly handles values including comments and lists.
-
-    Better error handling.
-
-    String interpolation.
-
-    Some options implemented.
-
-    You can pass a Section a dictionary to initialise it.
-
-    Setting a Section member to a dictionary will create a Section instance.
-
-    2005/06/26
-    ----------
-
-    Version 0.0.1
-
-    Experimental reader.
-
-    A reasonably elegant implementation - a basic reader in 160 lines of code.
-
-    *A programming language is a medium of expression.* - Paul Graham
-"""

=== modified file 'landscape/lib/persist.py'
--- landscape/lib/persist.py	2011-07-12 09:34:49 +0000
+++ landscape/lib/persist.py	2013-04-05 16:00:29 +0000
@@ -24,7 +24,7 @@
 import re
 
 
-__all__ = ["Persist", "PickleBackend", "BPickleBackend", "ConfigObjBackend",
+__all__ = ["Persist", "PickleBackend", "BPickleBackend",
            "path_string_to_tuple", "path_tuple_to_string", "RootedPersist",
            "PersistError", "PersistReadOnlyError"]
 
@@ -141,6 +141,9 @@
 
         If None is specified, then the filename passed during construction will
         be used.
+
+        If the destination file already exists, it will be renamed
+        to C{<filepath>.old}.
         """
         if filepath is None:
             if self.filename is None:
@@ -414,6 +417,21 @@
 
 
 def path_string_to_tuple(path):
+    """Convert a L{Persist} path string to a path tuple.
+
+    Examples:
+
+        >>> path_string_to_tuple("ab")
+        ("ab",)
+        >>> path_string_to_tuple("ab.cd")
+        ("ab", "cd"))
+        >>> path_string_to_tuple("ab[0][1]")
+        ("ab", 0, 1)
+        >>> path_string_to_tuple("ab[0].cd[1]")
+        ("ab", 0, "cd", 1)
+
+    Raises L{PersistError} if the given path string is invalid.
+    """
     if "." not in path and "[" not in path:
         return (path,)
     result = []
@@ -441,6 +459,35 @@
 
 
 class Backend(object):
+    """
+    Base class for L{Persist} backends implementing hierarchical storage
+    functionality.
+
+    Each node of the hierarchy is an object of type C{dict}, C{list}
+    or C{tuple}. A node can have zero or more children, each child can be
+    another node or a leaf value compatible with the backend's serialization
+    mechanism.
+
+    Each child element is associated with a unique key, that can be used to
+    get, set or remove the child itself from its containing node. If the node
+    object is of type C{dict}, then the child keys will be the keys of the
+    dictionary, otherwise if the node object is of type C{list} or C{tuple}
+    the child element keys are the indexes of the available items, or the value
+    of items theselves.
+
+    The root node object is always a C{dict}.
+
+    For example:
+
+        >>> root = backend.new()
+        >>> backend.set(root, "foo", "bar")
+        'bar'
+        >>> egg = backend.set(root, "egg", [1, 2, 3])
+        >>> backend.set(egg, 0, 10)
+        10
+        >>> root
+        {'foo': 'bar', 'egg': [10, 2, 3]}
+    """
 
     def new(self):
         raise NotImplementedError
@@ -452,6 +499,7 @@
         raise NotImplementedError
 
     def get(self, obj, elem, _marker=NOTHING):
+        """Lookup a child in the given node object."""
         if type(obj) is dict:
             newobj = obj.get(elem, _marker)
         elif type(obj) in (tuple, list):
@@ -469,6 +517,7 @@
         return newobj
 
     def set(self, obj, elem, value):
+        """Set the value of the given child in the given node object."""
         if type(obj) is dict:
             newobj = obj[elem] = value
         elif type(obj) is list and type(elem) is int:
@@ -485,6 +534,12 @@
         return newobj
 
     def remove(self, obj, elem, isvalue):
+        """Remove a the given child in the given node object.
+
+        @param isvalue: In case the node object is a C{list}, a boolean
+            indicating if C{elem} is the index of the child or the value
+            of the child itself.
+        """
         result = False
         if type(obj) is dict:
             if elem in obj:
@@ -505,20 +560,24 @@
         return result
 
     def copy(self, value):
+        """Copy a node or a value."""
         if type(value) in (dict, list):
             return copy.deepcopy(value)
         return value
 
     def empty(self, obj):
+        """Whether the given node object has no children."""
         return (not obj)
 
     def has(self, obj, elem):
+        """Whether the given node object contains the given child element."""
         contains = getattr(obj, "__contains__", None)
         if contains:
             return contains(elem)
         return NotImplemented
 
     def keys(self, obj):
+        """Return the keys of the child elements of the given node object."""
         keys = getattr(obj, "keys", None)
         if keys:
             return keys()
@@ -574,49 +633,4 @@
         finally:
             file.close()
 
-
-class ConfigObjBackend(Backend):
-
-    def __init__(self):
-        from landscape.lib import configobj
-        self.ConfigObj = configobj.ConfigObj
-        self.Section = configobj.Section
-
-    def new(self):
-        return self.ConfigObj(unrepr=True)
-
-    def load(self, filepath):
-        return self.ConfigObj(filepath, unrepr=True)
-
-    def save(self, filepath, map):
-        file = open(filepath, "w")
-        try:
-            map.write(file)
-        finally:
-            file.close()
-
-    def get(self, obj, elem, _marker=NOTHING):
-        if isinstance(obj, self.Section):
-            return obj.get(elem, _marker)
-        return Backend.get(self, obj, elem)
-
-    def set(self, obj, elem, value):
-        if isinstance(obj, self.Section):
-            obj[elem] = value
-            return obj[elem]
-        return Backend.set(self, obj, elem, value)
-
-    def remove(self, obj, elem, isvalue):
-        if isinstance(obj, self.Section):
-            if elem in obj:
-                del obj[elem]
-                return True
-            return False
-        return Backend.remove(self, obj, elem, isvalue)
-
-    def copy(self, value):
-        if isinstance(value, self.Section):
-            return value.dict()
-        return Backend.copy(self, value)
-
 # vim:ts=4:sw=4:et

=== modified file 'landscape/lib/tests/test_persist.py'
--- landscape/lib/tests/test_persist.py	2011-07-12 09:34:49 +0000
+++ landscape/lib/tests/test_persist.py	2013-04-05 16:00:29 +0000
@@ -3,7 +3,7 @@
 
 from landscape.lib.persist import (
     path_string_to_tuple, path_tuple_to_string, Persist, RootedPersist,
-    PickleBackend, ConfigObjBackend, PersistError, PersistReadOnlyError)
+    PickleBackend, PersistError, PersistReadOnlyError)
 from landscape.tests.helpers import LandscapeTest
 
 
@@ -444,12 +444,6 @@
         return Persist(PickleBackend(), *args, **kwargs)
 
 
-class ConfigObjPersistTest(GeneralPersistTest, SaveLoadPersistTest):
-
-    def build_persist(self, *args, **kwargs):
-        return Persist(ConfigObjBackend(), *args, **kwargs)
-
-
 class RootedPersistTest(GeneralPersistTest):
 
     def build_persist(self, *args, **kwargs):

=== removed file 'landscape/lib/twisted_amp.py'
--- landscape/lib/twisted_amp.py	2010-06-08 07:59:32 +0000
+++ landscape/lib/twisted_amp.py	1970-01-01 00:00:00 +0000
@@ -1,2223 +0,0 @@
-# -*- test-case-name: twisted.test.test_amp -*-
-# Copyright (c) 2005 Divmod, Inc.
-# Copyright (c) 2007 Twisted Matrix Laboratories.
-# See LICENSE for details.
-
-"""
-This module implements AMP, the Asynchronous Messaging Protocol.
-
-AMP is a protocol for sending multiple asynchronous request/response pairs over
-the same connection.  Requests and responses are both collections of key/value
-pairs.
-
-AMP is a very simple protocol which is not an application.  This module is a
-"protocol construction kit" of sorts; it attempts to be the simplest wire-level
-implementation of Deferreds.  AMP provides the following base-level features:
-
-    - Asynchronous request/response handling (hence the name)
-
-    - Requests and responses are both key/value pairs
-
-    - Binary transfer of all data: all data is length-prefixed.  Your
-      application will never need to worry about quoting.
-
-    - Command dispatching (like HTTP Verbs): the protocol is extensible, and
-      multiple AMP sub-protocols can be grouped together easily.
-
-The protocol implementation also provides a few additional features which are
-not part of the core wire protocol, but are nevertheless very useful:
-
-    - Tight TLS integration, with an included StartTLS command.
-
-    - Handshaking to other protocols: because AMP has well-defined message
-      boundaries and maintains all incoming and outgoing requests for you, you
-      can start a connection over AMP and then switch to another protocol.
-      This makes it ideal for firewall-traversal applications where you may
-      have only one forwarded port but multiple applications that want to use
-      it.
-
-Using AMP with Twisted is simple.  Each message is a command, with a response.
-You begin by defining a command type.  Commands specify their input and output
-in terms of the types that they expect to see in the request and response
-key-value pairs.  Here's an example of a command that adds two integers, 'a'
-and 'b'::
-
-    class Sum(amp.Command):
-        arguments = [('a', amp.Integer()),
-                     ('b', amp.Integer())]
-        response = [('total', amp.Integer())]
-
-Once you have specified a command, you need to make it part of a protocol, and
-define a responder for it.  Here's a 'JustSum' protocol that includes a
-responder for our 'Sum' command::
-
-    class JustSum(amp.AMP):
-        def sum(self, a, b):
-            total = a + b
-            print 'Did a sum: %d + %d = %d' % (a, b, total)
-            return {'total': total}
-        Sum.responder(sum)
-
-Later, when you want to actually do a sum, the following expression will return
-a Deferred which will fire with the result::
-
-    ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback(
-        lambda p: p.callRemote(Sum, a=13, b=81)).addCallback(
-            lambda result: result['total'])
-
-You can also define the propagation of specific errors in AMP.  For example,
-for the slightly more complicated case of division, we might have to deal with
-division by zero::
-
-    class Divide(amp.Command):
-        arguments = [('numerator', amp.Integer()),
-                     ('denominator', amp.Integer())]
-        response = [('result', amp.Float())]
-        errors = {ZeroDivisionError: 'ZERO_DIVISION'}
-
-The 'errors' mapping here tells AMP that if a responder to Divide emits a
-L{ZeroDivisionError}, then the other side should be informed that an error of
-the type 'ZERO_DIVISION' has occurred.  Writing a responder which takes
-advantage of this is very simple - just raise your exception normally::
-
-    class JustDivide(amp.AMP):
-        def divide(self, numerator, denominator):
-            result = numerator / denominator
-            print 'Divided: %d / %d = %d' % (numerator, denominator, total)
-            return {'result': result}
-        Divide.responder(divide)
-
-On the client side, the errors mapping will be used to determine what the
-'ZERO_DIVISION' error means, and translated into an asynchronous exception,
-which can be handled normally as any L{Deferred} would be::
-
-    def trapZero(result):
-        result.trap(ZeroDivisionError)
-        print "Divided by zero: returning INF"
-        return 1e1000
-    ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback(
-        lambda p: p.callRemote(Divide, numerator=1234,
-                               denominator=0)
-        ).addErrback(trapZero)
-
-For a complete, runnable example of both of these commands, see the files in
-the Twisted repository::
-
-    doc/core/examples/ampserver.py
-    doc/core/examples/ampclient.py
-
-On the wire, AMP is a protocol which uses 2-byte lengths to prefix keys and
-values, and empty keys to separate messages::
-
-    <2-byte length><key><2-byte length><value>
-    <2-byte length><key><2-byte length><value>
-    ...
-    <2-byte length><key><2-byte length><value>
-    <NUL><NUL>                  # Empty Key == End of Message
-
-And so on.  Because it's tedious to refer to lengths and NULs constantly, the
-documentation will refer to packets as if they were newline delimited, like
-so::
-
-    C: _command: sum
-    C: _ask: ef639e5c892ccb54
-    C: a: 13
-    C: b: 81
-
-    S: _answer: ef639e5c892ccb54
-    S: total: 94
-
-Notes:
-
-Values are limited to the maximum encodable size in a 16-bit length, 65535
-bytes.
-
-Keys are limited to the maximum encodable size in a 8-bit length, 255 bytes.
-Note that we still use 2-byte lengths to encode keys.  This small redundancy
-has several features:
-
-    - If an implementation becomes confused and starts emitting corrupt data,
-      or gets keys confused with values, many common errors will be
-      signalled immediately instead of delivering obviously corrupt packets.
-
-    - A single NUL will separate every key, and a double NUL separates
-      messages.  This provides some redundancy when debugging traffic dumps.
-
-    - NULs will be present at regular intervals along the protocol, providing
-      some padding for otherwise braindead C implementations of the protocol,
-      so that <stdio.h> string functions will see the NUL and stop.
-
-    - This makes it possible to run an AMP server on a port also used by a
-      plain-text protocol, and easily distinguish between non-AMP clients (like
-      web browsers) which issue non-NUL as the first byte, and AMP clients,
-      which always issue NUL as the first byte.
-
-"""
-
-__metaclass__ = type
-
-import types, warnings
-
-from cStringIO import StringIO
-from struct import pack
-
-from zope.interface import Interface, implements
-
-from twisted.python.reflect import accumulateClassDict
-from twisted.python.failure import Failure
-from twisted.python import log, filepath
-
-from twisted.internet.main import CONNECTION_LOST
-from twisted.internet.error import ConnectionLost
-from twisted.internet.defer import Deferred, maybeDeferred, fail
-from twisted.protocols.basic import Int16StringReceiver, StatefulStringProtocol
-
-#from twisted.internet._sslverify import problemsFromTransport
-
-# I'd like this to use the exposed public API, but for some reason, when it was
-# moved, these names were not exposed by internet.ssl.
-
-#from twisted.internet.ssl import CertificateOptions, Certificate, DN, KeyPair
-
-ASK = '_ask'
-ANSWER = '_answer'
-COMMAND = '_command'
-ERROR = '_error'
-ERROR_CODE = '_error_code'
-ERROR_DESCRIPTION = '_error_description'
-UNKNOWN_ERROR_CODE = 'UNKNOWN'
-UNHANDLED_ERROR_CODE = 'UNHANDLED'
-
-MAX_KEY_LENGTH = 0xff
-MAX_VALUE_LENGTH = 0xffff
-
-
-class IBoxSender(Interface):
-    """
-    A transport which can send L{AmpBox} objects.
-    """
-
-    def sendBox(box):
-        """
-        Send an L{AmpBox}.
-
-        @raise ProtocolSwitched: if the underlying protocol has been
-        switched.
-
-        @raise ConnectionLost: if the underlying connection has already been
-        lost.
-        """
-
-    def unhandledError(failure):
-        """
-        An unhandled error occurred in response to a box.  Log it
-        appropriately.
-
-        @param failure: a L{Failure} describing the error that occurred.
-        """
-
-
-
-class IBoxReceiver(Interface):
-    """
-    An application object which can receive L{AmpBox} objects and dispatch them
-    appropriately.
-    """
-
-    def startReceivingBoxes(boxSender):
-        """
-        The L{ampBoxReceived} method will start being called; boxes may be
-        responded to by responding to the given L{IBoxSender}.
-
-        @param boxSender: an L{IBoxSender} provider.
-        """
-
-
-    def ampBoxReceived(box):
-        """
-        A box was received from the transport; dispatch it appropriately.
-        """
-
-
-    def stopReceivingBoxes(reason):
-        """
-        No further boxes will be received on this connection.
-
-        @type reason: L{Failure}
-        """
-
-
-
-class IResponderLocator(Interface):
-    """
-    An application object which can look up appropriate responder methods for
-    AMP commands.
-    """
-
-    def locateResponder(self, name):
-        """
-        Locate a responder method appropriate for the named command.
-
-        @param name: the wire-level name (commandName) of the AMP command to be
-        responded to.
-
-        @return: a 1-argument callable that takes an L{AmpBox} with argument
-        values for the given command, and returns an L{AmpBox} containing
-        argument values for the named command, or a L{Deferred} that fires the
-        same.
-        """
-
-
-
-class AmpError(Exception):
-    """
-    Base class of all Amp-related exceptions.
-    """
-
-
-
-class ProtocolSwitched(Exception):
-    """
-    Connections which have been switched to other protocols can no longer
-    accept traffic at the AMP level.  This is raised when you try to send it.
-    """
-
-
-
-class OnlyOneTLS(AmpError):
-    """
-    This is an implementation limitation; TLS may only be started once per
-    connection.
-    """
-
-
-
-class NoEmptyBoxes(AmpError):
-    """
-    You can't have empty boxes on the connection.  This is raised when you
-    receive or attempt to send one.
-    """
-
-
-
-class InvalidSignature(AmpError):
-    """
-    You didn't pass all the required arguments.
-    """
-
-
-
-class TooLong(AmpError):
-    """
-    One of the protocol's length limitations was violated.
-
-    @ivar isKey: true if the string being encoded in a key position, false if
-    it was in a value position.
-
-    @ivar isLocal: Was the string encoded locally, or received too long from
-    the network?  (It's only physically possible to encode "too long" values on
-    the network for keys.)
-
-    @ivar value: The string that was too long.
-
-    @ivar keyName: If the string being encoded was in a value position, what
-    key was it being encoded for?
-    """
-
-    def __init__(self, isKey, isLocal, value, keyName=None):
-        AmpError.__init__(self)
-        self.isKey = isKey
-        self.isLocal = isLocal
-        self.value = value
-        self.keyName = keyName
-
-
-    def __repr__(self):
-        hdr = self.isKey and "key" or "value"
-        if not self.isKey:
-            hdr += ' ' + repr(self.keyName)
-        lcl = self.isLocal and "local" or "remote"
-        return "%s %s too long: %d" % (lcl, hdr, len(self.value))
-
-
-
-class BadLocalReturn(AmpError):
-    """
-    A bad value was returned from a local command; we were unable to coerce it.
-    """
-    def __init__(self, message, enclosed):
-        AmpError.__init__(self)
-        self.message = message
-        self.enclosed = enclosed
-
-
-    def __repr__(self):
-        return self.message + " " + self.enclosed.getBriefTraceback()
-
-    __str__ = __repr__
-
-
-
-class RemoteAmpError(AmpError):
-    """
-    This error indicates that something went wrong on the remote end of the
-    connection, and the error was serialized and transmitted to you.
-    """
-    def __init__(self, errorCode, description, fatal=False, local=None):
-        """Create a remote error with an error code and description.
-
-        @param errorCode: the AMP error code of this error.
-
-        @param description: some text to show to the user.
-
-        @param fatal: a boolean, true if this error should terminate the
-        connection.
-
-        @param local: a local Failure, if one exists.
-        """
-        if local:
-            localwhat = ' (local)'
-            othertb = local.getBriefTraceback()
-        else:
-            localwhat = ''
-            othertb = ''
-        Exception.__init__(self, "Code<%s>%s: %s%s" % (
-                errorCode, localwhat,
-                description, othertb))
-        self.local = local
-        self.errorCode = errorCode
-        self.description = description
-        self.fatal = fatal
-
-
-
-class UnknownRemoteError(RemoteAmpError):
-    """
-    This means that an error whose type we can't identify was raised from the
-    other side.
-    """
-    def __init__(self, description):
-        errorCode = UNKNOWN_ERROR_CODE
-        RemoteAmpError.__init__(self, errorCode, description)
-
-
-
-class MalformedAmpBox(AmpError):
-    """
-    This error indicates that the wire-level protocol was malformed.
-    """
-
-
-
-class UnhandledCommand(AmpError):
-    """
-    A command received via amp could not be dispatched.
-    """
-
-
-
-class IncompatibleVersions(AmpError):
-    """
-    It was impossible to negotiate a compatible version of the protocol with
-    the other end of the connection.
-    """
-
-
-PROTOCOL_ERRORS = {UNHANDLED_ERROR_CODE: UnhandledCommand}
-
-class AmpBox(dict):
-    """
-    I am a packet in the AMP protocol, much like a regular str:str dictionary.
-    """
-    __slots__ = []              # be like a regular dictionary, don't magically
-                                # acquire a __dict__...
-
-
-    def copy(self):
-        """
-        Return another AmpBox just like me.
-        """
-        newBox = self.__class__()
-        newBox.update(self)
-        return newBox
-
-
-    def serialize(self):
-        """
-        Convert me into a wire-encoded string.
-
-        @return: a str encoded according to the rules described in the module
-        docstring.
-        """
-        i = self.items()
-        i.sort()
-        L = []
-        w = L.append
-        for k, v in i:
-            if len(k) > MAX_KEY_LENGTH:
-                raise TooLong(True, True, k, None)
-            if len(v) > MAX_VALUE_LENGTH:
-                raise TooLong(False, True, v, k)
-            for kv in k, v:
-                w(pack("!H", len(kv)))
-                w(kv)
-        w(pack("!H", 0))
-        return ''.join(L)
-
-
-    def _sendTo(self, proto):
-        """
-        Serialize and send this box to a Amp instance.  By the time it is being
-        sent, several keys are required.  I must have exactly ONE of::
-
-            _ask
-            _answer
-            _error
-
-        If the '_ask' key is set, then the '_command' key must also be
-        set.
-
-        @param proto: an AMP instance.
-        """
-        proto.sendBox(self)
-
-    def __repr__(self):
-        return 'AmpBox(%s)' % (dict.__repr__(self),)
-
-# amp.Box => AmpBox
-
-Box = AmpBox
-
-class QuitBox(AmpBox):
-    """
-    I am an AmpBox that, upon being sent, terminates the connection.
-    """
-    __slots__ = []
-
-
-    def __repr__(self):
-        return 'QuitBox(**%s)' % (super(QuitBox, self).__repr__(),)
-
-
-    def _sendTo(self, proto):
-        """
-        Immediately call loseConnection after sending.
-        """
-        super(QuitBox, self)._sendTo(proto)
-        proto.transport.loseConnection()
-
-
-
-class _SwitchBox(AmpBox):
-    """
-    Implementation detail of ProtocolSwitchCommand: I am a AmpBox which sets
-    up state for the protocol to switch.
-    """
-
-    # DON'T set __slots__ here; we do have an attribute.
-
-    def __init__(self, innerProto, **kw):
-        """
-        Create a _SwitchBox with the protocol to switch to after being sent.
-
-        @param innerProto: the protocol instance to switch to.
-        @type innerProto: an IProtocol provider.
-        """
-        super(_SwitchBox, self).__init__(**kw)
-        self.innerProto = innerProto
-
-
-    def __repr__(self):
-        return '_SwitchBox(%r, **%s)' % (self.innerProto,
-                                         dict.__repr__(self),)
-
-
-    def _sendTo(self, proto):
-        """
-        Send me; I am the last box on the connection.  All further traffic will be
-        over the new protocol.
-        """
-        super(_SwitchBox, self)._sendTo(proto)
-        proto._lockForSwitch()
-        proto._switchTo(self.innerProto)
-
-
-
-class BoxDispatcher:
-    """
-    A L{BoxDispatcher} dispatches '_ask', '_answer', and '_error' L{AmpBox}es,
-    both incoming and outgoing, to their appropriate destinations.
-
-    Outgoing commands are converted into L{Deferred}s and outgoing boxes, and
-    associated tracking state to fire those L{Deferred} when '_answer' boxes
-    come back.  Incoming '_answer' and '_error' boxes are converted into
-    callbacks and errbacks on those L{Deferred}s, respectively.
-
-    Incoming '_ask' boxes are converted into method calls on a supplied method
-    locator.
-
-    @ivar _outstandingRequests: a dictionary mapping request IDs to
-    L{Deferred}s which were returned for those requests.
-
-    @ivar locator: an object with a L{locateResponder} method that locates a
-    responder function that takes a Box and returns a result (either a Box or a
-    Deferred which fires one).
-
-    @ivar boxSender: an object which can send boxes, via the L{_sendBox}
-    method, such as an L{AMP} instance.
-    @type boxSender: L{IBoxSender}
-    """
-
-    implements(IBoxReceiver)
-
-    _failAllReason = None
-    _outstandingRequests = None
-    _counter = 0L
-    boxSender = None
-
-    def __init__(self, locator):
-        self._outstandingRequests = {}
-        self.locator = locator
-
-
-    def startReceivingBoxes(self, boxSender):
-        """
-        The given boxSender is going to start calling boxReceived on this
-        L{BoxDispatcher}.
-
-        @param boxSender: The L{IBoxSender} to send command responses to.
-        """
-        self.boxSender = boxSender
-
-
-    def stopReceivingBoxes(self, reason):
-        """
-        No further boxes will be received here.  Terminate all currently
-        oustanding command deferreds with the given reason.
-        """
-        self.failAllOutgoing(reason)
-
-
-    def failAllOutgoing(self, reason):
-        """
-        Call the errback on all outstanding requests awaiting responses.
-
-        @param reason: the Failure instance to pass to those errbacks.
-        """
-        self._failAllReason = reason
-        OR = self._outstandingRequests.items()
-        self._outstandingRequests = None # we can never send another request
-        for key, value in OR:
-            value.errback(reason)
-
-
-    def _nextTag(self):
-        """
-        Generate protocol-local serial numbers for _ask keys.
-
-        @return: a string that has not yet been used on this connection.
-        """
-        self._counter += 1
-        return '%x' % (self._counter,)
-
-
-    def _sendBoxCommand(self, command, box, requiresAnswer=True):
-        """
-        Send a command across the wire with the given C{amp.Box}.
-
-        Mutate the given box to give it any additional keys (_command, _ask)
-        required for the command and request/response machinery, then send it.
-
-        If requiresAnswer is True, returns a C{Deferred} which fires when a
-        response is received. The C{Deferred} is fired with an C{amp.Box} on
-        success, or with an C{amp.RemoteAmpError} if an error is received.
-
-        If the Deferred fails and the error is not handled by the caller of
-        this method, the failure will be logged and the connection dropped.
-
-        @param command: a str, the name of the command to issue.
-
-        @param box: an AmpBox with the arguments for the command.
-
-        @param requiresAnswer: a boolean.  Defaults to True.  If True, return a
-        Deferred which will fire when the other side responds to this command.
-        If False, return None and do not ask the other side for acknowledgement.
-
-        @return: a Deferred which fires the AmpBox that holds the response to
-        this command, or None, as specified by requiresAnswer.
-
-        @raise ProtocolSwitched: if the protocol has been switched.
-        """
-        if self._failAllReason is not None:
-            return fail(self._failAllReason)
-        box[COMMAND] = command
-        tag = self._nextTag()
-        if requiresAnswer:
-            box[ASK] = tag
-        box._sendTo(self.boxSender)
-        if requiresAnswer:
-            result = self._outstandingRequests[tag] = Deferred()
-        else:
-            result = None
-        return result
-
-
-    def callRemoteString(self, command, requiresAnswer=True, **kw):
-        """
-        This is a low-level API, designed only for optimizing simple messages
-        for which the overhead of parsing is too great.
-
-        @param command: a str naming the command.
-
-        @param kw: arguments to the amp box.
-
-        @param requiresAnswer: a boolean.  Defaults to True.  If True, return a
-        Deferred which will fire when the other side responds to this command.
-        If False, return None and do not ask the other side for acknowledgement.
-
-        @return: a Deferred which fires the AmpBox that holds the response to
-        this command, or None, as specified by requiresAnswer.
-        """
-        box = Box(kw)
-        return self._sendBoxCommand(command, box)
-
-
-    def callRemote(self, commandType, *a, **kw):
-        """
-        This is the primary high-level API for sending messages via AMP.  Invoke it
-        with a command and appropriate arguments to send a message to this
-        connection's peer.
-
-        @param commandType: a subclass of Command.
-        @type commandType: L{type}
-
-        @param a: Positional (special) parameters taken by the command.
-        Positional parameters will typically not be sent over the wire.  The
-        only command included with AMP which uses positional parameters is
-        L{ProtocolSwitchCommand}, which takes the protocol that will be
-        switched to as its first argument.
-
-        @param kw: Keyword arguments taken by the command.  These are the
-        arguments declared in the command's 'arguments' attribute.  They will
-        be encoded and sent to the peer as arguments for the L{commandType}.
-
-        @return: If L{commandType} has a C{requiresAnswer} attribute set to
-        L{False}, then return L{None}.  Otherwise, return a L{Deferred} which
-        fires with a dictionary of objects representing the result of this
-        call.  Additionally, this L{Deferred} may fail with an exception
-        representing a connection failure, with L{UnknownRemoteError} if the
-        other end of the connection fails for an unknown reason, or with any
-        error specified as a key in L{commandType}'s C{errors} dictionary.
-        """
-
-        # XXX this takes command subclasses and not command objects on purpose.
-        # There's really no reason to have all this back-and-forth between
-        # command objects and the protocol, and the extra object being created
-        # (the Command instance) is pointless.  Command is kind of like
-        # Interface, and should be more like it.
-
-        # In other words, the fact that commandType is instantiated here is an
-        # implementation detail.  Don't rely on it.
-
-        co = commandType(*a, **kw)
-        return co._doCommand(self)
-
-
-    def unhandledError(self, failure):
-        """
-        This is a terminal callback called after application code has had a
-        chance to quash any errors.
-        """
-        return self.boxSender.unhandledError(failure)
-
-
-    def _answerReceived(self, box):
-        """
-        An AMP box was received that answered a command previously sent with
-        L{callRemote}.
-
-        @param box: an AmpBox with a value for its L{ANSWER} key.
-        """
-        question = self._outstandingRequests.pop(box[ANSWER])
-        question.addErrback(self.unhandledError)
-        question.callback(box)
-
-
-    def _errorReceived(self, box):
-        """
-        An AMP box was received that answered a command previously sent with
-        L{callRemote}, with an error.
-
-        @param box: an L{AmpBox} with a value for its L{ERROR}, L{ERROR_CODE},
-        and L{ERROR_DESCRIPTION} keys.
-        """
-        question = self._outstandingRequests.pop(box[ERROR])
-        question.addErrback(self.unhandledError)
-        errorCode = box[ERROR_CODE]
-        description = box[ERROR_DESCRIPTION]
-        if errorCode in PROTOCOL_ERRORS:
-            exc = PROTOCOL_ERRORS[errorCode](errorCode, description)
-        else:
-            exc = RemoteAmpError(errorCode, description)
-        question.errback(Failure(exc))
-
-
-    def _commandReceived(self, box):
-        """
-        @param box: an L{AmpBox} with a value for its L{COMMAND} and L{ASK}
-        keys.
-        """
-        cmd = box[COMMAND]
-        def formatAnswer(answerBox):
-            answerBox[ANSWER] = box[ASK]
-            return answerBox
-        def formatError(error):
-            if error.check(RemoteAmpError):
-                code = error.value.errorCode
-                desc = error.value.description
-                if error.value.fatal:
-                    errorBox = QuitBox()
-                else:
-                    errorBox = AmpBox()
-            else:
-                errorBox = QuitBox()
-                log.err(error) # here is where server-side logging happens
-                               # if the error isn't handled
-                code = UNKNOWN_ERROR_CODE
-                desc = "Unknown Error"
-            errorBox[ERROR] = box[ASK]
-            errorBox[ERROR_DESCRIPTION] = desc
-            errorBox[ERROR_CODE] = code
-            return errorBox
-        deferred = self.dispatchCommand(box)
-        if ASK in box:
-            deferred.addCallbacks(formatAnswer, formatError)
-            deferred.addCallback(self._safeEmit)
-        deferred.addErrback(self.unhandledError)
-
-
-    def ampBoxReceived(self, box):
-        """
-        An AmpBox was received, representing a command, or an answer to a
-        previously issued command (either successful or erroneous).  Respond to
-        it according to its contents.
-
-        @param box: an AmpBox
-
-        @raise NoEmptyBoxes: when a box is received that does not contain an
-        '_answer', '_command' / '_ask', or '_error' key; i.e. one which does not
-        fit into the command / response protocol defined by AMP.
-        """
-        if ANSWER in box:
-            self._answerReceived(box)
-        elif ERROR in box:
-            self._errorReceived(box)
-        elif COMMAND in box:
-            self._commandReceived(box)
-        else:
-            raise NoEmptyBoxes(box)
-
-
-    def _safeEmit(self, aBox):
-        """
-        Emit a box, ignoring L{ProtocolSwitched} and L{ConnectionLost} errors
-        which cannot be usefully handled.
-        """
-        try:
-            aBox._sendTo(self.boxSender)
-        except (ProtocolSwitched, ConnectionLost):
-            pass
-
-
-    def dispatchCommand(self, box):
-        """
-        A box with a _command key was received.
-
-        Dispatch it to a local handler call it.
-
-        @param proto: an AMP instance.
-        @param box: an AmpBox to be dispatched.
-        """
-        cmd = box[COMMAND]
-        responder = self.locator.locateResponder(cmd)
-        if responder is None:
-            return fail(RemoteAmpError(
-                    UNHANDLED_ERROR_CODE,
-                    "Unhandled Command: %r" % (cmd,),
-                    False,
-                    local=Failure(UnhandledCommand())))
-        return maybeDeferred(responder, box)
-
-
-
-class CommandLocator:
-    """
-    A L{CommandLocator} is a collection of responders to AMP L{Command}s, with
-    the help of the L{Command.responder} decorator.
-    """
-
-    class __metaclass__(type):
-        """
-        This metaclass keeps track of all of the Command.responder-decorated
-        methods defined since the last CommandLocator subclass was defined.  It
-        assumes (usually correctly, but unfortunately not necessarily so) that
-        those commands responders were all declared as methods of the class
-        being defined.  Note that this list can be incorrect if users use the
-        Command.responder decorator outside the context of a CommandLocator
-        class declaration.
-
-        The Command.responder decorator explicitly cooperates with this
-        metaclass.
-        """
-