Merge lp:~mbp/bzr/715000-more-fallbacks into lp:bzr/2.2

[Martin Pool]

follow-through from bug 715000: rename _fallback_vfs to _immediate_fallbacks to make it clear it's not the whole stack.

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2/9/2011 2:08 AM, Martin Pool wrote:
> Martin Pool has proposed merging lp:~mbp/bzr/715000-more-fallbacks into lp:bzr.
>
> Requested reviews:
> bzr-core (bzr-core)
>
> For more details, see:
> https://code.launchpad.net/~mbp/bzr/715000-more-fallbacks/+merge/49025
>
> follow-through from bug 715000: rename _fallback_vfs to _immediate_fallbacks to make it clear it's not the whole stack.

I'm ok with the change, but I would make it "_immediate_fallback_vfs" to
distinguish it from "_immediate_fallback_repositories".
Mostly to be easily greppable.

I don't personally find it better, mostly because the added length
doesn't add much. It seems a simple

  :ivar _fallback_vfs: Immediate fallbacks, note this is not transitive

Could be a better way to go.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1St0oACgkQJdeBCYSNAAOLHQCg0Yj/BsMKAzL4Sr5OhGQbxBoZ
9FMAnj/yOfdMjxhNnjLFiYvdnY9+ansx
=LcsQ
-----END PGP SIGNATURE-----

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2/9/2011 2:09 AM, Martin Pool wrote:
> Martin Pool has proposed merging lp:~mbp/bzr/715000-more-fallbacks into lp:bzr/2.2.
>
> Requested reviews:
> bzr-core (bzr-core)
>
> For more details, see:
> https://code.launchpad.net/~mbp/bzr/715000-more-fallbacks/+merge/49026
>
> follow-through from bug 715000: rename _fallback_vfs to _immediate_fallbacks to make it clear it's not the whole stack.

I don't really like changing the name of things in a stable series. Do
you feel this is really necessary?

 review: needs_information

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1St2gACgkQJdeBCYSNAAPIYwCeJ4s9T48rbxRduTts3rarhkPu
1UMAoKDzNFENu713ShbJuPiCF9zDJQdi
=F4wU
-----END PGP SIGNATURE-----

review: Needs Information

=== modified file '.bzrignore'
--- .bzrignore	2010-04-01 04:41:18 +0000
+++ .bzrignore	2011-02-09 08:08:20 +0000
@@ -19,7 +19,8 @@
 ./doc/**/*.html
 ./doc/developers/performance.png
 ./doc/en/user-reference/*.txt
-./doc/en/release-notes/*.txt
+./doc/en/release-notes/index.txt
+./doc/en/release-notes/NEWS.txt
 BRANCH-INFO
 # setup.py working directory
 ./build

=== modified file '.testr.conf'
--- .testr.conf	2010-02-28 10:08:29 +0000
+++ .testr.conf	2011-02-09 08:08:20 +0000
@@ -1,3 +1,4 @@
 [DEFAULT]
-test_command=./bzr selftest --subunit $IDOPTION
+test_command=./bzr selftest --subunit $IDOPTION $LISTOPT
 test_id_option=--load-list $IDFILE
+test_list_option=--list

=== modified file 'Makefile'
--- Makefile	2010-04-06 06:59:03 +0000
+++ Makefile	2011-02-09 08:08:20 +0000
@@ -39,8 +39,14 @@
 check: docs check-nodocs
 
 check-nodocs: extensions
+	set -e
 	# Generate a stream for PQM to watch.
+	-$(RM) -f selftest.log
 	$(PYTHON) -Werror -O ./bzr selftest --subunit $(tests) | tee selftest.log
+	# An empty log file should catch errors in the $(PYTHON)
+	# command above (the '|' swallow any errors since 'make'
+	# sees the 'tee' exit code for the whole line
+	if [ ! -s selftest.log ] ; then exit 1 ; fi
 	# Check that there were no errors reported.
 	subunit-stats < selftest.log
 
@@ -133,11 +139,13 @@
 	doc/developers/Makefile \
 	doc/developers/make.bat
 
+NEWS_FILES = $(wildcard doc/en/release-notes/bzr-*.txt)
+
 doc/en/user-reference/index.txt: $(MAN_DEPENDENCIES)
 	$(PYTHON) tools/generate_docs.py -o $@ rstx
 
-doc/en/release-notes/index.txt: NEWS tools/generate_release_notes.py
-	$(PYTHON) tools/generate_release_notes.py NEWS $@
+doc/en/release-notes/index.txt: $(NEWS_FILES) tools/generate_release_notes.py
+	$(PYTHON) tools/generate_release_notes.py $@ $(NEWS_FILES)
 
 doc/%/Makefile: doc/en/Makefile
 	$(PYTHON) -c "import shutil; shutil.copyfile('$<', '$@')"
@@ -296,10 +304,10 @@
 	$(rst2html) --stylesheet=default.css $< $@
 
 %.html: %.txt
-	$(rst2html) --stylesheet=../../default.css $< $@
+	$(rst2html) --stylesheet=../../default.css $< "$@"
 
-doc/en/release-notes/NEWS.txt: NEWS
-	$(PYTHON) -c "import shutil; shutil.copyfile('$<', '$@')"
+doc/en/release-notes/NEWS.txt: $(NEWS_FILES) tools/generate_release_notes.py
+	$(PYTHON) tools/generate_release_notes.py "$@" $(NEWS_FILES)
 
 upgrade_guide_dependencies =  $(wildcard $(addsuffix /*.txt, doc/en/upgrade-guide)) 
 

=== added file 'NEWS'
--- NEWS	1970-01-01 00:00:00 +0000
+++ NEWS	2011-02-09 08:08:20 +0000
@@ -0,0 +1,2 @@
+The NEWS file has been moved and split into multiple files (one per release
+series).  The NEWS files are now found in doc/en/release-notes/.

=== modified file 'README'
--- README	2010-01-29 14:09:05 +0000
+++ README	2011-02-09 08:08:20 +0000
@@ -36,7 +36,7 @@
 It also directly supports and encourages a large number of development best
 practices like refactoring and pre-commit regression testing. Users can
 choose between our command line tool and our cross-platform GUI application.
-For further details, see our website at http://bazaar-vcs.org/en.
+For further details, see our website at http://bazaar.canonical.com/en/
 
 Feedback
 ========

=== modified file 'bzr'
--- bzr	2011-02-04 14:04:18 +0000
+++ bzr	2011-02-09 08:08:20 +0000
@@ -23,7 +23,7 @@
 import warnings
 
 # update this on each release
-_script_version = (2, 2, 5)
+_script_version = (2, 4, 0)
 
 try:
     version_info = sys.version_info

=== modified file 'bzrlib/__init__.py'
--- bzrlib/__init__.py	2011-02-04 14:04:18 +0000
+++ bzrlib/__init__.py	2011-02-09 08:08:20 +0000
@@ -1,4 +1,4 @@
-# Copyright (C) 2005-2010 Canonical Ltd
+# Copyright (C) 2005-2011 Canonical Ltd
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -52,10 +52,10 @@
 # Python version 2.0 is (2, 0, 0, 'final', 0)."  Additionally we use a
 # releaselevel of 'dev' for unreleased under-development code.
 
-version_info = (2, 2, 5, 'dev', 0)
+version_info = (2, 4, 0, 'dev', 1)
 
 # API compatibility version
-api_minimum_version = (2, 2, 0)
+api_minimum_version = (2, 4, 0)
 
 
 def _format_version_tuple(version_info):
@@ -71,17 +71,17 @@
     1.0.0
     >>> print _format_version_tuple((1, 2, 0, 'dev', 0))
     1.2.0dev
-    >>> print bzrlib._format_version_tuple((1, 2, 0, 'dev', 1))
+    >>> print _format_version_tuple((1, 2, 0, 'dev', 1))
     1.2.0dev1
     >>> print _format_version_tuple((1, 1, 1, 'candidate', 2))
     1.1.1rc2
-    >>> print bzrlib._format_version_tuple((2, 1, 0, 'beta', 1))
+    >>> print _format_version_tuple((2, 1, 0, 'beta', 1))
     2.1b1
     >>> print _format_version_tuple((1, 4, 0))
     1.4.0
     >>> print _format_version_tuple((1, 4))
     1.4
-    >>> print bzrlib._format_version_tuple((2, 1, 0, 'final', 1))
+    >>> print _format_version_tuple((2, 1, 0, 'final', 1))
     Traceback (most recent call last):
     ...
     ValueError: version_info (2, 1, 0, 'final', 1) not valid

=== modified file 'bzrlib/_btree_serializer_pyx.pyx'
--- bzrlib/_btree_serializer_pyx.pyx	2010-02-17 17:11:16 +0000
+++ bzrlib/_btree_serializer_pyx.pyx	2011-02-09 08:08:20 +0000
@@ -33,6 +33,7 @@
     char *PyString_AsString(object p) except NULL
     object PyString_FromStringAndSize(char *, Py_ssize_t)
     PyObject *PyString_FromStringAndSize_ptr "PyString_FromStringAndSize" (char *, Py_ssize_t)
+    object PyString_FromFormat(char *, ...)
     int PyString_CheckExact(object s)
     int PyString_CheckExact_ptr "PyString_CheckExact" (PyObject *)
     Py_ssize_t PyString_Size(object p)
@@ -49,19 +50,31 @@
     PyObject *PyTuple_GET_ITEM_ptr_object "PyTuple_GET_ITEM" (object tpl, int index)
     void Py_INCREF(object)
     void Py_DECREF_ptr "Py_DECREF" (PyObject *)
+    void *PyMem_Malloc(size_t nbytes)
+    void PyMem_Free(void *)
+    void memset(void *, int, size_t)
 
 cdef extern from "string.h":
     void *memcpy(void *dest, void *src, size_t n)
     void *memchr(void *s, int c, size_t n)
+    int memcmp(void *s1, void *s2, size_t n)
     # GNU extension
     # void *memrchr(void *s, int c, size_t n)
     int strncmp(char *s1, char *s2, size_t n)
+    unsigned long strtoul(char *s1, char **out, int base)
+    long long strtoll(char *s1, char **out, int base)
+
 
 # It seems we need to import the definitions so that the pyrex compiler has
 # local names to access them.
 from _static_tuple_c cimport StaticTuple, \
     import_static_tuple_c, StaticTuple_New, \
-    StaticTuple_Intern, StaticTuple_SET_ITEM, StaticTuple_CheckExact
+    StaticTuple_Intern, StaticTuple_SET_ITEM, StaticTuple_CheckExact, \
+    StaticTuple_GET_SIZE, StaticTuple_GET_ITEM
+# This tells the test infrastructure that StaticTuple is a class, so we don't
+# have to worry about exception checking.
+## extern cdef class StaticTuple
+import sys
 
 
 # TODO: Find some way to import this from _dirstate_helpers
@@ -103,7 +116,6 @@
     Py_DECREF_ptr(py_str)
     return result
 
-from bzrlib import _static_tuple_c
 # This sets up the StaticTuple C_API functionality
 import_static_tuple_c()
 
@@ -315,6 +327,537 @@
     return parser.parse()
 
 
+# TODO: We can go from 8 byte offset + 4 byte length to a simple lookup,
+#       because the block_offset + length is likely to be repeated. However,
+#       the big win there is to cache across pages, and not just one page
+#       Though if we did cache in a page, we could certainly use a short int.
+#       And this goes from 40 bytes to 30 bytes.
+#       One slightly ugly option would be to cache block offsets in a global.
+#       However, that leads to thread-safety issues, etc.
+ctypedef struct gc_chk_sha1_record:
+    long long block_offset
+    unsigned int block_length
+    unsigned int record_start
+    unsigned int record_end
+    char sha1[20]
+
+
+cdef int _unhexbuf[256]
+cdef char *_hexbuf
+_hexbuf = '0123456789abcdef'
+
+cdef _populate_unhexbuf():
+    cdef int i
+    for i from 0 <= i < 256:
+        _unhexbuf[i] = -1
+    for i from 0 <= i < 10: # 0123456789 => map to the raw number
+        _unhexbuf[(i + c'0')] = i
+    for i from 10 <= i < 16: # abcdef => 10, 11, 12, 13, 14, 15, 16
+        _unhexbuf[(i - 10 + c'a')] = i
+    for i from 10 <= i < 16: # ABCDEF => 10, 11, 12, 13, 14, 15, 16
+        _unhexbuf[(i - 10 + c'A')] = i
+_populate_unhexbuf()
+
+
+cdef int _unhexlify_sha1(char *as_hex, char *as_bin): # cannot_raise
+    """Take the hex sha1 in as_hex and make it binary in as_bin
+    
+    Same as binascii.unhexlify, but working on C strings, not Python objects.
+    """
+    cdef int top
+    cdef int bot
+    cdef int i, j
+    cdef char *cur
+    
+    # binascii does this using isupper() and tolower() and ?: syntax. I'm
+    # guessing a simple lookup array should be faster.
+    j = 0
+    for i from 0 <= i < 20:
+        top = _unhexbuf[<unsigned char>(as_hex[j])]
+        j = j + 1
+        bot = _unhexbuf[<unsigned char>(as_hex[j])]
+        j = j + 1
+        if top == -1 or bot == -1:
+            return 0
+        as_bin[i] = <unsigned char>((top << 4) + bot);
+    return 1
+
+
+def _py_unhexlify(as_hex):
+    """For the test infrastructure, just thunks to _unhexlify_sha1"""
+    if len(as_hex) != 40 or not PyString_CheckExact(as_hex):
+        raise ValueError('not a 40-byte hex digest')
+    as_bin = PyString_FromStringAndSize(NULL, 20)
+    if _unhexlify_sha1(PyString_AS_STRING(as_hex), PyString_AS_STRING(as_bin)):
+        return as_bin
+    return None
+
+
+cdef void _hexlify_sha1(char *as_bin, char *as_hex): # cannot_raise
+    cdef int i, j
+    cdef char c
+
+    j = 0
+    for i from 0 <= i < 20:
+        c = as_bin[i]
+        as_hex[j] = _hexbuf[(c>>4)&0xf]
+        j = j + 1
+        as_hex[j] = _hexbuf[(c)&0xf]
+        j = j + 1
+
+
+def _py_hexlify(as_bin):
+    """For test infrastructure, thunk to _hexlify_sha1"""
+    if len(as_bin) != 20 or not PyString_CheckExact(as_bin):
+        raise ValueError('not a 20-byte binary digest')
+    as_hex = PyString_FromStringAndSize(NULL, 40)
+    _hexlify_sha1(PyString_AS_STRING(as_bin), PyString_AS_STRING(as_hex))
+    return as_hex
+
+
+cdef int _key_to_sha1(key, char *sha1): # cannot_raise
+    """Map a key into its sha1 content.
+
+    :param key: A tuple of style ('sha1:abcd...',)
+    :param sha1: A char buffer of 20 bytes
+    :return: 1 if this could be converted, 0 otherwise
+    """
+    cdef char *c_val
+    cdef PyObject *p_val
+
+    if StaticTuple_CheckExact(key) and StaticTuple_GET_SIZE(key) == 1:
+        p_val = <PyObject *>StaticTuple_GET_ITEM(key, 0)
+    elif (PyTuple_CheckExact(key) and PyTuple_GET_SIZE(key) == 1):
+        p_val = PyTuple_GET_ITEM_ptr_object(key, 0)
+    else:
+        # Not a tuple or a StaticTuple
+        return 0
+    if (PyString_CheckExact_ptr(p_val) and PyString_GET_SIZE_ptr(p_val) == 45):
+        c_val = PyString_AS_STRING_ptr(p_val)
+    else:
+        return 0
+    if strncmp(c_val, 'sha1:', 5) != 0:
+        return 0
+    if not _unhexlify_sha1(c_val + 5, sha1):
+        return 0
+    return 1
+
+
+def _py_key_to_sha1(key):
+    """Map a key to a simple sha1 string.
+
+    This is a testing thunk to the C function.
+    """
+    as_bin_sha = PyString_FromStringAndSize(NULL, 20)
+    if _key_to_sha1(key, PyString_AS_STRING(as_bin_sha)):
+        return as_bin_sha
+    return None
+
+
+cdef StaticTuple _sha1_to_key(char *sha1):
+    """Compute a ('sha1:abcd',) key for a given sha1."""
+    cdef StaticTuple key
+    cdef object hexxed
+    cdef char *c_buf
+    hexxed = PyString_FromStringAndSize(NULL, 45)
+    c_buf = PyString_AS_STRING(hexxed)
+    memcpy(c_buf, 'sha1:', 5)
+    _hexlify_sha1(sha1, c_buf+5)
+    key = StaticTuple_New(1)
+    Py_INCREF(hexxed)
+    StaticTuple_SET_ITEM(key, 0, hexxed)
+    # This is a bit expensive. To parse 120 keys takes 48us, to return them all
+    # can be done in 66.6us (so 18.6us to build them all).
+    # Adding simple hash() here brings it to 76.6us (so computing the hash
+    # value of 120keys is 10us), Intern is 86.9us (another 10us to look and add
+    # them to the intern structure.)
+    # However, since we only intern keys that are in active use, it is probably
+    # a win. Since they would have been read from elsewhere anyway.
+    # We *could* hang the PyObject form off of the gc_chk_sha1_record for ones
+    # that we have deserialized. Something to think about, at least.
+    key = StaticTuple_Intern(key)
+    return key
+
+
+def _py_sha1_to_key(sha1_bin):
+    """Test thunk to check the sha1 mapping."""
+    if not PyString_CheckExact(sha1_bin) or PyString_GET_SIZE(sha1_bin) != 20:
+        raise ValueError('sha1_bin must be a str of exactly 20 bytes')
+    return _sha1_to_key(PyString_AS_STRING(sha1_bin))
+
+
+cdef unsigned int _sha1_to_uint(char *sha1): # cannot_raise
+    cdef unsigned int val
+    # Must be in MSB, because that is how the content is sorted
+    val = (((<unsigned int>(sha1[0]) & 0xff) << 24)
+           | ((<unsigned int>(sha1[1]) & 0xff) << 16)
+           | ((<unsigned int>(sha1[2]) & 0xff) << 8)
+           | ((<unsigned int>(sha1[3]) & 0xff) << 0))
+    return val
+
+
+cdef _format_record_py24(gc_chk_sha1_record *record):
+    """Python2.4 PyString_FromFormat doesn't have %u.
+
+    It only has %d and %ld. We would really like to even have %llu, which
+    is only in python2.7. So we go back into casting to regular objects.
+    """
+    return "%s %s %s %s" % (record.block_offset, record.block_length,
+                            record.record_start, record.record_end)
+
+
+cdef _format_record(gc_chk_sha1_record *record):
+    # This is inefficient to go from a logical state back to a
+    # string, but it makes things work a bit better internally for now.
+    if record.block_offset >= 0xFFFFFFFF:
+        # %llu is what we really want, but unfortunately it was only added
+        # in python 2.7... :(
+        block_offset_str = str(record.block_offset)
+        value = PyString_FromFormat('%s %lu %lu %lu',
+                                PyString_AS_STRING(block_offset_str),
+                                record.block_length,
+                                record.record_start, record.record_end)
+    else:
+        value = PyString_FromFormat('%lu %lu %lu %lu',
+                                    <unsigned long>record.block_offset,
+                                    record.block_length,
+                                    record.record_start, record.record_end)
+    return value
+
+ctypedef object (*formatproc)(gc_chk_sha1_record *)
+cdef formatproc _record_formatter
+_record_formatter = _format_record
+if sys.version_info[:2] == (2, 4):
+    _record_formatter = _format_record_py24
+
+
+cdef class GCCHKSHA1LeafNode:
+    """Track all the entries for a given leaf node."""
+
+    cdef gc_chk_sha1_record *records
+    cdef public object last_key
+    cdef gc_chk_sha1_record *last_record
+    cdef public int num_records
+    # This is the number of bits to shift to get to the interesting byte. A
+    # value of 24 means that the very first byte changes across all keys.
+    # Anything else means that there is a common prefix of bits that we can
+    # ignore. 0 means that at least the first 3 bytes are identical, though
+    # that is going to be very rare
+    cdef public unsigned char common_shift
+    # This maps an interesting byte to the first record that matches.
+    # Equivalent to bisect.bisect_left(self.records, sha1), though only taking
+    # into account that one byte.
+    cdef unsigned char offsets[257]
+
+    def __sizeof__(self):
+        # :( Why doesn't Pyrex let me do a simple sizeof(GCCHKSHA1LeafNode)
+        # like Cython? Explicitly enumerating everything here seems to leave my
+        # size off by 2 (286 bytes vs 288 bytes actual). I'm guessing it is an
+        # alignment/padding issue. Oh well- at least we scale properly with
+        # num_records and are very close to correct, which is what I care
+        # about.
+        # If we ever decide to require cython:
+        # return (sizeof(GCCHKSHA1LeafNode)
+        #     + sizeof(gc_chk_sha1_record)*self.num_records)
+        return (sizeof(PyObject) + sizeof(void*) + sizeof(int)
+            + sizeof(gc_chk_sha1_record*) + sizeof(PyObject *)
+            + sizeof(gc_chk_sha1_record*) + sizeof(char)
+            + sizeof(unsigned char)*257
+            + sizeof(gc_chk_sha1_record)*self.num_records)
+
+    def __dealloc__(self):
+        if self.records != NULL:
+            PyMem_Free(self.records)
+            self.records = NULL
+
+    def __init__(self, bytes):
+        self._parse_bytes(bytes)
+        self.last_key = None
+        self.last_record = NULL
+
+    property min_key:
+        def __get__(self):
+            if self.num_records > 0:
+                return _sha1_to_key(self.records[0].sha1)
+            return None
+
+    property max_key:
+        def __get__(self):
+            if self.num_records > 0:
+                return _sha1_to_key(self.records[self.num_records-1].sha1)
+            return None
+
+    cdef StaticTuple _record_to_value_and_refs(self,
+                                               gc_chk_sha1_record *record):
+        """Extract the refs and value part of this record."""
+        cdef StaticTuple value_and_refs
+        cdef StaticTuple empty
+        value_and_refs = StaticTuple_New(2)
+        value = _record_formatter(record)
+        Py_INCREF(value)
+        StaticTuple_SET_ITEM(value_and_refs, 0, value)
+        # Always empty refs
+        empty = StaticTuple_New(0)
+        Py_INCREF(empty)
+        StaticTuple_SET_ITEM(value_and_refs, 1, empty)
+        return value_and_refs
+
+    cdef StaticTuple _record_to_item(self, gc_chk_sha1_record *record):
+        """Turn a given record back into a fully fledged item.
+        """
+        cdef StaticTuple item
+        cdef StaticTuple key
+        cdef StaticTuple value_and_refs
+        cdef object value
+        key = _sha1_to_key(record.sha1)
+        item = StaticTuple_New(2)
+        Py_INCREF(key)
+        StaticTuple_SET_ITEM(item, 0, key)
+        value_and_refs = self._record_to_value_and_refs(record)
+        Py_INCREF(value_and_refs)
+        StaticTuple_SET_ITEM(item, 1, value_and_refs)
+        return item
+
+    cdef gc_chk_sha1_record* _lookup_record(self, char *sha1) except? NULL:
+        """Find a gc_chk_sha1_record that matches the sha1 supplied."""
+        cdef int lo, hi, mid, the_cmp
+        cdef int offset
+
+        # TODO: We can speed up misses by comparing this sha1 to the common
+        #       bits, and seeing if the common prefix matches, if not, we don't
+        #       need to search for anything because it cannot match
+        # Use the offset array to find the closest fit for this entry
+        # follow that up with bisecting, since multiple keys can be in one
+        # spot
+        # Bisecting dropped us from 7000 comparisons to 582 (4.8/key), using
+        # the offset array dropped us from 23us to 20us and 156 comparisions
+        # (1.3/key)
+        offset = self._offset_for_sha1(sha1)
+        lo = self.offsets[offset]
+        hi = self.offsets[offset+1]
+        if hi == 255:
+            # if hi == 255 that means we potentially ran off the end of the
+            # list, so push it up to num_records
+            # note that if 'lo' == 255, that is ok, because we can start
+            # searching from that part of the list.
+            hi = self.num_records
+        local_n_cmp = 0
+        while lo < hi:
+            mid = (lo + hi) / 2
+            the_cmp = memcmp(self.records[mid].sha1, sha1, 20)
+            if the_cmp == 0:
+                return &self.records[mid]
+            elif the_cmp < 0:
+                lo = mid + 1
+            else:
+                hi = mid
+        return NULL
+
+    def __contains__(self, key):
+        cdef char sha1[20]
+        cdef gc_chk_sha1_record *record
+        if _key_to_sha1(key, sha1):
+            # If it isn't a sha1 key, then it won't be in this leaf node
+            record = self._lookup_record(sha1)
+            if record != NULL:
+                self.last_key = key
+                self.last_record = record
+                return True
+        return False
+
+    def __getitem__(self, key):
+        cdef char sha1[20]
+        cdef gc_chk_sha1_record *record
+        record = NULL
+        if self.last_record != NULL and key is self.last_key:
+            record = self.last_record
+        elif _key_to_sha1(key, sha1):
+            record = self._lookup_record(sha1)
+        if record == NULL:
+            raise KeyError('key %r is not present' % (key,))
+        return self._record_to_value_and_refs(record)
+
+    def __len__(self):
+        return self.num_records
+
+    def all_keys(self):
+        cdef int i
+        result = []
+        for i from 0 <= i < self.num_records:
+            PyList_Append(result, _sha1_to_key(self.records[i].sha1))
+        return result
+
+    def all_items(self):
+        cdef int i
+        result = []
+        for i from 0 <= i < self.num_records:
+            item = self._record_to_item(&self.records[i])
+            PyList_Append(result, item)
+        return result
+
+    cdef int _count_records(self, char *c_content, char *c_end): # cannot_raise
+        """Count how many records are in this section."""
+        cdef char *c_cur
+        cdef int num_records
+
+        c_cur = c_content
+        num_records = 0
+        while c_cur != NULL and c_cur < c_end:
+            c_cur = <char *>memchr(c_cur, c'\n', c_end - c_cur);
+            if c_cur == NULL:
+                break
+            c_cur = c_cur + 1
+            num_records = num_records + 1
+        return num_records
+
+    cdef _parse_bytes(self, bytes):
+        """Parse the string 'bytes' into content."""
+        cdef char *c_bytes
+        cdef char *c_cur
+        cdef char *c_end
+        cdef Py_ssize_t n_bytes
+        cdef int num_records
+        cdef int entry
+        cdef gc_chk_sha1_record *cur_record
+
+        if not PyString_CheckExact(bytes):
+            raise TypeError('We only support parsing plain 8-bit strings.')
+        # Pass 1, count how many records there will be
+        n_bytes = PyString_GET_SIZE(bytes)
+        c_bytes = PyString_AS_STRING(bytes)
+        c_end = c_bytes + n_bytes
+        if strncmp(c_bytes, 'type=leaf\n', 10):
+            raise ValueError("bytes did not start with 'type=leaf\\n': %r"
+                             % (bytes[:10],))
+        c_cur = c_bytes + 10
+        num_records = self._count_records(c_cur, c_end)
+        # Now allocate the memory for these items, and go to town
+        self.records = <gc_chk_sha1_record*>PyMem_Malloc(num_records *
+            (sizeof(unsigned short) + sizeof(gc_chk_sha1_record)))
+        self.num_records = num_records
+        cur_record = self.records
+        entry = 0
+        while c_cur != NULL and c_cur < c_end and entry < num_records:
+            c_cur = self._parse_one_entry(c_cur, c_end, cur_record)
+            cur_record = cur_record + 1
+            entry = entry + 1
+        if (entry != self.num_records
+            or c_cur != c_end
+            or cur_record != self.records + self.num_records):
+            raise ValueError('Something went wrong while parsing.')
+        # Pass 3: build the offset map
+        self._compute_common()
+
+    cdef char *_parse_one_entry(self, char *c_cur, char *c_end,
+                                gc_chk_sha1_record *cur_record) except NULL:
+        """Read a single sha record from the bytes.
+        :param c_cur: The pointer to the start of bytes
+        :param cur_record: 
+        """
+        cdef char *c_next
+        if strncmp(c_cur, 'sha1:', 5):
+            raise ValueError('line did not start with sha1: %r'
+                % (safe_string_from_size(c_cur, 10),))
+        c_cur = c_cur + 5
+        c_next = <char *>memchr(c_cur, c'\0', c_end - c_cur)
+        if c_next == NULL or (c_next - c_cur != 40):
+            raise ValueError('Line did not contain 40 hex bytes')
+        if not _unhexlify_sha1(c_cur, cur_record.sha1):
+            raise ValueError('We failed to unhexlify')
+        c_cur = c_next + 1
+        if c_cur[0] != c'\0':
+            raise ValueError('only 1 null, not 2 as expected')
+        c_cur = c_cur + 1
+        cur_record.block_offset = strtoll(c_cur, &c_next, 10)
+        if c_cur == c_next or c_next[0] != c' ':
+            raise ValueError('Failed to parse block offset')
+        c_cur = c_next + 1
+        cur_record.block_length = strtoul(c_cur, &c_next, 10)
+        if c_cur == c_next or c_next[0] != c' ':
+            raise ValueError('Failed to parse block length')
+        c_cur = c_next + 1
+        cur_record.record_start = strtoul(c_cur, &c_next, 10)
+        if c_cur == c_next or c_next[0] != c' ':
+            raise ValueError('Failed to parse block length')
+        c_cur = c_next + 1
+        cur_record.record_end = strtoul(c_cur, &c_next, 10)
+        if c_cur == c_next or c_next[0] != c'\n':
+            raise ValueError('Failed to parse record end')
+        c_cur = c_next + 1
+        return c_cur
+
+    cdef int _offset_for_sha1(self, char *sha1) except -1:
+        """Find the first interesting 8-bits of this sha1."""
+        cdef int this_offset
+        cdef unsigned int as_uint
+        as_uint = _sha1_to_uint(sha1)
+        this_offset = (as_uint >> self.common_shift) & 0xFF
+        return this_offset
+
+    def _get_offset_for_sha1(self, sha1):
+        return self._offset_for_sha1(PyString_AS_STRING(sha1))
+
+    cdef _compute_common(self):
+        cdef unsigned int first
+        cdef unsigned int this
+        cdef unsigned int common_mask
+        cdef unsigned char common_shift
+        cdef int i
+        cdef int offset, this_offset
+        cdef int max_offset
+        # The idea with the offset map is that we should be able to quickly
+        # jump to the key that matches a gives sha1. We know that the keys are
+        # in sorted order, and we know that a lot of the prefix is going to be
+        # the same across them.
+        # By XORing the records together, we can determine what bits are set in
+        # all of them
+        if self.num_records < 2:
+            # Everything is in common if you have 0 or 1 leaves
+            # So we'll always just shift to the first byte
+            self.common_shift = 24
+        else:
+            common_mask = 0xFFFFFFFF
+            first = _sha1_to_uint(self.records[0].sha1)
+            for i from 0 < i < self.num_records:
+                this = _sha1_to_uint(self.records[i].sha1)
+                common_mask = (~(first ^ this)) & common_mask
+            common_shift = 24
+            while common_mask & 0x80000000 and common_shift > 0:
+                common_mask = common_mask << 1
+                common_shift = common_shift - 1
+            self.common_shift = common_shift
+        offset = 0
+        max_offset = self.num_records
+        # We cap this loop at 254 records. All the other offsets just get
+        # filled with 0xff as the singleton saying 'too many'.
+        # It means that if we have >255 records we have to bisect the second
+        # half of the list, but this is going to be very rare in practice.
+        if max_offset > 255:
+            max_offset = 255
+        for i from 0 <= i < max_offset:
+            this_offset = self._offset_for_sha1(self.records[i].sha1)
+            while offset <= this_offset:
+                self.offsets[offset] = i
+                offset = offset + 1
+        while offset < 257:
+            self.offsets[offset] = max_offset
+            offset = offset + 1
+
+    def _get_offsets(self):
+        cdef int i
+        result = []
+        for i from 0 <= i < 257:
+            PyList_Append(result, self.offsets[i])
+        return result
+
+
+def _parse_into_chk(bytes, key_length, ref_list_length):
+    """Parse into a format optimized for chk records."""
+    assert key_length == 1
+    assert ref_list_length == 0
+    return GCCHKSHA1LeafNode(bytes)
+
+
 def _flatten_node(node, reference_lists):
     """Convert a node into the serialized form.
 

=== modified file 'bzrlib/_chk_map_pyx.pyx'
--- bzrlib/_chk_map_pyx.pyx	2010-05-16 15:18:43 +0000
+++ bzrlib/_chk_map_pyx.pyx	2011-02-09 08:08:20 +0000
@@ -413,7 +413,7 @@
     cdef Py_ssize_t byte_size, pos, file_id_len
 
     if not PyString_CheckExact(bytes):
-        raise TypeError('bytes must be a string')
+        raise TypeError('bytes must be a string, got %r' % (type(bytes),))
     byte_str = PyString_AS_STRING(bytes)
     byte_size = PyString_GET_SIZE(bytes)
     byte_end = byte_str + byte_size
@@ -421,7 +421,8 @@
     if cur_end == NULL:
         raise ValueError('No kind section found.')
     if cur_end[1] != c' ':
-        raise ValueError('Kind section should end with ": "')
+        raise ValueError(
+            'Kind section should end with ": ", got %r' % str(cur_end[:2],))
     file_id_str = cur_end + 2
     # file_id is now the data up until the next newline
     cur_end = <char*>memchr(file_id_str, c'\n', byte_end - file_id_str)

=== modified file 'bzrlib/_dirstate_helpers_pyx.pyx'
--- bzrlib/_dirstate_helpers_pyx.pyx	2010-05-20 02:57:52 +0000
+++ bzrlib/_dirstate_helpers_pyx.pyx	2011-02-09 08:08:20 +0000
@@ -1,4 +1,4 @@
-# Copyright (C) 2007, 2008, 2010 Canonical Ltd
+# Copyright (C) 2007-2010 Canonical Ltd
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -118,6 +118,11 @@
     # ??? memrchr is a GNU extension :(
     # void *memrchr(void *s, int c, size_t len)
 
+# cimport all of the definitions we will need to access
+from _static_tuple_c cimport import_static_tuple_c, StaticTuple, \
+    StaticTuple_New, StaticTuple_SET_ITEM
+
+import_static_tuple_c()
 
 cdef void* _my_memrchr(void *s, int c, size_t n): # cannot_raise
     # memrchr seems to be a GNU extension, so we have to implement it ourselves
@@ -610,7 +615,8 @@
         :param new_block: This is to let the caller know that it needs to
             create a new directory block to store the next entry.
         """
-        cdef object path_name_file_id_key
+        cdef StaticTuple path_name_file_id_key
+        cdef StaticTuple tmp
         cdef char *entry_size_cstr
         cdef unsigned long int entry_size
         cdef char* executable_cstr
@@ -650,10 +656,20 @@
         # Build up the key that will be used.
         # By using <object>(void *) Pyrex will automatically handle the
         # Py_INCREF that we need.
-        path_name_file_id_key = (<object>p_current_dirname[0],
-                                 self.get_next_str(),
-                                 self.get_next_str(),
-                                )
+        cur_dirname = <object>p_current_dirname[0]
+        # Use StaticTuple_New to pre-allocate, rather than creating a regular
+        # tuple and passing it to the StaticTuple constructor.
+        # path_name_file_id_key = StaticTuple(<object>p_current_dirname[0],
+        #                          self.get_next_str(),
+        #                          self.get_next_str(),
+        #                         )
+        tmp = StaticTuple_New(3)
+        Py_INCREF(cur_dirname); StaticTuple_SET_ITEM(tmp, 0, cur_dirname)
+        cur_basename = self.get_next_str()
+        cur_file_id = self.get_next_str()
+        Py_INCREF(cur_basename); StaticTuple_SET_ITEM(tmp, 1, cur_basename)
+        Py_INCREF(cur_file_id); StaticTuple_SET_ITEM(tmp, 2, cur_file_id)
+        path_name_file_id_key = tmp
 
         # Parse all of the per-tree information. current has the information in
         # the same location as parent trees. The only difference is that 'info'
@@ -677,7 +693,20 @@
             executable_cstr = self.get_next(&cur_size)
             is_executable = (executable_cstr[0] == c'y')
             info = self.get_next_str()
-            PyList_Append(trees, (
+            # TODO: If we want to use StaticTuple_New here we need to be pretty
+            #       careful. We are relying on a bit of Pyrex
+            #       automatic-conversion from 'int' to PyInt, and that doesn't
+            #       play well with the StaticTuple_SET_ITEM macro.
+            #       Timing doesn't (yet) show a worthwile improvement in speed
+            #       versus complexity and maintainability.
+            # tmp = StaticTuple_New(5)
+            # Py_INCREF(minikind); StaticTuple_SET_ITEM(tmp, 0, minikind)
+            # Py_INCREF(fingerprint); StaticTuple_SET_ITEM(tmp, 1, fingerprint)
+            # Py_INCREF(entry_size); StaticTuple_SET_ITEM(tmp, 2, entry_size)
+            # Py_INCREF(is_executable); StaticTuple_SET_ITEM(tmp, 3, is_executable)
+            # Py_INCREF(info); StaticTuple_SET_ITEM(tmp, 4, info)
+            # PyList_Append(trees, tmp)
+            PyList_Append(trees, StaticTuple(
                 minikind,     # minikind
                 fingerprint,  # fingerprint
                 entry_size,   # size

=== modified file 'bzrlib/_groupcompress_pyx.pyx'
--- bzrlib/_groupcompress_pyx.pyx	2010-02-17 17:11:16 +0000
+++ bzrlib/_groupcompress_pyx.pyx	2011-02-09 08:08:20 +0000
@@ -22,6 +22,8 @@
 
 
 cdef extern from "Python.h":
+    ctypedef struct PyObject:
+        pass
     ctypedef int Py_ssize_t # Required for older pyrex versions
     int PyString_CheckExact(object)
     char * PyString_AS_STRING(object)
@@ -53,6 +55,7 @@
              unsigned long *delta_size, unsigned long max_delta_size) nogil
     unsigned long get_delta_hdr_size(unsigned char **datap,
                                      unsigned char *top) nogil
+    unsigned long sizeof_delta_index(delta_index *index)
     Py_ssize_t DELTA_SIZE_MIN
 
 
@@ -91,8 +94,8 @@
     cdef readonly object _sources
     cdef source_info *_source_infos
     cdef delta_index *_index
+    cdef public unsigned long _source_offset
     cdef readonly unsigned int _max_num_sources
-    cdef public unsigned long _source_offset
 
     def __init__(self, source=None):
         self._sources = []
@@ -105,6 +108,22 @@
         if source is not None:
             self.add_source(source, 0)
 
+    def __sizeof__(self):
+        # We want to track the _source_infos allocations, but the referenced
+        # void* are actually tracked in _sources itself.
+        # XXX: Cython is capable of doing sizeof(class) and returning the size
+        #      of the underlying struct. Pyrex (<= 0.9.9) refuses, so we need
+        #      to do it manually. *sigh* Note that we might get it wrong
+        #      because of alignment issues.
+        cdef Py_ssize_t size
+        # PyObject start, vtable *, 3 object pointers, 2 C ints
+        size = ((sizeof(PyObject) + sizeof(void*) + 3*sizeof(PyObject*)
+                 + sizeof(unsigned long)
+                 + sizeof(unsigned int))
+                + (sizeof(source_info) * self._max_num_sources)
+                + sizeof_delta_index(self._index))
+        return size
+
     def __repr__(self):
         return '%s(%d, %d)' % (self.__class__.__name__,
             len(self._sources), self._source_offset)

=== modified file 'bzrlib/_simple_set_pyx.pyx'
--- bzrlib/_simple_set_pyx.pyx	2010-02-17 17:11:16 +0000
+++ bzrlib/_simple_set_pyx.pyx	2011-02-09 08:08:20 +0000
@@ -115,6 +115,20 @@
             raise MemoryError()
         memset(self._table, 0, n_bytes)
 
+    def __sizeof__(self):
+        # Note: Pyrex doesn't allow sizeof(class) so we re-implement it here.
+        # Bits are:
+        #   1: PyObject
+        #   2: vtable *
+        #   3: 3 Py_ssize_t
+        #   4: PyObject**
+        # Note that we might get alignment, etc, wrong, but at least this is
+        # better than no estimate at all
+        # return sizeof(SimpleSet) + (self._mask + 1) * (sizeof(PyObject*))
+        return (sizeof(PyObject) + sizeof(void*)
+                + 3*sizeof(Py_ssize_t) + sizeof(PyObject**)
+                + (self._mask + 1) * sizeof(PyObject*))
+
     def __dealloc__(self):
         if self._table != NULL:
             PyMem_Free(self._table)

=== modified file 'bzrlib/_static_tuple_c.pxd'
--- bzrlib/_static_tuple_c.pxd	2010-05-11 14:13:31 +0000
+++ bzrlib/_static_tuple_c.pxd	2011-02-09 08:08:20 +0000
@@ -33,6 +33,7 @@
     int import_static_tuple_c() except -1
     StaticTuple StaticTuple_New(Py_ssize_t)
     StaticTuple StaticTuple_Intern(StaticTuple)
+    StaticTuple StaticTuple_FromSequence(object)
 
     # Steals a reference and val must be a valid type, no checking is done
     void StaticTuple_SET_ITEM(StaticTuple key, Py_ssize_t offset, object val)

=== modified file 'bzrlib/branch.py'
--- bzrlib/branch.py	2010-08-13 07:32:06 +0000
+++ bzrlib/branch.py	2011-02-09 08:08:20 +0000
@@ -1,4 +1,4 @@
-# Copyright (C) 2005-2010 Canonical Ltd
+# Copyright (C) 2005-2011 Canonical Ltd
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -25,8 +25,11 @@
         bzrdir,
         cache_utf8,
         config as _mod_config,
+        controldir,
         debug,
         errors,
+        fetch,
+        graph as _mod_graph,
         lockdir,
         lockable_files,
         remote,
@@ -64,7 +67,7 @@
 BZR_BRANCH_FORMAT_6 = "Bazaar Branch Format 6 (bzr 0.15)\n"
 
 
-class Branch(bzrdir.ControlComponent):
+class Branch(controldir.ControlComponent):
     """Branch holding a history of revisions.
 
     :ivar base:
@@ -91,6 +94,7 @@
         self._revision_id_to_revno_cache = None
         self._partial_revision_id_to_revno_cache = {}
         self._partial_revision_history_cache = []
+        self._tags_bytes = None
         self._last_revision_info_cache = None
         self._merge_sorted_revisions_cache = None
         self._open_hook()
@@ -103,6 +107,13 @@
 
     def _activate_fallback_location(self, url):
         """Activate the branch/repository from url as a fallback repository."""
+        for existing_fallback_repo in self.repository._fallback_repositories:
+            if existing_fallback_repo.user_url == url:
+                # This fallback is already configured.  This probably only
+                # happens because BzrDir.sprout is a horrible mess.  To avoid
+                # confusing _unstack we don't add this a second time.
+                mutter('duplicate activation of fallback %r on %r', url, self)
+                return
         repo = self._get_fallback_repository(url)
         if repo.has_same_location(self.repository):
             raise errors.UnstackableLocationError(self.user_url, url)
@@ -226,6 +237,7 @@
             possible_transports=[self.bzrdir.root_transport])
         return a_branch.repository
 
+    @needs_read_lock
     def _get_tags_bytes(self):
         """Get the bytes of a serialised tags dict.
 
@@ -238,7 +250,9 @@
         :return: The bytes of the tags file.
         :seealso: Branch._set_tags_bytes.
         """
-        return self._transport.get_bytes('tags')
+        if self._tags_bytes is None:
+            self._tags_bytes = self._transport.get_bytes('tags')
+        return self._tags_bytes
 
     def _get_nick(self, local=False, possible_transports=None):
         config = self.get_config()
@@ -648,15 +662,22 @@
         raise errors.UnsupportedOperation(self.get_reference_info, self)
 
     @needs_write_lock
-    def fetch(self, from_branch, last_revision=None, pb=None):
+    def fetch(self, from_branch, last_revision=None, pb=None, fetch_spec=None):
         """Copy revisions from from_branch into this branch.
 
         :param from_branch: Where to copy from.
         :param last_revision: What revision to stop at (None for at the end
                               of the branch.
         :param pb: An optional progress bar to use.
+        :param fetch_spec: If specified, a SearchResult or
+            PendingAncestryResult that describes which revisions to copy.  This
+            allows copying multiple heads at once.  Mutually exclusive with
+            last_revision.
         :return: None
         """
+        if fetch_spec is not None and last_revision is not None:
+            raise AssertionError(
+                "fetch_spec and last_revision are mutually exclusive.")
         if self.base == from_branch.base:
             return (0, [])
         if pb is not None:
@@ -665,12 +686,12 @@
                 % "pb parameter to fetch()")
         from_branch.lock_read()
         try:
-            if last_revision is None:
+            if last_revision is None and fetch_spec is None:
                 last_revision = from_branch.last_revision()
                 last_revision = _mod_revision.ensure_null(last_revision)
             return self.repository.fetch(from_branch.repository,
                                          revision_id=last_revision,
-                                         pb=pb)
+                                         pb=pb, fetch_spec=fetch_spec)
         finally:
             from_branch.unlock()
 
@@ -804,7 +825,8 @@
             old_repository = self.repository
             if len(old_repository._fallback_repositories) != 1:
                 raise AssertionError("can't cope with fallback repositories "
-                    "of %r" % (self.repository,))
+                    "of %r (fallbacks: %r)" % (old_repository,
+                        old_repository._fallback_repositories))
             # Open the new repository object.
             # Repositories don't offer an interface to remove fallback
             # repositories today; take the conceptually simpler option and just
@@ -875,8 +897,12 @@
 
         :seealso: Branch._get_tags_bytes.
         """
-        return _run_with_write_locked_target(self, self._transport.put_bytes,
-            'tags', bytes)
+        return _run_with_write_locked_target(self, self._set_tags_bytes_locked,
+                bytes)
+
+    def _set_tags_bytes_locked(self, bytes):
+        self._tags_bytes = bytes
+        return self._transport.put_bytes('tags', bytes)
 
     def _cache_revision_history(self, rev_history):
         """Set the cached revision history to rev_history.
@@ -912,6 +938,7 @@
         self._merge_sorted_revisions_cache = None
         self._partial_revision_history_cache = []
         self._partial_revision_id_to_revno_cache = {}
+        self._tags_bytes = None
 
     def _gen_revision_history(self):
         """Return sequence of revision hashes on to this branch.
@@ -1002,7 +1029,7 @@
         return other_history[self_len:stop_revision]
 
     def update_revisions(self, other, stop_revision=None, overwrite=False,
-                         graph=None):
+                         graph=None, fetch_tags=True):
         """Pull in new perfect-fit revisions.
 
         :param other: Another Branch to pull from
@@ -1011,17 +1038,17 @@
             to see if it is a proper descendant.
         :param graph: A Graph object that can be used to query history
             information. This can be None.
+        :param fetch_tags: Flag that specifies if tags from other should be
+            fetched too.
         :return: None
         """
         return InterBranch.get(other, self).update_revisions(stop_revision,
-            overwrite, graph)
+            overwrite, graph, fetch_tags=fetch_tags)
 
+    @deprecated_method(deprecated_in((2, 4, 0)))
     def import_last_revision_info(self, source_repo, revno, revid):
         """Set the last revision info, importing from another repo if necessary.
 
-        This is used by the bound branch code to upload a revision to
-        the master branch first before updating the tip of the local branch.
-
         :param source_repo: Source repository to optionally fetch from
         :param revno: Revision number of the new tip
         :param revid: Revision id of the new tip
@@ -1030,6 +1057,28 @@
             self.repository.fetch(source_repo, revision_id=revid)
         self.set_last_revision_info(revno, revid)
 
+    def import_last_revision_info_and_tags(self, source, revno, revid):
+        """Set the last revision info, importing from another repo if necessary.
+
+        This is used by the bound branch code to upload a revision to
+        the master branch first before updating the tip of the local branch.
+        Revisions referenced by source's tags are also transferred.
+
+        :param source: Source branch to optionally fetch from
+        :param revno: Revision number of the new tip
+        :param revid: Revision id of the new tip
+        """
+        if not self.repository.has_same_location(source.repository):
+            try:
+                tags_to_fetch = set(source.tags.get_reverse_tag_dict())
+            except errors.TagsNotSupported:
+                tags_to_fetch = set()
+            fetch_spec = _mod_graph.NotInOtherForRevs(self.repository,
+                source.repository, [revid],
+                if_present_ids=tags_to_fetch).execute()
+            self.repository.fetch(source.repository, fetch_spec=fetch_spec)
+        self.set_last_revision_info(revno, revid)
+
     def revision_id_to_revno(self, revision_id):
         """Given a revision id, return its revno"""
         if _mod_revision.is_null(revision_id):
@@ -1256,7 +1305,8 @@
         return result
 
     @needs_read_lock
-    def sprout(self, to_bzrdir, revision_id=None, repository_policy=None):
+    def sprout(self, to_bzrdir, revision_id=None, repository_policy=None,
+            repository=None):
         """Create a new line of development from the branch, into to_bzrdir.
 
         to_bzrdir controls the branch format.
@@ -1267,7 +1317,7 @@
         if (repository_policy is not None and
             repository_policy.requires_stacking()):
             to_bzrdir._format.require_stacking(_skip_repo=True)
-        result = to_bzrdir.create_branch()
+        result = to_bzrdir.create_branch(repository=repository)
         result.lock_write()
         try:
             if repository_policy is not None:
@@ -1361,17 +1411,13 @@
         """Return the most suitable metadir for a checkout of this branch.
         Weaves are used if this branch's repository uses weaves.
         """
-        if isinstance(self.bzrdir, bzrdir.BzrDirPreSplitOut):
-            from bzrlib.repofmt import weaverepo
-            format = bzrdir.BzrDirMetaFormat1()
-            format.repository_format = weaverepo.RepositoryFormat7()
-        else:
-            format = self.repository.bzrdir.checkout_metadir()
-            format.set_branch_format(self._format)
+        format = self.repository.bzrdir.checkout_metadir()
+        format.set_branch_format(self._format)
         return format
 
     def create_clone_on_transport(self, to_transport, revision_id=None,
-        stacked_on=None, create_prefix=False, use_existing_dir=False):
+        stacked_on=None, create_prefix=False, use_existing_dir=False,
+        no_tree=None):
         """Create a clone of this branch and its bzrdir.
 
         :param to_transport: The transport to clone onto.
@@ -1390,7 +1436,8 @@
             revision_id = self.last_revision()
         dir_to = self.bzrdir.clone_on_transport(to_transport,
             revision_id=revision_id, stacked_on=stacked_on,
-            create_prefix=create_prefix, use_existing_dir=use_existing_dir)
+            create_prefix=create_prefix, use_existing_dir=use_existing_dir,
+            no_tree=no_tree)
         return dir_to.open_branch()
 
     def create_checkout(self, to_location, revision_id=None,
@@ -1521,7 +1568,7 @@
      * an open routine.
 
     Formats are placed in an dict by their format string for reference
-    during branch opening. Its not required that these be instances, they
+    during branch opening. It's not required that these be instances, they
     can be classes themselves with class methods - it simply depends on
     whether state is needed for a given format or not.
 
@@ -1536,6 +1583,9 @@
     _formats = {}
     """The known formats."""
 
+    _extra_formats = []
+    """Extra formats that can not be part of a metadir."""
+
     can_set_append_revisions_only = True
 
     def __eq__(self, other):
@@ -1576,7 +1626,7 @@
             if isinstance(fmt, MetaDirBranchFormatFactory):
                 fmt = fmt()
             result.append(fmt)
-        return result
+        return result + klass._extra_formats
 
     def get_reference(self, a_bzrdir, name=None):
         """Get the target reference of the branch in a_bzrdir.
@@ -1622,7 +1672,8 @@
             hook(params)
 
     def _initialize_helper(self, a_bzrdir, utf8_files, name=None,
-                           lock_type='metadir', set_format=True):
+                           repository=None, lock_type='metadir',
+                           set_format=True):
         """Initialize a branch in a bzrdir, with specified files
 
         :param a_bzrdir: The bzrdir to initialize the branch in
@@ -1662,11 +1713,12 @@
         finally:
             if lock_taken:
                 control_files.unlock()
-        branch = self.open(a_bzrdir, name, _found=True)
+        branch = self.open(a_bzrdir, name, _found=True,
+                found_repository=repository)
         self._run_post_branch_init_hooks(a_bzrdir, name, branch)
         return branch
 
-    def initialize(self, a_bzrdir, name=None):
+    def initialize(self, a_bzrdir, name=None, repository=None):
         """Create a branch of this format in a_bzrdir.
         
         :param name: Name of the colocated branch to create.
@@ -1706,7 +1758,8 @@
         """
         raise NotImplementedError(self.network_name)
 
-    def open(self, a_bzrdir, name=None, _found=False, ignore_fallbacks=False):
+    def open(self, a_bzrdir, name=None, _found=False, ignore_fallbacks=False,
+            found_repository=None):
         """Return the branch object for a_bzrdir
 
         :param a_bzrdir: A BzrDir that contains a branch.
@@ -1719,6 +1772,17 @@
         raise NotImplementedError(self.open)
 
     @classmethod
+    def register_extra_format(klass, format):
+        """Register a branch format that can not be part of a metadir.
+
+        This is mainly useful to allow custom branch formats, such as
+        older Bazaar formats and foreign formats, to be tested
+        """
+        klass._extra_formats.append(format)
+        network_format_registry.register(
+            format.network_name(), format.__class__)
+
+    @classmethod
     def register_format(klass, format):
         """Register a metadir format.
         
@@ -1750,6 +1814,10 @@
     def unregister_format(klass, format):
         del klass._formats[format.get_format_string()]
 
+    @classmethod
+    def unregister_extra_format(klass, format):
+        klass._extra_formats.remove(format)
+
     def __str__(self):
         return self.get_format_description().rstrip()
 
@@ -1818,7 +1886,7 @@
             "with a bzrlib.branch.PullResult object and only runs in the "
             "bzr client.", (0, 15), None))
         self.create_hook(HookPoint('pre_commit',
-            "Called after a commit is calculated but before it is is "
+            "Called after a commit is calculated but before it is "
             "completed. pre_commit is called with (local, master, old_revno, "
             "old_revid, future_revno, future_revid, tree_delta, future_tree"
             "). old_revid is NULL_REVISION for the first commit to a branch, "
@@ -2006,8 +2074,11 @@
         """See BranchFormat.get_format_description()."""
         return "Branch format 4"
 
-    def initialize(self, a_bzrdir, name=None):
+    def initialize(self, a_bzrdir, name=None, repository=None):
         """Create a branch of this format in a_bzrdir."""
+        if repository is not None:
+            raise NotImplementedError(
+                "initialize(repository=<not None>) on %r" % (self,))
         utf8_files = [('revision-history', ''),
                       ('branch-name', ''),
                       ]
@@ -2022,16 +2093,19 @@
         """The network name for this format is the control dirs disk label."""
         return self._matchingbzrdir.get_format_string()
 
-    def open(self, a_bzrdir, name=None, _found=False, ignore_fallbacks=False):
+    def open(self, a_bzrdir, name=None, _found=False, ignore_fallbacks=False,
+            found_repository=None):
         """See BranchFormat.open()."""
         if not _found:
             # we are being called directly and must probe.
             raise NotImplementedError
-        return BzrBranch(_format=self,
+        if found_repository is None:
+            found_repository = a_bzrdir.open_repository()
+        return BzrBranchPreSplitOut(_format=self,
                          _control_files=a_bzrdir._control_files,
                          a_bzrdir=a_bzrdir,
                          name=name,
-                         _repository=a_bzrdir.open_repository())
+                         _repository=found_repository)
 
     def __str__(self):
         return "Bazaar-NG branch format 4"
@@ -2051,7 +2125,8 @@
         """
         return self.get_format_string()
 
-    def open(self, a_bzrdir, name=None, _found=False, ignore_fallbacks=False):
+    def open(self, a_bzrdir, name=None, _found=False, ignore_fallbacks=False,
+            found_repository=None):
         """See BranchFormat.open()."""
         if not _found:
             format = BranchFormat.find_format(a_bzrdir, name=name)
@@ -2062,11 +2137,13 @@
         try:
             control_files = lockable_files.LockableFiles(transport, 'lock',
                                                          lockdir.LockDir)
+            if found_repository is None:
+                found_repository = a_bzrdir.find_repository()
             return self._branch_class()(_format=self,
                               _control_files=control_files,
                               name=name,
                               a_bzrdir=a_bzrdir,
-                              _repository=a_bzrdir.find_repository(),
+                              _repository=found_repository,
                               ignore_fallbacks=ignore_fallbacks)
         except errors.NoSuchFile:
             raise errors.NotBranchError(path=transport.base, bzrdir=a_bzrdir)
@@ -2104,12 +2181,12 @@
         """See BranchFormat.get_format_description()."""
         return "Branch format 5"
 
-    def initialize(self, a_bzrdir, name=None):
+    def initialize(self, a_bzrdir, name=None, repository=None):
         """Create a branch of this format in a_bzrdir."""
         utf8_files = [('revision-history', ''),
                       ('branch-name', ''),
                       ]
-        return self._initialize_helper(a_bzrdir, utf8_files, name)
+        return self._initialize_helper(a_bzrdir, utf8_files, name, repository)
 
     def supports_tags(self):
         return False
@@ -2137,13 +2214,13 @@
         """See BranchFormat.get_format_description()."""
         return "Branch format 6"
 
-    def initialize(self, a_bzrdir, name=None):
+    def initialize(self, a_bzrdir, name=None, repository=None):
         """Create a branch of this format in a_bzrdir."""
         utf8_files = [('last-revision', '0 null:\n'),
                       ('branch.conf', ''),
                       ('tags', ''),
                       ]
-        return self._initialize_helper(a_bzrdir, utf8_files, name)
+        return self._initialize_helper(a_bzrdir, utf8_files, name, repository)
 
     def make_tags(self, branch):
         """See bzrlib.branch.BranchFormat.make_tags()."""
@@ -2167,14 +2244,14 @@
         """See BranchFormat.get_format_description()."""
         return "Branch format 8"
 
-    def initialize(self, a_bzrdir, name=None):
+    def initialize(self, a_bzrdir, name=None, repository=None):
         """Create a branch of this format in a_bzrdir."""
         utf8_files = [('last-revision', '0 null:\n'),
                       ('branch.conf', ''),
                       ('tags', ''),
                       ('references', '')
                       ]
-        return self._initialize_helper(a_bzrdir, utf8_files, name)
+        return self._initialize_helper(a_bzrdir, utf8_files, name, repository)
 
     def __init__(self):
         super(BzrBranchFormat8, self).__init__()
@@ -2203,13 +2280,13 @@
     This format was introduced in bzr 1.6.
     """
 
-    def initialize(self, a_bzrdir, name=None):
+    def initialize(self, a_bzrdir, name=None, repository=None):
         """Create a branch of this format in a_bzrdir."""
         utf8_files = [('last-revision', '0 null:\n'),
                       ('branch.conf', ''),
                       ('tags', ''),
                       ]
-        return self._initialize_helper(a_bzrdir, utf8_files, name)
+        return self._initialize_helper(a_bzrdir, utf8_files, name, repository)
 
     def _branch_class(self):
         return BzrBranch7
@@ -2257,7 +2334,8 @@
         transport = a_bzrdir.get_branch_transport(None, name=name)
         location = transport.put_bytes('location', to_branch.base)
 
-    def initialize(self, a_bzrdir, name=None, target_branch=None):
+    def initialize(self, a_bzrdir, name=None, target_branch=None,
+            repository=None):
         """Create a branch of this format in a_bzrdir."""
         if target_branch is None:
             # this format does not implement branch itself, thus the implicit
@@ -2291,7 +2369,8 @@
         return clone
 
     def open(self, a_bzrdir, name=None, _found=False, location=None,
-             possible_transports=None, ignore_fallbacks=False):
+             possible_transports=None, ignore_fallbacks=False,
+             found_repository=None):
         """Return the branch that the branch reference in a_bzrdir points at.
 
         :param a_bzrdir: A BzrDir that contains a branch.
@@ -2349,10 +2428,7 @@
 BranchFormat.register_format(__format7)
 BranchFormat.register_format(__format8)
 BranchFormat.set_default_format(__format7)
-_legacy_formats = [BzrBranchFormat4(),
-    ]
-network_format_registry.register(
-    _legacy_formats[0].network_name(), _legacy_formats[0].__class__)
+BranchFormat.register_extra_format(BzrBranchFormat4())
 
 
 class BranchWriteLockResult(LogicalLockResult):
@@ -2634,7 +2710,7 @@
         result.target_branch = target
         result.old_revno, result.old_revid = target.last_revision_info()
         self.update_references(target)
-        if result.old_revid != self.last_revision():
+        if result.old_revid != stop_revision:
             # We assume that during 'push' this repository is closer than
             # the target.
             graph = self.repository.get_graph(target.repository)
@@ -2663,6 +2739,19 @@
                 mode=self.bzrdir._get_file_mode())
 
 
+class BzrBranchPreSplitOut(BzrBranch):
+
+    def _get_checkout_format(self):
+        """Return the most suitable metadir for a checkout of this branch.
+        Weaves are used if this branch's repository uses weaves.
+        """
+        from bzrlib.repofmt.weaverepo import RepositoryFormat7
+        from bzrlib.bzrdir import BzrDirMetaFormat1
+        format = BzrDirMetaFormat1()
+        format.repository_format = RepositoryFormat7()
+        return format
+
+
 class BzrBranch5(BzrBranch):
     """A format 5 branch. This supports new features over plain branches.
 
@@ -3102,8 +3191,12 @@
     :ivar tag_conflicts: A list of tag conflicts, see BasicTags.merge_to
     """
 
+    @deprecated_method(deprecated_in((2, 3, 0)))
     def __int__(self):
-        # DEPRECATED: pull used to return the change in revno
+        """Return the relative change in revno.
+
+        :deprecated: Use `new_revno` and `old_revno` instead.
+        """
         return self.new_revno - self.old_revno
 
     def report(self, to_file):
@@ -3134,8 +3227,12 @@
         target, otherwise it will be None.
     """
 
+    @deprecated_method(deprecated_in((2, 3, 0)))
     def __int__(self):
-        # DEPRECATED: push used to return the change in revno
+        """Return the relative change in revno.
+
+        :deprecated: Use `new_revno` and `old_revno` instead.
+        """
         return self.new_revno - self.old_revno
 
     def report(self, to_file):
@@ -3287,7 +3384,7 @@
 
     @needs_write_lock
     def update_revisions(self, stop_revision=None, overwrite=False,
-                         graph=None):
+                         graph=None, fetch_tags=True):
         """Pull in new perfect-fit revisions.
 
         :param stop_revision: Updated until the given revision
@@ -3295,6 +3392,8 @@
             to see if it is a proper descendant.
         :param graph: A Graph object that can be used to query history
             information. This can be None.
+        :param fetch_tags: Flag that specifies if tags from source should be
+            fetched too.
         :return: None
         """
         raise NotImplementedError(self.update_revisions)
@@ -3308,6 +3407,15 @@
         """
         raise NotImplementedError(self.push)
 
+    @needs_write_lock
+    def copy_content_into(self, revision_id=None):
+        """Copy the content of source into target
+
+        revision_id: if not None, the revision history in the new branch will
+                     be truncated to end with revision_id.
+        """
+        raise NotImplementedError(self.copy_content_into)
+
 
 class GenericInterBranch(InterBranch):
     """InterBranch implementation that uses public Branch functions."""
@@ -3326,7 +3434,7 @@
         if isinstance(format, remote.RemoteBranchFormat):
             format._ensure_real()
             return format._custom_format
-        return format                                                                                                  
+        return format
 
     @needs_write_lock
     def copy_content_into(self, revision_id=None):
@@ -3349,7 +3457,7 @@
 
     @needs_write_lock
     def update_revisions(self, stop_revision=None, overwrite=False,
-        graph=None):
+        graph=None, fetch_tags=True):
         """See InterBranch.update_revisions()."""
         other_revno, other_last_revision = self.source.last_revision_info()
         stop_revno = None # unknown
@@ -3367,7 +3475,18 @@
         # case of having something to pull, and so that the check for
         # already merged can operate on the just fetched graph, which will
         # be cached in memory.
-        self.target.fetch(self.source, stop_revision)
+        if fetch_tags:
+            fetch_spec_factory = fetch.FetchSpecFactory()
+            fetch_spec_factory.source_branch = self.source
+            fetch_spec_factory.source_branch_stop_revision_id = stop_revision
+            fetch_spec_factory.source_repo = self.source.repository
+            fetch_spec_factory.target_repo = self.target.repository
+            fetch_spec_factory.target_repo_kind = fetch.TargetRepoKinds.PREEXISTING
+            fetch_spec = fetch_spec_factory.make_fetch_spec()
+        else:
+            fetch_spec = _mod_graph.NotInOtherForRevs(self.target.repository,
+                self.source.repository, revision_ids=[stop_revision]).execute()
+        self.target.fetch(self.source, fetch_spec=fetch_spec)
         # Check to see if one is an ancestor of the other
         if not overwrite:
             if graph is None:
@@ -3401,7 +3520,8 @@
         if local and not bound_location:
             raise errors.LocalRequiresBoundBranch()
         master_branch = None
-        if not local and bound_location and self.source.user_url != bound_location:
+        source_is_master = (self.source.user_url == bound_location)
+        if not local and bound_location and not source_is_master:
             # not pulling from master, so we need to update master.
             master_branch = self.target.get_master_branch(possible_transports)
             master_branch.lock_write()
@@ -3413,7 +3533,8 @@
             return self._pull(overwrite,
                 stop_revision, _hook_master=master_branch,
                 run_hooks=run_hooks,
-                _override_hook_target=_override_hook_target)
+                _override_hook_target=_override_hook_target,
+                merge_tags_to_master=not source_is_master)
         finally:
             if master_branch:
                 master_branch.unlock()
@@ -3462,7 +3583,7 @@
                 # push into the master from the source branch.
                 self.source._basic_push(master_branch, overwrite, stop_revision)
                 # and push into the target branch from the source. Note that we
-                # push from the source branch again, because its considered the
+                # push from the source branch again, because it's considered the
                 # highest bandwidth repository.
                 result = self.source._basic_push(self.target, overwrite,
                     stop_revision)
@@ -3486,7 +3607,8 @@
 
     def _pull(self, overwrite=False, stop_revision=None,
              possible_transports=None, _hook_master=None, run_hooks=True,
-             _override_hook_target=None, local=False):
+             _override_hook_target=None, local=False,
+             merge_tags_to_master=True):
         """See Branch.pull.
 
         This function is the core worker, used by GenericInterBranch.pull to
@@ -3527,7 +3649,7 @@
             # so a tags implementation that versions tags can only 
             # pull in the most recent changes. -- JRV20090506
             result.tag_conflicts = self.source.tags.merge_to(self.target.tags,
-                overwrite)
+                overwrite, ignore_master=not merge_tags_to_master)
             result.new_revno, result.new_revid = self.target.last_revision_info()
             if _hook_master:
                 result.master_branch = _hook_master

=== modified file 'bzrlib/branchbuilder.py'
--- bzrlib/branchbuilder.py	2010-02-27 12:27:33 +0000
+++ bzrlib/branchbuilder.py	2011-02-09 08:08:20 +0000
@@ -21,6 +21,7 @@
     commit,
     errors,
     memorytree,
+    revision,
     )
 
 
@@ -186,7 +187,10 @@
         :return: The revision_id of the new commit
         """
         if parent_ids is not None:
-            base_id = parent_ids[0]
+            if len(parent_ids) == 0:
+                base_id = revision.NULL_REVISION
+            else:
+                base_id = parent_ids[0]
             if base_id != self._branch.last_revision():
                 self._move_branch_pointer(base_id,
                     allow_leftmost_as_ghost=allow_leftmost_as_ghost)

=== modified file 'bzrlib/btree_index.py'
--- bzrlib/btree_index.py	2010-06-20 11:18:38 +0000
+++ bzrlib/btree_index.py	2011-02-09 08:08:20 +0000
@@ -602,10 +602,10 @@
         """In memory index's have no known corruption at the moment."""
 
 
-class _LeafNode(object):
+class _LeafNode(dict):
     """A leaf node for a serialised B+Tree index."""
 
-    __slots__ = ('keys', 'min_key', 'max_key')
+    __slots__ = ('min_key', 'max_key', '_keys')
 
     def __init__(self, bytes, key_length, ref_list_length):
         """Parse bytes to create a leaf node object."""
@@ -617,7 +617,20 @@
             self.max_key = key_list[-1][0]
         else:
             self.min_key = self.max_key = None
-        self.keys = dict(key_list)
+        super(_LeafNode, self).__init__(key_list)
+        self._keys = dict(self)
+
+    def all_items(self):
+        """Return a sorted list of (key, (value, refs)) items"""
+        items = self.items()
+        items.sort()
+        return items
+
+    def all_keys(self):
+        """Return a sorted list of all keys."""
+        keys = self.keys()
+        keys.sort()
+        return keys
 
 
 class _InternalNode(object):
@@ -672,6 +685,7 @@
         self._recommended_pages = self._compute_recommended_pages()
         self._root_node = None
         self._base_offset = offset
+        self._leaf_factory = _LeafNode
         # Default max size is 100,000 leave values
         self._leaf_value_cache = None # lru_cache.LRUCache(100*1000)
         if unlimited_cache:
@@ -950,7 +964,7 @@
         """Cache directly from key => value, skipping the btree."""
         if self._leaf_value_cache is not None:
             for node in nodes.itervalues():
-                for key, value in node.keys.iteritems():
+                for key, value in node.all_items():
                     if key in self._leaf_value_cache:
                         # Don't add the rest of the keys, we've seen this node
                         # before.
@@ -980,10 +994,10 @@
         if self._row_offsets[-1] == 1:
             # There is only the root node, and we read that via key_count()
             if self.node_ref_lists:
-                for key, (value, refs) in sorted(self._root_node.keys.items()):
+                for key, (value, refs) in self._root_node.all_items():
                     yield (self, key, value, refs)
             else:
-                for key, (value, refs) in sorted(self._root_node.keys.items()):
+                for key, (value, refs) in self._root_node.all_items():
                     yield (self, key, value)
             return
         start_of_leaves = self._row_offsets[-2]
@@ -999,11 +1013,11 @@
         # for spilling index builds to disk.
         if self.node_ref_lists:
             for _, node in nodes:
-                for key, (value, refs) in sorted(node.keys.items()):
+                for key, (value, refs) in node.all_items():
                     yield (self, key, value, refs)
         else:
             for _, node in nodes:
-                for key, (value, refs) in sorted(node.keys.items()):
+                for key, (value, refs) in node.all_items():
                     yield (self, key, value)
 
     @staticmethod
@@ -1170,8 +1184,8 @@
                 continue
             node = nodes[node_index]
             for next_sub_key in sub_keys:
-                if next_sub_key in node.keys:
-                    value, refs = node.keys[next_sub_key]
+                if next_sub_key in node:
+                    value, refs = node[next_sub_key]
                     if self.node_ref_lists:
                         yield (self, next_sub_key, value, refs)
                     else:
@@ -1245,14 +1259,13 @@
             # sub_keys is all of the keys we are looking for that should exist
             # on this page, if they aren't here, then they won't be found
             node = nodes[node_index]
-            node_keys = node.keys
             parents_to_check = set()
             for next_sub_key in sub_keys:
-                if next_sub_key not in node_keys:
+                if next_sub_key not in node:
                     # This one is just not present in the index at all
                     missing_keys.add(next_sub_key)
                 else:
-                    value, refs = node_keys[next_sub_key]
+                    value, refs = node[next_sub_key]
                     parent_keys = refs[ref_list_num]
                     parent_map[next_sub_key] = parent_keys
                     parents_to_check.update(parent_keys)
@@ -1265,8 +1278,8 @@
             while parents_to_check:
                 next_parents_to_check = set()
                 for key in parents_to_check:
-                    if key in node_keys:
-                        value, refs = node_keys[key]
+                    if key in node:
+                        value, refs = node[key]
                         parent_keys = refs[ref_list_num]
                         parent_map[key] = parent_keys
                         next_parents_to_check.update(parent_keys)
@@ -1546,7 +1559,8 @@
                     continue
             bytes = zlib.decompress(data)
             if bytes.startswith(_LEAF_FLAG):
-                node = _LeafNode(bytes, self._key_length, self.node_ref_lists)
+                node = self._leaf_factory(bytes, self._key_length,
+                                          self.node_ref_lists)
             elif bytes.startswith(_INTERNAL_FLAG):
                 node = _InternalNode(bytes)
             else:
@@ -1571,8 +1585,11 @@
             pass
 
 
+_gcchk_factory = _LeafNode
+
 try:
     from bzrlib import _btree_serializer_pyx as _btree_serializer
+    _gcchk_factory = _btree_serializer._parse_into_chk
 except ImportError, e:
     osutils.failed_to_load_extension(e)
     from bzrlib import _btree_serializer_py as _btree_serializer

=== modified file 'bzrlib/bugtracker.py'
--- bzrlib/bugtracker.py	2010-04-30 11:03:59 +0000
+++ bzrlib/bugtracker.py	2011-02-09 08:08:20 +0000
@@ -93,7 +93,7 @@
 --fixes`` to mark bugs in that tracker as being fixed by that commit. For
 example::
 
-    bugzilla_squid_url = http://www.squid-cache.org/bugs
+    bugzilla_squid_url = http://bugs.squid-cache.org
 
 would allow ``bzr commit --fixes squid:1234`` to mark Squid's bug 1234 as
 fixed.
@@ -127,7 +127,13 @@
 
     bugtracker_cpan_url = http://rt.cpan.org/Public/Bug/Display.html?id={id}
 
-for CPAN's RT bug tracker.
+would allow ``bzr commit --fixes cpan:1234`` to mark bug 1234 in CPAN's
+RT bug tracker as fixed, or::
+
+    bugtracker_hudson_url = http://issues.hudson-ci.org/browse/{id}
+
+would allow ``bzr commit --fixes hudson:HUDSON-1234`` to mark bug HUDSON-1234
+in Hudson's JIRA bug tracker as fixed.
 """
 
 
@@ -228,14 +234,13 @@
     UniqueIntegerBugTracker('gnome', 'http://bugzilla.gnome.org/show_bug.cgi?id='))
 
 
-class URLParametrizedIntegerBugTracker(IntegerBugTracker):
+class URLParametrizedBugTracker(BugTracker):
     """A type of bug tracker that can be found on a variety of different sites,
     and thus needs to have the base URL configured.
 
     Looks for a config setting in the form '<type_name>_<abbreviation>_url'.
-    `type_name` is the name of the type of tracker (e.g. 'bugzilla' or 'trac')
-    and `abbreviation` is a short name for the particular instance (e.g.
-    'squid' or 'apache').
+    `type_name` is the name of the type of tracker and `abbreviation`
+    is a short name for the particular instance.
     """
 
     def get(self, abbreviation, branch):
@@ -256,6 +261,16 @@
         return urlutils.join(self._base_url, self._bug_area) + str(bug_id)
 
 
+class URLParametrizedIntegerBugTracker(IntegerBugTracker, URLParametrizedBugTracker):
+    """A type of bug tracker that can be found on a variety of different sites,
+    and thus needs to have the base URL configured, but only allows integer bug IDs.
+
+    Looks for a config setting in the form '<type_name>_<abbreviation>_url'.
+    `type_name` is the name of the type of tracker (e.g. 'bugzilla' or 'trac')
+    and `abbreviation` is a short name for the particular instance (e.g.
+    'squid' or 'apache').
+    """
+
 tracker_registry.register(
     'trac', URLParametrizedIntegerBugTracker('trac', 'ticket/'))
 
@@ -264,7 +279,7 @@
     URLParametrizedIntegerBugTracker('bugzilla', 'show_bug.cgi?id='))
 
 
-class GenericBugTracker(URLParametrizedIntegerBugTracker):
+class GenericBugTracker(URLParametrizedBugTracker):
     """Generic bug tracker specified by an URL template."""
 
     def __init__(self):

=== modified file 'bzrlib/builtins.py'
--- bzrlib/builtins.py	2010-07-28 07:05:19 +0000
+++ bzrlib/builtins.py	2011-02-09 08:08:20 +0000
@@ -1,4 +1,4 @@
-# Copyright (C) 2005-2010 Canonical Ltd
+# Copyright (C) 2005-2011 Canonical Ltd
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -20,7 +20,6 @@
 
 from bzrlib.lazy_import import lazy_import
 lazy_import(globals(), """
-import codecs
 import cStringIO
 import sys
 import time
@@ -33,7 +32,7 @@
     bzrdir,
     directory_service,
     delta,
-    config,
+    config as _mod_config,
     errors,
     globbing,
     hooks,
@@ -75,14 +74,11 @@
 from bzrlib.trace import mutter, note, warning, is_quiet, get_verbosity_level
 
 
+@symbol_versioning.deprecated_function(symbol_versioning.deprecated_in((2, 3, 0)))
 def tree_files(file_list, default_branch=u'.', canonicalize=True,
     apply_view=True):
-    try:
-        return internal_tree_files(file_list, default_branch, canonicalize,
-            apply_view)
-    except errors.FileInWrongBranch, e:
-        raise errors.BzrCommandError("%s is not in the same branch as %s" %
-                                     (e.path, file_list[0]))
+    return internal_tree_files(file_list, default_branch, canonicalize,
+        apply_view)
 
 
 def tree_files_for_add(file_list):
@@ -152,10 +148,13 @@
 
 # XXX: Bad function name; should possibly also be a class method of
 # WorkingTree rather than a function.
+@symbol_versioning.deprecated_function(symbol_versioning.deprecated_in((2, 3, 0)))
 def internal_tree_files(file_list, default_branch=u'.', canonicalize=True,
     apply_view=True):
     """Convert command-line paths to a WorkingTree and relative paths.
 
+    Deprecated: use WorkingTree.open_containing_paths instead.
+
     This is typically used for command-line processors that take one or
     more filenames, and infer the workingtree that contains them.
 
@@ -171,53 +170,10 @@
 
     :return: workingtree, [relative_paths]
     """
-    if file_list is None or len(file_list) == 0:
-        tree = WorkingTree.open_containing(default_branch)[0]
-        if tree.supports_views() and apply_view:
-            view_files = tree.views.lookup_view()
-            if view_files:
-                file_list = view_files
-                view_str = views.view_display_str(view_files)
-                note("Ignoring files outside view. View is %s" % view_str)
-        return tree, file_list
-    tree = WorkingTree.open_containing(file_list[0])[0]
-    return tree, safe_relpath_files(tree, file_list, canonicalize,
-        apply_view=apply_view)
-
-
-def safe_relpath_files(tree, file_list, canonicalize=True, apply_view=True):
-    """Convert file_list into a list of relpaths in tree.
-
-    :param tree: A tree to operate on.
-    :param file_list: A list of user provided paths or None.
-    :param apply_view: if True and a view is set, apply it or check that
-        specified files are within it
-    :return: A list of relative paths.
-    :raises errors.PathNotChild: When a provided path is in a different tree
-        than tree.
-    """
-    if file_list is None:
-        return None
-    if tree.supports_views() and apply_view:
-        view_files = tree.views.lookup_view()
-    else:
-        view_files = []
-    new_list = []
-    # tree.relpath exists as a "thunk" to osutils, but canonical_relpath
-    # doesn't - fix that up here before we enter the loop.
-    if canonicalize:
-        fixer = lambda p: osutils.canonical_relpath(tree.basedir, p)
-    else:
-        fixer = tree.relpath
-    for filename in file_list:
-        try:
-            relpath = fixer(osutils.dereference_path(filename))
-            if  view_files and not osutils.is_inside_any(view_files, relpath):
-                raise errors.FileOutsideView(filename, view_files)
-            new_list.append(relpath)
-        except errors.PathNotChild:
-            raise errors.FileInWrongBranch(tree.branch, filename)
-    return new_list
+    return WorkingTree.open_containing_paths(
+        file_list, default_directory='.',
+        canonicalize=True,
+        apply_view=True)
 
 
 def _get_view_info_for_change_reporter(tree):
@@ -294,8 +250,12 @@
     To skip the display of pending merge information altogether, use
     the no-pending option or specify a file/directory.
 
-    If a revision argument is given, the status is calculated against
-    that revision, or between two revisions if two are provided.
+    To compare the working directory to a specific revision, pass a
+    single revision to the revision argument.
+
+    To see which files have changed in a specific revision, or between
+    two revisions, pass a revision range to the revision argument.
+    This will produce the same results as calling 'bzr diff --summarize'.
     """
 
     # TODO: --no-recurse, --recurse options
@@ -323,7 +283,7 @@
             raise errors.BzrCommandError('bzr status --revision takes exactly'
                                          ' one or two revision specifiers')
 
-        tree, relfile_list = tree_files(file_list)
+        tree, relfile_list = WorkingTree.open_containing_paths(file_list)
         # Avoid asking for specific files when that is not needed.
         if relfile_list == ['']:
             relfile_list = None
@@ -368,7 +328,8 @@
         if revision_id is None and revision is None:
             raise errors.BzrCommandError('You must supply either'
                                          ' --revision or a revision_id')
-        b = WorkingTree.open_containing(directory)[0].branch
+
+        b = bzrdir.BzrDir.open_containing_tree_or_branch(directory)[1]
 
         revisions = b.repository.revisions
         if revisions is None:
@@ -521,6 +482,59 @@
             d.destroy_workingtree()
 
 
+class cmd_repair_workingtree(Command):
+    __doc__ = """Reset the working tree state file.
+
+    This is not meant to be used normally, but more as a way to recover from
+    filesystem corruption, etc. This rebuilds the working inventory back to a
+    'known good' state. Any new modifications (adding a file, renaming, etc)
+    will be lost, though modified files will still be detected as such.
+
+    Most users will want something more like "bzr revert" or "bzr update"
+    unless the state file has become corrupted.
+
+    By default this attempts to recover the current state by looking at the
+    headers of the state file. If the state file is too corrupted to even do
+    that, you can supply --revision to force the state of the tree.
+    """
+
+    takes_options = ['revision', 'directory',
+        Option('force',
+               help='Reset the tree even if it doesn\'t appear to be'
+                    ' corrupted.'),
+    ]
+    hidden = True
+
+    def run(self, revision=None, directory='.', force=False):
+        tree, _ = WorkingTree.open_containing(directory)
+        self.add_cleanup(tree.lock_tree_write().unlock)
+        if not force:
+            try:
+                tree.check_state()
+            except errors.BzrError:
+                pass # There seems to be a real error here, so we'll reset
+            else:
+                # Refuse
+                raise errors.BzrCommandError(
+                    'The tree does not appear to be corrupt. You probably'
+                    ' want "bzr revert" instead. Use "--force" if you are'
+                    ' sure you want to reset the working tree.')
+        if revision is None:
+            revision_ids = None
+        else:
+            revision_ids = [r.as_revision_id(tree.branch) for r in revision]
+        try:
+            tree.reset_state(revision_ids)
+        except errors.BzrError, e:
+            if revision_ids is None:
+                extra = (', the header appears corrupt, try passing -r -1'
+                         ' to set the state to the last commit')
+            else:
+                extra = ''
+            raise errors.BzrCommandError('failed to reset the tree state'
+                                         + extra)
+
+
 class cmd_revno(Command):
     __doc__ = """Show current revision number.
 
@@ -761,7 +775,7 @@
             raise errors.BzrCommandError('invalid kind %r specified' % (kind,))
 
         revision = _get_one_revision('inventory', revision)
-        work_tree, file_list = tree_files(file_list)
+        work_tree, file_list = WorkingTree.open_containing_paths(file_list)
         self.add_cleanup(work_tree.lock_read().unlock)
         if revision is not None:
             tree = revision.as_tree(work_tree.branch)
@@ -832,7 +846,7 @@
             names_list = []
         if len(names_list) < 2:
             raise errors.BzrCommandError("missing file argument")
-        tree, rel_names = tree_files(names_list, canonicalize=False)
+        tree, rel_names = WorkingTree.open_containing_paths(names_list, canonicalize=False)
         self.add_cleanup(tree.lock_tree_write().unlock)
         self._run(tree, names_list, rel_names, after)
 
@@ -843,7 +857,8 @@
         if after:
             raise errors.BzrCommandError('--after cannot be specified with'
                                          ' --auto.')
-        work_tree, file_list = tree_files(names_list, default_branch='.')
+        work_tree, file_list = WorkingTree.open_containing_paths(
+            names_list, default_directory='.')
         self.add_cleanup(work_tree.lock_tree_write().unlock)
         rename_map.RenameMap.guess_renames(work_tree, dry_run)
 
@@ -966,13 +981,16 @@
                  "branch.  Local pulls are not applied to "
                  "the master branch."
             ),
+        Option('show-base',
+            help="Show base revision text in conflicts.")
         ]
     takes_args = ['location?']
     encoding_type = 'replace'
 
     def run(self, location=None, remember=False, overwrite=False,
             revision=None, verbose=False,
-            directory=None, local=False):
+            directory=None, local=False,
+            show_base=False):
         # FIXME: too much stuff is in the command class
         revision_id = None
         mergeable = None
@@ -987,6 +1005,9 @@
             branch_to = Branch.open_containing(directory)[0]
             self.add_cleanup(branch_to.lock_write().unlock)
 
+        if tree_to is None and show_base:
+            raise errors.BzrCommandError("Need working tree for --show-base.")
+
         if local and not branch_to.get_bound_location():
             raise errors.LocalRequiresBoundBranch()
 
@@ -1037,7 +1058,8 @@
                 view_info=view_info)
             result = tree_to.pull(
                 branch_from, overwrite, revision_id, change_reporter,
-                possible_transports=possible_transports, local=local)
+                possible_transports=possible_transports, local=local,
+                show_base=show_base)
         else:
             result = branch_to.pull(
                 branch_from, overwrite, revision_id, local=local)
@@ -1047,6 +1069,10 @@
             log.show_branch_change(
                 branch_to, self.outf, result.old_revno,
                 result.old_revid)
+        if getattr(result, 'tag_conflicts', None):
+            return 1
+        else:
+            return 0
 
 
 class cmd_push(Command):
@@ -1099,6 +1125,9 @@
         Option('strict',
                help='Refuse to push if there are uncommitted changes in'
                ' the working tree, --no-strict disables the check.'),
+        Option('no-tree',
+               help="Don't populate the working tree, even for protocols"
+               " that support it."),
         ]
     takes_args = ['location?']
     encoding_type = 'replace'
@@ -1106,7 +1135,7 @@
     def run(self, location=None, remember=False, overwrite=False,
         create_prefix=False, verbose=False, revision=None,
         use_existing_dir=False, directory=None, stacked_on=None,
-        stacked=False, strict=None):
+        stacked=False, strict=None, no_tree=False):
         from bzrlib.push import _show_push_branch
 
         if directory is None:
@@ -1158,7 +1187,7 @@
         _show_push_branch(br_from, revision_id, location, self.outf,
             verbose=verbose, overwrite=overwrite, remember=remember,
             stacked_on=stacked_on, create_prefix=create_prefix,
-            use_existing_dir=use_existing_dir)
+            use_existing_dir=use_existing_dir, no_tree=no_tree)
 
 
 class cmd_branch(Command):
@@ -1177,8 +1206,10 @@
 
     _see_also = ['checkout']
     takes_args = ['from_location', 'to_location?']
-    takes_options = ['revision', Option('hardlink',
-        help='Hard-link working tree files where possible.'),
+    takes_options = ['revision',
+        Option('hardlink', help='Hard-link working tree files where possible.'),
+        Option('files-from', type=str,
+               help="Get file contents from this tree."),
         Option('no-tree',
             help="Create a branch without a working-tree."),
         Option('switch',
@@ -1202,11 +1233,19 @@
 
     def run(self, from_location, to_location=None, revision=None,
             hardlink=False, stacked=False, standalone=False, no_tree=False,
-            use_existing_dir=False, switch=False, bind=False):
+            use_existing_dir=False, switch=False, bind=False,
+            files_from=None):
         from bzrlib import switch as _mod_switch
         from bzrlib.tag import _merge_tags_if_possible
         accelerator_tree, br_from = bzrdir.BzrDir.open_tree_or_branch(
             from_location)
+        if not (hardlink or files_from):
+            # accelerator_tree is usually slower because you have to read N
+            # files (no readahead, lots of seeks, etc), but allow the user to
+            # explicitly request it
+            accelerator_tree = None
+        if files_from is not None and files_from != from_location:
+            accelerator_tree = WorkingTree.open(files_from)
         revision = _get_one_revision('branch', revision)
         self.add_cleanup(br_from.lock_read().unlock)
         if revision is not None:
@@ -1319,8 +1358,13 @@
             to_location = branch_location
         accelerator_tree, source = bzrdir.BzrDir.open_tree_or_branch(
             branch_location)
+        if not (hardlink or files_from):
+            # accelerator_tree is usually slower because you have to read N
+            # files (no readahead, lots of seeks, etc), but allow the user to
+            # explicitly request it
+            accelerator_tree = None
         revision = _get_one_revision('checkout', revision)
-        if files_from is not None:
+        if files_from is not None and files_from != branch_location:
             accelerator_tree = WorkingTree.open(files_from)
         if revision is not None:
             revision_id = revision.as_revision_id(source)
@@ -1382,16 +1426,22 @@
     If you want to discard your local changes, you can just do a
     'bzr revert' instead of 'bzr commit' after the update.
 
+    If you want to restore a file that has been removed locally, use
+    'bzr revert' instead of 'bzr update'.
+
     If the tree's branch is bound to a master branch, it will also update
     the branch from the master.
     """
 
     _see_also = ['pull', 'working-trees', 'status-flags']
     takes_args = ['dir?']
-    takes_options = ['revision']
+    takes_options = ['revision',
+                     Option('show-base',
+                            help="Show base revision text in conflicts."),
+                     ]
     aliases = ['up']
 
-    def run(self, dir='.', revision=None):
+    def run(self, dir='.', revision=None, show_base=None):
         if revision is not None and len(revision) != 1:
             raise errors.BzrCommandError(
                         "bzr update --revision takes exactly one revision")
@@ -1437,7 +1487,8 @@
                 change_reporter,
                 possible_transports=possible_transports,
                 revision=revision_id,
-                old_tip=old_tip)
+                old_tip=old_tip,
+                show_base=show_base)
         except errors.NoSuchRevision, e:
             raise errors.BzrCommandError(
                                   "branch has no revision %s\n"
@@ -1505,10 +1556,11 @@
 class cmd_remove(Command):
     __doc__ = """Remove files or directories.
 
-    This makes bzr stop tracking changes to the specified files. bzr will delete
-    them if they can easily be recovered using revert. If no options or
-    parameters are given bzr will scan for files that are being tracked by bzr
-    but missing in your tree and stop tracking them for you.
+    This makes Bazaar stop tracking changes to the specified files. Bazaar will
+    delete them if they can easily be recovered using revert otherwise they
+    will be backed up (adding an extention of the form .~#~). If no options or
+    parameters are given Bazaar will scan for files that are being tracked by
+    Bazaar but missing in your tree and stop tracking them for you.
     """
     takes_args = ['file*']
     takes_options = ['verbose',
@@ -1516,17 +1568,23 @@
         RegistryOption.from_kwargs('file-deletion-strategy',
             'The file deletion mode to be used.',
             title='Deletion Strategy', value_switches=True, enum_switch=False,
-            safe='Only delete files if they can be'
-                 ' safely recovered (default).',
+            safe='Backup changed files (default).',
             keep='Delete from bzr but leave the working copy.',
+            no_backup='Don\'t backup changed files.',
             force='Delete all the specified files, even if they can not be '
-                'recovered and even if they are non-empty directories.')]
+                'recovered and even if they are non-empty directories. '
+                '(deprecated, use no-backup)')]
     aliases = ['rm', 'del']
     encoding_type = 'replace'
 
     def run(self, file_list, verbose=False, new=False,
         file_deletion_strategy='safe'):
-        tree, file_list = tree_files(file_list)
+        if file_deletion_strategy == 'force':
+            note("(The --force option is deprecated, rather use --no-backup "
+                "in future.)")
+            file_deletion_strategy = 'no-backup'
+
+        tree, file_list = WorkingTree.open_containing_paths(file_list)
 
         if file_list is not None:
             file_list = [f for f in file_list]
@@ -1552,7 +1610,7 @@
             file_deletion_strategy = 'keep'
         tree.remove(file_list, verbose=verbose, to_file=self.outf,
             keep_files=file_deletion_strategy=='keep',
-            force=file_deletion_strategy=='force')
+            force=(file_deletion_strategy=='no-backup'))
 
 
 class cmd_file_id(Command):
@@ -1620,11 +1678,17 @@
 
     _see_also = ['check']
     takes_args = ['branch?']
+    takes_options = [
+        Option('canonicalize-chks',
+               help='Make sure CHKs are in canonical form (repairs '
+                    'bug 522637).',
+               hidden=True),
+        ]
 
-    def run(self, branch="."):
+    def run(self, branch=".", canonicalize_chks=False):
         from bzrlib.reconcile import reconcile
         dir = bzrdir.BzrDir.open(branch)
-        reconcile(dir)
+        reconcile(dir, canonicalize_chks=canonicalize_chks)
 
 
 class cmd_revision_history(Command):
@@ -1707,10 +1771,12 @@
                 ),
          Option('append-revisions-only',
                 help='Never change revnos or the existing log.'
-                '  Append revisions to it only.')
+                '  Append revisions to it only.'),
+         Option('no-tree',
+                'Create a branch without a working tree.')
          ]
     def run(self, location=None, format=None, append_revisions_only=False,
-            create_prefix=False):
+            create_prefix=False, no_tree=False):
         if format is None:
             format = bzrdir.format_registry.make_bzrdir('default')
         if location is None:
@@ -1739,8 +1805,13 @@
         except errors.NotBranchError:
             # really a NotBzrDir error...
             create_branch = bzrdir.BzrDir.create_branch_convenience
+            if no_tree:
+                force_new_tree = False
+            else:
+                force_new_tree = None
             branch = create_branch(to_transport.base, format=format,
-                                   possible_transports=[to_transport])
+                                   possible_transports=[to_transport],
+                                   force_new_tree=force_new_tree)
             a_bzrdir = branch.bzrdir
         else:
             from bzrlib.transport.local import LocalTransport
@@ -1750,7 +1821,8 @@
                         raise errors.BranchExistsWithoutWorkingTree(location)
                 raise errors.AlreadyBranchError(location)
             branch = a_bzrdir.create_branch()
-            a_bzrdir.create_workingtree()
+            if not no_tree:
+                a_bzrdir.create_workingtree()
         if append_revisions_only:
             try:
                 branch.set_append_revisions_only(True)
@@ -1850,6 +1922,13 @@
     "bzr diff -p1" is equivalent to "bzr diff --prefix old/:new/", and
     produces patches suitable for "patch -p1".
 
+    Note that when using the -r argument with a range of revisions, the
+    differences are computed between the two specified revisions.  That
+    is, the command does not show the changes introduced by the first 
+    revision in the range.  This differs from the interpretation of 
+    revision ranges used by "bzr log" which includes the first revision
+    in the range.
+
     :Exit values:
         1 - changed
         2 - unrepresentable changes
@@ -1873,7 +1952,11 @@
 
             bzr diff -r1..3 xxx
 
-        To see the changes introduced in revision X::
+        The changes introduced by revision 2 (equivalent to -r1..2)::
+
+            bzr diff -c2
+
+        To see the changes introduced by revision X::
         
             bzr diff -cX
 
@@ -1883,9 +1966,10 @@
 
             bzr diff -r<chosen_parent>..X
 
-        The changes introduced by revision 2 (equivalent to -r1..2)::
+        The changes between the current revision and the previous revision
+        (equivalent to -c-1 and -r-2..-1)
 
-            bzr diff -c2
+            bzr diff -r-2..
 
         Show just the differences for file NEWS::
 
@@ -1906,6 +1990,10 @@
         Same as 'bzr diff' but prefix paths with old/ and new/::
 
             bzr diff --prefix old/:new/
+            
+        Show the differences using a custom diff program with options::
+        
+            bzr diff --using /usr/bin/diff --diff-options -wu
     """
     _see_also = ['status']
     takes_args = ['file*']
@@ -1931,9 +2019,10 @@
             type=unicode,
             ),
         RegistryOption('format',
+            short_name='F',
             help='Diff format to use.',
             lazy_registry=('bzrlib.diff', 'format_registry'),
-            value_switches=False, title='Diff format'),
+            title='Diff format'),
         ]
     aliases = ['di', 'dif']
     encoding_type = 'exact'
@@ -2020,7 +2109,9 @@
     @display_command
     def run(self, null=False, directory=u'.'):
         tree = WorkingTree.open_containing(directory)[0]
+        self.add_cleanup(tree.lock_read().unlock)
         td = tree.changes_from(tree.basis_tree())
+        self.cleanup_now()
         for path, id, kind, text_modified, meta_modified in td.modified:
             if null:
                 self.outf.write(path + '\0')
@@ -2656,8 +2747,13 @@
     Patterns prefixed with '!!' act as regular ignore patterns, but have
     precedence over the '!' exception patterns.
 
-    Note: ignore patterns containing shell wildcards must be quoted from
-    the shell on Unix.
+    :Notes: 
+        
+    * Ignore patterns containing shell wildcards must be quoted from
+      the shell on Unix.
+
+    * Ignore patterns starting with "#" act as comments in the ignore file.
+      To ignore patterns that begin with that character, use the "RE:" prefix.
 
     :Examples:
         Ignore the top level Makefile::
@@ -2672,6 +2768,10 @@
 
             bzr ignore "!special.class"
 
+        Ignore files whose name begins with the "#" character::
+
+            bzr ignore "RE:^#"
+
         Ignore .o files under the lib directory::
 
             bzr ignore "lib/**/*.o"
@@ -3117,7 +3217,7 @@
 
         properties = {}
 
-        tree, selected_list = tree_files(selected_list)
+        tree, selected_list = WorkingTree.open_containing_paths(selected_list)
         if selected_list == ['']:
             # workaround - commit of root of tree should be exactly the same
             # as just default commit in that tree, and succeed even though
@@ -3158,9 +3258,9 @@
         def get_message(commit_obj):
             """Callback to get commit message"""
             if file:
-                f = codecs.open(file, 'rt', osutils.get_user_encoding())
+                f = open(file)
                 try:
-                    my_message = f.read()
+                    my_message = f.read().decode(osutils.get_user_encoding())
                 finally:
                     f.close()
             elif message is not None:
@@ -3197,7 +3297,7 @@
                         reporter=None, verbose=verbose, revprops=properties,
                         authors=author, timestamp=commit_stamp,
                         timezone=offset,
-                        exclude=safe_relpath_files(tree, exclude))
+                        exclude=tree.safe_relpath_files(exclude))
         except PointlessCommit:
             raise errors.BzrCommandError("No changes to commit."
                               " Use --unchanged to commit anyhow.")
@@ -3287,27 +3387,62 @@
 
 
 class cmd_upgrade(Command):
-    __doc__ = """Upgrade branch storage to current format.
-
-    The check command or bzr developers may sometimes advise you to run
-    this command. When the default format has changed you may also be warned
-    during other operations to upgrade.
+    __doc__ = """Upgrade a repository, branch or working tree to a newer format.
+
+    When the default format has changed after a major new release of
+    Bazaar, you may be informed during certain operations that you
+    should upgrade. Upgrading to a newer format may improve performance
+    or make new features available. It may however limit interoperability
+    with older repositories or with older versions of Bazaar.
+
+    If you wish to upgrade to a particular format rather than the
+    current default, that can be specified using the --format option.
+    As a consequence, you can use the upgrade command this way to
+    "downgrade" to an earlier format, though some conversions are
+    a one way process (e.g. changing from the 1.x default to the
+    2.x default) so downgrading is not always possible.
+
+    A backup.bzr.~#~ directory is created at the start of the conversion
+    process (where # is a number). By default, this is left there on
+    completion. If the conversion fails, delete the new .bzr directory
+    and rename this one back in its place. Use the --clean option to ask
+    for the backup.bzr directory to be removed on successful conversion.
+    Alternatively, you can delete it by hand if everything looks good
+    afterwards.
+
+    If the location given is a shared repository, dependent branches
+    are also converted provided the repository converts successfully.
+    If the conversion of a branch fails, remaining branches are still
+    tried.
+
+    For more information on upgrades, see the Bazaar Upgrade Guide,
+    http://doc.bazaar.canonical.com/latest/en/upgrade-guide/.
     """
 
-    _see_also = ['check']
+    _see_also = ['check', 'reconcile', 'formats']
     takes_args = ['url?']
     takes_options = [
-                    RegistryOption('format',
-                        help='Upgrade to a specific format.  See "bzr help'
-                             ' formats" for details.',
-                        lazy_registry=('bzrlib.bzrdir', 'format_registry'),
-                        converter=lambda name: bzrdir.format_registry.make_bzrdir(name),
-                        value_switches=True, title='Branch format'),
-                    ]
+        RegistryOption('format',
+            help='Upgrade to a specific format.  See "bzr help'
+                 ' formats" for details.',
+            lazy_registry=('bzrlib.bzrdir', 'format_registry'),
+            converter=lambda name: bzrdir.format_registry.make_bzrdir(name),
+            value_switches=True, title='Branch format'),
+        Option('clean',
+            help='Remove the backup.bzr directory if successful.'),
+        Option('dry-run',
+            help="Show what would be done, but don't actually do anything."),
+    ]
 
-    def run(self, url='.', format=None):
+    def run(self, url='.', format=None, clean=False, dry_run=False):
         from bzrlib.upgrade import upgrade
-        upgrade(url, format)
+        exceptions = upgrade(url, format, clean_up=clean, dry_run=dry_run)
+        if exceptions:
+            if len(exceptions) == 1:
+                # Compatibility with historical behavior
+                raise exceptions[0]
+            else:
+                return 3
 
 
 class cmd_whoami(Command):
@@ -3340,7 +3475,7 @@
                 try:
                     c = Branch.open_containing(u'.')[0].get_config()
                 except errors.NotBranchError:
-                    c = config.GlobalConfig()
+                    c = _mod_config.GlobalConfig()
             else:
                 c = Branch.open(directory).get_config()
             if email:
@@ -3349,9 +3484,13 @@
                 self.outf.write(c.username() + '\n')
             return
 
+        if email:
+            raise errors.BzrCommandError("--email can only be used to display existing "
+                                         "identity")
+
         # display a warning if an email address isn't included in the given name.
         try:
-            config.extract_email_address(name)
+            _mod_config.extract_email_address(name)
         except errors.NoEmailInUsername, e:
             warning('"%s" does not seem to contain an email address.  '
                     'This is allowed, but not recommended.', name)
@@ -3363,7 +3502,7 @@
             else:
                 c = Branch.open(directory).get_config()
         else:
-            c = config.GlobalConfig()
+            c = _mod_config.GlobalConfig()
         c.set_user_option('email', name)
 
 
@@ -3436,13 +3575,13 @@
                 'bzr alias --remove expects an alias to remove.')
         # If alias is not found, print something like:
         # unalias: foo: not found
-        c = config.GlobalConfig()
+        c = _mod_config.GlobalConfig()
         c.unset_alias(alias_name)
 
     @display_command
     def print_aliases(self):
         """Print out the defined aliases in a similar format to bash."""
-        aliases = config.GlobalConfig().get_aliases()
+        aliases = _mod_config.GlobalConfig().get_aliases()
         for key, value in sorted(aliases.iteritems()):
             self.outf.write('bzr alias %s="%s"\n' % (key, value))
 
@@ -3458,7 +3597,7 @@
 
     def set_alias(self, alias_name, alias_command):
         """Save the alias in the global config."""
-        c = config.GlobalConfig()
+        c = _mod_config.GlobalConfig()
         c.set_alias(alias_name, alias_command)
 
 
@@ -3499,6 +3638,9 @@
     If you set BZR_TEST_PDB=1 when running selftest, failing tests will drop
     into a pdb postmortem session.
 
+    The --coverage=DIRNAME global option produces a report with covered code
+    indicated.
+
     :Examples:
         Run only tests relating to 'ignore'::
 
@@ -3588,10 +3730,7 @@
             randomize=None, exclude=None, strict=False,
             load_list=None, debugflag=None, starting_with=None, subunit=False,
             parallel=None, lsprof_tests=False):
-        from bzrlib.tests import selftest
-
-        # Make deprecation warnings visible, unless -Werror is set
-        symbol_versioning.activate_deprecation_warnings(override=False)
+        from bzrlib import tests
 
         if testspecs_list is not None:
             pattern = '|'.join(testspecs_list)
@@ -3638,7 +3777,14 @@
                           "starting_with": starting_with
                           }
         selftest_kwargs.update(self.additional_selftest_args)
-        result = selftest(**selftest_kwargs)
+
+        # Make deprecation warnings visible, unless -Werror is set
+        cleanup = symbol_versioning.activate_deprecation_warnings(
+            override=False)
+        try:
+            result = tests.selftest(**selftest_kwargs)
+        finally:
+            cleanup()
         return int(not result)
 
 
@@ -3701,16 +3847,20 @@
     with bzr send. If neither is specified, the default is the upstream branch
     or the branch most recently merged using --remember.
 
-    When merging a branch, by default the tip will be merged. To pick a different
-    revision, pass --revision. If you specify two values, the first will be used as
-    BASE and the second one as OTHER. Merging individual revisions, or a subset of
-    available revisions, like this is commonly referred to as "cherrypicking".
-
-    Revision numbers are always relative to the branch being merged.
-
-    By default, bzr will try to merge in all new work from the other
-    branch, automatically determining an appropriate base.  If this
-    fails, you may need to give an explicit base.
+    When merging from a branch, by default bzr will try to merge in all new
+    work from the other branch, automatically determining an appropriate base
+    revision.  If this fails, you may need to give an explicit base.
+
+    To pick a different ending revision, pass "--revision OTHER".  bzr will
+    try to merge in all new work up to and including revision OTHER.
+
+    If you specify two values, "--revision BASE..OTHER", only revisions BASE
+    through OTHER, excluding BASE but including OTHER, will be merged.  If this
+    causes some revisions to be skipped, i.e. if the destination branch does
+    not already contain revision BASE, such a merge is commonly referred to as
+    a "cherrypick".
+
+    Revision numbers are always relative to the source branch.
 
     Merge will do its best to combine the changes in two branches, but there
     are some kinds of problems only a human can fix.  When it encounters those,
@@ -3740,7 +3890,7 @@
     you to apply each diff hunk and file change, similar to "shelve".
 
     :Examples:
-        To merge the latest revision from bzr.dev::
+        To merge all new revisions from bzr.dev::
 
             bzr merge ../bzr.dev
 
@@ -3980,7 +4130,9 @@
         if ((remember or tree.branch.get_submit_branch() is None) and
              user_location is not None):
             tree.branch.set_submit_branch(other_branch.base)
-        _merge_tags_if_possible(other_branch, tree.branch)
+        # Merge tags (but don't set them in the master branch yet, the user
+        # might revert this merge).  Commit will propagate them.
+        _merge_tags_if_possible(other_branch, tree.branch, ignore_master=True)
         merger = _mod_merge.Merger.from_revision_ids(pb, tree,
             other_revision_id, base_revision_id, other_branch, base_branch)
         if other_path != '':
@@ -4087,7 +4239,7 @@
         from bzrlib.conflicts import restore
         if merge_type is None:
             merge_type = _mod_merge.Merge3Merger
-        tree, file_list = tree_files(file_list)
+        tree, file_list = WorkingTree.open_containing_paths(file_list)
         self.add_cleanup(tree.lock_write().unlock)
         parents = tree.get_parent_ids()
         if len(parents) != 2:
@@ -4154,9 +4306,10 @@
     last committed revision is used.
 
     To remove only some changes, without reverting to a prior version, use
-    merge instead.  For example, "merge . --revision -2..-3" will remove the
-    changes introduced by -2, without affecting the changes introduced by -1.
-    Or to remove certain changes on a hunk-by-hunk basis, see the Shelf plugin.
+    merge instead.  For example, "merge . -r -2..-3" (don't forget the ".")
+    will remove the changes introduced by the second last commit (-2), without
+    affecting the changes introduced by the last commit (-1).  To remove
+    certain changes on a hunk-by-hunk basis, see the shelve command.
 
     By default, any files that have been manually changed will be backed up
     first.  (Files changed only by merge are not backed up.)  Backup files have
@@ -4192,7 +4345,7 @@
     target branches.
     """
 
-    _see_also = ['cat', 'export']
+    _see_also = ['cat', 'export', 'merge', 'shelve']
     takes_options = [
         'revision',
         Option('no-backup', "Do not save backups of reverted files."),
@@ -4203,7 +4356,7 @@
 
     def run(self, revision=None, no_backup=False, file_list=None,
             forget_merges=None):
-        tree, file_list = tree_files(file_list)
+        tree, file_list = WorkingTree.open_containing_paths(file_list)
         self.add_cleanup(tree.lock_tree_write().unlock)
         if forget_merges:
             tree.set_parent_ids(tree.get_parent_ids()[:1])
@@ -4491,26 +4644,9 @@
 
     @display_command
     def run(self, verbose=False):
-        import bzrlib.plugin
-        from inspect import getdoc
-        result = []
-        for name, plugin in bzrlib.plugin.plugins().items():
-            version = plugin.__version__
-            if version == 'unknown':
-                version = ''
-            name_ver = '%s %s' % (name, version)
-            d = getdoc(plugin.module)
-            if d:
-                doc = d.split('\n')[0]
-            else:
-                doc = '(no description)'
-            result.append((name_ver, doc, plugin.path()))
-        for name_ver, doc, path in sorted(result):
-            self.outf.write("%s\n" % name_ver)
-            self.outf.write("   %s\n" % doc)
-            if verbose:
-                self.outf.write("   %s\n" % path)
-            self.outf.write("\n")
+        from bzrlib import plugin
+        self.outf.writelines(
+            plugin.describe_plugins(show_paths=verbose))
 
 
 class cmd_testament(Command):
@@ -4812,8 +4948,11 @@
             self.outf.write('The above revision(s) will be removed.\n')
 
         if not force:
-            if not ui.ui_factory.get_boolean('Are you sure'):
-                self.outf.write('Canceled')
+            if not ui.ui_factory.confirm_action(
+                    'Uncommit these revisions',
+                    'bzrlib.builtins.uncommit',
+                    {}):
+                self.outf.write('Canceled\n')
                 return 0
 
         mutter('Uncommitting from {%s} to {%s}',
@@ -4825,7 +4964,10 @@
 
 
 class cmd_break_lock(Command):
-    __doc__ = """Break a dead lock on a repository, branch or working directory.
+    __doc__ = """Break a dead lock.
+
+    This command breaks a lock on a repository, branch, working directory or
+    config file.
 
     CAUTION: Locks should only be broken when you are sure that the process
     holding the lock has been stopped.
@@ -4836,17 +4978,33 @@
     :Examples:
         bzr break-lock
         bzr break-lock bzr+ssh://example.com/bzr/foo
+        bzr break-lock --conf ~/.bazaar
     """
+
     takes_args = ['location?']
+    takes_options = [
+        Option('config',
+               help='LOCATION is the directory where the config lock is.'),
+        Option('force',
+            help='Do not ask for confirmation before breaking the lock.'),
+        ]
 
-    def run(self, location=None, show=False):
+    def run(self, location=None, config=False, force=False):
         if location is None:
             location = u'.'
-        control, relpath = bzrdir.BzrDir.open_containing(location)
-        try:
-            control.break_lock()
-        except NotImplementedError:
-            pass
+        if force:
+            ui.ui_factory = ui.ConfirmationUserInterfacePolicy(ui.ui_factory,
+                None,
+                {'bzrlib.lockdir.break': True})
+        if config:
+            conf = _mod_config.LockableConfig(file_name=location)
+            conf.break_lock()
+        else:
+            control, relpath = bzrdir.BzrDir.open_containing(location)
+            try:
+                control.break_lock()
+            except NotImplementedError:
+                pass
 
 
 class cmd_wait_until_signalled(Command):
@@ -4937,7 +5095,7 @@
     not part of it.  (Such trees can be produced by "bzr split", but also by
     running "bzr branch" with the target inside a tree.)
 
-    The result is a combined tree, with the subtree no longer an independant
+    The result is a combined tree, with the subtree no longer an independent
     part.  This is marked as a merge of the subtree into the containing tree,
     and all history is preserved.
     """
@@ -5339,7 +5497,7 @@
             if tag_name is None:
                 raise errors.BzrCommandError("No tag specified to delete.")
             branch.tags.delete_tag(tag_name)
-            self.outf.write('Deleted tag %s.\n' % tag_name)
+            note('Deleted tag %s.' % tag_name)
         else:
             if revision:
                 if len(revision) != 1:
@@ -5357,7 +5515,7 @@
             if (not force) and branch.tags.has_tag(tag_name):
                 raise errors.TagAlreadyExists(tag_name)
             branch.tags.set_tag(tag_name, revision_id)
-            self.outf.write('Created tag %s.\n' % tag_name)
+            note('Created tag %s.' % tag_name)
 
 
 class cmd_tags(Command):
@@ -5370,22 +5528,17 @@
     takes_options = [
         custom_help('directory',
             help='Branch whose tags should be displayed.'),
-        RegistryOption.from_kwargs('sort',
+        RegistryOption('sort',
             'Sort tags by different criteria.', title='Sorting',
-            alpha='Sort tags lexicographically (default).',
-            time='Sort tags chronologically.',
+            lazy_registry=('bzrlib.tag', 'tag_sort_methods')
             ),
         'show-ids',
         'revision',
     ]
 
     @display_command
-    def run(self,
-            directory='.',
-            sort='alpha',
-            show_ids=False,
-            revision=None,
-            ):
+    def run(self, directory='.', sort=None, show_ids=False, revision=None):
+        from bzrlib.tag import tag_sort_methods
         branch, relpath = Branch.open_containing(directory)
 
         tags = branch.tags.get_tag_dict().items()
@@ -5400,19 +5553,9 @@
             # only show revisions between revid1 and revid2 (inclusive)
             tags = [(tag, revid) for tag, revid in tags if
                 graph.is_between(revid, revid1, revid2)]
-        if sort == 'alpha':
-            tags.sort()
-        elif sort == 'time':
-            timestamps = {}
-            for tag, revid in tags:
-                try:
-                    revobj = branch.repository.get_revision(revid)
-                except errors.NoSuchRevision:
-                    timestamp = sys.maxint # place them at the end
-                else:
-                    timestamp = revobj.timestamp
-                timestamps[revid] = timestamp
-            tags.sort(key=lambda x: timestamps[x[1]])
+        if sort is None:
+            sort = tag_sort_methods.get()
+        sort(branch, tags)
         if not show_ids:
             # [ (tag, revid), ... ] -> [ (tag, dotted_revno), ... ]
             for index, (tag, revid) in enumerate(tags):
@@ -5705,7 +5848,8 @@
             name=None,
             switch=None,
             ):
-        tree, file_list = tree_files(file_list, apply_view=False)
+        tree, file_list = WorkingTree.open_containing_paths(file_list,
+            apply_view=False)
         current_view, view_dict = tree.views.get_view_info()
         if name is None:
             name = current_view
@@ -5815,7 +5959,7 @@
             location = "."
         branch = Branch.open_containing(location)[0]
         branch.bzrdir.destroy_branch()
-        
+
 
 class cmd_shelve(Command):
     __doc__ = """Temporarily set aside some changes from the current tree.
@@ -5840,6 +5984,18 @@
 
     You can put multiple items on the shelf, and by default, 'unshelve' will
     restore the most recently shelved changes.
+
+    For complicated changes, it is possible to edit the changes in a separate
+    editor program to decide what the file remaining in the working copy
+    should look like.  To do this, add the configuration option
+
+        change_editor = PROGRAM @new_path @old_path
+
+    where @new_path is replaced with the path of the new version of the 
+    file and @old_path is replaced with the path of the old version of 
+    the file.  The PROGRAM should save the new file with the desired 
+    contents of the file in the working tree.
+        
     """
 
     takes_args = ['file*']
@@ -5857,12 +6013,12 @@
         Option('destroy',
                help='Destroy removed changes instead of shelving them.'),
     ]
-    _see_also = ['unshelve']
+    _see_also = ['unshelve', 'configuration']
 
     def run(self, revision=None, all=False, file_list=None, message=None,
-            writer=None, list=False, destroy=False, directory=u'.'):
+            writer=None, list=False, destroy=False, directory=None):
         if list:
-            return self.run_for_list()
+            return self.run_for_list(directory=directory)
         from bzrlib.shelf_ui import Shelver
         if writer is None:
             writer = bzrlib.option.diff_writer_registry.get()
@@ -5876,8 +6032,10 @@
         except errors.UserAbort:
             return 0
 
-    def run_for_list(self):
-        tree = WorkingTree.open_containing('.')[0]
+    def run_for_list(self, directory=None):
+        if directory is None:
+            directory = u'.'
+        tree = WorkingTree.open_containing(directory)[0]
         self.add_cleanup(tree.lock_read().unlock)
         manager = tree.get_shelf_manager()
         shelves = manager.active_shelves()
@@ -6012,10 +6170,12 @@
     # be only called once.
     for (name, aliases, module_name) in [
         ('cmd_bundle_info', [], 'bzrlib.bundle.commands'),
+        ('cmd_config', [], 'bzrlib.config'),
         ('cmd_dpush', [], 'bzrlib.foreign'),
         ('cmd_version_info', [], 'bzrlib.cmd_version_info'),
         ('cmd_resolve', ['resolved'], 'bzrlib.conflicts'),
         ('cmd_conflicts', [], 'bzrlib.conflicts'),
         ('cmd_sign_my_commits', [], 'bzrlib.sign_my_commits'),
+        ('cmd_test_script', [], 'bzrlib.cmd_test_script'),
         ]:
         builtin_command_registry.register_lazy(name, aliases, module_name)

=== modified file 'bzrlib/bundle/__init__.py'
--- bzrlib/bundle/__init__.py	2010-02-17 17:11:16 +0000
+++ bzrlib/bundle/__init__.py	2011-02-09 08:08:20 +0000
@@ -16,35 +16,26 @@
 
 from cStringIO import StringIO
 
-from bzrlib.symbol_versioning import deprecated_function, deprecated_in
 from bzrlib.lazy_import import lazy_import
 lazy_import(globals(), """
 from bzrlib import (
     errors,
+    transport as _mod_transport,
     urlutils,
     )
 from bzrlib.bundle import serializer as _serializer
 from bzrlib.merge_directive import MergeDirective
-from bzrlib.transport import (
-    do_catching_redirections,
-    get_transport,
-    )
 """)
 from bzrlib.trace import note
 
 
-@deprecated_function(deprecated_in((1, 12, 0)))
-def read_bundle_from_url(url):
-    return read_mergeable_from_url(url, _do_directive=False)
-
-
 def read_mergeable_from_url(url, _do_directive=True, possible_transports=None):
     """Read mergable object from a given URL.
 
     :return: An object supporting get_target_revision.  Raises NotABundle if
         the target is not a mergeable type.
     """
-    child_transport = get_transport(url,
+    child_transport = _mod_transport.get_transport(url,
         possible_transports=possible_transports)
     transport = child_transport.clone('..')
     filename = transport.relpath(child_transport.base)
@@ -63,11 +54,11 @@
                                        exclude_trailing_slash=False)
         if not filename:
             raise errors.NotABundle('A directory cannot be a bundle')
-        return get_transport(url)
+        return _mod_transport.get_transport(url)
 
     try:
-        bytef, transport = do_catching_redirections(get_bundle, transport,
-                                                    redirected_transport)
+        bytef, transport = _mod_transport.do_catching_redirections(
+            get_bundle, transport, redirected_transport)
     except errors.TooManyRedirections:
         raise errors.NotABundle(transport.clone(filename).base)
     except (errors.ConnectionReset, errors.ConnectionError), e:

=== modified file 'bzrlib/bundle/bundle_data.py'
--- bzrlib/bundle/bundle_data.py	2010-06-16 12:47:51 +0000
+++ bzrlib/bundle/bundle_data.py	2011-02-09 08:08:20 +0000
@@ -289,7 +289,7 @@
     def _validate_revision(self, inventory, revision_id):
         """Make sure all revision entries match their checksum."""
 
-        # This is a mapping from each revision id to it's sha hash
+        # This is a mapping from each revision id to its sha hash
         rev_to_sha1 = {}
 
         rev = self.get_revision(revision_id)
@@ -715,12 +715,11 @@
                 ie.symlink_target = self.get_symlink_target(file_id)
             ie.revision = revision_id
 
-            if kind in ('directory', 'symlink'):
-                ie.text_size, ie.text_sha1 = None, None
-            else:
+            if kind == 'file':
                 ie.text_size, ie.text_sha1 = self.get_size_and_sha1(file_id)
-            if (ie.text_size is None) and (kind == 'file'):
-                raise BzrError('Got a text_size of None for file_id %r' % file_id)
+                if ie.text_size is None:
+                    raise BzrError(
+                        'Got a text_size of None for file_id %r' % file_id)
             inv.add(ie)
 
         sorted_entries = self.sorted_path_id()

=== modified file 'bzrlib/bundle/serializer/__init__.py'
--- bzrlib/bundle/serializer/__init__.py	2009-03-23 14:59:43 +0000
+++ bzrlib/bundle/serializer/__init__.py	2011-02-09 08:08:20 +0000
@@ -1,4 +1,4 @@
-# Copyright (C) 2005, 2006 Canonical Ltd
+# Copyright (C) 2005, 2006, 2007, 2009, 2010 Canonical Ltd
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -21,7 +21,10 @@
 from StringIO import StringIO
 import re
 
-import bzrlib.errors as errors
+from bzrlib import (
+    errors,
+    pyutils,
+    )
 from bzrlib.diff import internal_diff
 from bzrlib.revision import NULL_REVISION
 # For backwards-compatibility
@@ -151,17 +154,6 @@
         """
         raise NotImplementedError
 
-    def write(self, source, revision_ids, forced_bases, f):
-        """Write the bundle to the supplied file.
-
-        DEPRECATED: see write_bundle
-        :param source: A source for revision information
-        :param revision_ids: The list of revision ids to serialize
-        :param forced_bases: A dict of revision -> base that overrides default
-        :param f: The file to output to
-        """
-        raise NotImplementedError
-
     def _write_bundle(self, repository, revision_id, base_revision_id, out):
         """Helper function for translating write_bundle to write"""
         forced_bases = {revision_id:base_revision_id}
@@ -202,8 +194,7 @@
     :param overwrite: Should this version override a default
     """
     def _loader(version):
-        mod = __import__(module, globals(), locals(), [classname])
-        klass = getattr(mod, classname)
+        klass = pyutils.get_named_object(module, classname)
         return klass(version)
     register(version, _loader, overwrite=overwrite)
 

=== modified file 'bzrlib/bundle/serializer/v4.py'
--- bzrlib/bundle/serializer/v4.py	2009-08-04 14:08:32 +0000
+++ bzrlib/bundle/serializer/v4.py	2011-02-09 08:08:20 +0000
@@ -1,4 +1,4 @@
-# Copyright (C) 2007 Canonical Ltd
+# Copyright (C) 2007-2010 Canonical Ltd
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -30,11 +30,49 @@
     serializer,
     trace,
     ui,
+    versionedfile as _mod_versionedfile,
     )
 from bzrlib.bundle import bundle_data, serializer as bundle_serializer
 from bzrlib import bencode
 
 
+class _MPDiffInventoryGenerator(_mod_versionedfile._MPDiffGenerator):
+    """Generate Inventory diffs serialized inventories."""
+
+    def __init__(self, repo, inventory_keys):
+        super(_MPDiffInventoryGenerator, self).__init__(repo.inventories,
+            inventory_keys)
+        self.repo = repo
+        self.sha1s = {}
+
+    def iter_diffs(self):
+        """Compute the diffs one at a time."""
+        # This is instead of compute_diffs() since we guarantee our ordering of
+        # inventories, we don't have to do any buffering
+        self._find_needed_keys()
+        # We actually use a slightly different ordering. We grab all of the
+        # parents first, and then grab the ordered requests.
+        needed_ids = [k[-1] for k in self.present_parents]
+        needed_ids.extend([k[-1] for k in self.ordered_keys])
+        inv_to_str = self.repo._serializer.write_inventory_to_string
+        for inv in self.repo.iter_inventories(needed_ids):
+            revision_id = inv.revision_id
+            key = (revision_id,)
+            if key in self.present_parents:
+                # Not a key we will transmit, which is a shame, since because
+                # of that bundles don't work with stacked branches
+                parent_ids = None
+            else:
+                parent_ids = [k[-1] for k in self.parent_map[key]]
+            as_bytes = inv_to_str(inv)
+            self._process_one_record(key, (as_bytes,))
+            if parent_ids is None:
+                continue
+            diff = self.diffs.pop(key)
+            sha1 = osutils.sha_string(as_bytes)
+            yield revision_id, parent_ids, sha1, diff
+
+
 class BundleWriter(object):
     """Writer for bundle-format files.
 
@@ -348,48 +386,10 @@
         the other side.
         """
         inventory_key_order = [(r,) for r in revision_order]
-        parent_map = self.repository.inventories.get_parent_map(
-                            inventory_key_order)
-        missing_keys = set(inventory_key_order).difference(parent_map)
-        if missing_keys:
-            raise errors.RevisionNotPresent(list(missing_keys)[0],
-                                            self.repository.inventories)
-        inv_to_str = self.repository._serializer.write_inventory_to_string
-        # Make sure that we grab the parent texts first
-        just_parents = set()
-        map(just_parents.update, parent_map.itervalues())
-        just_parents.difference_update(parent_map)
-        # Ignore ghost parents
-        present_parents = self.repository.inventories.get_parent_map(
-                            just_parents)
-        ghost_keys = just_parents.difference(present_parents)
-        needed_inventories = list(present_parents) + inventory_key_order
-        needed_inventories = [k[-1] for k in needed_inventories]
-        all_lines = {}
-        for inv in self.repository.iter_inventories(needed_inventories):
-            revision_id = inv.revision_id
-            key = (revision_id,)
-            as_bytes = inv_to_str(inv)
-            # The sha1 is validated as the xml/textual form, not as the
-            # form-in-the-repository
-            sha1 = osutils.sha_string(as_bytes)
-            as_lines = osutils.split_lines(as_bytes)
-            del as_bytes
-            all_lines[key] = as_lines
-            if key in just_parents:
-                # We don't transmit those entries
-                continue
-            # Create an mpdiff for this text, and add it to the output
-            parent_keys = parent_map[key]
-            # See the comment in VF.make_mpdiffs about how this effects
-            # ordering when there are ghosts present. I think we have a latent
-            # bug
-            parent_lines = [all_lines[p_key] for p_key in parent_keys
-                            if p_key not in ghost_keys]
-            diff = multiparent.MultiParent.from_lines(
-                as_lines, parent_lines)
+        generator = _MPDiffInventoryGenerator(self.repository,
+                                              inventory_key_order)
+        for revision_id, parent_ids, sha1, diff in generator.iter_diffs():
             text = ''.join(diff.to_patch())
-            parent_ids = [k[-1] for k in parent_keys]
             self.bundle.add_multiparent_record(text, sha1, parent_ids,
                                                'inventory', revision_id, None)
 

=== modified file 'bzrlib/bzrdir.py'
--- bzrlib/bzrdir.py	2010-11-26 18:13:30 +0000
+++ bzrlib/bzrdir.py	2011-02-09 08:08:20 +0000
@@ -1,4 +1,4 @@
-# Copyright (C) 2006-2010 Canonical Ltd
+# Copyright (C) 2006-2011 Canonical Ltd
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -34,20 +34,22 @@
 from bzrlib.lazy_import import lazy_import
 lazy_import(globals(), """
 from stat import S_ISDIR
-import textwrap
 
 import bzrlib
 from bzrlib import (
     branch,
     config,
+    controldir,
     errors,
     graph,
     lockable_files,
     lockdir,
     osutils,
+    pyutils,
     remote,
     repository,
     revision as _mod_revision,
+    transport as _mod_transport,
     ui,
     urlutils,
     versionedfile,
@@ -65,14 +67,16 @@
     )
 from bzrlib.repofmt import pack_repo
 from bzrlib.smart.client import _SmartClient
-from bzrlib.store.versioned import WeaveStore
+from bzrlib.store.versioned import VersionedFileStore
 from bzrlib.transactions import WriteTransaction
 from bzrlib.transport import (
     do_catching_redirections,
-    get_transport,
     local,
     )
-from bzrlib.weave import Weave
+from bzrlib.weave import (
+    WeaveFile,
+    Weave,
+    )
 """)
 
 from bzrlib.trace import (
@@ -86,42 +90,13 @@
     registry,
     symbol_versioning,
     )
-    
-    
-class ControlComponent(object):
-    """Abstract base class for control directory components.
-    
-    This provides interfaces that are common across bzrdirs, 
-    repositories, branches, and workingtree control directories.
-    
-    They all expose two urls and transports: the *user* URL is the 
-    one that stops above the control directory (eg .bzr) and that 
-    should normally be used in messages, and the *control* URL is
-    under that in eg .bzr/checkout and is used to read the control
-    files.
-    
-    This can be used as a mixin and is intended to fit with 
-    foreign formats.
-    """
-    
-    @property
-    def control_transport(self):
-        raise NotImplementedError
-   
-    @property
-    def control_url(self):
-        return self.control_transport.base
-    
-    @property
-    def user_transport(self):
-        raise NotImplementedError
-        
-    @property
-    def user_url(self):
-        return self.user_transport.base
-    
-
-class BzrDir(ControlComponent):
+from bzrlib.symbol_versioning import (
+    deprecated_in,
+    deprecated_method,
+    )
+
+
+class BzrDir(controldir.ControlDir):
     """A .bzr control diretory.
 
     BzrDir instances let you create or open any of the things that can be
@@ -158,10 +133,6 @@
                     return
         thing_to_unlock.break_lock()
 
-    def can_convert_format(self):
-        """Return true if this bzrdir is one whose format we can convert from."""
-        return True
-
     def check_conversion_target(self, target_format):
         """Check that a bzrdir as a whole can be converted to a new format."""
         # The only current restriction is that the repository content can be 
@@ -202,28 +173,9 @@
                 format.get_format_description(),
                 basedir)
 
-    def clone(self, url, revision_id=None, force_new_repo=False,
-              preserve_stacking=False):
-        """Clone this bzrdir and its contents to url verbatim.
-
-        :param url: The url create the clone at.  If url's last component does
-            not exist, it will be created.
-        :param revision_id: The tip revision-id to use for any branch or
-            working tree.  If not None, then the clone operation may tune
-            itself to download less data.
-        :param force_new_repo: Do not use a shared repository for the target
-                               even if one is available.
-        :param preserve_stacking: When cloning a stacked branch, stack the
-            new branch on top of the other branch's stacked-on branch.
-        """
-        return self.clone_on_transport(get_transport(url),
-                                       revision_id=revision_id,
-                                       force_new_repo=force_new_repo,
-                                       preserve_stacking=preserve_stacking)
-
     def clone_on_transport(self, transport, revision_id=None,
         force_new_repo=False, preserve_stacking=False, stacked_on=None,
-        create_prefix=False, use_existing_dir=True):
+        create_prefix=False, use_existing_dir=True, no_tree=False):
         """Clone this bzrdir and its contents to transport verbatim.
 
         :param transport: The transport for the location to produce the clone
@@ -241,12 +193,12 @@
         """
         # Overview: put together a broad description of what we want to end up
         # with; then make as few api calls as possible to do it.
-        
+
         # We may want to create a repo/branch/tree, if we do so what format
         # would we want for each:
         require_stacking = (stacked_on is not None)
         format = self.cloning_metadir(require_stacking)
-        
+
         # Figure out what objects we want:
         try:
             local_repo = self.find_repository()
@@ -271,7 +223,7 @@
         # we should look up the policy needs first, or just use it as a hint,
         # or something.
         if local_repo:
-            make_working_trees = local_repo.make_working_trees()
+            make_working_trees = local_repo.make_working_trees() and not no_tree
             want_shared = local_repo.is_shared()
             repo_format_name = format.repository_format.network_name()
         else:
@@ -326,26 +278,8 @@
     # TODO: This should be given a Transport, and should chdir up; otherwise
     # this will open a new connection.
     def _make_tail(self, url):
-        t = get_transport(url)
-        t.ensure_base()
-
-    @classmethod
-    def create(cls, base, format=None, possible_transports=None):
-        """Create a new BzrDir at the url 'base'.
-
-        :param format: If supplied, the format of branch to create.  If not
-            supplied, the default is used.
-        :param possible_transports: If supplied, a list of transports that
-            can be reused to share a remote connection.
-        """
-        if cls is not BzrDir:
-            raise AssertionError("BzrDir.create always creates the default"
-                " format, not one of %r" % cls)
-        t = get_transport(base, possible_transports)
-        t.ensure_base()
-        if format is None:
-            format = BzrDirFormat.get_default_format()
-        return format.initialize_on_transport(t)
+        t = _mod_transport.get_transport(url)
+        t.ensure_base()
 
     @staticmethod
     def find_bzrdirs(transport, evaluate=None, list_current=None):
@@ -388,15 +322,6 @@
                 for subdir in sorted(subdirs, reverse=True):
                     pending.append(current_transport.clone(subdir))
 
-    def list_branches(self):
-        """Return a sequence of all branches local to this control directory.
-
-        """
-        try:
-            return [self.open_branch()]
-        except (errors.NotBranchError, errors.NoRepositoryPresent):
-            return []
-
     @staticmethod
     def find_branches(transport):
         """Find all branches under a transport.
@@ -425,29 +350,6 @@
                 ret.extend(branches)
         return ret
 
-    def destroy_repository(self):
-        """Destroy the repository in this BzrDir"""
-        raise NotImplementedError(self.destroy_repository)
-
-    def create_branch(self, name=None):
-        """Create a branch in this BzrDir.
-
-        :param name: Name of the colocated branch to create, None for
-            the default branch.
-
-        The bzrdir's format will control what branch format is created.
-        For more control see BranchFormatXX.create(a_bzrdir).
-        """
-        raise NotImplementedError(self.create_branch)
-
-    def destroy_branch(self, name=None):
-        """Destroy a branch in this BzrDir.
-        
-        :param name: Name of the branch to destroy, None for the default 
-            branch.
-        """
-        raise NotImplementedError(self.destroy_branch)
-
     @staticmethod
     def create_branch_and_repo(base, force_new_repo=False, format=None):
         """Create a new BzrDir, Branch and Repository at the url 'base'.
@@ -566,7 +468,7 @@
         """
         if force_new_tree:
             # check for non local urls
-            t = get_transport(base, possible_transports)
+            t = _mod_transport.get_transport(base, possible_transports)
             if not isinstance(t, local.LocalTransport):
                 raise errors.NotLocalUrl(base)
         bzrdir = BzrDir.create(base, format, possible_transports)
@@ -594,7 +496,7 @@
         :param format: Override for the bzrdir format to create.
         :return: The WorkingTree object.
         """
-        t = get_transport(base)
+        t = _mod_transport.get_transport(base)
         if not isinstance(t, local.LocalTransport):
             raise errors.NotLocalUrl(base)
         bzrdir = BzrDir.create_branch_and_repo(base,
@@ -602,42 +504,30 @@
                                                format=format).bzrdir
         return bzrdir.create_workingtree()
 
-    def create_workingtree(self, revision_id=None, from_branch=None,
-        accelerator_tree=None, hardlink=False):
-        """Create a working tree at this BzrDir.
-
-        :param revision_id: create it as of this revision id.
-        :param from_branch: override bzrdir branch (for lightweight checkouts)
-        :param accelerator_tree: A tree which can be used for retrieving file
-            contents more quickly than the revision tree, i.e. a workingtree.
-            The revision tree will be used for cases where accelerator_tree's
-            content is different.
+    @deprecated_method(deprecated_in((2, 3, 0)))
+    def generate_backup_name(self, base):
+        return self._available_backup_name(base)
+
+    def _available_backup_name(self, base):
+        """Find a non-existing backup file name based on base.
+
+        See bzrlib.osutils.available_backup_name about race conditions.
         """
-        raise NotImplementedError(self.create_workingtree)
+        return osutils.available_backup_name(base, self.root_transport.has)
 
     def backup_bzrdir(self):
         """Backup this bzr control directory.
 
         :return: Tuple with old path name and new path name
         """
-        def name_gen(base='backup.bzr'):
-            counter = 1
-            name = "%s.~%d~" % (base, counter)
-            while self.root_transport.has(name):
-                counter += 1
-                name = "%s.~%d~" % (base, counter)
-            return name
 
-        backup_dir=name_gen()
         pb = ui.ui_factory.nested_progress_bar()
         try:
-            # FIXME: bug 300001 -- the backup fails if the backup directory
-            # already exists, but it should instead either remove it or make
-            # a new backup directory.
-            #
             old_path = self.root_transport.abspath('.bzr')
+            backup_dir = self._available_backup_name('backup.bzr')
             new_path = self.root_transport.abspath(backup_dir)
-            ui.ui_factory.note('making backup of %s\n  to %s' % (old_path, new_path,))
+            ui.ui_factory.note('making backup of %s\n  to %s'
+                               % (old_path, new_path,))
             self.root_transport.copy_tree('.bzr', backup_dir)
             return (old_path, new_path)
         finally:
@@ -668,21 +558,6 @@
                 else:
                     pass
 
-    def destroy_workingtree(self):
-        """Destroy the working tree at this BzrDir.
-
-        Formats that do not support this may raise UnsupportedOperation.
-        """
-        raise NotImplementedError(self.destroy_workingtree)
-
-    def destroy_workingtree_metadata(self):
-        """Destroy the control files for the working tree at this BzrDir.
-
-        The contents of working tree files are not affected.
-        Formats that do not support this may raise UnsupportedOperation.
-        """
-        raise NotImplementedError(self.destroy_workingtree_metadata)
-
     def _find_containing(self, evaluate):
         """Find something in a containing control directory.
 
@@ -737,33 +612,6 @@
             raise errors.NoRepositoryPresent(self)
         return found_repo
 
-    def get_branch_reference(self, name=None):
-        """Return the referenced URL for the branch in this bzrdir.
-
-        :param name: Optional colocated branch name
-        :raises NotBranchError: If there is no Branch.
-        :raises NoColocatedBranchSupport: If a branch name was specified
-            but colocated branches are not supported.
-        :return: The URL the branch in this bzrdir references if it is a
-            reference branch, or None for regular branches.
-        """
-        if name is not None:
-            raise errors.NoColocatedBranchSupport(self)
-        return None
-
-    def get_branch_transport(self, branch_format, name=None):
-        """Get the transport for use by branch format in this BzrDir.
-
-        Note that bzr dirs that do not support format strings will raise
-        IncompatibleFormat if the branch format they are given has
-        a format string, and vice versa.
-
-        If branch_format is None, the transport is returned with no
-        checking. If it is not None, then the returned transport is
-        guaranteed to point to an existing directory ready for use.
-        """
-        raise NotImplementedError(self.get_branch_transport)
-
     def _find_creation_modes(self):
         """Determine the appropriate modes for files and directories.
 
@@ -808,32 +656,6 @@
             self._find_creation_modes()
         return self._dir_mode
 
-    def get_repository_transport(self, repository_format):
-        """Get the transport for use by repository format in this BzrDir.
-
-        Note that bzr dirs that do not support format strings will raise
-        IncompatibleFormat if the repository format they are given has
-        a format string, and vice versa.
-
-        If repository_format is None, the transport is returned with no
-        checking. If it is not None, then the returned transport is
-        guaranteed to point to an existing directory ready for use.
-        """
-        raise NotImplementedError(self.get_repository_transport)
-
-    def get_workingtree_transport(self, tree_format):
-        """Get the transport for use by workingtree format in this BzrDir.
-
-        Note that bzr dirs that do not support format strings will raise
-        IncompatibleFormat if the workingtree format they are given has a
-        format string, and vice versa.
-
-        If workingtree_format is None, the transport is returned with no
-        checking. If it is not None, then the returned transport is
-        guaranteed to point to an existing directory ready for use.
-        """
-        raise NotImplementedError(self.get_workingtree_transport)
-
     def get_config(self):
         """Get configuration for this BzrDir."""
         return config.BzrDirConfig(self)
@@ -857,11 +679,11 @@
         self.transport = _transport.clone('.bzr')
         self.root_transport = _transport
         self._mode_check_done = False
-        
+
     @property 
     def user_transport(self):
         return self.root_transport
-        
+
     @property
     def control_transport(self):
         return self.transport
@@ -873,9 +695,7 @@
 
         This is true IF and ONLY IF the filename is part of the namespace reserved
         for bzr control dirs. Currently this is the '.bzr' directory in the root
-        of the root_transport. it is expected that plugins will need to extend
-        this in the future - for instance to make bzr talk with svn working
-        trees.
+        of the root_transport. 
         """
         # this might be better on the BzrDirFormat class because it refers to
         # all the possible bzrdir disk formats.
@@ -885,17 +705,6 @@
         # add new tests for it to the appropriate place.
         return filename == '.bzr' or filename.startswith('.bzr/')
 
-    def needs_format_conversion(self, format=None):
-        """Return true if this bzrdir needs convert_format run on it.
-
-        For instance, if the repository format is out of date but the
-        branch and working tree are not, this should return True.
-
-        :param format: Optional parameter indicating a specific desired
-                       format we plan to arrive at.
-        """
-        raise NotImplementedError(self.needs_format_conversion)
-
     @staticmethod
     def open_unsupported(base):
         """Open a branch which is not supported."""
@@ -907,7 +716,7 @@
 
         :param _unsupported: a private parameter to the BzrDir class.
         """
-        t = get_transport(base, possible_transports=possible_transports)
+        t = _mod_transport.get_transport(base, possible_transports)
         return BzrDir.open_from_transport(t, _unsupported=_unsupported)
 
     @staticmethod
@@ -924,7 +733,7 @@
         # the redirections.
         base = transport.base
         def find_format(transport):
-            return transport, BzrDirFormat.find_format(
+            return transport, controldir.ControlDirFormat.find_format(
                 transport, _server_formats=_server_formats)
 
         def redirected(transport, e, redirection_notice):
@@ -945,17 +754,6 @@
         BzrDir._check_supported(format, _unsupported)
         return format.open(transport, _found=True)
 
-    def open_branch(self, name=None, unsupported=False,
-                    ignore_fallbacks=False):
-        """Open the branch object at this BzrDir if one is present.
-
-        If unsupported is True, then no longer supported branch formats can
-        still be opened.
-
-        TODO: static convenience version of this?
-        """
-        raise NotImplementedError(self.open_branch)
-
     @staticmethod
     def open_containing(url, possible_transports=None):
         """Open an existing branch which contains url.
@@ -963,7 +761,7 @@
         :param url: url to search from.
         See open_containing_from_transport for more detail.
         """
-        transport = get_transport(url, possible_transports)
+        transport = _mod_transport.get_transport(url, possible_transports)
         return BzrDir.open_containing_from_transport(transport)
 
     @staticmethod
@@ -999,27 +797,6 @@
                 raise errors.NotBranchError(path=url)
             a_transport = new_t
 
-    def _get_tree_branch(self, name=None):
-        """Return the branch and tree, if any, for this bzrdir.
-
-        :param name: Name of colocated branch to open.
-
-        Return None for tree if not present or inaccessible.
-        Raise NotBranchError if no branch is present.
-        :return: (tree, branch)
-        """
-        try:
-            tree = self.open_workingtree()
-        except (errors.NoWorkingTree, errors.NotLocalUrl):
-            tree = None
-            branch = self.open_branch(name=name)
-        else:
-            if name is not None:
-                branch = self.open_branch(name=name)
-            else:
-                branch = tree.branch
-        return tree, branch
-
     @classmethod
     def open_tree_or_branch(klass, location):
         """Return the branch and working tree at a location.
@@ -1071,59 +848,6 @@
                 raise errors.NotBranchError(location)
         return tree, branch, branch.repository, relpath
 
-    def open_repository(self, _unsupported=False):
-        """Open the repository object at this BzrDir if one is present.
-
-        This will not follow the Branch object pointer - it's strictly a direct
-        open facility. Most client code should use open_branch().repository to
-        get at a repository.
-
-        :param _unsupported: a private parameter, not part of the api.
-        TODO: static convenience version of this?
-        """
-        raise NotImplementedError(self.open_repository)
-
-    def open_workingtree(self, _unsupported=False,
-                         recommend_upgrade=True, from_branch=None):
-        """Open the workingtree object at this BzrDir if one is present.
-
-        :param recommend_upgrade: Optional keyword parameter, when True (the
-            default), emit through the ui module a recommendation that the user
-            upgrade the working tree when the workingtree being opened is old
-            (but still fully supported).
-        :param from_branch: override bzrdir branch (for lightweight checkouts)
-        """
-        raise NotImplementedError(self.open_workingtree)
-
-    def has_branch(self, name=None):
-        """Tell if this bzrdir contains a branch.
-
-        Note: if you're going to open the branch, you should just go ahead
-        and try, and not ask permission first.  (This method just opens the
-        branch and discards it, and that's somewhat expensive.)
-        """
-        try:
-            self.open_branch(name)
-            return True
-        except errors.NotBranchError:
-            return False
-
-    def has_workingtree(self):
-        """Tell if this bzrdir contains a working tree.
-
-        This will still raise an exception if the bzrdir has a workingtree that
-        is remote & inaccessible.
-
-        Note: if you're going to open the working tree, you should just go ahead
-        and try, and not ask permission first.  (This method just opens the
-        workingtree and discards it, and that's somewhat expensive.)
-        """
-        try:
-            self.open_workingtree(recommend_upgrade=False)
-            return True
-        except errors.NoWorkingTree:
-            return False
-
     def _cloning_metadir(self):
         """Produce a metadir suitable for cloning with.
 
@@ -1187,192 +911,23 @@
             format.require_stacking()
         return format
 
-    def checkout_metadir(self):
-        return self.cloning_metadir()
-
-    def sprout(self, url, revision_id=None, force_new_repo=False,
-               recurse='down', possible_transports=None,
-               accelerator_tree=None, hardlink=False, stacked=False,
-               source_branch=None, create_tree_if_local=True):
-        """Create a copy of this bzrdir prepared for use as a new line of
-        development.
-
-        If url's last component does not exist, it will be created.
-
-        Attributes related to the identity of the source branch like
-        branch nickname will be cleaned, a working tree is created
-        whether one existed before or not; and a local branch is always
-        created.
-
-        if revision_id is not None, then the clone operation may tune
-            itself to download less data.
-        :param accelerator_tree: A tree which can be used for retrieving file
-            contents more quickly than the revision tree, i.e. a workingtree.
-            The revision tree will be used for cases where accelerator_tree's
-            content is different.
-        :param hardlink: If true, hard-link files from accelerator_tree,
-            where possible.
-        :param stacked: If true, create a stacked branch referring to the
-            location of this control directory.
-        :param create_tree_if_local: If true, a working-tree will be created
-            when working locally.
+    @classmethod
+    def create(cls, base, format=None, possible_transports=None):
+        """Create a new BzrDir at the url 'base'.
+
+        :param format: If supplied, the format of branch to create.  If not
+            supplied, the default is used.
+        :param possible_transports: If supplied, a list of transports that
+            can be reused to share a remote connection.
         """
-        target_transport = get_transport(url, possible_transports)
-        target_transport.ensure_base()
-        cloning_format = self.cloning_metadir(stacked)
-        # Create/update the result branch
-        result = cloning_format.initialize_on_transport(target_transport)
-        # if a stacked branch wasn't requested, we don't create one
-        # even if the origin was stacked
-        stacked_branch_url = None
-        if source_branch is not None:
-            if stacked:
-                stacked_branch_url = self.root_transport.base
-            source_repository = source_branch.repository
-        else:
-            try:
-                source_branch = self.open_branch()
-                source_repository = source_branch.repository
-                if stacked:
-                    stacked_branch_url = self.root_transport.base
-            except errors.NotBranchError:
-                source_branch = None
-                try:
-                    source_repository = self.open_repository()
-                except errors.NoRepositoryPresent:
-                    source_repository = None
-        repository_policy = result.determine_repository_policy(
-            force_new_repo, stacked_branch_url, require_stacking=stacked)
-        result_repo, is_new_repo = repository_policy.acquire_repository()
-        is_stacked = stacked or (len(result_repo._fallback_repositories) != 0)
-        if is_new_repo and revision_id is not None and not is_stacked:
-            fetch_spec = graph.PendingAncestryResult(
-                [revision_id], source_repository)
-        else:
-            fetch_spec = None
-        if source_repository is not None:
-            # Fetch while stacked to prevent unstacked fetch from
-            # Branch.sprout.
-            if fetch_spec is None:
-                result_repo.fetch(source_repository, revision_id=revision_id)
-            else:
-                result_repo.fetch(source_repository, fetch_spec=fetch_spec)
-
-        if source_branch is None:
-            # this is for sprouting a bzrdir without a branch; is that
-            # actually useful?
-            # Not especially, but it's part of the contract.
-            result_branch = result.create_branch()
-        else:
-            result_branch = source_branch.sprout(result,
-                revision_id=revision_id, repository_policy=repository_policy)
-        mutter("created new branch %r" % (result_branch,))
-
-        # Create/update the result working tree
-        if (create_tree_if_local and
-            isinstance(target_transport, local.LocalTransport) and
-            (result_repo is None or result_repo.make_working_trees())):
-            wt = result.create_workingtree(accelerator_tree=accelerator_tree,
-                hardlink=hardlink)
-            wt.lock_write()
-            try:
-                if wt.path2id('') is None:
-                    try:
-                        wt.set_root_id(self.open_workingtree.get_root_id())
-                    except errors.NoWorkingTree:
-                        pass
-            finally:
-                wt.unlock()
-        else:
-            wt = None
-        if recurse == 'down':
-            if wt is not None:
-                basis = wt.basis_tree()
-                basis.lock_read()
-                subtrees = basis.iter_references()
-            elif result_branch is not None:
-                basis = result_branch.basis_tree()
-                basis.lock_read()
-                subtrees = basis.iter_references()
-            elif source_branch is not None:
-                basis = source_branch.basis_tree()
-                basis.lock_read()
-                subtrees = basis.iter_references()
-            else:
-                subtrees = []
-                basis = None
-            try:
-                for path, file_id in subtrees:
-                    target = urlutils.join(url, urlutils.escape(path))
-                    sublocation = source_branch.reference_parent(file_id, path)
-                    sublocation.bzrdir.sprout(target,
-                        basis.get_reference_revision(file_id, path),
-                        force_new_repo=force_new_repo, recurse=recurse,
-                        stacked=stacked)
-            finally:
-                if basis is not None:
-                    basis.unlock()
-        return result
-
-    def push_branch(self, source, revision_id=None, overwrite=False, 
-        remember=False, create_prefix=False):
-        """Push the source branch into this BzrDir."""
-        br_to = None
-        # If we can open a branch, use its direct repository, otherwise see
-        # if there is a repository without a branch.
-        try:
-            br_to = self.open_branch()
-        except errors.NotBranchError:
-            # Didn't find a branch, can we find a repository?
-            repository_to = self.find_repository()
-        else:
-            # Found a branch, so we must have found a repository
-            repository_to = br_to.repository
-
-        push_result = PushResult()
-        push_result.source_branch = source
-        if br_to is None:
-            # We have a repository but no branch, copy the revisions, and then
-            # create a branch.
-            repository_to.fetch(source.repository, revision_id=revision_id)
-            br_to = source.clone(self, revision_id=revision_id)
-            if source.get_push_location() is None or remember:
-                source.set_push_location(br_to.base)
-            push_result.stacked_on = None
-            push_result.branch_push_result = None
-            push_result.old_revno = None
-            push_result.old_revid = _mod_revision.NULL_REVISION
-            push_result.target_branch = br_to
-            push_result.master_branch = None
-            push_result.workingtree_updated = False
-        else:
-            # We have successfully opened the branch, remember if necessary:
-            if source.get_push_location() is None or remember:
-                source.set_push_location(br_to.base)
-            try:
-                tree_to = self.open_workingtree()
-            except errors.NotLocalUrl:
-                push_result.branch_push_result = source.push(br_to, 
-                    overwrite, stop_revision=revision_id)
-                push_result.workingtree_updated = False
-            except errors.NoWorkingTree:
-                push_result.branch_push_result = source.push(br_to,
-                    overwrite, stop_revision=revision_id)
-                push_result.workingtree_updated = None # Not applicable
-            else:
-                tree_to.lock_write()
-                try:
-                    push_result.branch_push_result = source.push(
-                        tree_to.branch, overwrite, stop_revision=revision_id)
-                    tree_to.update()
-                finally:
-                    tree_to.unlock()
-                push_result.workingtree_updated = True
-            push_result.old_revno = push_result.branch_push_result.old_revno
-            push_result.old_revid = push_result.branch_push_result.old_revid
-            push_result.target_branch = \
-                push_result.branch_push_result.target_branch
-        return push_result
+        if cls is not BzrDir:
+            raise AssertionError("BzrDir.create always creates the"
+                "default format, not one of %r" % cls)
+        t = _mod_transport.get_transport(base, possible_transports)
+        t.ensure_base()
+        if format is None:
+            format = controldir.ControlDirFormat.get_default_format()
+        return format.initialize_on_transport(t)
 
 
 class BzrDirHooks(hooks.Hooks):
@@ -1448,7 +1003,7 @@
     def cloning_metadir(self, require_stacking=False):
         """Produce a metadir suitable for cloning with."""
         if require_stacking:
-            return format_registry.make_bzrdir('1.6')
+            return controldir.format_registry.make_bzrdir('1.6')
         return self._format.__class__()
 
     def clone(self, url, revision_id=None, force_new_repo=False,
@@ -1474,8 +1029,11 @@
             tree.clone(result)
         return result
 
-    def create_branch(self, name=None):
+    def create_branch(self, name=None, repository=None):
         """See BzrDir.create_branch."""
+        if repository is not None:
+            raise NotImplementedError(
+                "create_branch(repository=<not None>) on %r" % (self,))
         return self._format.get_branch_format().initialize(self, name=name)
 
     def destroy_branch(self, name=None):
@@ -1711,9 +1269,10 @@
         """See BzrDir.can_convert_format()."""
         return True
 
-    def create_branch(self, name=None):
+    def create_branch(self, name=None, repository=None):
         """See BzrDir.create_branch."""
-        return self._format.get_branch_format().initialize(self, name=name)
+        return self._format.get_branch_format().initialize(self, name=name,
+                repository=repository)
 
     def destroy_branch(self, name=None):
         """See BzrDir.create_branch."""
@@ -1741,7 +1300,10 @@
         wt = self.open_workingtree(recommend_upgrade=False)
         repository = wt.branch.repository
         empty = repository.revision_tree(_mod_revision.NULL_REVISION)
-        wt.revert(old_tree=empty)
+        # We ignore the conflicts returned by wt.revert since we're about to
+        # delete the wt metadata anyway, all that should be left here are
+        # detritus. But see bug #634470 about subtree .bzr dirs.
+        conflicts = wt.revert(old_tree=empty)
         self.destroy_workingtree_metadata()
 
     def destroy_workingtree_metadata(self):
@@ -1891,13 +1453,66 @@
         return config.TransportConfig(self.transport, 'control.conf')
 
 
-class BzrDirFormat(object):
-    """An encapsulation of the initialization and open routines for a format.
-
-    Formats provide three things:
-     * An initialization routine,
-     * a format string,
-     * an open routine.
+class BzrProber(controldir.Prober):
+    """Prober for formats that use a .bzr/ control directory."""
+
+    _formats = {}
+    """The known .bzr formats."""
+
+    @classmethod
+    def register_bzrdir_format(klass, format):
+        klass._formats[format.get_format_string()] = format
+
+    @classmethod
+    def unregister_bzrdir_format(klass, format):
+        del klass._formats[format.get_format_string()]
+
+    @classmethod
+    def probe_transport(klass, transport):
+        """Return the .bzrdir style format present in a directory."""
+        try:
+            format_string = transport.get_bytes(".bzr/branch-format")
+        except errors.NoSuchFile:
+            raise errors.NotBranchError(path=transport.base)
+        try:
+            return klass._formats[format_string]
+        except KeyError:
+            raise errors.UnknownFormatError(format=format_string, kind='bzrdir')
+
+
+controldir.ControlDirFormat.register_prober(BzrProber)
+
+
+class RemoteBzrProber(controldir.Prober):
+    """Prober for remote servers that provide a Bazaar smart server."""
+
+    @classmethod
+    def probe_transport(klass, transport):
+        """Return a RemoteBzrDirFormat object if it looks possible."""
+        try:
+            medium = transport.get_smart_medium()
+        except (NotImplementedError, AttributeError,
+                errors.TransportNotPossible, errors.NoSmartMedium,
+                errors.SmartProtocolError):
+            # no smart server, so not a branch for this format type.
+            raise errors.NotBranchError(path=transport.base)
+        else:
+            # Decline to open it if the server doesn't support our required
+            # version (3) so that the VFS-based transport will do it.
+            if medium.should_probe():
+                try:
+                    server_version = medium.protocol_version()
+                except errors.SmartProtocolError:
+                    # Apparently there's no usable smart server there, even though
+                    # the medium supports the smart protocol.
+                    raise errors.NotBranchError(path=transport.base)
+                if server_version != '2':
+                    raise errors.NotBranchError(path=transport.base)
+            return RemoteBzrDirFormat()
+
+
+class BzrDirFormat(controldir.ControlDirFormat):
+    """ControlDirFormat base class for .bzr/ directories.
 
     Formats are placed in a dict by their format string for reference
     during bzrdir opening. These should be subclasses of BzrDirFormat
@@ -1906,104 +1521,17 @@
     Once a format is deprecated, just deprecate the initialize and open
     methods on the format class. Do not deprecate the object, as the
     object will be created every system load.
-
-    :cvar colocated_branches: Whether this formats supports colocated branches.
-    """
-
-    _default_format = None
-    """The default format used for new .bzr dirs."""
-
-    _formats = {}
-    """The known formats."""
-
-    _control_formats = []
-    """The registered control formats - .bzr, ....
-
-    This is a list of BzrDirFormat objects.
-    """
-
-    _control_server_formats = []
-    """The registered control server formats, e.g. RemoteBzrDirs.
-
-    This is a list of BzrDirFormat objects.
     """
 
     _lock_file_name = 'branch-lock'
 
-    colocated_branches = False
-    """Whether co-located branches are supported for this control dir format.
-    """
-
     # _lock_class must be set in subclasses to the lock type, typ.
     # TransportLock or LockDir
 
-    @classmethod
-    def find_format(klass, transport, _server_formats=True):
-        """Return the format present at transport."""
-        if _server_formats:
-            formats = klass._control_server_formats + klass._control_formats
-        else:
-            formats = klass._control_formats
-        for format in formats:
-            try:
-                return format.probe_transport(transport)
-            except errors.NotBranchError:
-                # this format does not find a control dir here.
-                pass
-        raise errors.NotBranchError(path=transport.base)
-
-    @classmethod
-    def probe_transport(klass, transport):
-        """Return the .bzrdir style format present in a directory."""
-        try:
-            format_string = transport.get_bytes(".bzr/branch-format")
-        except errors.NoSuchFile:
-            raise errors.NotBranchError(path=transport.base)
-        try:
-            return klass._formats[format_string]
-        except KeyError:
-            raise errors.UnknownFormatError(format=format_string, kind='bzrdir')
-
-    @classmethod
-    def get_default_format(klass):
-        """Return the current default format."""
-        return klass._default_format
-
     def get_format_string(self):
         """Return the ASCII format string that identifies this format."""
         raise NotImplementedError(self.get_format_string)
 
-    def get_format_description(self):
-        """Return the short description for this format."""
-        raise NotImplementedError(self.get_format_description)
-
-    def get_converter(self, format=None):
-        """Return the converter to use to convert bzrdirs needing converts.
-
-        This returns a bzrlib.bzrdir.Converter object.
-
-        This should return the best upgrader to step this format towards the
-        current default format. In the case of plugins we can/should provide
-        some means for them to extend the range of returnable converters.
-
-        :param format: Optional format to override the default format of the
-                       library.
-        """
-        raise NotImplementedError(self.get_converter)
-
-    def initialize(self, url, possible_transports=None):
-        """Create a bzr control dir at this url and return an opened copy.
-
-        While not deprecated, this method is very specific and its use will
-        lead to many round trips to setup a working environment. See
-        initialize_on_transport_ex for a [nearly] all-in-one method.
-
-        Subclasses should typically override initialize_on_transport
-        instead of this method.
-        """
-        return self.initialize_on_transport(get_transport(url,
-                                                          possible_transports))
-
     def initialize_on_transport(self, transport):
         """Initialize a new bzrdir in the base directory of a Transport."""
         try:
@@ -2141,7 +1669,7 @@
         utf8_files = [('README',
                        "This is a Bazaar control directory.\n"
                        "Do not change any files in this directory.\n"
-                       "See http://bazaar-vcs.org/ for more information about Bazaar.\n"),
+                       "See http://bazaar.canonical.com/ for more information about Bazaar.\n"),
                       ('branch-format', self.get_format_string()),
                       ]
         # NB: no need to escape relative paths that are url safe.
@@ -2157,55 +1685,13 @@
             control_files.unlock()
         return self.open(transport, _found=True)
 
-    def is_supported(self):
-        """Is this format supported?
-
-        Supported formats must be initializable and openable.
-        Unsupported formats may not support initialization or committing or
-        some other features depending on the reason for not being supported.
-        """
-        return True
-
-    def network_name(self):
-        """A simple byte string uniquely identifying this format for RPC calls.
-
-        Bzr control formats use thir disk format string to identify the format
-        over the wire. Its possible that other control formats have more
-        complex detection requirements, so we permit them to use any unique and
-        immutable string they desire.
-        """
-        raise NotImplementedError(self.network_name)
-
-    def same_model(self, target_format):
-        return (self.repository_format.rich_root_data ==
-            target_format.rich_root_data)
-
-    @classmethod
-    def known_formats(klass):
-        """Return all the known formats.
-
-        Concrete formats should override _known_formats.
-        """
-        # There is double indirection here to make sure that control
-        # formats used by more than one dir format will only be probed
-        # once. This can otherwise be quite expensive for remote connections.
-        result = set()
-        for format in klass._control_formats:
-            result.update(format._known_formats())
-        return result
-
-    @classmethod
-    def _known_formats(klass):
-        """Return the known format instances for this control format."""
-        return set(klass._formats.values())
-
     def open(self, transport, _found=False):
         """Return an instance of this format for the dir transport points at.
 
         _found is a private parameter, do not use it.
         """
         if not _found:
-            found_format = BzrDirFormat.find_format(transport)
+            found_format = controldir.ControlDirFormat.find_format(transport)
             if not isinstance(found_format, self.__class__):
                 raise AssertionError("%s was asked to open %s, but it seems to need "
                         "format %s"
@@ -2225,40 +1711,10 @@
 
     @classmethod
     def register_format(klass, format):
-        klass._formats[format.get_format_string()] = format
+        BzrProber.register_bzrdir_format(format)
         # bzr native formats have a network name of their format string.
-        network_format_registry.register(format.get_format_string(), format.__class__)
-
-    @classmethod
-    def register_control_format(klass, format):
-        """Register a format that does not use '.bzr' for its control dir.
-
-        TODO: This should be pulled up into a 'ControlDirFormat' base class
-        which BzrDirFormat can inherit from, and renamed to register_format
-        there. It has been done without that for now for simplicity of
-        implementation.
-        """
-        klass._control_formats.append(format)
-
-    @classmethod
-    def register_control_server_format(klass, format):
-        """Register a control format for client-server environments.
-
-        These formats will be tried before ones registered with
-        register_control_format.  This gives implementations that decide to the
-        chance to grab it before anything looks at the contents of the format
-        file.
-        """
-        klass._control_server_formats.append(format)
-
-    @classmethod
-    def _set_default_format(klass, format):
-        """Set default format (for testing behavior of defaults only)"""
-        klass._default_format = format
-
-    def __str__(self):
-        # Trim the newline
-        return self.get_format_description().rstrip()
+        controldir.network_format_registry.register(format.get_format_string(), format.__class__)
+        controldir.ControlDirFormat.register_format(format)
 
     def _supply_sub_formats_to(self, other_format):
         """Give other_format the same values for sub formats as this has.
@@ -2274,11 +1730,9 @@
 
     @classmethod
     def unregister_format(klass, format):
-        del klass._formats[format.get_format_string()]
-
-    @classmethod
-    def unregister_control_format(klass, format):
-        klass._control_formats.remove(format)
+        BzrProber.unregister_bzrdir_format(format)
+        controldir.ControlDirFormat.unregister_format(format)
+        controldir.network_format_registry.remove(format.get_format_string())
 
 
 class BzrDirFormat4(BzrDirFormat):
@@ -2392,7 +1846,8 @@
         return ConvertBzrDir5To6()
 
     def _initialize_for_clone(self, url):
-        return self.initialize_on_transport(get_transport(url), _cloning=True)
+        return self.initialize_on_transport(
+            _mod_transport.get_transport(url), _cloning=True)
 
     def initialize_on_transport(self, transport, _cloning=False):
         """Format 5 dirs always have working tree, branch and repository.
@@ -2452,7 +1907,8 @@
         return ConvertBzrDir6ToMeta()
 
     def _initialize_for_clone(self, url):
-        return self.initialize_on_transport(get_transport(url), _cloning=True)
+        return self.initialize_on_transport(
+            _mod_transport.get_transport(url), _cloning=True)
 
     def initialize_on_transport(self, transport, _cloning=False):
         """Format 6 dirs always have working tree, branch and repository.
@@ -2687,25 +2143,13 @@
                                   __set_workingtree_format)
 
 
-network_format_registry = registry.FormatRegistry()
-"""Registry of formats indexed by their network name.
-
-The network name for a BzrDirFormat is an identifier that can be used when
-referring to formats with smart server operations. See
-BzrDirFormat.network_name() for more detail.
-"""
-
-
-# Register bzr control format
-BzrDirFormat.register_control_format(BzrDirFormat)
-
 # Register bzr formats
 BzrDirFormat.register_format(BzrDirFormat4())
 BzrDirFormat.register_format(BzrDirFormat5())
 BzrDirFormat.register_format(BzrDirFormat6())
 __default_format = BzrDirMetaFormat1()
 BzrDirFormat.register_format(__default_format)
-BzrDirFormat._default_format = __default_format
+controldir.ControlDirFormat._default_format = __default_format
 
 
 class Converter(object):
@@ -2813,9 +2257,11 @@
             mode=self.bzrdir._get_file_mode())
 
     def _write_all_weaves(self):
-        controlweaves = WeaveStore(self.bzrdir.transport, prefixed=False)
+        controlweaves = VersionedFileStore(self.bzrdir.transport, prefixed=False,
+                                           versionedfile_class=WeaveFile)
         weave_transport = self.bzrdir.transport.clone('weaves')
-        weaves = WeaveStore(weave_transport, prefixed=False)
+        weaves = VersionedFileStore(weave_transport, prefixed=False,
+                            versionedfile_class=WeaveFile)
         transaction = WriteTransaction()
 
         try:
@@ -2939,7 +2385,6 @@
         previous_entries = dict((head, parent_candiate_entries[head]) for head
             in heads)
         self.snapshot_ie(previous_entries, ie, w, rev_id)
-        del ie.text_id
 
     def get_parent_map(self, revision_ids):
         """See graph.StackedParentsProvider.get_parent_map"""
@@ -3247,6 +2692,8 @@
 class RemoteBzrDirFormat(BzrDirMetaFormat1):
     """Format representing bzrdirs accessed via a smart server"""
 
+    supports_workingtrees = False
+
     def __init__(self):
         BzrDirMetaFormat1.__init__(self)
         # XXX: It's a bit ugly that the network name is here, because we'd
@@ -3261,7 +2708,7 @@
 
     def get_format_description(self):
         if self._network_name:
-            real_format = network_format_registry.get(self._network_name)
+            real_format = controldir.network_format_registry.get(self._network_name)
             return 'Remote: ' + real_format.get_format_description()
         return 'bzr remote bzrdir'
 
@@ -3274,30 +2721,6 @@
         else:
             raise AssertionError("No network name set.")
 
-    @classmethod
-    def probe_transport(klass, transport):
-        """Return a RemoteBzrDirFormat object if it looks possible."""
-        try:
-            medium = transport.get_smart_medium()
-        except (NotImplementedError, AttributeError,
-                errors.TransportNotPossible, errors.NoSmartMedium,
-                errors.SmartProtocolError):
-            # no smart server, so not a branch for this format type.
-            raise errors.NotBranchError(path=transport.base)
-        else:
-            # Decline to open it if the server doesn't support our required
-            # version (3) so that the VFS-based transport will do it.
-            if medium.should_probe():
-                try:
-                    server_version = medium.protocol_version()
-                except errors.SmartProtocolError:
-                    # Apparently there's no usable smart server there, even though
-                    # the medium supports the smart protocol.
-                    raise errors.NotBranchError(path=transport.base)
-                if server_version != '2':
-                    raise errors.NotBranchError(path=transport.base)
-            return klass()
-
     def initialize_on_transport(self, transport):
         try:
             # hand off the request to the smart server
@@ -3502,190 +2925,7 @@
         BzrDirMetaFormat1._set_repository_format) #.im_func)
 
 
-BzrDirFormat.register_control_server_format(RemoteBzrDirFormat)
-
-
-class BzrDirFormatInfo(object):
-
-    def __init__(self, native, deprecated, hidden, experimental):
-        self.deprecated = deprecated
-        self.native = native
-        self.hidden = hidden
-        self.experimental = experimental
-
-
-class BzrDirFormatRegistry(registry.Registry):
-    """Registry of user-selectable BzrDir subformats.
-
-    Differs from BzrDirFormat._control_formats in that it provides sub-formats,
-    e.g. BzrDirMeta1 with weave repository.  Also, it's more user-oriented.
-    """
-
-    def __init__(self):
-        """Create a BzrDirFormatRegistry."""
-        self._aliases = set()
-        self._registration_order = list()
-        super(BzrDirFormatRegistry, self).__init__()
-
-    def aliases(self):
-        """Return a set of the format names which are aliases."""
-        return frozenset(self._aliases)
-
-    def register_metadir(self, key,
-             repository_format, help, native=True, deprecated=False,
-             branch_format=None,
-             tree_format=None,
-             hidden=False,
-             experimental=False,
-             alias=False):
-        """Register a metadir subformat.
-
-        These all use a BzrDirMetaFormat1 bzrdir, but can be parameterized
-        by the Repository/Branch/WorkingTreeformats.
-
-        :param repository_format: The fully-qualified repository format class
-            name as a string.
-        :param branch_format: Fully-qualified branch format class name as
-            a string.
-        :param tree_format: Fully-qualified tree format class name as
-            a string.
-        """
-        # This should be expanded to support setting WorkingTree and Branch
-        # formats, once BzrDirMetaFormat1 supports that.
-        def _load(full_name):
-            mod_name, factory_name = full_name.rsplit('.', 1)
-            try:
-                mod = __import__(mod_name, globals(), locals(),
-                        [factory_name])
-            except ImportError, e:
-                raise ImportError('failed to load %s: %s' % (full_name, e))
-            try:
-                factory = getattr(mod, factory_name)
-            except AttributeError:
-                raise AttributeError('no factory %s in module %r'
-                    % (full_name, mod))
-            return factory()
-
-        def helper():
-            bd = BzrDirMetaFormat1()
-            if branch_format is not None:
-                bd.set_branch_format(_load(branch_format))
-            if tree_format is not None:
-                bd.workingtree_format = _load(tree_format)
-            if repository_format is not None:
-                bd.repository_format = _load(repository_format)
-            return bd
-        self.register(key, helper, help, native, deprecated, hidden,
-            experimental, alias)
-
-    def register(self, key, factory, help, native=True, deprecated=False,
-                 hidden=False, experimental=False, alias=False):
-        """Register a BzrDirFormat factory.
-
-        The factory must be a callable that takes one parameter: the key.
-        It must produce an instance of the BzrDirFormat when called.
-
-        This function mainly exists to prevent the info object from being
-        supplied directly.
-        """
-        registry.Registry.register(self, key, factory, help,
-            BzrDirFormatInfo(native, deprecated, hidden, experimental))
-        if alias:
-            self._aliases.add(key)
-        self._registration_order.append(key)
-
-    def register_lazy(self, key, module_name, member_name, help, native=True,
-        deprecated=False, hidden=False, experimental=False, alias=False):
-        registry.Registry.register_lazy(self, key, module_name, member_name,
-            help, BzrDirFormatInfo(native, deprecated, hidden, experimental))
-        if alias:
-            self._aliases.add(key)
-        self._registration_order.append(key)
-
-    def set_default(self, key):
-        """Set the 'default' key to be a clone of the supplied key.
-
-        This method must be called once and only once.
-        """
-        registry.Registry.register(self, 'default', self.get(key),
-            self.get_help(key), info=self.get_info(key))
-        self._aliases.add('default')
-
-    def set_default_repository(self, key):
-        """Set the FormatRegistry default and Repository default.
-
-        This is a transitional method while Repository.set_default_format
-        is deprecated.
-        """
-        if 'default' in self:
-            self.remove('default')
-        self.set_default(key)
-        format = self.get('default')()
-
-    def make_bzrdir(self, key):
-        return self.get(key)()
-
-    def help_topic(self, topic):
-        output = ""
-        default_realkey = None
-        default_help = self.get_help('default')
-        help_pairs = []
-        for key in self._registration_order:
-            if key == 'default':
-                continue
-            help = self.get_help(key)
-            if help == default_help:
-                default_realkey = key
-            else:
-                help_pairs.append((key, help))
-
-        def wrapped(key, help, info):
-            if info.native:
-                help = '(native) ' + help
-            return ':%s:\n%s\n\n' % (key,
-                textwrap.fill(help, initial_indent='    ',
-                    subsequent_indent='    ',
-                    break_long_words=False))
-        if default_realkey is not None:
-            output += wrapped(default_realkey, '(default) %s' % default_help,
-                              self.get_info('default'))
-        deprecated_pairs = []
-        experimental_pairs = []
-        for key, help in help_pairs:
-            info = self.get_info(key)
-            if info.hidden:
-                continue
-            elif info.deprecated:
-                deprecated_pairs.append((key, help))
-            elif info.experimental:
-                experimental_pairs.append((key, help))
-            else:
-                output += wrapped(key, help, info)
-        output += "\nSee :doc:`formats-help` for more about storage formats."
-        other_output = ""
-        if len(experimental_pairs) > 0:
-            other_output += "Experimental formats are shown below.\n\n"
-            for key, help in experimental_pairs:
-                info = self.get_info(key)
-                other_output += wrapped(key, help, info)
-        else:
-            other_output += \
-                "No experimental formats are available.\n\n"
-        if len(deprecated_pairs) > 0:
-            other_output += "\nDeprecated formats are shown below.\n\n"
-            for key, help in deprecated_pairs:
-                info = self.get_info(key)
-                other_output += wrapped(key, help, info)
-        else:
-            other_output += \
-                "\nNo deprecated formats are available.\n\n"
-        other_output += \
-                "\nSee :doc:`formats-help` for more about storage formats."
-
-        if topic == 'other-formats':
-            return other_output
-        else:
-            return output
+controldir.ControlDirFormat.register_server_prober(RemoteBzrProber)
 
 
 class RepositoryAcquisitionPolicy(object):
@@ -3845,33 +3085,73 @@
         return self._repository, False
 
 
-# Please register new formats after old formats so that formats
-# appear in chronological order and format descriptions can build
-# on previous ones.
-format_registry = BzrDirFormatRegistry()
+def register_metadir(registry, key,
+         repository_format, help, native=True, deprecated=False,
+         branch_format=None,
+         tree_format=None,
+         hidden=False,
+         experimental=False,
+         alias=False):
+    """Register a metadir subformat.
+
+    These all use a BzrDirMetaFormat1 bzrdir, but can be parameterized
+    by the Repository/Branch/WorkingTreeformats.
+
+    :param repository_format: The fully-qualified repository format class
+        name as a string.
+    :param branch_format: Fully-qualified branch format class name as
+        a string.
+    :param tree_format: Fully-qualified tree format class name as
+        a string.
+    """
+    # This should be expanded to support setting WorkingTree and Branch
+    # formats, once BzrDirMetaFormat1 supports that.
+    def _load(full_name):
+        mod_name, factory_name = full_name.rsplit('.', 1)
+        try:
+            factory = pyutils.get_named_object(mod_name, factory_name)
+        except ImportError, e:
+            raise ImportError('failed to load %s: %s' % (full_name, e))
+        except AttributeError:
+            raise AttributeError('no factory %s in module %r'
+                % (full_name, sys.modules[mod_name]))
+        return factory()
+
+    def helper():
+        bd = BzrDirMetaFormat1()
+        if branch_format is not None:
+            bd.set_branch_format(_load(branch_format))
+        if tree_format is not None:
+            bd.workingtree_format = _load(tree_format)
+        if repository_format is not None:
+            bd.repository_format = _load(repository_format)
+        return bd
+    registry.register(key, helper, help, native, deprecated, hidden,
+        experimental, alias)
+
 # The pre-0.8 formats have their repository format network name registered in
 # repository.py. MetaDir formats have their repository format network name
 # inferred from their disk format string.
-format_registry.register('weave', BzrDirFormat6,
+controldir.format_registry.register('weave', BzrDirFormat6,
     'Pre-0.8 format.  Slower than knit and does not'
     ' support checkouts or shared repositories.',
     hidden=True,
     deprecated=True)
-format_registry.register_metadir('metaweave',
+register_metadir(controldir.format_registry, 'metaweave',
     'bzrlib.repofmt.weaverepo.RepositoryFormat7',
     'Transitional format in 0.8.  Slower than knit.',
     branch_format='bzrlib.branch.BzrBranchFormat5',
     tree_format='bzrlib.workingtree.WorkingTreeFormat3',
     hidden=True,
     deprecated=True)
-format_registry.register_metadir('knit',
+register_metadir(controldir.format_registry, 'knit',
     'bzrlib.repofmt.knitrepo.RepositoryFormatKnit1',
     'Format using knits.  Recommended for interoperation with bzr <= 0.14.',
     branch_format='bzrlib.branch.BzrBranchFormat5',
     tree_format='bzrlib.workingtree.WorkingTreeFormat3',
     hidden=True,
     deprecated=True)
-format_registry.register_metadir('dirstate',
+register_metadir(controldir.format_registry, 'dirstate',
     'bzrlib.repofmt.knitrepo.RepositoryFormatKnit1',
     help='New in 0.15: Fast local operations. Compatible with bzr 0.8 and '
         'above when accessed over the network.',
@@ -3881,7 +3161,7 @@
     tree_format='bzrlib.workingtree.WorkingTreeFormat4',
     hidden=True,
     deprecated=True)
-format_registry.register_metadir('dirstate-tags',
+register_metadir(controldir.format_registry, 'dirstate-tags',
     'bzrlib.repofmt.knitrepo.RepositoryFormatKnit1',
     help='New in 0.15: Fast local operations and improved scaling for '
         'network operations. Additionally adds support for tags.'
@@ -3890,7 +3170,7 @@
     tree_format='bzrlib.workingtree.WorkingTreeFormat4',
     hidden=True,
     deprecated=True)
-format_registry.register_metadir('rich-root',
+register_metadir(controldir.format_registry, 'rich-root',
     'bzrlib.repofmt.knitrepo.RepositoryFormatKnit4',
     help='New in 1.0.  Better handling of tree roots.  Incompatible with'
         ' bzr < 1.0.',
@@ -3898,7 +3178,7 @@
     tree_format='bzrlib.workingtree.WorkingTreeFormat4',
     hidden=True,
     deprecated=True)
-format_registry.register_metadir('dirstate-with-subtree',
+register_metadir(controldir.format_registry, 'dirstate-with-subtree',
     'bzrlib.repofmt.knitrepo.RepositoryFormatKnit3',
     help='New in 0.15: Fast local operations and improved scaling for '
         'network operations. Additionally adds support for versioning nested '
@@ -3908,7 +3188,7 @@
     experimental=True,
     hidden=True,
     )
-format_registry.register_metadir('pack-0.92',
+register_metadir(controldir.format_registry, 'pack-0.92',
     'bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack1',
     help='New in 0.92: Pack-based format with data compatible with '
         'dirstate-tags format repositories. Interoperates with '
@@ -3917,7 +3197,7 @@
     branch_format='bzrlib.branch.BzrBranchFormat6',
     tree_format='bzrlib.workingtree.WorkingTreeFormat4',
     )
-format_registry.register_metadir('pack-0.92-subtree',
+register_metadir(controldir.format_registry, 'pack-0.92-subtree',
     'bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack3',
     help='New in 0.92: Pack-based format with data compatible with '
         'dirstate-with-subtree format repositories. Interoperates with '
@@ -3928,7 +3208,7 @@
     hidden=True,
     experimental=True,
     )
-format_registry.register_metadir('rich-root-pack',
+register_metadir(controldir.format_registry, 'rich-root-pack',
     'bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack4',
     help='New in 1.0: A variant of pack-0.92 that supports rich-root data '
          '(needed for bzr-svn and bzr-git).',
@@ -3936,7 +3216,7 @@
     tree_format='bzrlib.workingtree.WorkingTreeFormat4',
     hidden=True,
     )
-format_registry.register_metadir('1.6',
+register_metadir(controldir.format_registry, '1.6',
     'bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack5',
     help='A format that allows a branch to indicate that there is another '
          '(stacked) repository that should be used to access data that is '
@@ -3945,7 +3225,7 @@
     tree_format='bzrlib.workingtree.WorkingTreeFormat4',
     hidden=True,
     )
-format_registry.register_metadir('1.6.1-rich-root',
+register_metadir(controldir.format_registry, '1.6.1-rich-root',
     'bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack5RichRoot',
     help='A variant of 1.6 that supports rich-root data '
          '(needed for bzr-svn and bzr-git).',
@@ -3953,7 +3233,7 @@
     tree_format='bzrlib.workingtree.WorkingTreeFormat4',
     hidden=True,
     )
-format_registry.register_metadir('1.9',
+register_metadir(controldir.format_registry, '1.9',
     'bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack6',
     help='A repository format using B+tree indexes. These indexes '
          'are smaller in size, have smarter caching and provide faster '
@@ -3962,7 +3242,7 @@
     tree_format='bzrlib.workingtree.WorkingTreeFormat4',
     hidden=True,
     )
-format_registry.register_metadir('1.9-rich-root',
+register_metadir(controldir.format_registry, '1.9-rich-root',
     'bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack6RichRoot',
     help='A variant of 1.9 that supports rich-root data '
          '(needed for bzr-svn and bzr-git).',
@@ -3970,13 +3250,13 @@
     tree_format='bzrlib.workingtree.WorkingTreeFormat4',
     hidden=True,
     )
-format_registry.register_metadir('1.14',
+register_metadir(controldir.format_registry, '1.14',
     'bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack6',
     help='A working-tree format that supports content filtering.',
     branch_format='bzrlib.branch.BzrBranchFormat7',
     tree_format='bzrlib.workingtree.WorkingTreeFormat5',
     )
-format_registry.register_metadir('1.14-rich-root',
+register_metadir(controldir.format_registry, '1.14-rich-root',
     'bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack6RichRoot',
     help='A variant of 1.14 that supports rich-root data '
          '(needed for bzr-svn and bzr-git).',
@@ -3984,22 +3264,8 @@
     tree_format='bzrlib.workingtree.WorkingTreeFormat5',
     )
 # The following un-numbered 'development' formats should always just be aliases.
-format_registry.register_metadir('development-rich-root',
-    'bzrlib.repofmt.groupcompress_repo.RepositoryFormatCHK1',
-    help='Current development format. Supports rich roots. Can convert data '
-        'to and from rich-root-pack (and anything compatible with '
-        'rich-root-pack) format repositories. Repositories and branches in '
-        'this format can only be read by bzr.dev. Please read '
-        'http://doc.bazaar.canonical.com/latest/developers/development-repo.html '
-        'before use.',
-    branch_format='bzrlib.branch.BzrBranchFormat7',
-    tree_format='bzrlib.workingtree.WorkingTreeFormat6',
-    experimental=True,
-    alias=True,
-    hidden=True,
-    )
-format_registry.register_metadir('development-subtree',
-    'bzrlib.repofmt.pack_repo.RepositoryFormatPackDevelopment2Subtree',
+register_metadir(controldir.format_registry, 'development-subtree',
+    'bzrlib.repofmt.groupcompress_repo.RepositoryFormat2aSubtree',
     help='Current development format, subtree variant. Can convert data to and '
         'from pack-0.92-subtree (and anything compatible with '
         'pack-0.92-subtree) format repositories. Repositories and branches in '
@@ -4014,33 +3280,25 @@
                  # This current non-alias status is simply because we did not introduce a
                  # chk based subtree format.
     )
+register_metadir(controldir.format_registry, 'development5-subtree',
+    'bzrlib.repofmt.pack_repo.RepositoryFormatPackDevelopment2Subtree',
+    help='Development format, subtree variant. Can convert data to and '
+        'from pack-0.92-subtree (and anything compatible with '
+        'pack-0.92-subtree) format repositories. Repositories and branches in '
+        'this format can only be read by bzr.dev. Please read '
+        'http://doc.bazaar.canonical.com/latest/developers/development-repo.html '
+        'before use.',
+    branch_format='bzrlib.branch.BzrBranchFormat7',
+    tree_format='bzrlib.workingtree.WorkingTreeFormat6',
+    experimental=True,
+    hidden=True,
+    alias=False,
+    )
 
 # And the development formats above will have aliased one of the following:
-format_registry.register_metadir('development6-rich-root',
-    'bzrlib.repofmt.groupcompress_repo.RepositoryFormatCHK1',
-    help='pack-1.9 with 255-way hashed CHK inv, group compress, rich roots '
-        'Please read '
-        'http://doc.bazaar.canonical.com/latest/developers/development-repo.html '
-        'before use.',
-    branch_format='bzrlib.branch.BzrBranchFormat7',
-    tree_format='bzrlib.workingtree.WorkingTreeFormat6',
-    hidden=True,
-    experimental=True,
-    )
-
-format_registry.register_metadir('development7-rich-root',
-    'bzrlib.repofmt.groupcompress_repo.RepositoryFormatCHK2',
-    help='pack-1.9 with 255-way hashed CHK inv, bencode revision, group compress, '
-        'rich roots. Please read '
-        'http://doc.bazaar.canonical.com/latest/developers/development-repo.html '
-        'before use.',
-    branch_format='bzrlib.branch.BzrBranchFormat7',
-    tree_format='bzrlib.workingtree.WorkingTreeFormat6',
-    hidden=True,
-    experimental=True,
-    )
-
-format_registry.register_metadir('2a',
+
+# Finally, the current format.
+register_metadir(controldir.format_registry, '2a',
     'bzrlib.repofmt.groupcompress_repo.RepositoryFormat2a',
     help='First format for bzr 2.0 series.\n'
         'Uses group-compress storage.\n'
@@ -4049,12 +3307,12 @@
         # 'rich roots. Supported by bzr 1.16 and later.',
     branch_format='bzrlib.branch.BzrBranchFormat7',
     tree_format='bzrlib.workingtree.WorkingTreeFormat6',
-    experimental=True,
+    experimental=False,
     )
 
 # The following format should be an alias for the rich root equivalent 
 # of the default format