Merge lp:~vila/bzr/389648-hook-calls-base into lp:~bzr/bzr/trunk-old

[Vincent Ladeuil]

[Vincent Ladeuil (vila)]

I finalized a patch proposed by bialix with a more focused test.
Like bialix tests it catches the two actual bugs in bzr but in addition,
it will catch any such bugs in plugins.

[Alexander Belchenko (bialix)]

Something wrong with the diff.

[Aaron Bentley (abentley)]

> Something wrong with the diff.

I think you want to propose against lp:~bzr-pqm/bzr/bzr.dev, not lp:~bzr/bzr/trunk, which is a 1.9-format mirror from before the 2a upgrade.

[John A Meinel (jameinel)]

As mentioned above, the target was wrong, it should be 'lp:bzr' which is ~bzr-pqm/bzr/bzr.dev, not ~bzr/bzr/trunk.

Anyway, I did the review locally. It was still a bit hard to follow, because a lot of 'formatting' updates. Stuff like:

-class TestHook(TestCase):
+class TestHook(tests.TestCase):

I wonder if this wouldn't be like the whitespace fixes, and we should have "one big patch" across the codebase to fix them, rather than getting them piecemeal. Then again, the same arguments apply for and against that were present for the whitespace updates...

Anyway, I think the basic fix here was:

1) Add
+ super(InfoHooks, self).__init__()

To a couple of the hook instances

2) Add
+ self.assertEqual("No hook name", new_hooks.get_hook_name(None))
To the tests to ensure that the internal dictionary is created and able to return a value for an unnamed hook.

Which seems just fine.

I should comment that using "super()" is not always correct anymore. To start with, it only matters when you have multiple inheritance. But even more importantly, python 2.6 'broke' super(...).__init__() if there are any arguments. Because object.__init__() no longer allows arguments. (So as near as I can tell, there is no way to safely __init__ a multiple inheritance structure that wants arguments passed as part of init. Unless maybe you do something like use a single abstract base class that all classes inherit from, that *doesn't* call super?)

Anyway, IMO super().__init__() was broken in 2.6, though certainly someone thought it was the right thing to do.

review: Approve

[Vincent Ladeuil (vila)]

>>>>> "jam" == John A Meinel <email address hidden> writes:

    jam> Review: Approve
    jam> As mentioned above, the target was wrong, it should be 'lp:bzr'
    jam> which is ~bzr-pqm/bzr/bzr.dev, not ~bzr/bzr/trunk.

Amazing, obviously I clicked the wrong button... no idea why nor
when... that wasn't intended anyway.

    jam> Anyway, I did the review locally. It was still a bit hard to
    jam> follow, because a lot of 'formatting' updates. Stuff like:

    jam> -class TestHook(TestCase):
    jam> +class TestHook(tests.TestCase):

Yeah :-(

In the past I've sent these trivial updates directly without
review, but then you commented that may be I should at least send
them so that people know about them even without waiting for an
approval.

I tried a middle term approach here, because I thought there was
only a few of them (but you call them 'a lot' :-)...

I don't mind fixing them as I encounter them, I don't want to
make the reviews harder than they have to either, but I'd like a
minimal effort workflow to get these fixes landed too.

    jam> I wonder if this wouldn't be like the whitespace fixes, and we
    jam> should have "one big patch" across the codebase to fix them,
    jam> rather than getting them piecemeal. Then again, the same
    jam> arguments apply for and against that were present for the
    jam> whitespace updates...

Yeah, exactly the same arguments. But again, I feel that I spend
less energy fixing them that discussing how to fix them all,
so...

    jam> Anyway, I think the basic fix here was:

You're right.

<snip/>

    jam> Which seems just fine.

Ok.

    jam> I should comment that using "super()" is not always
    jam> correct anymore.

    jam> To start with, it only matters when you have multiple
    jam> inheritance. But even more importantly, python 2.6
    jam> 'broke' super(...).__init__() if there are any
    jam> arguments.

I disagree. python-2.6 is just stricter and catch bogus calls.

    jam> Because object.__init__() no longer allows
    jam> arguments. (So as near as I can tell, there is no way to
    jam> safely __init__ a multiple inheritance structure that
    jam> wants arguments passed as part of init.

Yes there is, you handle arguments where they are expected and
you don't pass them where they are not.

The cases I had to fix when I addressed python-2.6 compatibility
weren't that complex once I understood what was going on.

And as I remember the only case that seemed controversial at the
time was indeed an abuse.

<snip/>

    jam> Anyway, IMO super().__init__() was broken in 2.6, though
    jam> certainly someone thought it was the right thing to do.

Well, it would be good to agree on that and decide whether we
want to use super() or not.

I refrained to fix all calls to Hooks.__init__ because I
considered it was slightly out of scope for this patch but only
slightly.

If you can provide examples in the actual code base where calling
super().__init__ will be bogus, I'll be happy to revise my
opinion.

        Vincent

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

...
>
> jam> I should comment that using "super()" is not always
> jam> correct anymore.
>
>
> jam> To start with, it only matters when you have multiple
> jam> inheritance. But even more importantly, python 2.6
> jam> 'broke' super(...).__init__() if there are any
> jam> arguments.
>
> I disagree. python-2.6 is just stricter and catch bogus calls.
>
> jam> Because object.__init__() no longer allows
> jam> arguments. (So as near as I can tell, there is no way to
> jam> safely __init__ a multiple inheritance structure that
> jam> wants arguments passed as part of init.
>
> Yes there is, you handle arguments where they are expected and
> you don't pass them where they are not.
>
> The cases I had to fix when I addressed python-2.6 compatibility
> weren't that complex once I understood what was going on.
>
> And as I remember the only case that seemed controversial at the
> time was indeed an abuse.

By the way, don't block landing this on us resolving super() issues. I'm
moving the super() discussion over to the regular mailing list.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkr10xMACgkQJdeBCYSNAANU8QCdGUx5dQkzaNrZV3f3p0SAQNj5
q1MAoM5oqOsF0jXtbfuy7hNYgOC1kV13
=uKqH
-----END PGP SIGNATURE-----

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm trying to split up a few discussion points into different emails.
Sorry if things get a bit broken. Thunderbird isn't as nice as 'bzr
shelve' :)

Vincent Ladeuil wrote:
>>>>>> "jam" == John A Meinel <email address hidden> writes:
>
> jam> Review: Approve
> jam> As mentioned above, the target was wrong, it should be 'lp:bzr'
> jam> which is ~bzr-pqm/bzr/bzr.dev, not ~bzr/bzr/trunk.
>
> Amazing, obviously I clicked the wrong button... no idea why nor
> when... that wasn't intended anyway.
>
> jam> Anyway, I did the review locally. It was still a bit hard to
> jam> follow, because a lot of 'formatting' updates. Stuff like:
>
> jam> -class TestHook(TestCase):
> jam> +class TestHook(tests.TestCase):
>
> Yeah :-(
>
> In the past I've sent these trivial updates directly without
> review, but then you commented that may be I should at least send
> them so that people know about them even without waiting for an
> approval.
>
> I tried a middle term approach here, because I thought there was
> only a few of them (but you call them 'a lot' :-)...

Your fix was about 2 lines long, and it was about 10 lines of
'tests.TestCase' changes.

so not many, but *more* of those trivial changes than there were actual
content changes.

>
> I don't mind fixing them as I encounter them, I don't want to
> make the reviews harder than they have to either, but I'd like a
> minimal effort workflow to get these fixes landed too.
>
> jam> I wonder if this wouldn't be like the whitespace fixes, and we
> jam> should have "one big patch" across the codebase to fix them,
> jam> rather than getting them piecemeal. Then again, the same
> jam> arguments apply for and against that were present for the
> jam> whitespace updates...
>
> Yeah, exactly the same arguments. But again, I feel that I spend
> less energy fixing them that discussing how to fix them all,
> so...

Style patches are always a bit contentious. Far to easy for people to
bikeshed on. Which is why we say "PEP 8" but even that doesn't cover all
the cases of things you want to do.

Similarly stripping the whitespace from all files is trivially easy to
do. (We now have about 300? files that have trailing whitespace again.)

I'd like to say just land trivial cleanups. But on the flip side, your
'trivial cleanups' may cause merge conflicts in any pending work people
are doing. However, we don't want to hold cleaning up the code hostage
for work that may land in the future either.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkr108YACgkQJdeBCYSNAANHTwCgxsrLQOM7+0sySJtYKRW2NSaI
QooAnRk6sP/DUSgjDQjf2+n8NDgKzU7A
=OUbh
-----END PGP SIGNATURE-----

=== modified file '.bzrignore'
--- .bzrignore	2009-07-31 19:56:19 +0000
+++ .bzrignore	2009-11-03 09:40:55 +0000
@@ -5,10 +5,21 @@
 ./.bzr.log
 # Generated files
 CHANGELOG
+# generated documents
 bzr.1
 ./doc/*.html
-./doc/_build/
+./doc/*/_build/
+./doc/*/Makefile
+./doc/*/make.bat
 ./tutorial.html
+./build_doc_website
+./html_docs
+./pretty_docs
+./api
+./doc/**/*.html
+./doc/developers/performance.png
+./doc/en/user-reference/*.txt
+./doc/en/release-notes/*.txt
 BRANCH-INFO
 # setup.py working directory
 ./build
@@ -35,11 +46,7 @@
 ./dist
 # performance history data file
 ./.perf_history
-./html_docs
-./pretty_docs
-./api
-doc/**/*.html
-doc/developers/performance.png
+# Pyrex
 bzrlib/_annotator_pyx.c
 bzrlib/_bencode_pyx.c
 bzrlib/_btree_serializer_pyx.c
@@ -51,16 +58,12 @@
 bzrlib/_known_graph_pyx.c
 bzrlib/_readdir_pyx.c
 bzrlib/_rio_pyx.c
+bzrlib/_simple_set_pyx.c
+bzrlib/_simple_set_pyx.h
+bzrlib/_simple_set_pyx_api.h
 bzrlib/_walkdirs_win32.c
-doc/en/release-notes/NEWS.txt
-doc/en/developer-guide/HACKING.txt
-doc/en/user-reference/bzr_man.txt
 # built extension modules
 bzrlib/_*.dll
 bzrlib/_*.so
 bzrlib/_*.pyd
-# generated help topics
-doc/en/user-reference/*.txt
-./doc/en/user-guide/latex_prepared
-./doc/en/user-guide/*.pdf
 ./.ccache
 
=== modified file 'Makefile'
--- Makefile	2009-08-27 00:53:27 +0000
+++ Makefile	2009-11-03 09:40:55 +0000
@@ -75,44 +75,161 @@
 # these are treated as phony so they'll always be rebuilt - it's pretty quick
 .PHONY: TAGS tags
 
+
 ### Documentation ###
 
-# set PRETTY to get docs that look like the Bazaar web site
-ifdef PRETTY
-rst2html := $(PYTHON) tools/rst2prettyhtml.py doc/bazaar-vcs.org.kid 
-else
-rst2html := $(PYTHON) tools/rst2html.py --link-stylesheet --footnote-references=superscript --halt=warning
-endif
+# Default to plain documentation for maximum backwards compatibility.
+# (Post 2.0, the defaults will most likely be Sphinx-style instead.)
+
+docs: docs-plain
+
+clean-docs: clean-plain
+
+html-docs: html-plain
+
+
+### Man-page Documentation ###
+
+MAN_DEPENDENCIES = bzrlib/builtins.py \
+	$(wildcard bzrlib/*.py) \
+	$(wildcard bzrlib/*/*.py) \
+	tools/generate_docs.py \
+	$(wildcard $(addsuffix /*.txt, bzrlib/help_topics/en)) 
+
+MAN_PAGES = man1/bzr.1
+man1/bzr.1: $(MAN_DEPENDENCIES)
+	$(PYTHON) tools/generate_docs.py -o $@ man
+
+
+### Sphinx-style Documentation ###
+
+# Build the documentation. To keep the dependencies down to a minimum
+# for distro packagers, we only build the html documentation by default.
+# Sphinx 0.6 or later is preferred for the best rendering, though
+# Sphinx 0.4 or later should work. See http://sphinx.pocoo.org/index.html
+# for installation instructions.
+docs-sphinx: html-sphinx
+
+# Clean out generated documentation
+clean-sphinx:
+	cd doc/en && make clean
+	cd doc/es && make clean
+	cd doc/ru && make clean
+	cd doc/developers && make clean
+
+SPHINX_DEPENDENCIES = \
+        doc/en/release-notes/index.txt \
+        doc/en/user-reference/index.txt \
+	doc/es/Makefile \
+	doc/es/make.bat \
+	doc/ru/Makefile \
+	doc/ru/make.bat \
+	doc/developers/Makefile \
+	doc/developers/make.bat
+
+doc/en/user-reference/index.txt: $(MAN_DEPENDENCIES)
+	$(PYTHON) tools/generate_docs.py -o $@ rstx
+
+doc/en/release-notes/index.txt: NEWS tools/generate_release_notes.py
+	$(PYTHON) tools/generate_release_notes.py NEWS $@
+
+doc/%/Makefile: doc/en/Makefile
+	$(PYTHON) -c "import shutil; shutil.copyfile('$<', '$@')"
+
+doc/%/make.bat: doc/en/make.bat
+	$(PYTHON) -c "import shutil; shutil.copyfile('$<', '$@')"
+
+# Build the html docs using Sphinx.
+html-sphinx: $(SPHINX_DEPENDENCIES)
+	cd doc/en && make html
+	cd doc/es && make html
+	cd doc/ru && make html
+	cd doc/developers && make html
+
+# Build the PDF docs using Sphinx. This requires numerous LaTeX
+# packages. See http://sphinx.pocoo.org/builders.html for details.
+# Note: We don't currently build PDFs for the Russian docs because
+# they require additional packages to be installed (to handle
+# Russian hyphenation rules, etc.)
+pdf-sphinx: $(SPHINX_DEPENDENCIES)
+	cd doc/en && make latex
+	cd doc/es && make latex
+	cd doc/developers && make latex
+	cd doc/en/_build/latex && make all-pdf
+	cd doc/es/_build/latex && make all-pdf
+	cd doc/developers/_build/latex && make all-pdf
+
+# Build the CHM (Windows Help) docs using Sphinx.
+# Note: HtmlHelp Workshop needs to be used on the generated hhp files
+# to generate the final chm files.
+chm-sphinx: $(SPHINX_DEPENDENCIES)
+	cd doc/en && make htmlhelp
+	cd doc/es && make htmlhelp
+	cd doc/ru && make htmlhelp
+	cd doc/developers && make htmlhelp
+
+
+### Documentation Website ###
+
+# Where to build the website
+DOC_WEBSITE_BUILD = build_doc_website
+
+# Build and package docs into a website, complete with downloads.
+doc-website: html-sphinx pdf-sphinx
+	$(PYTHON) tools/package_docs.py doc/en $(DOC_WEBSITE_BUILD)
+	$(PYTHON) tools/package_docs.py doc/es $(DOC_WEBSITE_BUILD)
+	$(PYTHON) tools/package_docs.py doc/ru $(DOC_WEBSITE_BUILD)
+	$(PYTHON) tools/package_docs.py doc/developers $(DOC_WEBSITE_BUILD)
+
+
+### Plain Documentation ###
+
+# While Sphinx is the preferred tool for building documentation, we still
+# support our "plain" html documentation so that Sphinx is not a hard
+# dependency for packagers on older platforms.
+
+rst2html = $(PYTHON) tools/rst2html.py --link-stylesheet --footnote-references=superscript --halt=warning
 
 # translate txt docs to html
-derived_txt_files := \
+derived_txt_files = \
 	doc/en/user-reference/bzr_man.txt \
 	doc/en/release-notes/NEWS.txt
-txt_files := \
+txt_all = \
 	doc/en/tutorials/tutorial.txt \
 	doc/en/tutorials/using_bazaar_with_launchpad.txt \
 	doc/en/tutorials/centralized_workflow.txt \
+        $(wildcard doc/es/tutorials/*.txt) \
         $(wildcard doc/ru/tutorials/*.txt) \
 	$(wildcard doc/*/mini-tutorial/index.txt) \
-	$(wildcard doc/*/user-guide/index.txt) \
+	$(wildcard doc/*/user-guide/index-plain.txt) \
+	$(wildcard doc/es/guia-usario/*.txt) \
 	$(derived_txt_files) \
-	doc/en/developer-guide/HACKING.txt \
 	doc/en/upgrade-guide/index.txt \
-	$(wildcard doc/es/guia-usario/*.txt) \
-	doc/es/mini-tutorial/index.txt \
 	doc/index.txt \
 	$(wildcard doc/index.*.txt)
-non_txt_files := \
+txt_nohtml = \
+	doc/en/user-guide/index.txt \
+	doc/es/user-guide/index.txt \
+	doc/ru/user-guide/index.txt
+txt_files = $(filter-out $(txt_nohtml), $(txt_all))
+htm_files = $(patsubst %.txt, %.html, $(txt_files)) 
+
+non_txt_files = \
        doc/default.css \
-       $(wildcard doc/*/quick-reference/bzr-quick-reference.svg) \
-       $(wildcard doc/*/quick-reference/bzr-quick-reference.png) \
-       $(wildcard doc/*/quick-reference/bzr-quick-reference.pdf) \
+       $(wildcard doc/*/bzr-en-quick-reference.svg) \
+       $(wildcard doc/*/bzr-en-quick-reference.png) \
+       $(wildcard doc/*/bzr-en-quick-reference.pdf) \
+       $(wildcard doc/*/bzr-es-quick-reference.svg) \
+       $(wildcard doc/*/bzr-es-quick-reference.png) \
+       $(wildcard doc/*/bzr-es-quick-reference.pdf) \
+       $(wildcard doc/*/bzr-ru-quick-reference.svg) \
+       $(wildcard doc/*/bzr-ru-quick-reference.png) \
+       $(wildcard doc/*/bzr-ru-quick-reference.pdf) \
        $(wildcard doc/*/user-guide/images/*.png)
-htm_files := $(patsubst %.txt, %.html, $(txt_files)) 
 
 # doc/developers/*.txt files that should *not* be individually
 # converted to HTML
-dev_txt_nohtml := \
+dev_txt_nohtml = \
 	doc/developers/add.txt \
 	doc/developers/annotate.txt \
 	doc/developers/bundle-creation.txt \
@@ -120,46 +237,37 @@
 	doc/developers/diff.txt \
 	doc/developers/directory-fingerprints.txt \
 	doc/developers/gc.txt \
+	doc/developers/implementation-notes.txt \
 	doc/developers/incremental-push-pull.txt \
+	doc/developers/index.txt \
 	doc/developers/initial-push-pull.txt \
 	doc/developers/merge-scaling.txt \
+	doc/developers/miscellaneous-notes.txt \
 	doc/developers/missing.txt \
 	doc/developers/performance-roadmap-rationale.txt \
 	doc/developers/performance-use-case-analysis.txt \
 	doc/developers/planned-change-integration.txt \
 	doc/developers/planned-performance-changes.txt \
+	doc/developers/plans.txt \
+	doc/developers/process.txt \
 	doc/developers/revert.txt \
+	doc/developers/specifications.txt \
 	doc/developers/status.txt \
 	doc/developers/uncommit.txt
 
-dev_txt_all := $(wildcard $(addsuffix /*.txt, doc/developers))
-dev_txt_files := $(filter-out $(dev_txt_nohtml), $(dev_txt_all))
-dev_htm_files := $(patsubst %.txt, %.html, $(dev_txt_files)) 
-
-doc/%/user-guide/index.html: $(wildcard $(addsuffix /*.txt, doc/%/user-guide)) 
-	$(rst2html) --stylesheet=../../default.css $(dir $@)index.txt $@
-
-# Set the paper size for PDF files.
-# Options:  'a4' (ISO A4 size), 'letter' (US Letter size)
-PAPERSIZE = a4
-# TODO: Add generation for Russian PDF
-PDF_DOCS := doc/en/user-guide/user-guide.$(PAPERSIZE).pdf
-
-# Copy and modify the RST sources, and convert SVG images to PDF
-# files for use a images in the LaTeX-generated PDF.
-# Then generate the PDF output from the modified RST sources.
-doc/en/user-guide/user-guide.$(PAPERSIZE).pdf: $(wildcard $(addsuffix /*.txt, doc/en/user-guide))
-	mkdir -p doc/en/user-guide/latex_prepared
-	$(PYTHON) tools/prepare_for_latex.py \
-	    --out-dir=doc/en/user-guide/latex_prepared \
-	    --in-dir=doc/en/user-guide
-	cd doc/en/user-guide/latex_prepared && \
-	    $(PYTHON) ../../../../tools/rst2pdf.py \
-	        --documentoptions=10pt,$(PAPERSIZE)paper \
-	        --input-encoding=UTF-8:strict --output-encoding=UTF-8:strict \
-	        --strict --title="Bazaar User Guide" \
-	        index.txt ../user-guide.$(PAPERSIZE).pdf
-
+dev_txt_all = $(wildcard $(addsuffix /*.txt, doc/developers))
+dev_txt_files = $(filter-out $(dev_txt_nohtml), $(dev_txt_all))
+dev_htm_files = $(patsubst %.txt, %.html, $(dev_txt_files)) 
+
+doc/en/user-guide/index-plain.html: $(wildcard $(addsuffix /*.txt, doc/en/user-guide)) 
+	$(rst2html) --stylesheet=../../default.css $(dir $@)index-plain.txt $@
+
+#doc/es/user-guide/index.html: $(wildcard $(addsuffix /*.txt, doc/es/user-guide)) 
+#	$(rst2html) --stylesheet=../../default.css $(dir $@)index.txt $@
+#
+#doc/ru/user-guide/index.html: $(wildcard $(addsuffix /*.txt, doc/ru/user-guide)) 
+#	$(rst2html) --stylesheet=../../default.css $(dir $@)index.txt $@
+#
 doc/developers/%.html: doc/developers/%.txt
 	$(rst2html) --stylesheet=../default.css $< $@
 
@@ -172,55 +280,42 @@
 %.html: %.txt
 	$(rst2html) --stylesheet=../../default.css $< $@
 
-MAN_DEPENDENCIES = bzrlib/builtins.py \
-	$(wildcard bzrlib/*.py) \
-	$(wildcard bzrlib/*/*.py) \
-	tools/generate_docs.py \
-	$(wildcard $(addsuffix /*.txt, bzrlib/help_topics/en)) 
-
 doc/en/user-reference/bzr_man.txt: $(MAN_DEPENDENCIES)
 	$(PYTHON) tools/generate_docs.py -o $@ rstx
 
 doc/en/release-notes/NEWS.txt: NEWS
 	$(PYTHON) -c "import shutil; shutil.copyfile('$<', '$@')"
 
-MAN_PAGES = man1/bzr.1
-man1/bzr.1: $(MAN_DEPENDENCIES)
-	$(PYTHON) tools/generate_docs.py -o $@ man
-
 upgrade_guide_dependencies =  $(wildcard $(addsuffix /*.txt, doc/en/upgrade-guide)) 
 
 doc/en/upgrade-guide/index.html: $(upgrade_guide_dependencies)
 	$(rst2html) --stylesheet=../../default.css $(dir $@)index.txt $@
 
-# build a png of our performance task list
-# 
-# this is no longer built by default; you can build it if you want to look at it
-doc/developers/performance.png: doc/developers/performance.dot
-	@echo Generating $@
-	@dot -Tpng $< -o$@ || echo "Dot not installed; skipping generation of $@"
-
 derived_web_docs = $(htm_files) $(dev_htm_files) 
 WEB_DOCS = $(derived_web_docs) $(non_txt_files)
 ALL_DOCS = $(derived_web_docs) $(MAN_PAGES)
 
 # the main target to build all the docs
-docs: $(ALL_DOCS)
+docs-plain: $(ALL_DOCS)
 
 # produce a tree containing just the final docs, ready for uploading to the web
-HTMLDIR := html_docs
-html-docs: docs
+HTMLDIR = html_docs
+html-plain: docs-plain
 	$(PYTHON) tools/win32/ostools.py copytree $(WEB_DOCS) $(HTMLDIR)
 
-# Produce PDF documents.  Requires pdfLaTeX, rubber, and Inkscape.
-pdf-docs: $(PDF_DOCS)
-
 # clean produced docs
-clean-docs:
+clean-plain:
 	$(PYTHON) tools/win32/ostools.py remove $(ALL_DOCS) \
 	    $(HTMLDIR) $(derived_txt_files)
-	rm -f doc/*/user-guide/*.pdf
-	rm -rf doc/*/user-guide/latex_prepared
+
+
+### Miscellaneous Documentation Targets ###
+
+# build a png of our performance task list
+# this is no longer built by default; you can build it if you want to look at it
+doc/developers/performance.png: doc/developers/performance.dot
+	@echo Generating $@
+	@dot -Tpng $< -o$@ || echo "Dot not installed; skipping generation of $@"
 
 
 ### Windows Support ###
@@ -230,7 +325,7 @@
 # These are files that need to be copied into the build location to boostrap
 # the build process.
 # Note that the path is relative to tools/win32
-BUILDOUT_FILES := buildout.cfg \
+BUILDOUT_FILES = buildout.cfg \
 	buildout-templates/bin/build-installer.bat.in \
 	ostools.py bootstrap.py
 
@@ -296,9 +391,12 @@
 	$(PYTHON) tools/win32/ostools.py remove bzr-*win32.exe
 	$(PYTHON) tools/win32/ostools.py remove dist
 
+
+### Packaging Targets ###
+
 .PHONY: dist dist-upload-escudero check-dist-tarball
 
-# build a distribution tarball and zip file.
+# build a distribution source tarball
 #
 # this method of copying the pyrex generated files is a bit ugly; it would be
 # nicer to generate it from distutils.
@@ -308,15 +406,12 @@
 	expbasedir=`mktemp -t -d tmp_bzr_dist.XXXXXXXXXX` && \
 	expdir=$$expbasedir/bzr-$$version && \
 	tarball=$$PWD/../bzr-$$version.tar.gz && \
-	zipball=$$PWD/../bzr-$$version.zip && \
 	$(MAKE) clean && \
 	$(MAKE) && \
 	bzr export $$expdir && \
-	cp bzrlib/*.c $$expdir/bzrlib/. && \
+	cp bzrlib/*.c bzrlib/*.h $$expdir/bzrlib/. && \
 	tar cfz $$tarball -C $$expbasedir bzr-$$version && \
-	(cd $$expbasedir && zip -r $$zipball bzr-$$version) && \
 	gpg --detach-sign $$tarball && \
-	gpg --detach-sign $$zipball && \
 	rm -rf $$expbasedir
 
 # run all tests in a previously built tarball
@@ -334,15 +429,10 @@
 dist-upload-escudero:
 	version=`./bzr version --short` && \
 	tarball=../bzr-$$version.tar.gz && \
-	zipball=../bzr-$$version.zip && \
-	scp $$zipball $$zipball.sig $$tarball $$tarball.sig \
+	scp $$tarball $$tarball.sig \
 	    escudero.ubuntu.com:/srv/bazaar.canonical.com/www/releases/src \
 		&& \
 	echo verifying over http... && \
-	curl http://bazaar-vcs.org/releases/src/bzr-$$version.zip \
-		| diff -s - $$zipball && \
-	curl http://bazaar-vcs.org/releases/src/bzr-$$version.zip.sig \
-		| diff -s - $$zipball.sig 
 	curl http://bazaar-vcs.org/releases/src/bzr-$$version.tar.gz \
 		| diff -s - $$tarball && \
 	curl http://bazaar-vcs.org/releases/src/bzr-$$version.tar.gz.sig \
 
=== modified file 'NEWS'
--- NEWS	2009-09-02 18:07:58 +0000
+++ NEWS	2009-11-03 09:40:55 +0000
@@ -2,44 +2,409 @@
 Bazaar Release Notes
 ####################
 
-
 .. contents:: List of Releases
    :depth: 1
 
-In Development
-##############
-
-Compatibility Breaks
-********************
-
-New Features
-************
-
-Bug Fixes
-*********
-
-* ``bzr check`` in pack-0.92, 1.6 and 1.9 format repositories will no
-  longer report incorrect errors about ``Missing inventory ('TREE_ROOT', ...)``
-  (Robert Collins, #416732)
+
+bzr 2.1.0b3 (not released yet)
+##############################
+
+:Codename:
+:2.1.0b3: ???
+
+Compatibility Breaks
+********************
+
+New Features
+************
+
+Bug Fixes
+*********
+
+* Hooks daughter classes should always call the base constructor.
+  (Alexander Belchenko, Vincent Ladeuil, #389648) 
+
+Improvements
+************
+
+Documentation
+*************
+
+API Changes
+***********
+
+Internals
+*********
+
+Testing
+*******
+
+* KnownFailure is now signalled to ``ExtendedTestResult`` using the same
+  method that Python 2.7 uses - ``addExpectedFailure``. (Robert Collins)
+
+* TestNotApplicable is now handled within the TestCase.run method rather
+  than being looked for within ``ExtendedTestResult.addError``. This
+  provides better handling with other ``TestResult`` objects, degrading to
+  sucess rather than error. (Robert Collins)
+
+* The private method ``_testConcluded`` on ``ExtendedTestResult`` has been
+  removed - it was empty and unused. (Robert Collins)
+
+* UnavailableFeature is now handled within the TestCase.run method rather
+  than being looked for within addError. If the Result object does not
+  have an addNotSupported method, addSkip is attempted instead, and
+  failing that addSuccess. (Robert Collins)
+
+* When a TestResult does not have an addSkip method, skipped tests are now
+  reported as successful tests, rather than as errors. This change is
+  to make it possible to get a clean test run with a less capable
+  TestResult. (Robert Collins)
+
+
+
+bzr 2.0.3 (not released yet)
+############################
+
+:Codename: template
+:2.0.3: ???
+
+Compatibility Breaks
+********************
+
+New Features
+************
+
+Bug Fixes
+*********
+
+Improvements
+************
+
+Documentation
+*************
+
+API Changes
+***********
+
+Internals
+*********
+
+Testing
+*******
+
+
+bzr 2.1.0b2
+###########
+
+:Codename: a load off my mind
+:2.1.0b2: 2009-11-02
+
+This is our second feature-filled release since 2.0, pushing us down the
+path to a 2.1.0. Once again, all bugfixes in 2.0.2 are present in 2.1.0b2.
+
+Key highlights in this release are: improved handling of
+failures-during-cleanup for commit, fixing a long-standing bug with
+``bzr+http`` and shared repositories, and a new StaticTuple datatype,
+allowing us to reduce memory consumption (50%) and garbage collector
+overhead (40% faster) for many operations.
+
+Bug Fixes
+*********
+
+* ``bzr+http`` servers no longer give spurious jail break errors when
+  serving branches inside a shared repository.  (Andrew Bennetts, #348308)
+
+* Errors during commit are handled more robustly so that knock-on errors
+  are less likely to occur, and will not obscure the original error if
+  they do occur.  This fixes some causes of ``TooManyConcurrentRequests``
+  and similar errors.  (Andrew Bennetts, #429747, #243391)
+
+* TreeTransform.adjust_path updates the limbo paths of descendants of adjusted
+  files.  (Aaron Bentley)
+
+* Unicode paths are now handled correctly and consistently by the smart
+  server.  (Andrew Bennetts, Michael Hudson, #458762)
+
+Improvements
+************
+
+* When reading index files, we now use a ``StaticTuple`` rather than a
+  plain ``tuple`` object. This generally gives a 20% decrease in peak
+  memory, and can give a performance boost up to 40% on large projects.
+  (John Arbash Meinel)
+
+* Peak memory under certain operations has been reduced significantly.
+  (eg, 'bzr branch launchpad standalone' is cut in half)
+  (John Arbash Meinel)
+
+Documentation
+*************
+
+* Filtered views user documentation upgraded to refer to format 2a
+  instead of pre-2.0 formats. (Ian Clatworthy)
+
+API Changes
+***********
+
+* Remove deprecated ``CLIUIFactory``.  (Martin Pool)
+
+* ``UIFactory`` now has new ``show_error``, ``show_message`` and
+  ``show_warning`` methods, which can be hooked by non-text UIs.  
+  (Martin Pool)
+
+Internals
+*********
+
+* Added ``bzrlib._simple_set_pyx``. This is a hybrid between a Set and a
+  Dict (it only holds keys, but you can lookup the object located at a
+  given key). It has significantly reduced memory consumption versus the
+  builtin objects (1/2 the size of Set, 1/3rd the size of Dict). This is
+  used as the interning structure for StaticTuple objects.
+  (John Arbash Meinel)
+
+* ``bzrlib._static_tuple_c.StaticTuple`` is now available and used by
+  the btree index parser and the chk map parser. This class functions
+  similarly to ``tuple`` objects. However, it can only point to a limited
+  collection of types.  (Currently StaticTuple, str, unicode, None, bool,
+  int, long, float, but not subclasses).  This allows us to remove it from
+  the garbage collector (it cannot be in a cycle), it also allows us to
+  intern the objects. In testing, this can reduce peak memory by 20-40%,
+  and significantly improve performance by removing objects from being
+  inspected by the garbage collector.  (John Arbash Meinel)
+
+* ``GroupCompressBlock._ensure_content()`` will now release the
+  ``zlib.decompressobj()`` when the first request is for all of the
+  content. (Previously it would only be released if you made a request for
+  part of the content, and then all of it later.) This turns out to be a
+  significant memory savings, as a ``zstream`` carries around approx 260kB
+  of internal state and buffers. (For branching bzr.dev this drops peak
+  memory from 382MB => 345MB.) (John Arbash Meinel)
+
+* When streaming content between ``2a`` format repositories, we now clear
+  caches from earlier versioned files. (So 'revisions' is cleared when we
+  start reading 'inventories', etc.) This can have a significant impact on
+  peak memory for initial copies (~200MB). (John Arbash Meinel)
+
+
+bzr 2.0.2
+#########
+
+:Codename: after the scare
+:2.0.2: 2009-11-02
+
+The second in our "let's keep the stable bugfixes flowing" series. As
+expected this has a few (~9) bugfixes relative to 2.0.1, and no major api
+changes or features.
+
+Bug Fixes
+*********
+
+* Avoid "NoneType has no attribute st_mode" error when files disappear
+  from a directory while it's being read.  (Martin Pool, #446033)
+
+* Content filters are now applied correctly after revert.
+  (Ian Clatworthy)
+
+* Diff parsing handles "Binary files differ" hunks.  (Aaron Bentley, #436325)
+
+* Fetching from stacked pre-2a repository via a smart server no longer
+  fails intermittently with "second push failed to complete".
+  (Andrew Bennetts, #437626)
+
+* Fix typos left after test_selftest refactoring.
+  (Vincent Ladeuil, Matt Nordhoff, #461149)
+
+* Fixed ``ObjectNotLocked`` errors during ``bzr log -r NNN somefile``.
+  (Andrew Bennetts, #445171)
+  
+* PreviewTree file names are not limited by the encoding of the temp
+  directory's filesystem. (Aaron Bentley, #436794)
+
+Improvements
+************
+
+* ``bzr log`` now read-locks branches exactly once, so makes better use of
+  data caches.  (Andrew Bennetts)
+
+Documentation
+*************
+
+* Filtered views user documentation upgraded to refer to format 2a
+  instead of pre-2.0 formats. (Ian Clatworthy)
+
+
+bzr 2.1.0b1
+###########
+
+:Codename: While the cat is away
+:2.1.0b1: 2009-10-14
+
+This is the first development release in the new split "stable" and
+"development" series. As such, the release is a snapshot of bzr.dev
+without creating a release candidate first. This release includes a
+fair amount of internal changes, with deprecated code being removed,
+and several new feature developments. People looking for a stable code
+base with only bugfixes should focus on the 2.0.1 release. All bugfixes
+present in 2.0.1 are present in 2.1.0b1.
+
+Highlights include support for ``bzr+ssh://host/~/homedir`` style urls,
+finer control over the plugin search path via extended BZR_PLUGIN_PATH
+syntax, visible warnings when extension modules fail to load, and improved
+error handling during unlocking.
+
+
+New Features
+************
+
+* Bazaar can now send mail through Apple OS X Mail.app. 
+  (Brian de Alwis)
+
+* ``bzr+ssh`` and ``bzr`` paths can now be relative to home directories
+  specified in the URL.  Paths starting with a path segment of ``~`` are
+  relative to the home directory of the user running the server, and paths
+  starting with ``~user`` are relative to the home directory of the named
+  user.  For example, for a user "bob" with a home directory of
+  ``/home/bob``, these URLs are all equivalent:
+
+  * ``bzr+ssh://bob@host/~/repo``
+  * ``bzr+ssh://bob@host/~bob/repo``
+  * ``bzr+ssh://bob@host/home/bob/repo``
+
+  If ``bzr serve`` was invoked with a ``--directory`` argument, then no
+  home directories outside that directory will be accessible via this
+  method.
+
+  This is a feature of ``bzr serve``, so pre-2.1 clients will
+  automatically benefit from this feature when ``bzr`` on the server is
+  upgraded.  (Andrew Bennetts, #109143)
+
+* Extensions can now be compiled if either Cython or Pyrex is available.
+  Currently Pyrex is preferred, but that may change in the future.
+  (Arkanes)
+
+* Give more control on BZR_PLUGIN_PATH by providing a way to refer to or
+  disable the user, site and core plugin directories.
+  (Vincent Ladeuil, #412930, #316192, #145612)
+
+Bug Fixes
+*********
+
+* Bazaar's native protocol code now correctly handles EINTR, which most
+  noticeably occurs if you break in to the debugger while connected to a
+  bzr+ssh server.  You can now can continue from the debugger (by typing
+  'c') and the process continues.  However, note that pressing C-\ in the
+  shell may still kill the SSH process, which is bug 162509, so you must
+  sent a signal to the bzr process specifically, for example by typing
+  ``kill -QUIT PID`` in another shell.  (Martin Pool, #341535)
+
+* ``bzr add`` in a tree that has files with ``\r`` or ``\n`` in the
+  filename will issue a warning and skip over those files.
+  (Robert Collins, #3918)
+
+* ``bzr dpush`` now aborts if uncommitted changes (including pending merges)
+  are present in the working tree. The configuration option ``dpush_strict``
+  can be used to set the default for this behavior.
+  (Vincent Ladeuil, #438158)
+
+* ``bzr merge`` and ``bzr remove-tree`` now requires --force if pending
+  merges are present in the working tree.
+  (Vincent Ladeuil, #426344)
+
+* Clearer message when Bazaar runs out of memory, instead of a ``MemoryError``
+  traceback.  (Martin Pool, #109115)
+
+* Don't give a warning on Windows when failing to import ``_readdir_pyx``
+  as it is never built. (John Arbash Meinel, #430645)
 
 * Don't restrict the command name used to run the test suite.
   (Vincent Ladeuil, #419950)
 
+* ftp transports were built differently when the kerberos python module was
+  present leading to obscure failures related to ASCII/BINARY modes.
+  (Vincent Ladeuil, #443041)
+
+* Network streams now decode adjacent records of the same type into a
+  single stream, reducing layering churn. (Robert Collins)
+
+* PreviewTree behaves correctly when get_file_mtime is invoked on an unmodified
+  file. (Aaron Bentley, #251532)
+
+* Registry objects should not use iteritems() when asked to use items().
+  (Vincent Ladeuil, #430510)
+
+* Weave based repositories couldn't be cloned when committers were using
+  domains or user ids embedding '.sig'. Now they can.
+  (Matthew Fuller, Vincent Ladeuil, #430868)
+
 Improvements
 ************
 
+* Revision specifiers can now be given in a more DWIM form, without
+  needing explicit prefixes for specifiers like tags or revision id's.
+  See ``bzr help revisionspec`` for full details.  (Matthew Fuller)
+
+* Bazaar gives a warning before exiting, and writes into ``.bzr.log``, if 
+  compiled extensions can't be loaded.  This typically indicates a
+  packaging or installation problem.  In this case Bazaar will keep
+  running using pure-Python versions, but this may be substantially
+  slower.  The warning can be disabled by setting
+  ``ignore_missing_extensions = True`` in ``bazaar.conf``.
+  See also <https://answers.launchpad.net/bzr/+faq/703>.
+  (Martin Pool, #406113, #430529)
+
+* Secondary errors that occur during Branch.unlock and Repository.unlock
+  no longer obscure the original error.  These methods now use a new
+  decorator, ``only_raises``.  This fixes many causes of
+  ``TooManyConcurrentRequests`` and similar errors.
+  (Andrew Bennetts, #429747)
+
 Documentation
 *************
 
+* Describe the new shell-like test feature. (Vincent Ladeuil)
+
+* Help on hooks no longer says 'Not deprecated' for hooks that are
+  currently supported. (Ian Clatworthy, #422415)
+
 API Changes
 ***********
 
+* ``bzrlib.user_encoding`` has been removed; use
+  ``bzrlib.osutils.get_user_encoding`` instead.  (Martin Pool)
+
 * ``bzrlib.tests`` now uses ``stopTestRun`` for its ``TestResult``
   subclasses - the same as python's unittest module. (Robert Collins)
+  
+* ``diff._get_trees_to_diff`` has been renamed to 
+  ``diff.get_trees_and_branches_to_diff``. It is now a public API, and it 
+  returns the old and new branches. (Gary van der Merwe)
+
+* ``bzrlib.trace.log_error``, ``error`` and ``info`` have been deprecated.
+  (Martin Pool)
+
+* ``MutableTree.has_changes()`` does not require a tree parameter anymore. It
+  now defaults to comparing to the basis tree. It now checks for pending
+  merges too.  ``Merger.check_basis`` has been deprecated and replaced by the
+  corresponding has_changes() calls. ``Merge.compare_basis``,
+  ``Merger.file_revisions`` and ``Merger.ensure_revision_trees`` have also
+  been deprecated.
+  (Vincent Ladeuil, #440631)
+
+* ``ProgressTask.note`` is deprecated.
+  (Martin Pool)
 
 Internals
 *********
 
+* Added ``-Drelock`` debug flag.  It will ``note`` a message every time a
+  repository or branch object is unlocked then relocked the same way.
+  (Andrew Bennetts)
+  
+* ``BTreeLeafParser.extract_key`` has been tweaked slightly to reduce
+  mallocs while parsing the index (approx 3=>1 mallocs per key read).
+  This results in a 10% speedup while reading an index.
+  (John Arbash Meinel)
+
 * The ``bzrlib.lsprof`` module has a new class ``BzrProfiler`` which makes
   profiling in some situations like callbacks and generators easier.
   (Robert Collins)
@@ -50,27 +415,195 @@
 * Passing ``--lsprof-tests -v`` to bzr selftest will cause lsprof output to
   be output for every test. Note that this is very verbose! (Robert Collins)
 
+* Setting ``BZR_TEST_PDB=1`` when running selftest will cause a pdb
+  post_mortem to be triggered when a test failure occurs. (Robert Collins)
+
+* Shell-like tests can now be written. Code in ``bzrlib/tests/script.py`` ,
+  documentation in ``developers/testing.txt`` for details.
+  (Vincent Ladeuil)
+
+* Some tests could end up with the same id, that was dormant for
+  a long time.
+  (Vincent Ladeuil, #442980)
+
+* Stop showing the number of tests due to missing features in the test
+  progress bar.  (Martin Pool)
+
 * Test parameterisation now does a shallow copy, not a deep copy of the test
   to be parameterised. This is not expected to break external use of test
   parameterisation, and is substantially faster. (Robert Collins)
 
-
-bzr 2.0rc2
-##########
-
-Bug Fixes
-*********
+* Tests that try to open a bzr dir on an arbitrary transport will now
+  fail unless they have explicitly permitted the transport via
+  ``self.permit_url``. The standard test factories such as ``self.get_url``
+  will permit the urls they provide automatically, so only exceptional
+  tests should need to do this. (Robert Collins)
+
+* The break-in test no longer cares about clean shutdown of the child,
+  instead it is happy if the debugger starts up. (Robert  Collins)
+
+* The full test suite is expected to pass when the C extensions are not
+  present. (Vincent Ladeuil, #430749)
+
+
+bzr 2.0.1
+#########
+
+:Codename: Stability First
+:2.0.1: 2009-10-14
+
+The first of our new ongoing bugfix-only stable releases has arrived. It
+includes a collection of 12 bugfixes applied to bzr 2.0.0, but does not
+include any of the feature development in the 2.1.0 series.
+
+
+Bug Fixes
+*********
+
+* ``bzr add`` in a tree that has files with ``\r`` or ``\n`` in the
+  filename will issue a warning and skip over those files.
+  (Robert Collins, #3918)
+
+* bzr will attempt to authenticate with SSH servers that support
+  ``keyboard-interactive`` auth but not ``password`` auth when using
+  Paramiko.   (Andrew Bennetts, #433846)
+
+* Fixed fetches from a stacked branch on a smart server that were failing
+  with some combinations of remote and local formats.  This was causing
+  "unknown object type identifier 60" errors.  (Andrew Bennetts, #427736)
+
+* Fixed ``ObjectNotLocked`` errors when doing some log and diff operations
+  on branches via a smart server.  (Andrew Bennetts, #389413)
+
+* Handle things like ``bzr add foo`` and ``bzr rm foo`` when the tree is
+  at the root of a drive. ``osutils._cicp_canonical_relpath`` always
+  assumed that ``abspath()`` returned a path that did not have a trailing
+  ``/``, but that is not true when working at the root of the filesystem.
+  (John Arbash Meinel, Jason Spashett, #322807)
+
+* Hide deprecation warnings for 'final' releases for python2.6.
+  (John Arbash Meinel, #440062)
+
+* Improve the time for ``bzr log DIR`` for 2a format repositories.
+  We had been using the same code path as for <2a formats, which required
+  iterating over all objects in all revisions.
+  (John Arbash Meinel, #374730)
+
+* Make sure that we unlock the tree if we fail to create a TreeTransform
+  object when doing a merge, and there is limbo, or pending-deletions
+  directory.  (Gary van der Merwe, #427773)
+
+* Occasional IndexError on renamed files have been fixed. Operations that
+  set a full inventory in the working tree will now go via the
+  apply_inventory_delta code path which is simpler and easier to
+  understand than dirstates set_state_from_inventory method. This may
+  have a small performance impact on operations built on _write_inventory,
+  but such operations are already doing full tree scans, so no radical
+  performance change should be observed. (Robert Collins, #403322)
+
+* Retrieving file text or mtime from a _PreviewTree has good performance when
+  there are many changes.  (Aaron Bentley)
+
+* The CHK index pages now use an unlimited cache size. With a limited
+  cache and a large project, the random access of chk pages could cause us
+  to download the entire cix file many times.
+  (John Arbash Meinel, #402623)
+
+* When a file kind becomes unversionable after being added, a sensible
+  error will be shown instead of a traceback. (Robert Collins, #438569)
+
+Documentation
+*************
+
+* Improved README. (Ian Clatworthy)
+
+* Improved upgrade documentation for Launchpad branches.
+  (Barry Warsaw)
+
+
+bzr 2.0.0
+#########
+
+:2.0.0: 2009-09-22
+:Codename: Instant Karma
+
+This release of Bazaar makes the 2a (previously 'brisbane-core') format
+the default when new branches or repositories are created.  This format is
+substantially smaller and faster for many operations.  Most of the work in
+this release focuses on bug fixes and stabilization, covering both 2a and
+previous formats.  (See the Upgrade Guide for information on migrating
+existing projects.)
+
+This release also improves the documentation content and presentation,
+including adding Windows HtmlHelp manuals.
+
+The Bazaar team decided that 2.0 will be a long-term supported release,
+with bugfix-only 2.0.x releases based on it, continuing for at least six
+months or until the following stable release.
+
+Changes from 2.0.0rc2 to final
+******************************
+
+* Officially branded as 2.0.0 rather than 2.0 to clarify between things
+  that "want to happen on the 2.0.x stable series" versus things that want
+  to "land in 2.0.0". (Changes how bzrlib._format_version_tuple() handles
+  micro = 0.) (John Arbash Meinel)
+
+
+bzr 2.0.0rc2
+############
+
+:2.0.0rc2: 2009-09-10
+
+New Features
+************
+
+* Added post_commit hook for mutable trees. This allows the keywords
+  plugin to expand keywords on files changed by the commit.
+  (Ian Clatworthy, #408841)
+
+Bug Fixes
+*********
+
+* Bazaar's native protocol code now correctly handles EINTR, which most
+  noticeably occurs if you break in to the debugger while connected to a
+  bzr+ssh server.  You can now can continue from the debugger (by typing
+  'c') and the process continues.  However, note that pressing C-\ in the
+  shell may still kill the SSH process, which is bug 162509, so you must
+  sent a signal to the bzr process specifically, for example by typing
+  ``kill -QUIT PID`` in another shell.  (Martin Pool, #341535)
 
 * ``bzr check`` in pack-0.92, 1.6 and 1.9 format repositories will no
   longer report incorrect errors about ``Missing inventory ('TREE_ROOT', ...)``
   (Robert Collins, #416732)
 
+* ``bzr info -v`` on a 2a format still claimed that it was a "Development
+  format" (John Arbash Meinel, #424392)
+
 * ``bzr log stacked-branch`` shows the full log including
   revisions that are in the fallback repository. (Regressed in 2.0rc1).
   (John Arbash Meinel, #419241)
 
-* Fix a segmentation fault when computing the ``merge_sort`` of a graph
-  that has a ghost in the mainline ancestry.
+* Clearer message when Bazaar runs out of memory, instead of a ``MemoryError``
+  traceback.  (Martin Pool, #109115)
+
+* Conversion to 2a will create a single pack for all the new revisions (as
+  long as it ran without interruption). This improves both ``bzr upgrade``
+  and ``bzr pull`` or ``bzr merge`` from local branches in older formats.
+  The autopack logic that occurs every 100 revisions during local
+  conversions was not returning that pack's identifier, which resulted in
+  the partial packs created during the conversion not being consolidated
+  at the end of the conversion process. (Robert Collins, #423818)
+
+* Fetches from 2a to 2a are now again requested in 'groupcompress' order.
+  Groups that are seen as 'underutilized' will be repacked on-the-fly.
+  This means that when the source is fully packed, there is minimal
+  overhead during the fetch, but if the source is poorly packed the result
+  is a fairly well packed repository (not as good as 'bzr pack' but
+  good-enough.) (Robert Collins, John Arbash Meinel, #402652)
+
+* Fix a potential segmentation fault when doing 'log' of a branch that had
+  ghosts in its mainline.  (Evaluating None as a tuple is bad.)
   (John Arbash Meinel, #419241)
 
 * ``groupcompress`` sort order is now more stable, rather than relying on
@@ -81,28 +614,41 @@
   bugfix vs 2.0rc1, and all 2.0rc1 users should upgrade to 2.0rc2 before
   converting repositories. (Robert Collins, #422849)
 
+* Network streams now decode adjacent records of the same type into a
+  single stream, reducing layering churn. (Robert Collins)
+
+* Prevent some kinds of incomplete data from being committed to a 2a
+  repository, such as revisions without inventories, a missing chk_bytes
+  record for an inventory, or a missing text referenced by an inventory.
+  (Andrew Bennetts, #423506, #406687)
+  
 Documentation
 *************
 
+* Fix assertion error about "_remember_remote_is_before" when pushing to
+  older smart servers.
+  (Andrew Bennetts, #418931)
+
+* Help on hooks no longer says 'Not deprecated' for hooks that are
+  currently supported. (Ian Clatworthy, #422415)
+
+* PDF and CHM (Windows HtmlHelp) formats are now supported for the
+  user documentation. The HTML documentation is better broken up into
+  topics. (Ian Clatworthy)
+
+* The developer and foreign language documents are now separated
+  out so that searching in the HTML and CHM files produces more
+  useful results. (Ian Clatworthy)
+
 * The main table of contents now provides links to the new Migration Docs
   and Plugins Guide. (Ian Clatworthy)
 
 
-bzr 2.0rc1
-##########
-
+bzr 2.0.0rc1
+############
 
 :Codename: no worries
-:2.0rc1: 2009-08-26
-
-This release of Bazaar makes 2a 'brisbane-core' format the
-default.  Most of the work in this release now focuses on bug
-fixes and stabilization, covering both 2a and previous formats.
-
-The Bazaar team decided that 2.0 will be a long-term supported
-release, with bugfix-only releases based on it continuing for at
-least six months or until the following stable release (we said
-that previously, but that's worth repeating).
+:2.0.0rc1: 2009-08-26
 
 Compatibility Breaks
 ********************
@@ -110,7 +656,7 @@
 * The default format for bzr is now ``2a``. This format brings many
   significant performance and size improvements. bzr can pull from
   any existing repository into a ``2a`` one, but can only transfer
-  into ``rich-root`` repositories from ``2a``. The Upgrade guide
+  from ``2a`` into ``rich-root`` repositories. The Upgrade guide
   has more information about this change. (Robert Collins)
 
 * On Windows auto-detection of Putty's plink.exe is disabled.
@@ -175,7 +721,7 @@
 * ``bzr shelve`` and ``bzr unshelve`` now work on windows.
   (Robert Collins, #305006)
 
-* Commit of specific files no longer prevents using the the iter_changes
+* Commit of specific files no longer prevents using the iter_changes
   codepath. On 2a repositories, commit of specific files should now be as
   fast, or slightly faster, than a full commit. (Robert Collins)
 
@@ -202,8 +748,17 @@
 Testing
 *******
 
-bzr 1.18.1 NOT RELEASED YET
-###########################
+bzr 1.18.1
+##########
+
+:Codename:     nein nein nein!
+:1.18.1:       2009-09-09
+
+This release fixes two small but worthwhile bugs relevant to users on
+Microsoft Windows: some commands that failed on with locking errors will
+now work, and a bug that caused poor performance after committing a file
+with line-ending conversion has now been fixed.  It also fixes a bug in
+pushing to older servers.
 
 Bug Fixes
 *********
@@ -212,13 +767,17 @@
   conversion will commit too many copies a file.
   (Martin Pool, #415508)
 
+* Fix assertion error about ``_remember_remote_is_before`` in
+  ``set_tags_bytes`` when pushing to older smart servers.  
+  (Andrew Bennetts, Alexander Belchenko, #418931)
+
 Improvements
 ************
 
-* ``bzr push`` locally on windows will no longer give a locking error with
+* ``bzr push`` locally on Windows will no longer give a locking error with
   dirstate based formats. (Robert Collins)
 
-* ``bzr shelve`` and ``bzr unshelve`` now work on windows.
+* ``bzr shelve`` and ``bzr unshelve`` now work on Windows.
   (Robert Collins, #305006)
 
 API Changes
@@ -583,8 +1142,8 @@
   knit format repositories.  (Andrew Bennetts, #405653)
   
 
-bzr 1.17 "So late it's brunch" 2009-07-20
-#########################################
+bzr 1.17
+########
 :Codename: so-late-its-brunch
 :1.17rc1: 2009-07-13
 :1.17: 2009-07-20
@@ -833,8 +1392,10 @@
   this class and the UI.  (Martin Pool)
 
 
-bzr 1.16.1 2009-06-26
-#####################
+bzr 1.16.1
+##########
+
+:Released: 2009-06-26
 
 End user testing of the 2a format revealed two serious bugs. The first,
 #365615, caused bzr to raise AbsentContentFactory errors when autopacking.
@@ -889,8 +1450,8 @@
   (Robert Collins, #376748)
 
 
-bzr 1.16 "It's yesterday in California" 2009-06-18
-##################################################
+bzr 1.16
+########
 :Codename: yesterday-in-california
 :1.16rc1: 2009-06-11
 :1.16: 2009-06-18
@@ -929,7 +1490,7 @@
 ************
 
 * A new repository format ``2a`` has been added.  This is a beta release
-  of the the brisbane-core (aka group-compress) project.  This format now
+  of the brisbane-core (aka group-compress) project.  This format now
   suitable for wider testing by advanced users willing to deal with some
   bugs.  We would appreciate test reports, either positive or negative.
   Format 2a is substantially smaller and faster for many operations on
@@ -1658,7 +2219,7 @@
 
 * Added ``bzrlib.inventory_delta`` module.  This will be used for
   serializing and deserializing inventory deltas for more efficient
-  streaming on the the network.  (Robert Collins, Andrew Bennetts)
+  streaming on the network.  (Robert Collins, Andrew Bennetts)
 
 * ``Branch._get_config`` has been added, which splits out access to the
   specific config file from the branch. This is used to let RemoteBranch
@@ -2367,8 +2928,11 @@
   (Vincent Ladeuil)
 
 
-bzr 1.11 "Eyes up!" 2009-01-19
-##############################
+bzr 1.11
+########
+
+:Codename: "Eyes up!"
+:Released: 2009-01-19
 
 This first monthly release of Bazaar for 2009 improves Bazaar's operation
 in Windows, Mac OS X, and other situations where file names are matched
@@ -2394,8 +2958,11 @@
 
 
 
-bzr 1.11rc1 "Eyes up!" 2009-01-09
-#################################
+bzr 1.11rc1
+###########
+
+:Codename: "Eyes up!"
+:Released: 2009-01-09
 
 Changes
 *******
@@ -2631,8 +3198,10 @@
 
 
 
-bzr 1.10 2008-12-05
-###################
+bzr 1.10
+########
+
+:Released: 2008-12-05
 
 Bazaar 1.10 has several performance improvements for copying revisions
 (especially for small updates to large projects).  There has also been a
@@ -2657,8 +3226,10 @@
   topologically. (John Arbash Meinel, #304841)
 
 
-bzr 1.10rc1 2008-11-28
-######################
+bzr 1.10rc1
+###########
+
+:Released: 2008-11-28
 
 This release of Bazaar focuses on performance improvements when pushing
 and pulling revisions, both locally and to remote networks.  The popular
@@ -2778,8 +3349,10 @@
 * Doctests now only report the first failure.  (Martin Pool)
 
 
-bzr 1.9 2008-11-07
-##################
+bzr 1.9
+#######
+
+:Released: 2008-11-07
 
 This release of Bazaar adds a new repository format, ``1.9``, with smaller
 and more efficient index files.  This format can be specified when
@@ -2801,8 +3374,10 @@
   (John Arbash Meinel, #293746)
 
 
-bzr 1.9rc1 2008-10-31
-#####################
+bzr 1.9rc1
+##########
+
+:Released: 2008-10-31
 
 New Features
 ************
@@ -2919,8 +3494,10 @@
   configuration of authetication credentials.
 
 
-bzr 1.8 2008-10-16
-##################
+bzr 1.8
+#######
+
+:Released: 2008-10-16
 
 Bazaar 1.8 includes several fixes that improve working tree performance,
 display of revision logs, and merges.  The bzr testsuite now passes on OS
@@ -2941,8 +3518,10 @@
   that platform.  (Alexander Belchenko, Martin Pool, #277481)
 
 
-bzr 1.8rc1 2008-10-07
-#####################
+bzr 1.8rc1
+##########
+
+:Released: 2008-10-07
 
 Changes
 *******
@@ -3140,14 +3719,18 @@
   (Vincent Ladeuil)
 
 
-bzr 1.7.1 2008-10-01
-####################
+bzr 1.7.1
+#########
+
+:Released:  2008-10-01
 
 No changes from 1.7.1rc1.
 
 
-bzr 1.7.1rc1 2008-09-24
-#######################
+bzr 1.7.1rc1
+############
+
+:Released: 2008-09-24
 
 This release just includes an update to how the merge algorithm handles
 file paths when we encounter complex history.
@@ -3162,8 +3745,10 @@
   (John Arbash Meinel)
 
 
-bzr 1.7 2008-09-23
-##################
+bzr 1.7
+#######
+
+:Released: 2008-09-23
 
 This release includes many bug fixes and a few performance and feature
 improvements.  ``bzr rm`` will now scan for missing files and remove them,
@@ -3180,8 +3765,10 @@
   packaging qbzr. (Mark Hammond)
 
 
-bzr 1.7rc2 2008-09-17
-#####################
+bzr 1.7rc2
+##########
+
+:Released: 2008-09-17
 
 A few bug fixes from 1.7rc1. The biggest change is a new
 ``RemoteBranch.get_stacked_on_url`` rpc. This allows clients that are
@@ -3205,8 +3792,10 @@
   (Andrew Bennetts)
 
 
-bzr 1.7rc1 2008-09-09
-#####################
+bzr 1.7rc1
+##########
+
+:Released: 2008-09-09
 
 This release candidate for bzr 1.7 has several bug fixes and a few
 performance and feature improvements.  ``bzr rm`` will now scan for
@@ -3403,16 +3992,20 @@
   clients now use this mechanism.  (Neil Martinsen-Burrell)
 
 
-bzr 1.6.1 2008-09-05
-####################
+bzr 1.6.1
+#########
+
+:Released: 2008-09-05
 
 A couple regressions were found in the 1.6 release. There was a
 performance issue when using ``bzr+ssh`` to branch large repositories,
 and some problems with stacking and ``rich-root`` capable repositories.
 
 
-bzr 1.6.1rc2 2008-09-03
-#######################
+bzr 1.6.1rc2
+############
+
+:Released: 2008-09-03
 
 Bug Fixes
 *********
@@ -3423,8 +4016,10 @@
   (John Arbash Meinel, #264321)
 
 
-bzr 1.6.1rc1 2008-08-29
-#######################
+bzr 1.6.1rc1
+############
+
+:Released: 2008-08-29
 
 This release fixes a few regressions found in the 1.6 client. Fetching
 changes was using an O(N^2) buffering algorithm, so for large projects it
@@ -3464,8 +4059,10 @@
   (John Arbash Meinel, #262333)
 
 
-bzr 1.6 2008-08-25
-##################
+bzr 1.6
+#######
+
+:Released: 2008-08-25
 
 Finally, the long awaited bzr 1.6 has been released. This release includes
 new features like Stacked Branches, improved weave merge, and an updated
@@ -3478,8 +4075,10 @@
 TortoiseBzr in the standalone Windows installer.
 
 
-bzr 1.6rc5 2008-08-19
-#####################
+bzr 1.6rc5
+##########
+
+:Released: 2008-08-19
 
 Bug Fixes
 *********
@@ -3491,8 +4090,10 @@
   (This change was reverted when merged to bzr.dev)
 
 
-bzr 1.6rc4 2008-08-18
-#####################
+bzr 1.6rc4
+##########
+
+:Released: 2008-08-18
 
 Bug Fixes
 *********
@@ -3502,8 +4103,10 @@
   rather than preserving deltas.  (John Arbash Meinel, #256757)
 
 
-bzr 1.6rc3 2008-08-14
-#####################
+bzr 1.6rc3
+##########
+
+:Released: 2008-08-14
 
 Changes
 *******
@@ -3536,8 +4139,10 @@
   development which is substantially faster. (Robert Collins)
 
 
-bzr 1.6rc2 2008-08-13
-#####################
+bzr 1.6rc2
+##########
+
+:Released: 2008-08-13
 
 This release candidate has a few minor bug fixes, and some regression
 fixes for Windows.
@@ -3575,8 +4180,10 @@
   will get the same results.  (John Arbash Meinel, #232188)
 
 
-bzr 1.6rc1 2008-08-06
-#####################
+bzr 1.6rc1
+##########
+
+:Released: 2008-08-06
 
 This release candidate for bzr 1.6 solidifies the new branch stacking
 feature.  Bazaar now recommends that users upgrade all knit repositories,
@@ -3712,8 +4319,10 @@
   (Ian Clatworthy)
 
 
-bzr 1.6beta3 2008-07-17
-#######################
+bzr 1.6beta3
+############
+
+:Released: 2008-07-17
 
 This release adds a new 'stacked branches' feature allowing branches to
 share storage without being in the same repository or on the same machine.
@@ -3885,8 +4494,10 @@
   (Robert Collins)
 
 
-bzr 1.6beta2 2008-06-10
-#######################
+bzr 1.6beta2
+############
+
+:Released: 2008-06-10
 
 This release contains further progress towards our 1.6 goals of shallow
 repositories, and contains a fix for some user-affecting bugs in the
@@ -3969,9 +4580,10 @@
 * Knit record serialisation is now stricter on what it will accept, to
   guard against potential internal bugs, or broken input. (Robert Collins)
 
-bzr 1.6beta1 2008-06-02
-#######################
+bzr 1.6beta1
+############
 
+:Released: 2008-06-02
 
 Commands that work on the revision history such as push, pull, missing,
 uncommit and log are now substantially faster.  This release adds a
@@ -4189,8 +4801,10 @@
   (Martin Pool)
 
 
-bzr 1.5 2008-05-16
-##################
+bzr 1.5
+#######
+
+:Released: 2008-05-16
 
 This release of Bazaar includes several updates to the documentation, and fixes
 to prepare for making rich root support the default format. Many bugs have been
@@ -4211,8 +4825,10 @@
   (Ian Clatworthy)
 
 
-bzr 1.5rc1 2008-05-09
-#####################
+bzr 1.5rc1
+##########
+
+:Released: 2008-05-09
 
 Changes
 *******
@@ -4327,7 +4943,7 @@
   exception. (Andrew Bennetts)
 
 * New ``--debugflag``/``-E`` option to ``bzr selftest`` for setting
-  options for debugging tests, these are complementary to the the -D
+  options for debugging tests, these are complementary to the -D
   options.  The ``-Dselftest_debug`` global option has been replaced by the
   ``-E=allow_debug`` option for selftest. (Andrew Bennetts)
 
@@ -4382,8 +4998,10 @@
   (Martin Pool)
 
 
-bzr 1.4 2008-04-28
-##################
+bzr 1.4 
+#######
+
+:Released: 2008-04-28
 
 This release of Bazaar includes handy improvements to the speed of log and
 status, new options for several commands, improved documentation, and better
@@ -4402,8 +5020,10 @@
   (John Arbash Meinel, Andrew Bennetts, #214894)
 
 
-bzr 1.4rc2 2008-04-21
-#####################
+bzr 1.4rc2
+##########
+
+:Released: 2008-04-21
 
 Bug Fixes
 *********
@@ -4426,8 +5046,10 @@
   (Robert Collins, John Arbash Meinel)
 
 
-bzr 1.4rc1 2008-04-11
-#####################
+bzr 1.4rc1
+##########
+
+:Released: 2008-04-11
 
 Changes
 *******
@@ -4714,14 +5336,18 @@
   ``Graph.get_parent_map`` returns a dict of key:parents. (Robert Collins)
 
 
-bzr 1.3.1 2008-04-09
-####################
+bzr 1.3.1
+#########
+
+:Released: 2008-04-09
 
 No changes from 1.3.1rc1.
 
 
-bzr 1.3.1rc1 2008-04-04
-#######################
+bzr 1.3.1rc1
+############
+
+:Released: 2008-04-04
 
 Bug Fixes
 *********
@@ -4732,8 +5358,10 @@
   (#208418, Andrew Bennetts, Martin Pool, Robert Collins)
 
 
-bzr 1.3 2008-03-20
-##################
+bzr 1.3
+#######
+
+:Released: 2008-03-20
 
 Bazaar has become part of the GNU project <http://www.gnu.org>
 
@@ -4749,8 +5377,10 @@
   (#202778, Martin Pool)
 
 
-bzr 1.3rc1 2008-03-16
-#####################
+bzr 1.3rc1
+##########
+
+:Released: 2008-03-16
 
 Notes When Upgrading
 ********************
@@ -4936,8 +5566,10 @@
   format. (Robert Collins)
 
 
-bzr 1.2 2008-02-15
-##################
+bzr 1.2
+#######
+
+:Released: 2008-02-15
 
 Bug Fixes
 *********
@@ -4945,8 +5577,10 @@
 * Fix failing test in Launchpad plugin. (Martin Pool)
 
 
-bzr 1.2rc1 2008-02-13
-#####################
+bzr 1.2rc1
+##########
+
+:Released: 2008-02-13
 
 Notes When Upgrading
 ********************
@@ -5147,13 +5781,17 @@
   revision names etc. (Robert Collins)
 
 
-bzr 1.1 2008-01-15
-##################
+bzr 1.1
+#######
+
+:Released: 2008-01-15
 
 (no changes from 1.1rc1)
 
-bzr 1.1rc1 2008-01-05
-#####################
+bzr 1.1rc1
+##########
+
+:Released: 2008-01-05
 
 Changes
 *******
@@ -5363,8 +6001,10 @@
   replaced by the new helper methods added in this release. (Robert Collins)
 
 
-bzr 1.0 2007-12-14
-##################
+bzr 1.0
+#######
+
+:Released: 2007-12-14
 
 Documentation
 *************
@@ -5381,8 +6021,10 @@
   (Ian Clatworthy)
 
 
-bzr 1.0rc3 2007-12-11
-#####################
+bzr 1.0rc3
+##########
+
+:Released: 2007-12-11
 
 Changes
 *******
@@ -5423,8 +6065,10 @@
   (Vincent Ladeuil)
 
 
-bzr 1.0rc2 2007-12-07
-#####################
+bzr 1.0rc2
+##########
+
+:Released: 2007-12-07
 
 Improvements
 ************
@@ -5498,8 +6142,10 @@
   ``started`` methods. (Matt Nordhoff)
 
 
-bzr 1.0rc1 2007-11-30
-#####################
+bzr 1.0rc1
+##########
+
+:Released: 2007-11-30
 
 Notes When Upgrading
 ********************
@@ -5775,8 +6421,10 @@
 * InventoryEntry.diff is now deprecated.  Please use diff.DiffTree instead.
 
 
-bzr 0.92 2007-11-05
-###################
+bzr 0.92
+########
+
+:Released: 2007-11-05
 
 Changes
 *******
@@ -5784,8 +6432,10 @@
   * New uninstaller on Win32.  (Alexander Belchenko)
 
 
-bzr 0.92rc1 2007-10-29
-######################
+bzr 0.92rc1
+###########
+
+:Released: 2007-10-29
 
 Changes
 *******
@@ -6104,8 +6754,10 @@
   raises a Python warning.  (Martin Pool)
 
 
-bzr 0.91 2007-09-26
-###################
+bzr 0.91
+########
+
+:Released: 2007-09-26
 
 Bug Fixes
 *********
@@ -6126,15 +6778,19 @@
   (Andrew Bennetts)
 
 
-bzr 0.91rc2 2007-09-11
-######################
+bzr 0.91rc2
+###########
+
+:Released: 2007-09-11
 
 * Replaced incorrect tarball for previous release; a debug statement was left
   in bzrlib/remote.py.
 
 
-bzr 0.91rc1 2007-09-11
-######################
+bzr 0.91rc1
+###########
+
+:Released: 2007-09-11
 
 Changes
 *******
@@ -6461,8 +7117,10 @@
   [] to revert all files is deprecated.  (Aaron Bentley)
 
 
-bzr 0.90 2007-08-28
-###################
+bzr 0.90
+########
+
+:Released: 2007-08-28
 
 Improvements
 ************
@@ -6485,8 +7143,10 @@
   and call ``create_repository`` on that.  (Martin Pool)
 
 
-bzr 0.90rc1 2007-08-14
-######################
+bzr 0.90rc1
+###########
+
+:Released: 2007-08-14
 
 Bugfixes
 ********
@@ -6758,8 +7418,10 @@
   parameter.  Varargs-style parameters are deprecated. (Aaron Bentley)
 
 
-bzr 0.18  2007-07-17
-####################
+bzr 0.18
+########
+
+:Released:  2007-07-17
 
 Bugfixes
 ********
@@ -6767,8 +7429,10 @@
 * Fix 'bzr add' crash under Win32 (Kuno Meyer)
 
 
-bzr 0.18rc1  2007-07-10
-#######################
+bzr 0.18rc1
+###########
+
+:Released:  2007-07-10
 
 Bugfixes
 ********
@@ -7019,8 +7683,10 @@
   (Vincent Ladeuil, #110448)
 
 
-bzr 0.17  2007-06-18
-####################
+bzr 0.17
+########
+
+:Released:  2007-06-18
 
 Bugfixes
 ********
@@ -7032,8 +7698,10 @@
   (Aaron Bentley, Ian Clatworthy, #120930)
 
 
-bzr 0.17rc1  2007-06-12
-#######################
+bzr 0.17rc1
+###########
+
+:Released:  2007-06-12
 
 Notes When Upgrading
 ********************
@@ -7134,8 +7802,10 @@
   your repository. (Previously Branch6 only supported revisions in your
   mainline). (John Arbash Meinel, #115343)
 
-bzr 0.16  2007-05-07
-####################
+bzr 0.16
+########
+
+:Released:  2007-05-07
 
 Bugfixes
 ********
@@ -7165,8 +7835,10 @@
   diff`` and ``bzr status`` significantly improving the speed of
   both. (John Arbash Meinel)
 
-bzr 0.16rc2  2007-04-30
-#######################
+bzr 0.16rc2
+###########
+
+:Released:  2007-04-30
 
 Bugfixes
 ********
@@ -7189,8 +7861,10 @@
 * ``WorkingTree4._iter_changes`` should not descend into unversioned
   directories. (John Arbash Meinel, #110399)
 
-bzr 0.16rc1  2007-04-26
-#######################
+bzr 0.16rc1
+###########
+
+:Released:  2007-04-26
 
 Notes When Upgrading
 ********************
@@ -7497,8 +8171,10 @@
   implementation of new auth schemes for both http and proxy.
   (Vincent Ladeuil)
 
-bzr 0.15 2007-04-01
-###################
+bzr 0.15
+########
+
+:Released: 2007-04-01
 
 Bugfixes
 ********
@@ -7510,8 +8186,10 @@
   checking out a branch that contains an old-format working tree.
   (Martin Pool)
 
-bzr 0.15rc3  2007-03-26
-#######################
+bzr 0.15rc3
+###########
+
+:Released:  2007-03-26
 
 Changes
 *******
@@ -7586,8 +8264,10 @@
 * Correctly handles mutiple permanent http redirections.
   (vila, #88780)
 
-bzr 0.15rc2  2007-03-14
-#######################
+bzr 0.15rc2
+###########
+
+:Released:  2007-03-14
 
 Notes When Upgrading
 ********************
@@ -7640,8 +8320,10 @@
   (Wouter van Heyst, #53483)
 
 
-bzr 0.15rc1  2007-03-07
-#######################
+bzr 0.15rc1
+###########
+
+:Released:  2007-03-07
 
 Surprises
 *********
@@ -7871,8 +8553,10 @@
   before the rest of the suite.  (Martin Pool)
 
 
-bzr 0.14  2007-01-23
-####################
+bzr 0.14
+########
+
+:Released:  2007-01-23
 
 Improvements
 ************
@@ -7891,8 +8575,10 @@
   it is in is below a repository. (James Westby, #77306)
 
 
-bzr 0.14rc1  2007-01-16
-#######################
+bzr 0.14rc1
+###########
+
+:Released:  2007-01-16
 
 Improvements
 ************
@@ -8026,14 +8712,18 @@
   (Alexander Belchenko, #68124)
 
 
-bzr 0.13  2006-12-05
-####################
+bzr 0.13
+########
+
+:Released:  2006-12-05
 
 No changes from 0.13rc
 
 
-bzr 0.13rc1  2006-11-27
-#######################
+bzr 0.13rc1
+###########
+
+:Released:  2006-11-27
 
 Improvements
 ************
@@ -8158,8 +8848,10 @@
 * TestingHTTPRequestHandler really handles the Range header
   (previously it was ignoring it and returning the whole file,).
 
-bzr 0.12  2006-10-30
-####################
+bzr 0.12
+########
+
+:Released:  2006-10-30
 
 Internals
 *********
@@ -8168,8 +8860,10 @@
   and remove benchmarks that take longer than 10min to run.
   (John Arbash Meinel)
 
-bzr 0.12rc1  2006-10-23
-#######################
+bzr 0.12rc1
+###########
+
+:Released:  2006-10-23
 
 Improvements
 ************
@@ -8290,13 +8984,17 @@
   option to set the BzrDir, Repository and Branch formats of the
   created objects. (Robert Collins, John Arbash Meinel)
 
-bzr 0.11  2006-10-02
-####################
+bzr 0.11
+########
+
+:Released:  2006-10-02
 
 * Smart server transport test failures on windows fixed. (Lukáš Lalinský).
 
-bzr 0.11rc2  2006-09-27
-#######################
+bzr 0.11rc2
+###########
+
+:Released:  2006-09-27
 
 Bug Fixes
 *********
@@ -8306,8 +9004,10 @@
 * Commit performance regression fixed. (Aaron Bentley, Robert Collins, John
   Arbash Meinel).
 
-bzr 0.11rc1  2006-09-25
-#######################
+bzr 0.11rc1
+###########
+
+:Released:  2006-09-25
 
 Improvements
 ************
@@ -8500,8 +9200,10 @@
   allow upgrades to a richer interface than the VFS one provided by
   Transport. (Andrew Bennetts, Martin Pool)
 
-bzr 0.10  2006-08-29
-####################
+bzr 0.10
+########
+
+:Released:  2006-08-29
 
 Improvements
 ************
@@ -8578,8 +9280,10 @@
   need to be installed. This should help make the life of packagers
   easier. (John Arbash Meinel)
 
-bzr 0.9.0  2006-08-11
-#####################
+bzr 0.9.0
+#########
+
+:Released:  2006-08-11
 
 Surprises
 *********
@@ -8846,16 +9550,20 @@
 
 * "RevisionTree" is now in bzrlib/revisiontree.py. (Robert Collins)
 
-bzr 0.8.2  2006-05-17
-#####################
+bzr 0.8.2
+#########
+
+:Released:  2006-05-17
 
 Bug Fixes
 *********
 
 * setup.py failed to install launchpad plugin.  (Martin Pool)
 
-bzr 0.8.1  2006-05-16
-#####################
+bzr 0.8.1
+#########
+
+:Released:  2006-05-16
 
 Bug Fixes
 *********
@@ -8908,8 +9616,10 @@
 * Fix test case for bzr info in upgrading a standalone branch to metadir,
   uses bzrlib api now. (Olaf Conradi)
 
-bzr 0.8  2006-05-08
-###################
+bzr 0.8
+#######
+
+:Released:  2006-05-08
 
 Notes When Upgrading
 ********************
@@ -9170,8 +9880,10 @@
 * ``run_bzr`` and ``run_bzr_captured`` now accept a 'stdin="foo"'
   parameter which will provide String("foo") to the command as its stdin.
 
-bzr 0.7 2006-01-09
-##################
+bzr 0.7
+#######
+
+:Released: 2006-01-09
 
 Changes
 *******
@@ -9466,8 +10178,10 @@
 * ``bzrlib.osutils.safe_unicode`` now exists to provide parameter coercion
   for functions that need unicode strings. (Robert Collins)
 
-bzr 0.6 2005-10-28
-##################
+bzr 0.6
+#######
+
+:Released: 2005-10-28
 
 Improvements
 ************
@@ -9682,8 +10396,10 @@
   [-1] in the revision-history. (Andres Salomon)
 
 
-bzr 0.1.1 2005-10-12
-####################
+bzr 0.1.1
+#########
+
+:Released: 2005-10-12
 
 Bug Fixes
 *********
@@ -9700,8 +10416,10 @@
 * Avoid some unnecessary http operations in branch and pull.
 
 
-bzr 0.1 2005-10-11
-##################
+bzr 0.1
+#######
+
+:Released: 2005-10-11
 
 Notes
 *****
@@ -9830,8 +10548,10 @@
   of tests to run, e.g. ``bzr selftest test_weave``
 
 
-bzr 0.0.9 2005-09-23
-####################
+bzr 0.0.9
+#########
+
+:Released: 2005-09-23
 
 Bug Fixes
 *********
@@ -9872,8 +10592,11 @@
   another (used by pull, merged, etc.)
 
 
-bzr 0.0.8 2005-09-20
-####################
+bzr 0.0.8
+#########
+
+:Released: 2005-09-20
+
 
 Improvements
 ************
@@ -9923,8 +10646,10 @@
 * Quieten warnings about locking; patch from Matt Lavin.
 
 
-bzr-0.0.7 2005-09-02
-####################
+bzr-0.0.7
+#########
+
+:Released: 2005-09-02
 
 New Features
 ************
@@ -9978,8 +10703,10 @@
 
 
 
-bzr-0.0.6 2005-08-18
-####################
+bzr-0.0.6
+#########
+
+:Released: 2005-08-18
 
 New Features
 ************
@@ -10064,8 +10791,10 @@
 * Fix bugs in committing only selected files or within a subdirectory.
 
 
-bzr-0.0.5  2005-06-15
-#####################
+bzr-0.0.5
+#########
+
+:Released:  2005-06-15
 
 Changes
 *******
@@ -10206,8 +10935,10 @@
   2.4 is now only recommended.
 
 
-bzr-0.0.4  2005-04-22
-#####################
+bzr-0.0.4
+#########
+
+:Released:  2005-04-22
 
 Enhancements
 ************
@@ -10275,8 +11006,10 @@
   from QuantumG.
 
 
-bzr-0.0.3  2005-04-06
-#####################
+bzr-0.0.3
+#########
+
+:Released:  2005-04-06
 
 Enhancements
 ************
@@ -10330,8 +11063,11 @@
 * Win32 fixes from Steve Brown.
 
 
-bzr-0.0.2  "black cube"  2005-03-31
-###################################
+bzr-0.0.2
+#########
+
+:Codename: "black cube"
+:Released: 2005-03-31
 
 Enhancements
 ************
@@ -10359,8 +11095,10 @@
   the root directory only.
 
 
-bzr-0.0.1  2005-03-26
-#####################
+bzr-0.0.1
+#########
+
+:Released:  2005-03-26
 
 Enhancements
 ************
@@ -10389,8 +11127,10 @@
   supported).
 
 
-bzr-0.0.0.69  2005-03-22
-########################
+bzr-0.0.0.69
+############
+
+:Released:  2005-03-22
 
 Enhancements
 ************
@@ -10400,5 +11140,38 @@
 * Storage of local versions: init, add, remove, rm, info, log,
   diff, status, etc.
 
+
+bzr ?.?.? (not released yet)
+############################
+
+:Codename: template
+:2.0.2: ???
+
+Compatibility Breaks
+********************
+
+New Features
+************
+
+Bug Fixes
+*********
+
+Improvements
+************
+
+Documentation
+*************
+
+API Changes
+***********
+
+Internals
+*********
+
+Testing
+*******
+
+
+
 ..
    vim: tw=74 ft=rst ff=unix
 
=== modified file 'README'
--- README	2008-03-16 14:01:20 +0000
+++ README	2009-11-03 09:40:55 +0000
@@ -3,72 +3,44 @@
 =================
 
 Bazaar (``bzr``) is a decentralized revision control system, designed to be
-easy for developers and end users alike.  Bazaar is part of the GNU project to
-develop a complete free operating system.
-
-To install Bazaar from source, follow the instructions in the INSTALL
-file. Otherwise, you may want to check your distribution package manager
-for ready-to-install packages, or http://bazaar-vcs.org/DistroDownloads.
-
-To learn how to use Bazaar, check the documentation in the doc/ directory.
-Once installed, you can also run 'bzr help'. An always up-to-date and more
-complete set of documents can be found in the Bazaar website, at:
-
-    http://bazaar-vcs.org/Documentation
+easy for developers and end users alike.  Bazaar is part of the GNU project
+to develop a complete free operating system.
+
+To install Bazaar, follow the instructions given at
+http://bazaar-vcs.org/Download. Ready-to-install packages are available
+for most popular operating systems or you can install from source.
+
+To learn how to use Bazaar, see the official documentation at:
+
+    http://doc.bazaar-vcs.org/en/
+
+For additional training materials including screencasts and slides,
+visit our community wiki documentation page at:
+
+    http://bazaar-vcs.org/Documentation/
 
 Bazaar is written in Python, and is sponsored by Canonical Limited, the
 founders of Ubuntu and Launchpad. Bazaar is Free Software, and is released
 under the GNU General Public License.
 
-Bazaar was formerly known as Bazaar-NG.  It's the successor to ``baz``, a fork
-of GNU arch, but shares no code.  (To upgrade from Baz, use the ``baz-import``
-command in the bzrtools plugin.)
-
 Bazaar highlights
 =================
 
-* Easy to use and intuitive.
-
-  Only five commands are needed to do all basic operations, and all
-  commands have documentation accessible via 'bzr help command'.
-  Bazaar's interface is also easy to learn for CVS and Subversion users.
-
-* Robust and reliable.
-
-  Bazaar is developed under an extensive test suite. Branches can be
-  checked and verified for integrity at any time, and revisions can be
-  signed with PGP/GnuPG.
-
-* Publish branches with HTTP.
-
-  Branches can be hosted on an HTTP server with no need for special
-  software on the server side. Branches can be uploaded by bzr itself
-  over SSH (SFTP), or with rsync.
-
-* Adapts to multiple environments.
-
-  Bazaar runs on Linux and Windows, fully supports Unicode filenames,
-  and suits different development models, including centralized.
-
-* Easily extended and customized.
-
-  A rich Python interface is provided for extending and embedding,
-  including a plugin interface. There are already many available plugins,
-  most of them registered at http://bazaar-vcs.org/PluginRegistry.
-
-* Smart merging.
-
-  Changes will never be merged more than once, conflicts will be
-  minimized, and identical changes are dealt with well.
-
-* Vibrant and active community.
-
-  Help with Bazaar is obtained easily, via the mailing list, or the IRC
-  channel.
-
-
-Registration and Feedback
-=========================
+Bazaar directly supports both central version control (like cvs/svn) and
+distributed version control (like git/hg). Developers can organize their
+workspace in whichever way they want on a per project basis including:
+
+* checkouts (like svn)
+* feature branches (like hg)
+* shared working tree (like git).
+
+It also directly supports and encourages a large number of development best
+practices like refactoring and pre-commit regression testing. Users can
+choose between our command line tool and our cross-platform GUI application.
+For further details, see our website at http://bazaar-vcs.org/en.
+
+Feedback
+========
 
 If you encounter any problems with Bazaar, need help understanding it, or would
 like to offer suggestions or feedback, please get in touch with us:
@@ -76,7 +48,7 @@
 * Ask a question through our web support interface, at 
   https://answers.launchpad.net/bzr/
 
-* Report bugs at https://bugs.edge.launchpad.net/bzr/+filebug
+* Report bugs at https://bugs.launchpad.net/bzr/+filebug
 
 * Write to us at [e-mail] 
   You can join the list at <https://lists.ubuntu.com/mailman/listinfo/bazaar>.
@@ -85,12 +57,8 @@
 
 * Talk to us in irc://irc.ubuntu.com/bzr
 
-* And see http://bazaar-vcs.org/BzrSupport for more.
-
-If you would like to help us improve Bazaar by telling us about yourself and
-what we could do better, please register and complete the online survey here:
-http://www.surveymonkey.com/s.aspx?sm=L94RvLswhKdktrxiHWiX3g_3d_3d.  
-Registration is completely optional. 
-
-Enjoy,
+Our mission is to make a version control tool that developers LOVE to use
+and that casual contributors feel confident with. Please let us know how
+we're going.
+
 The Bazaar Team
 
=== modified file 'bzrlib/__init__.py'
--- bzrlib/__init__.py	2009-08-30 21:34:42 +0000
+++ bzrlib/__init__.py	2009-11-03 09:40:55 +0000
@@ -31,16 +31,10 @@
     import bzrlib.lazy_regex
     bzrlib.lazy_regex.install_lazy_compile()
 
-from bzrlib.osutils import get_user_encoding
-
 
 IGNORE_FILENAME = ".bzrignore"
 
 
-# XXX: Deprecated as of bzr-1.17 use osutils.get_user_encoding() directly
-user_encoding = get_user_encoding()
-
-
 __copyright__ = "Copyright 2005, 2006, 2007, 2008, 2009 Canonical Ltd."
 
 # same format as sys.version_info: "A tuple containing the five components of
@@ -50,10 +44,11 @@
 # Python version 2.0 is (2, 0, 0, 'final', 0)."  Additionally we use a
 # releaselevel of 'dev' for unreleased under-development code.
 
-version_info = (2, 1, 0, 'dev', 0)
+version_info = (2, 1, 0, 'dev', 3)
 
 # API compatibility version: bzrlib is currently API compatible with 1.15.
-api_minimum_version = (1, 17, 0)
+api_minimum_version = (2, 1, 0)
+
 
 def _format_version_tuple(version_info):
     """Turn a version number 2, 3 or 5-tuple into a short string.
@@ -65,23 +60,29 @@
     zero for final releases.
 
     >>> print _format_version_tuple((1, 0, 0, 'final', 0))
-    1.0
+    1.0.0
     >>> print _format_version_tuple((1, 2, 0, 'dev', 0))
-    1.2dev
+    1.2.0dev
+    >>> print bzrlib._format_version_tuple((1, 2, 0, 'dev', 1))
+    1.2.0dev1
     >>> print _format_version_tuple((1, 1, 1, 'candidate', 2))
     1.1.1rc2
     >>> print bzrlib._format_version_tuple((2, 1, 0, 'beta', 1))
-    2.1b1
+    2.1.0b1
     >>> print _format_version_tuple((1, 4, 0))
-    1.4
+    1.4.0
     >>> print _format_version_tuple((1, 4))
     1.4
+    >>> print bzrlib._format_version_tuple((2, 1, 0, 'final', 1))
+    Traceback (most recent call last):
+    ...
+    ValueError: version_info (2, 1, 0, 'final', 1) not valid
     >>> print _format_version_tuple((1, 4, 0, 'wibble', 0))
     Traceback (most recent call last):
     ...
     ValueError: version_info (1, 4, 0, 'wibble', 0) not valid
     """
-    if len(version_info) == 2 or version_info[2] == 0:
+    if len(version_info) == 2:
         main_version = '%d.%d' % version_info[:2]
     else:
         main_version = '%d.%d.%d' % version_info[:3]
@@ -96,6 +97,8 @@
         sub_string = ''
     elif release_type == 'dev' and sub == 0:
         sub_string = 'dev'
+    elif release_type == 'dev':
+        sub_string = 'dev' + str(sub)
     elif release_type in ('alpha', 'beta'):
         sub_string = release_type[0] + str(sub)
     elif release_type == 'candidate':
@@ -103,7 +106,6 @@
     else:
         raise ValueError("version_info %r not valid" % (version_info,))
 
-    version_string = '%d.%d.%d.%s.%d' % tuple(version_info)
     return main_version + sub_string
 
 
 
=== modified file 'bzrlib/_bencode_pyx.pyx'
--- bzrlib/_bencode_pyx.pyx	2009-06-05 01:48:32 +0000
+++ bzrlib/_bencode_pyx.pyx	2009-11-03 09:40:55 +0000
@@ -58,6 +58,13 @@
     void D_UPDATE_TAIL(Decoder, int n)
     void E_UPDATE_TAIL(Encoder, int n)
 
+# To maintain compatibility with older versions of pyrex, we have to use the
+# relative import here, rather than 'bzrlib._static_tuple_c'
+from _static_tuple_c cimport StaticTuple, StaticTuple_CheckExact, \
+    import_static_tuple_c
+
+import_static_tuple_c()
+
 
 cdef class Decoder:
     """Bencode decoder"""
@@ -371,7 +378,8 @@
                 self._encode_int(x)
             elif PyLong_CheckExact(x):
                 self._encode_long(x)
-            elif PyList_CheckExact(x) or PyTuple_CheckExact(x):
+            elif (PyList_CheckExact(x) or PyTuple_CheckExact(x)
+                  or StaticTuple_CheckExact(x)):
                 self._encode_list(x)
             elif PyDict_CheckExact(x):
                 self._encode_dict(x)
 
=== modified file 'bzrlib/_btree_serializer_pyx.pyx'
--- bzrlib/_btree_serializer_pyx.pyx	2009-06-22 12:52:39 +0000
+++ bzrlib/_btree_serializer_pyx.pyx	2009-11-03 09:40:55 +0000
@@ -1,4 +1,4 @@
-# Copyright (C) 2008 Canonical Ltd
+# Copyright (C) 2008, 2009 Canonical Ltd
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -38,11 +38,16 @@
     Py_ssize_t PyString_Size(object p)
     Py_ssize_t PyString_GET_SIZE_ptr "PyString_GET_SIZE" (PyObject *)
     char * PyString_AS_STRING_ptr "PyString_AS_STRING" (PyObject *)
+    char * PyString_AS_STRING(object)
+    Py_ssize_t PyString_GET_SIZE(object)
     int PyString_AsStringAndSize_ptr(PyObject *, char **buf, Py_ssize_t *len)
     void PyString_InternInPlace(PyObject **)
     int PyTuple_CheckExact(object t)
+    object PyTuple_New(Py_ssize_t n_entries)
+    void PyTuple_SET_ITEM(object, Py_ssize_t offset, object) # steals the ref
     Py_ssize_t PyTuple_GET_SIZE(object t)
     PyObject *PyTuple_GET_ITEM_ptr_object "PyTuple_GET_ITEM" (object tpl, int index)
+    void Py_INCREF(object)
     void Py_DECREF_ptr "Py_DECREF" (PyObject *)
 
 cdef extern from "string.h":
@@ -52,6 +57,12 @@
     # void *memrchr(void *s, int c, size_t n)
     int strncmp(char *s1, char *s2, size_t n)
 
+# It seems we need to import the definitions so that the pyrex compiler has
+# local names to access them.
+from _static_tuple_c cimport StaticTuple, \
+    import_static_tuple_c, StaticTuple_New, \
+    StaticTuple_Intern, StaticTuple_SET_ITEM, StaticTuple_CheckExact
+
 
 # TODO: Find some way to import this from _dirstate_helpers
 cdef void* _my_memrchr(void *s, int c, size_t n):
@@ -68,6 +79,7 @@
         pos = pos - 1
     return NULL
 
+
 # TODO: Import this from _dirstate_helpers when it is merged
 cdef object safe_string_from_size(char *s, Py_ssize_t size):
     if size < 0:
@@ -91,6 +103,10 @@
     Py_DECREF_ptr(py_str)
     return result
 
+from bzrlib import _static_tuple_c
+# This sets up the StaticTuple C_API functionality
+import_static_tuple_c()
+
 
 cdef class BTreeLeafParser:
     """Parse the leaf nodes of a BTree index.
@@ -130,6 +146,7 @@
         self._cur_str = NULL
         self._end_str = NULL
         self._header_found = 0
+        # keys are tuples
 
     cdef extract_key(self, char * last):
         """Extract a key.
@@ -139,15 +156,14 @@
         """
         cdef char *temp_ptr
         cdef int loop_counter
-        # keys are tuples
-        loop_counter = 0
-        key_segments = []
-        while loop_counter < self.key_length:
-            loop_counter = loop_counter + 1
+        cdef StaticTuple key
+
+        key = StaticTuple_New(self.key_length)
+        for loop_counter from 0 <= loop_counter < self.key_length:
             # grab a key segment
             temp_ptr = <char*>memchr(self._start, c'\0', last - self._start)
             if temp_ptr == NULL:
-                if loop_counter == self.key_length:
+                if loop_counter + 1 == self.key_length:
                     # capture to last
                     temp_ptr = last
                 else:
@@ -157,15 +173,20 @@
                                                    last - self._start)))
                     raise AssertionError(failure_string)
             # capture the key string
-            # TODO: Consider using PyIntern_FromString, the only caveat is that
-            # it assumes a NULL-terminated string, so we have to check if
-            # temp_ptr[0] == c'\0' or some other char.
-            key_element = safe_interned_string_from_size(self._start,
+            if (self.key_length == 1
+                and (temp_ptr - self._start) == 45
+                and strncmp(self._start, 'sha1:', 5) == 0):
+                key_element = safe_string_from_size(self._start,
+                                                    temp_ptr - self._start)
+            else:
+                key_element = safe_interned_string_from_size(self._start,
                                                          temp_ptr - self._start)
             # advance our pointer
             self._start = temp_ptr + 1
-            PyList_Append(key_segments, key_element)
-        return tuple(key_segments)
+            Py_INCREF(key_element)
+            StaticTuple_SET_ITEM(key, loop_counter, key_element)
+        key = StaticTuple_Intern(key)
+        return key
 
     cdef int process_line(self) except -1:
         """Process a line in the bytes."""
@@ -174,6 +195,7 @@
         cdef char *ref_ptr
         cdef char *next_start
         cdef int loop_counter
+        cdef Py_ssize_t str_len
 
         self._start = self._cur_str
         # Find the next newline
@@ -186,14 +208,13 @@
             # And the next string is right after it
             self._cur_str = last + 1
             # The last character is right before the '\n'
-            last = last
 
         if last == self._start:
             # parsed it all.
             return 0
         if last < self._start:
             # Unexpected error condition - fail
-            return -1
+            raise AssertionError("last < self._start")
         if 0 == self._header_found:
             # The first line in a leaf node is the header "type=leaf\n"
             if strncmp("type=leaf", self._start, last - self._start) == 0:
@@ -202,36 +223,48 @@
             else:
                 raise AssertionError('Node did not start with "type=leaf": %r'
                     % (safe_string_from_size(self._start, last - self._start)))
-                return -1
 
         key = self.extract_key(last)
         # find the value area
         temp_ptr = <char*>_my_memrchr(self._start, c'\0', last - self._start)
         if temp_ptr == NULL:
             # Invalid line
-            return -1
+            raise AssertionError("Failed to find the value area")
         else:
-            # capture the value string
-            value = safe_string_from_size(temp_ptr + 1, last - temp_ptr - 1)
+            # Because of how conversions were done, we ended up with *lots* of
+            # values that are identical. These are all of the 0-length nodes
+            # that are referred to by the TREE_ROOT (and likely some other
+            # directory nodes.) For example, bzr has 25k references to
+            # something like '12607215 328306 0 0', which ends up consuming 1MB
+            # of memory, just for those strings.
+            str_len = last - temp_ptr - 1
+            if (str_len > 4
+                and strncmp(" 0 0", last - 4, 4) == 0):
+                # This drops peak mem for bzr.dev from 87.4MB => 86.2MB
+                # For Launchpad 236MB => 232MB
+                value = safe_interned_string_from_size(temp_ptr + 1, str_len)
+            else:
+                value = safe_string_from_size(temp_ptr + 1, str_len)
             # shrink the references end point
             last = temp_ptr
+
         if self.ref_list_length:
-            ref_lists = []
+            ref_lists = StaticTuple_New(self.ref_list_length)
             loop_counter = 0
             while loop_counter < self.ref_list_length:
                 ref_list = []
                 # extract a reference list
                 loop_counter = loop_counter + 1
                 if last < self._start:
-                    return -1
+                    raise AssertionError("last < self._start")
                 # find the next reference list end point:
                 temp_ptr = <char*>memchr(self._start, c'\t', last - self._start)
                 if temp_ptr == NULL:
                     # Only valid for the last list
                     if loop_counter != self.ref_list_length:
                         # Invalid line
-                        return -1
-                        raise AssertionError("invalid key")
+                        raise AssertionError(
+                            "invalid key, loop_counter != self.ref_list_length")
                     else:
                         # scan to the end of the ref list area
                         ref_ptr = last
@@ -248,18 +281,20 @@
                     if temp_ptr == NULL:
                         # key runs to the end
                         temp_ptr = ref_ptr
+
                     PyList_Append(ref_list, self.extract_key(temp_ptr))
-                PyList_Append(ref_lists, tuple(ref_list))
+                ref_list = StaticTuple_Intern(StaticTuple(*ref_list))
+                Py_INCREF(ref_list)
+                StaticTuple_SET_ITEM(ref_lists, loop_counter - 1, ref_list)
                 # prepare for the next reference list
                 self._start = next_start
-            ref_lists = tuple(ref_lists)
-            node_value = (value, ref_lists)
+            node_value = StaticTuple(value, ref_lists)
         else:
             if last != self._start:
                 # unexpected reference data present
-                return -1
-            node_value = (value, ())
-        PyList_Append(self.keys, (key, node_value))
+                raise AssertionError("unexpected reference data present")
+            node_value = StaticTuple(value, StaticTuple())
+        PyList_Append(self.keys, StaticTuple(key, node_value))
         return 0
 
     def parse(self):
@@ -294,7 +329,6 @@
     cdef Py_ssize_t flat_len
     cdef Py_ssize_t key_len
     cdef Py_ssize_t node_len
-    cdef PyObject * val
     cdef char * value
     cdef Py_ssize_t value_len
     cdef char * out
@@ -303,13 +337,12 @@
     cdef int first_ref_list
     cdef int first_reference
     cdef int i
-    cdef PyObject *ref_bit
     cdef Py_ssize_t ref_bit_len
 
-    if not PyTuple_CheckExact(node):
-        raise TypeError('We expected a tuple() for node not: %s'
+    if not PyTuple_CheckExact(node) and not StaticTuple_CheckExact(node):
+        raise TypeError('We expected a tuple() or StaticTuple() for node not: %s'
             % type(node))
-    node_len = PyTuple_GET_SIZE(node)
+    node_len = len(node)
     have_reference_lists = reference_lists
     if have_reference_lists:
         if node_len != 4:
@@ -318,8 +351,17 @@
     elif node_len < 3:
         raise ValueError('Without ref_lists, we need at least 3 entries not: %s'
             % len(node))
-    # I don't expect that we can do faster than string.join()
-    string_key = '\0'.join(<object>PyTuple_GET_ITEM_ptr_object(node, 1))
+    # TODO: We can probably do better than string.join(), namely
+    #       when key has only 1 item, we can just grab that string
+    #       And when there are 2 items, we could do a single malloc + len() + 1
+    #       also, doing .join() requires a PyObject_GetAttrString call, which
+    #       we could also avoid.
+    # TODO: Note that pyrex 0.9.6 generates fairly crummy code here, using the
+    #       python object interface, versus 0.9.8+ which uses a helper that
+    #       checks if this supports the sequence interface.
+    #       We *could* do more work on our own, and grab the actual items
+    #       lists. For now, just ask people to use a better compiler. :)
+    string_key = '\0'.join(node[1])
 
     # TODO: instead of using string joins, precompute the final string length,
     #       and then malloc a single string and copy everything in.
@@ -336,7 +378,7 @@
     refs_len = 0
     if have_reference_lists:
         # Figure out how many bytes it will take to store the references
-        ref_lists = <object>PyTuple_GET_ITEM_ptr_object(node, 3)
+        ref_lists = node[3]
         next_len = len(ref_lists) # TODO: use a Py function
         if next_len > 0:
             # If there are no nodes, we don't need to do any work
@@ -350,31 +392,31 @@
                     # references
                     refs_len = refs_len + (next_len - 1)
                     for reference in ref_list:
-                        if not PyTuple_CheckExact(reference):
+                        if (not PyTuple_CheckExact(reference)
+                            and not StaticTuple_CheckExact(reference)):
                             raise TypeError(
                                 'We expect references to be tuples not: %s'
                                 % type(reference))
-                        next_len = PyTuple_GET_SIZE(reference)
+                        next_len = len(reference)
                         if next_len > 0:
                             # We will need (len - 1) '\x00' characters to
                             # separate the reference key
                             refs_len = refs_len + (next_len - 1)
-                            for i from 0 <= i < next_len:
-                                ref_bit = PyTuple_GET_ITEM_ptr_object(reference, i)
-                                if not PyString_CheckExact_ptr(ref_bit):
+                            for ref_bit in reference:
+                                if not PyString_CheckExact(ref_bit):
                                     raise TypeError('We expect reference bits'
                                         ' to be strings not: %s'
                                         % type(<object>ref_bit))
-                                refs_len = refs_len + PyString_GET_SIZE_ptr(ref_bit)
+                                refs_len = refs_len + PyString_GET_SIZE(ref_bit)
 
     # So we have the (key NULL refs NULL value LF)
     key_len = PyString_Size(string_key)
-    val = PyTuple_GET_ITEM_ptr_object(node, 2)
-    if not PyString_CheckExact_ptr(val):
+    val = node[2]
+    if not PyString_CheckExact(val):
         raise TypeError('Expected a plain str for value not: %s'
-                        % type(<object>val))
-    value = PyString_AS_STRING_ptr(val)
-    value_len = PyString_GET_SIZE_ptr(val)
+                        % type(val))
+    value = PyString_AS_STRING(val)
+    value_len = PyString_GET_SIZE(val)
     flat_len = (key_len + 1 + refs_len + 1 + value_len + 1)
     line = PyString_FromStringAndSize(NULL, flat_len)
     # Get a pointer to the new buffer
@@ -396,14 +438,14 @@
                     out[0] = c'\r'
                     out = out + 1
                 first_reference = 0
-                next_len = PyTuple_GET_SIZE(reference)
+                next_len = len(reference)
                 for i from 0 <= i < next_len:
                     if i != 0:
                         out[0] = c'\x00'
                         out = out + 1
-                    ref_bit = PyTuple_GET_ITEM_ptr_object(reference, i)
-                    ref_bit_len = PyString_GET_SIZE_ptr(ref_bit)
-                    memcpy(out, PyString_AS_STRING_ptr(ref_bit), ref_bit_len)
+                    ref_bit = reference[i]
+                    ref_bit_len = PyString_GET_SIZE(ref_bit)
+                    memcpy(out, PyString_AS_STRING(ref_bit), ref_bit_len)
                     out = out + ref_bit_len
     out[0] = c'\0'
     out = out  + 1
 
=== modified file 'bzrlib/_chk_map_py.py'
--- bzrlib/_chk_map_py.py	2009-04-09 20:23:07 +0000
+++ bzrlib/_chk_map_py.py	2009-11-03 09:40:55 +0000
@@ -19,6 +19,8 @@
 import zlib
 import struct
 
+from bzrlib.static_tuple import StaticTuple
+
 _LeafNode = None
 _InternalNode = None
 _unknown = None
@@ -93,7 +95,7 @@
         value_lines = lines[pos:pos+num_value_lines]
         pos += num_value_lines
         value = '\n'.join(value_lines)
-        items[tuple(elements[:-1])] = value
+        items[StaticTuple.from_sequence(elements[:-1])] = value
     if len(items) != length:
         raise AssertionError("item count (%d) mismatch for key %s,"
             " bytes %r" % (length, key, bytes))
@@ -141,7 +143,7 @@
     for line in lines[5:]:
         line = common_prefix + line
         prefix, flat_key = line.rsplit('\x00', 1)
-        items[prefix] = (flat_key,)
+        items[prefix] = StaticTuple(flat_key,)
     if len(items) == 0:
         raise AssertionError("We didn't find any item for %s" % key)
     result._items = items
@@ -155,4 +157,3 @@
     result._node_width = len(prefix)
     result._search_prefix = common_prefix
     return result
-
 
=== modified file 'bzrlib/_chk_map_pyx.pyx'
--- bzrlib/_chk_map_pyx.pyx	2009-06-22 12:52:39 +0000
+++ bzrlib/_chk_map_pyx.pyx	2009-11-03 09:40:54 +0000
@@ -29,9 +29,8 @@
 
 cdef extern from "Python.h":
     ctypedef int Py_ssize_t # Required for older pyrex versions
-    struct _PyObject:
+    ctypedef struct PyObject:
         pass
-    ctypedef _PyObject PyObject
     int PyTuple_CheckExact(object p)
     Py_ssize_t PyTuple_GET_SIZE(object t)
     int PyString_CheckExact(object)
@@ -52,6 +51,18 @@
     char *PyString_AS_STRING_ptr "PyString_AS_STRING" (PyObject *s)
     object PyString_FromStringAndSize(char*, Py_ssize_t)
 
+# cimport all of the definitions we will need to access
+from _static_tuple_c cimport StaticTuple,\
+    import_static_tuple_c, StaticTuple_New, \
+    StaticTuple_Intern, StaticTuple_SET_ITEM, StaticTuple_CheckExact
+
+cdef extern from "_static_tuple_c.h":
+    # Defined explicitly rather than cimport-ing. Trying to use cimport, the
+    # type for PyObject is a different class that happens to have the same
+    # name...
+    PyObject * StaticTuple_GET_ITEM_ptr "StaticTuple_GET_ITEM" (StaticTuple,
+                                                                Py_ssize_t)
+
 cdef extern from "zlib.h":
     ctypedef unsigned long uLong
     ctypedef unsigned int uInt
@@ -60,8 +71,14 @@
     uLong crc32(uLong crc, Bytef *buf, uInt len)
 
 
+# Set up the StaticTuple C_API functionality
+import_static_tuple_c()
+
+cdef object _LeafNode
 _LeafNode = None
+cdef object _InternalNode
 _InternalNode = None
+cdef object _unknown
 _unknown = None
 
 # We shouldn't just copy this from _dirstate_helpers_pyx
@@ -91,9 +108,9 @@
     cdef char *c_out
     cdef PyObject *bit
 
-    if not PyTuple_CheckExact(key):
-        raise TypeError('key %r is not a tuple' % (key,))
-    num_bits = PyTuple_GET_SIZE(key)
+    if not StaticTuple_CheckExact(key):
+        raise TypeError('key %r is not a StaticTuple' % (key,))
+    num_bits = len(key)
     # 4 bytes per crc32, and another 1 byte between bits
     num_out_bytes = (9 * num_bits) - 1
     out = PyString_FromStringAndSize(NULL, num_out_bytes)
@@ -105,7 +122,7 @@
         # We use the _ptr variant, because GET_ITEM returns a borrowed
         # reference, and Pyrex assumes that returned 'object' are a new
         # reference
-        bit = PyTuple_GET_ITEM_ptr(key, i)
+        bit = StaticTuple_GET_ITEM_ptr(key, i)
         if not PyString_CheckExact_ptr(bit):
             raise TypeError('Bit %d of %r is not a string' % (i, key))
         c_bit = <Bytef *>PyString_AS_STRING_ptr(bit)
@@ -129,9 +146,9 @@
     cdef char *c_out
     cdef PyObject *bit
 
-    if not PyTuple_CheckExact(key):
-        raise TypeError('key %r is not a tuple' % (key,))
-    num_bits = PyTuple_GET_SIZE(key)
+    if not StaticTuple_CheckExact(key):
+        raise TypeError('key %r is not a StaticTuple' % (key,))
+    num_bits = len(key)
     # 4 bytes per crc32, and another 1 byte between bits
     num_out_bytes = (5 * num_bits) - 1
     out = PyString_FromStringAndSize(NULL, num_out_bytes)
@@ -140,10 +157,10 @@
         if i > 0:
             c_out[0] = c'\x00'
             c_out = c_out + 1
-        bit = PyTuple_GET_ITEM_ptr(key, i)
+        bit = StaticTuple_GET_ITEM_ptr(key, i)
         if not PyString_CheckExact_ptr(bit):
-            raise TypeError('Bit %d of %r is not a string: %r' % (i, key,
-            <object>bit))
+            raise TypeError('Bit %d of %r is not a string: %r'
+                            % (i, key, <object>bit))
         c_bit = <Bytef *>PyString_AS_STRING_ptr(bit)
         c_len = PyString_GET_SIZE_ptr(bit)
         crc_val = crc32(0, c_bit, c_len)
@@ -195,6 +212,7 @@
     cdef char *prefix, *value_start, *prefix_tail
     cdef char *next_null, *last_null, *line_start
     cdef char *c_entry, *entry_start
+    cdef StaticTuple entry_bits
 
     if _LeafNode is None:
         from bzrlib import chk_map
@@ -265,12 +283,14 @@
             if next_line == NULL:
                 raise ValueError('missing trailing newline')
             cur = next_line + 1
-        entry_bits = PyTuple_New(width)
+        entry_bits = StaticTuple_New(width)
         for i from 0 <= i < num_prefix_bits:
+            # TODO: Use PyList_GetItem, or turn prefix_bits into a
+            #       tuple/StaticTuple
             entry = prefix_bits[i]
             # SET_ITEM 'steals' a reference
             Py_INCREF(entry)
-            PyTuple_SET_ITEM(entry_bits, i, entry)
+            StaticTuple_SET_ITEM(entry_bits, i, entry)
         value = PyString_FromStringAndSize(value_start, next_line - value_start)
         # The next entry bit needs the 'tail' from the prefix, and first part
         # of the line
@@ -288,7 +308,7 @@
             memcpy(c_entry + prefix_tail_len, line_start, next_null - line_start)
         Py_INCREF(entry)
         i = num_prefix_bits
-        PyTuple_SET_ITEM(entry_bits, i, entry)
+        StaticTuple_SET_ITEM(entry_bits, i, entry)
         while next_null != last_null: # We have remaining bits
             i = i + 1
             if i > width:
@@ -301,11 +321,12 @@
             entry = PyString_FromStringAndSize(entry_start,
                                                next_null - entry_start)
             Py_INCREF(entry)
-            PyTuple_SET_ITEM(entry_bits, i, entry)
+            StaticTuple_SET_ITEM(entry_bits, i, entry)
         if len(entry_bits) != width:
             raise AssertionError(
                 'Incorrect number of elements (%d vs %d)'
                 % (len(entry_bits)+1, width + 1))
+        entry_bits = StaticTuple_Intern(entry_bits)
         PyDict_SetItem(items, entry_bits, value)
     if len(items) != length:
         raise ValueError("item count (%d) mismatch for key %s,"
@@ -343,6 +364,8 @@
         _unknown = chk_map._unknown
     result = _InternalNode(search_key_func=search_key_func)
 
+    if not StaticTuple_CheckExact(key):
+        raise TypeError('key %r is not a StaticTuple' % (key,))
     if not PyString_CheckExact(bytes):
         raise TypeError('bytes must be a plain string not %s' % (type(bytes),))
 
@@ -384,7 +407,8 @@
         memcpy(c_item_prefix + prefix_length, cur, next_null - cur)
         flat_key = PyString_FromStringAndSize(next_null + 1,
                                               next_line - next_null - 1)
-        PyDict_SetItem(items, item_prefix, (flat_key,))
+        flat_key = StaticTuple(flat_key).intern()
+        PyDict_SetItem(items, item_prefix, flat_key)
         cur = next_line + 1
     assert len(items) > 0
     result._items = items
@@ -398,4 +422,3 @@
     result._node_width = len(item_prefix)
     result._search_prefix = PyString_FromStringAndSize(prefix, prefix_length)
     return result
-
 
=== modified file 'bzrlib/_dirstate_helpers_pyx.pyx'
--- bzrlib/_dirstate_helpers_pyx.pyx	2009-08-28 05:00:33 +0000
+++ bzrlib/_dirstate_helpers_pyx.pyx	2009-11-03 09:40:54 +0000
@@ -1202,7 +1202,9 @@
                         content_change = 0
                     target_exec = False
                 else:
-                    raise Exception, "unknown kind %s" % path_info[2]
+                    if path is None:
+                        path = self.pathjoin(old_dirname, old_basename)
+                    raise errors.BadFileKindError(path, path_info[2])
             if source_minikind == c'd':
                 if path is None:
                     old_path = path = self.pathjoin(old_dirname, old_basename)
 
=== added file 'bzrlib/_export_c_api.h'
--- bzrlib/_export_c_api.h	1970-01-01 00:00:00 +0000
+++ bzrlib/_export_c_api.h	2009-11-03 09:40:54 +0000
@@ -0,0 +1,104 @@
+/* Copyright (C) 2009 Canonical Ltd
+ * 
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+
+/* This file contains helper functions for exporting a C API for a CPython
+ * extension module.
+ */
+
+#ifndef _EXPORT_C_API_H_
+#define _EXPORT_C_API_H_
+
+static const char *_C_API_NAME = "_C_API";
+
+/**
+ * Add a C function to the modules _C_API
+ * This wraps the function in a PyCObject, and inserts that into a dict.
+ * The key of the dict is the function name, and the description is the
+ * signature of the function.
+ * This is generally called during a modules init_MODULE function.
+ *
+ * @param   module  A Python module (the one being initialized)
+ * @param   funcname The name of the function being exported
+ * @param   func    A pointer to the function
+ * @param   signature The C signature of the function
+ * @return  0 if everything is successful, -1 if there is a problem. An
+ *          exception should also be set
+ */
+static int
+_export_function(PyObject *module, char *funcname, void *func, char *signature)
+{
+    PyObject *d = NULL;
+    PyObject *c_obj = NULL;
+
+    /* (char *) is because python2.4 declares this api as 'char *' rather than
+     * const char* which it really is.
+     */
+    d = PyObject_GetAttrString(module, (char *)_C_API_NAME);
+    if (!d) {
+        PyErr_Clear();
+        d = PyDict_New();
+        if (!d)
+            goto bad;
+        Py_INCREF(d);
+        if (PyModule_AddObject(module, (char *)_C_API_NAME, d) < 0)
+            goto bad;
+    }
+    c_obj = PyCObject_FromVoidPtrAndDesc(func, signature, 0);
+    if (!c_obj)
+        goto bad;
+    if (PyDict_SetItemString(d, funcname, c_obj) < 0)
+        goto bad;
+    Py_DECREF(d);
+    return 0;
+bad:
+    Py_XDECREF(c_obj);
+    Py_XDECREF(d);
+    return -1;
+}
+
+/* Note:
+ *  It feels like more could be done here. Specifically, if you look at
+ *  _static_tuple_c.h you can see some boilerplate where we have:
+ * #ifdef STATIC_TUPLE_MODULE  // are we exporting or importing
+ * static RETVAL FUNCNAME PROTO;
+ * #else
+ * static RETVAL (*FUNCNAME) PROTO;
+ * #endif
+ * 
+ * And then in _static_tuple_c.c we have
+ * int setup_c_api()
+ * {
+ *   _export_function(module, #FUNCNAME, FUNCNAME, #PROTO);
+ * }
+ *
+ * And then in _static_tuple_c.h import_##MODULE
+ * struct function_definition functions[] = {
+ *   {#FUNCNAME, (void **)&FUNCNAME, #RETVAL #PROTO},
+ *   ...
+ *   {NULL}};
+ *
+ * And some similar stuff for types. However, this would mean that we would
+ * need a way for the C preprocessor to build up a list of definitions to be
+ * generated, and then expand that list at the appropriate time.
+ * I would guess there would be a way to do this, but probably not without a
+ * lot of magic, and the end result probably wouldn't be very pretty to
+ * maintain. Perhaps python's dynamic nature has left me jaded about writing
+ * boilerplate....
+ */
+
+#endif // _EXPORT_C_API_H_
 
=== added file 'bzrlib/_import_c_api.h'
--- bzrlib/_import_c_api.h	1970-01-01 00:00:00 +0000
+++ bzrlib/_import_c_api.h	2009-11-03 09:40:54 +0000
@@ -0,0 +1,189 @@
+/* Copyright (C) 2009 Canonical Ltd
+ * 
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef _IMPORT_C_API_H_
+#define _IMPORT_C_API_H_
+
+/**
+ * Helper functions to eliminate some of the boilerplate when importing a C API
+ * from a CPython extension module.
+ *
+ * For more information see _export_c_api.h
+ */
+
+static const char *_C_API_NAME = "_C_API";
+
+/**
+ * Import a function from the _C_API_NAME dict that is part of module.
+ *
+ * @param   module  The Python module we are importing from
+ *                  the attribute _C_API_NAME will be used as a dictionary
+ *                  containing the function pointer we are looking for.
+ * @param   funcname    Name of the function we want to import
+ * @param   func    A pointer to the function handle where we will store the
+ *                  function.
+ * @param   signature   The C signature of the function. This is validated
+ *                      against the signature stored in the C api, to make sure
+ *                      there is no versioning skew.
+ */
+static int _import_function(PyObject *module, const char *funcname,
+                            void **func, const char *signature)
+{
+    PyObject *d = NULL;
+    PyObject *c_obj = NULL;
+    const char *desc = NULL;
+
+    /* (char *) because Python2.4 defines this as (char *) rather than
+     * (const char *)
+     */
+    d = PyObject_GetAttrString(module, (char *)_C_API_NAME);
+    if (!d) {
+        // PyObject_GetAttrString sets an appropriate exception
+        goto bad;
+    }
+    c_obj = PyDict_GetItemString(d, funcname);
+    if (!c_obj) {
+        // PyDict_GetItemString does not set an exception
+        PyErr_Format(PyExc_AttributeError,
+            "Module %s did not export a function named %s\n",
+            PyModule_GetName(module), funcname);
+        goto bad;
+    }
+    desc = (char *)PyCObject_GetDesc(c_obj);
+    if (!desc || strcmp(desc, signature) != 0) {
+        if (desc == NULL) {
+            desc = "<null>";
+        }
+        PyErr_Format(PyExc_TypeError,
+            "C function %s.%s has wrong signature (expected %s, got %s)",
+                PyModule_GetName(module), funcname, signature, desc);
+        goto bad;
+    }
+    *func = PyCObject_AsVoidPtr(c_obj);
+    Py_DECREF(d);
+    return 0;
+bad:
+    Py_XDECREF(d);
+    return -1;
+}
+
+
+/**
+ * Get a pointer to an exported PyTypeObject.
+ *
+ * @param   module        The Python module we are importing from
+ * @param   class_name    Attribute of the module that should reference the
+ *                        Type object. Note that a PyTypeObject is the python
+ *                        description of the type, not the raw C structure.
+ * @return  A Pointer to the requested type object. On error NULL will be
+ *          returned and an exception will be set.
+ */
+static PyTypeObject *
+_import_type(PyObject *module, const char *class_name)
+{
+    PyObject *type = NULL;
+
+    type = PyObject_GetAttrString(module, (char *)class_name);
+    if (!type) {
+        goto bad;
+    }
+    if (!PyType_Check(type)) {
+        PyErr_Format(PyExc_TypeError,
+            "%s.%s is not a type object",
+            PyModule_GetName(module), class_name);
+        goto bad;
+    }
+    return (PyTypeObject *)type;
+bad:
+    Py_XDECREF(type);
+    return NULL;
+}
+
+
+struct function_description
+{
+    const char *name;
+    void **pointer;
+    const char *signature;
+};
+
+struct type_description
+{
+    const char *name;
+    PyTypeObject **pointer;
+};
+
+/**
+ * Helper for importing several functions and types in a data-driven manner.
+ *
+ * @param   module  The name of the module we will be importing
+ * @param   functions   A list of function_description objects, describing the
+ *                      functions being imported.
+ *                      The list should be terminated with {NULL} to indicate
+ *                      there are no more functions to import.
+ * @param   types       A list of type_description objects describing type
+ *                      objects that we want to import. The list should be
+ *                      terminated with {NULL} to indicate there are no more
+ *                      types to import.
+ * @return  0 on success, -1 on error and an exception should be set.
+ */
+
+static int
+_import_extension_module(const char *module_name,
+                         struct function_description *functions,
+                         struct type_description *types)
+{
+    PyObject *module = NULL;
+    struct function_description *cur_func;
+    struct type_description *cur_type;
+    int ret_code;
+    
+    module = PyImport_ImportModule((char *)module_name);
+    if (!module)
+        goto bad;
+    if (functions != NULL) {
+        cur_func = functions;
+        while (cur_func->name != NULL) {
+            ret_code = _import_function(module, cur_func->name,
+                                        cur_func->pointer,
+                                        cur_func->signature);
+            if (ret_code < 0)
+                goto bad;
+            cur_func++;
+        }
+    }
+    if (types != NULL) {
+        PyTypeObject *type_p = NULL;
+        cur_type = types;
+        while (cur_type->name != NULL)  {
+            type_p = _import_type(module, cur_type->name);
+            if (type_p == NULL)
+                goto bad;
+            *(cur_type->pointer) = type_p;
+            cur_type++;
+        }
+    }
+    
+    Py_XDECREF(module);
+    return 0;
+bad:
+    Py_XDECREF(module);
+    return -1;
+}
+
+
+#endif // _IMPORT_C_API_H_
 
=== modified file 'bzrlib/_known_graph_py.py'
--- bzrlib/_known_graph_py.py	2009-08-25 18:45:40 +0000
+++ bzrlib/_known_graph_py.py	2009-11-03 09:40:55 +0000
@@ -281,3 +281,26 @@
                  in tsort.merge_sort(as_parent_map, tip_key,
                                      mainline_revisions=None,
                                      generate_revno=True)]
+    
+    def get_parent_keys(self, key):
+        """Get the parents for a key
+        
+        Returns a list containg the parents keys. If the key is a ghost,
+        None is returned. A KeyError will be raised if the key is not in
+        the graph.
+        
+        :param keys: Key to check (eg revision_id)
+        :return: A list of parents
+        """
+        return self._nodes[key].parent_keys
+
+    def get_child_keys(self, key):
+        """Get the children for a key
+        
+        Returns a list containg the children keys. A KeyError will be raised
+        if the key is not in the graph.
+        
+        :param keys: Key to check (eg revision_id)
+        :return: A list of children
+        """
+        return self._nodes[key].child_keys
 
=== modified file 'bzrlib/_known_graph_pyx.pyx'
--- bzrlib/_known_graph_pyx.pyx	2009-09-02 13:32:52 +0000
+++ bzrlib/_known_graph_pyx.pyx	2009-11-03 09:40:54 +0000
@@ -88,6 +88,18 @@
                 PyList_Append(keys, child.key)
             return keys
 
+    property parent_keys:
+        def __get__(self):
+            if self.parents is None:
+                return None
+            
+            cdef _KnownGraphNode parent
+
+            keys = []
+            for parent in self.parents:
+                PyList_Append(keys, parent.key)
+            return keys
+    
     cdef clear_references(self):
         self.parents = None
         self.children = None
@@ -549,6 +561,29 @@
         #       shown a specific impact, yet.
         sorter = _MergeSorter(self, tip_key)
         return sorter.topo_order()
+    
+    def get_parent_keys(self, key):
+        """Get the parents for a key
+        
+        Returns a list containg the parents keys. If the key is a ghost,
+        None is returned. A KeyError will be raised if the key is not in
+        the graph.
+        
+        :param keys: Key to check (eg revision_id)
+        :return: A list of parents
+        """
+        return self._nodes[key].parent_keys 
+
+    def get_child_keys(self, key):
+        """Get the children for a key
+        
+        Returns a list containg the children keys. A KeyError will be raised
+        if the key is not in the graph.
+        
+        :param keys: Key to check (eg revision_id)
+        :return: A list of children
+        """
+        return self._nodes[key].child_keys    
 
 
 cdef class _MergeSortNode:
 
=== modified file 'bzrlib/_patiencediff_c.c'
--- bzrlib/_patiencediff_c.c	2009-03-23 14:59:43 +0000
+++ bzrlib/_patiencediff_c.c	2009-11-03 09:40:55 +0000
@@ -298,7 +298,7 @@
         apos = SENTINEL;
         /* loop through all lines in the linked list */
         for (i = h[equiv].a_pos; i != SENTINEL; i = lines_a[i].next) {
-            /* the index is lower than alo, the the next line */
+            /* the index is lower than alo, continue to the next line */
             if (i < alo) {
                 h[equiv].a_pos = i;
                 continue;
@@ -319,7 +319,7 @@
         /* check for duplicates of this line in lines_b[blo:bhi] */
         /* loop through all lines in the linked list */
         for (i = h[equiv].b_pos; i != SENTINEL; i = lines_b[i].next) {
-            /* the index is lower than blo, the the next line */
+            /* the index is lower than blo, continue to the next line */
             if (i < blo) {
                 h[equiv].b_pos = i;
                 continue;
 
=== modified file 'bzrlib/_readdir_pyx.pyx'
--- bzrlib/_readdir_pyx.pyx	2009-07-27 04:24:36 +0000
+++ bzrlib/_readdir_pyx.pyx	2009-11-03 09:40:54 +0000
@@ -343,8 +343,10 @@
                             raise OSError(errno, "lstat: " + strerror(errno),
                                 path + "/" + entry.d_name)
                         else:
-                            kind = _missing
-                            statvalue = None
+                            # the file seems to have disappeared after being
+                            # seen by readdir - perhaps a transient temporary
+                            # file.  there's no point returning it.
+                            continue
                     # We append a 5-tuple that can be modified in-place by the C
                     # api:
                     # inode to sort on (to replace with top_path)
 
=== added file 'bzrlib/_simple_set_pyx.pxd'
--- bzrlib/_simple_set_pyx.pxd	1970-01-01 00:00:00 +0000
+++ bzrlib/_simple_set_pyx.pxd	2009-11-03 09:40:54 +0000
@@ -0,0 +1,91 @@
+# Copyright (C) 2009 Canonical Ltd
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+"""Interface definition of a class like PySet but without caching the hash.
+
+This is generally useful when you want to 'intern' objects, etc. Note that this
+differs from Set in that we:
+  1) Don't have all of the .intersection, .difference, etc functions
+  2) Do return the object from the set via queries
+     eg. SimpleSet.add(key) => saved_key and SimpleSet[key] => saved_key
+"""
+
+cdef extern from "Python.h":
+    ctypedef struct PyObject:
+        pass
+
+
+cdef public api class SimpleSet [object SimpleSetObject, type SimpleSet_Type]:
+    """A class similar to PySet, but with simpler implementation.
+
+    The main advantage is that this class uses only 2N memory to store N
+    objects rather than 4N memory. The main trade-off is that we do not cache
+    the hash value of saved objects. As such, it is assumed that computing the
+    hash will be cheap (such as strings or tuples of strings, etc.)
+
+    This also differs in that you can get back the objects that are stored
+    (like a dict), but we also don't implement the complete list of 'set'
+    operations (difference, intersection, etc).
+    """
+    # Data structure definition:
+    #   This is a basic hash table using open addressing.
+    #       http://en.wikipedia.org/wiki/Open_addressing
+    #   Basically that means we keep an array of pointers to Python objects
+    #   (called a table). Each location in the array is called a 'slot'.
+    #
+    #   An empty slot holds a NULL pointer, a slot where there was an item
+    #   which was then deleted will hold a pointer to _dummy, and a filled slot
+    #   points at the actual object which fills that slot.
+    #
+    #   The table is always a power of two, and the default location where an
+    #   object is inserted is at hash(object) & (table_size - 1)
+    #
+    #   If there is a collision, then we search for another location. The
+    #   specific algorithm is in _lookup. We search until we:
+    #       find the object
+    #       find an equivalent object (by tp_richcompare(obj1, obj2, Py_EQ))
+    #       find a NULL slot
+    #
+    #   When an object is deleted, we set its slot to _dummy. this way we don't
+    #   have to track whether there was a collision, and find the corresponding
+    #   keys. (The collision resolution algorithm makes that nearly impossible
+    #   anyway, because it depends on the upper bits of the hash.)
+    #   The main effect of this, is that if we find _dummy, then we can insert
+    #   an object there, but we have to keep searching until we find NULL to
+    #   know that the object is not present elsewhere.
+
+    cdef Py_ssize_t _used   # active
+    cdef Py_ssize_t _fill   # active + dummy
+    cdef Py_ssize_t _mask   # Table contains (mask+1) slots, a power of 2
+    cdef PyObject **_table  # Pyrex/Cython doesn't support arrays to 'object'
+                            # so we manage it manually
+
+    cdef PyObject *_get(self, object key) except? NULL
+    cdef object _add(self, key)
+    cdef int _discard(self, key) except -1
+    cdef int _insert_clean(self, PyObject *key) except -1
+    cdef Py_ssize_t _resize(self, Py_ssize_t min_unused) except -1
+
+
+# TODO: might want to export the C api here, though it is all available from
+#       the class object...
+cdef api SimpleSet SimpleSet_New()
+cdef api object SimpleSet_Add(object self, object key)
+cdef api int SimpleSet_Contains(object self, object key) except -1
+cdef api int SimpleSet_Discard(object self, object key) except -1
+cdef api PyObject *SimpleSet_Get(SimpleSet self, object key) except? NULL
+cdef api Py_ssize_t SimpleSet_Size(object self) except -1
+cdef api int SimpleSet_Next(object self, Py_ssize_t *pos, PyObject **key)
 
=== added file 'bzrlib/_simple_set_pyx.pyx'
--- bzrlib/_simple_set_pyx.pyx	1970-01-01 00:00:00 +0000
+++ bzrlib/_simple_set_pyx.pyx	2009-11-03 09:40:54 +0000
@@ -0,0 +1,590 @@
+# Copyright (C) 2009 Canonical Ltd
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+"""Definition of a class that is similar to Set with some small changes."""
+
+cdef extern from "python-compat.h":
+    pass
+
+cdef extern from "Python.h":
+    ctypedef unsigned long size_t
+    ctypedef long (*hashfunc)(PyObject*) except -1
+    ctypedef object (*richcmpfunc)(PyObject *, PyObject *, int)
+    ctypedef int (*visitproc)(PyObject *, void *)
+    ctypedef int (*traverseproc)(PyObject *, visitproc, void *)
+    int Py_EQ
+    void Py_INCREF(PyObject *)
+    void Py_DECREF(PyObject *)
+    ctypedef struct PyTypeObject:
+        hashfunc tp_hash
+        richcmpfunc tp_richcompare
+        traverseproc tp_traverse
+
+    PyTypeObject *Py_TYPE(PyObject *)
+    # Note: *Don't* use hash(), Pyrex 0.9.8.5 thinks it returns an 'int', and
+    #       thus silently truncates to 32-bits on 64-bit machines.
+    long PyObject_Hash(PyObject *) except -1
+        
+    void *PyMem_Malloc(size_t nbytes)
+    void PyMem_Free(void *)
+    void memset(void *, int, size_t)
+
+
+# Dummy is an object used to mark nodes that have been deleted. Since
+# collisions require us to move a node to an alternative location, if we just
+# set an entry to NULL on delete, we won't find any relocated nodes.
+# We have to use _dummy_obj because we need to keep a refcount to it, but we
+# also use _dummy as a pointer, because it avoids having to put <PyObject*> all
+# over the code base.
+cdef object _dummy_obj
+cdef PyObject *_dummy
+_dummy_obj = object()
+_dummy = <PyObject *>_dummy_obj
+
+
+cdef object _NotImplemented
+_NotImplemented = NotImplemented
+
+
+cdef int _is_equal(PyObject *this, long this_hash, PyObject *other) except -1:
+    cdef long other_hash
+
+    if this == other:
+        return 1
+    other_hash = PyObject_Hash(other)
+    if other_hash != this_hash:
+        return 0
+
+    # This implements a subset of the PyObject_RichCompareBool functionality.
+    # Namely it:
+    #   1) Doesn't try to do anything with old-style classes
+    #   2) Assumes that both objects have a tp_richcompare implementation, and
+    #      that if that is not enough to compare equal, then they are not
+    #      equal. (It doesn't try to cast them both to some intermediate form
+    #      that would compare equal.)
+    res = Py_TYPE(this).tp_richcompare(this, other, Py_EQ)
+    if res is _NotImplemented:
+        res = Py_TYPE(other).tp_richcompare(other, this, Py_EQ)
+        if res is _NotImplemented:
+            return 0
+    if res:
+        return 1
+    return 0
+
+
+cdef public api class SimpleSet [object SimpleSetObject, type SimpleSet_Type]:
+    """This class can be used to track canonical forms for objects.
+
+    It is similar in function to the interned dictionary that is used by
+    strings. However:
+
+      1) It assumes that hash(obj) is cheap, so does not need to inline a copy
+         of it
+      2) It only stores one reference to the object, rather than 2 (key vs
+         key:value)
+
+    As such, it uses 1/3rd the amount of memory to store a pointer to the
+    interned object.
+    """
+    # Attributes are defined in the .pxd file
+    DEF DEFAULT_SIZE=1024
+
+    def __init__(self):
+        cdef Py_ssize_t size, n_bytes
+
+        size = DEFAULT_SIZE
+        self._mask = size - 1
+        self._used = 0
+        self._fill = 0
+        n_bytes = sizeof(PyObject*) * size;
+        self._table = <PyObject **>PyMem_Malloc(n_bytes)
+        if self._table == NULL:
+            raise MemoryError()
+        memset(self._table, 0, n_bytes)
+
+    def __dealloc__(self):
+        if self._table != NULL:
+            PyMem_Free(self._table)
+            self._table = NULL
+
+    property used:
+        def __get__(self):
+            return self._used
+
+    property fill:
+        def __get__(self):
+            return self._fill
+
+    property mask:
+        def __get__(self):
+            return self._mask
+
+    def _memory_size(self):
+        """Return the number of bytes of memory consumed by this class."""
+        return sizeof(self) + (sizeof(PyObject*)*(self._mask + 1))
+
+    def __len__(self):
+        return self._used
+
+    def _test_lookup(self, key):
+        cdef PyObject **slot
+
+        slot = _lookup(self, key)
+        if slot[0] == NULL:
+            res = '<null>'
+        elif slot[0] == _dummy:
+            res = '<dummy>'
+        else:
+            res = <object>slot[0]
+        return <int>(slot - self._table), res
+
+    def __contains__(self, key):
+        """Is key present in this SimpleSet."""
+        cdef PyObject **slot
+
+        slot = _lookup(self, key)
+        if slot[0] == NULL or slot[0] == _dummy:
+            return False
+        return True
+
+    cdef PyObject *_get(self, object key) except? NULL:
+        """Return the object (or nothing) define at the given location."""
+        cdef PyObject **slot
+
+        slot = _lookup(self, key)
+        if slot[0] == NULL or slot[0] == _dummy:
+            return NULL
+        return slot[0]
+
+    def __getitem__(self, key):
+        """Return a stored item that is equivalent to key."""
+        cdef PyObject *py_val
+
+        py_val = self._get(key)
+        if py_val == NULL:
+            raise KeyError("Key %s is not present" % key)
+        val = <object>(py_val)
+        return val
+
+    cdef int _insert_clean(self, PyObject *key) except -1:
+        """Insert a key into self.table.
+
+        This is only meant to be used during times like '_resize',
+        as it makes a lot of assuptions about keys not already being present,
+        and there being no dummy entries.
+        """
+        cdef size_t i, n_lookup
+        cdef long the_hash
+        cdef PyObject **table, **slot
+        cdef Py_ssize_t mask
+
+        mask = self._mask
+        table = self._table
+
+        the_hash = PyObject_Hash(key)
+        i = the_hash
+        for n_lookup from 0 <= n_lookup <= <size_t>mask: # Don't loop forever
+            slot = &table[i & mask]
+            if slot[0] == NULL:
+                slot[0] = key
+                self._fill = self._fill + 1
+                self._used = self._used + 1
+                return 1
+            i = i + 1 + n_lookup
+        raise RuntimeError('ran out of slots.')
+
+    def _py_resize(self, min_used):
+        """Do not use this directly, it is only exposed for testing."""
+        return self._resize(min_used)
+
+    cdef Py_ssize_t _resize(self, Py_ssize_t min_used) except -1:
+        """Resize the internal table.
+
+        The final table will be big enough to hold at least min_used entries.
+        We will copy the data from the existing table over, leaving out dummy
+        entries.
+
+        :return: The new size of the internal table
+        """
+        cdef Py_ssize_t new_size, n_bytes, remaining
+        cdef PyObject **new_table, **old_table, **slot
+
+        new_size = DEFAULT_SIZE
+        while new_size <= min_used and new_size > 0:
+            new_size = new_size << 1
+        # We rolled over our signed size field
+        if new_size <= 0:
+            raise MemoryError()
+        # Even if min_used == self._mask + 1, and we aren't changing the actual
+        # size, we will still run the algorithm so that dummy entries are
+        # removed
+        # TODO: Test this
+        # if new_size < self._used:
+        #     raise RuntimeError('cannot shrink SimpleSet to something'
+        #                        ' smaller than the number of used slots.')
+        n_bytes = sizeof(PyObject*) * new_size;
+        new_table = <PyObject **>PyMem_Malloc(n_bytes)
+        if new_table == NULL:
+            raise MemoryError()
+
+        old_table = self._table
+        self._table = new_table
+        memset(self._table, 0, n_bytes)
+        self._mask = new_size - 1
+        self._used = 0
+        remaining = self._fill
+        self._fill = 0
+
+        # Moving everything to the other table is refcount neutral, so we don't
+        # worry about it.
+        slot = old_table
+        while remaining > 0:
+            if slot[0] == NULL: # unused slot
+                pass 
+            elif slot[0] == _dummy: # dummy slot
+                remaining = remaining - 1
+            else: # active slot
+                remaining = remaining - 1
+                self._insert_clean(slot[0])
+            slot = slot + 1
+        PyMem_Free(old_table)
+        return new_size
+
+    def add(self, key):
+        """Similar to set.add(), start tracking this key.
+        
+        There is one small difference, which is that we return the object that
+        is stored at the given location. (which is closer to the
+        dict.setdefault() functionality.)
+        """
+        return self._add(key)
+
+    cdef object _add(self, key):
+        cdef PyObject **slot, *py_key
+        cdef int added
+
+        py_key = <PyObject *>key
+        if (Py_TYPE(py_key).tp_richcompare == NULL
+            or Py_TYPE(py_key).tp_hash == NULL):
+            raise TypeError('Types added to SimpleSet must implement'
+                            ' both tp_richcompare and tp_hash')
+        added = 0
+        # We need at least one empty slot
+        assert self._used < self._mask
+        slot = _lookup(self, key)
+        if (slot[0] == NULL):
+            Py_INCREF(py_key)
+            self._fill = self._fill + 1
+            self._used = self._used + 1
+            slot[0] = py_key
+            added = 1
+        elif (slot[0] == _dummy):
+            Py_INCREF(py_key)
+            self._used = self._used + 1
+            slot[0] = py_key
+            added = 1
+        # No else: clause. If _lookup returns a pointer to
+        # a live object, then we already have a value at this location.
+        retval = <object>(slot[0])
+        # PySet and PyDict use a 2-3rds full algorithm, we'll follow suit
+        if added and (self._fill * 3) >= ((self._mask + 1) * 2):
+            # However, we always work for a load factor of 2:1
+            self._resize(self._used * 2)
+        # Even if we resized and ended up moving retval into a different slot,
+        # it is still the value that is held at the slot equivalent to 'key',
+        # so we can still return it
+        return retval
+
+    def discard(self, key):
+        """Remove key from the set, whether it exists or not.
+
+        :return: False if the item did not exist, True if it did
+        """
+        if self._discard(key):
+            return True
+        return False
+
+    cdef int _discard(self, key) except -1:
+        cdef PyObject **slot, *py_key
+
+        slot = _lookup(self, key)
+        if slot[0] == NULL or slot[0] == _dummy:
+            return 0
+        self._used = self._used - 1
+        Py_DECREF(slot[0])
+        slot[0] = _dummy
+        # PySet uses the heuristic: If more than 1/5 are dummies, then resize
+        #                           them away
+        #   if ((so->_fill - so->_used) * 5 < so->mask)
+        # However, we are planning on using this as an interning structure, in
+        # which we will be putting a lot of objects. And we expect that large
+        # groups of them are going to have the same lifetime.
+        # Dummy entries hurt a little bit because they cause the lookup to keep
+        # searching, but resizing is also rather expensive
+        # For now, we'll just use their algorithm, but we may want to revisit
+        # it
+        if ((self._fill - self._used) * 5 > self._mask):
+            self._resize(self._used * 2)
+        return 1
+
+    def __iter__(self):
+        return _SimpleSet_iterator(self)
+
+
+cdef class _SimpleSet_iterator:
+    """Iterator over the SimpleSet structure."""
+
+    cdef Py_ssize_t pos
+    cdef SimpleSet set
+    cdef Py_ssize_t _used # track if things have been mutated while iterating
+    cdef Py_ssize_t len # number of entries left
+
+    def __init__(self, obj):
+        self.set = obj
+        self.pos = 0
+        self._used = self.set._used
+        self.len = self.set._used
+
+    def __iter__(self):
+        return self
+
+    def __next__(self):
+        cdef Py_ssize_t mask, i
+        cdef PyObject *key
+
+        if self.set is None:
+            raise StopIteration
+        if self.set._used != self._used:
+            # Force this exception to continue to be raised
+            self._used = -1
+            raise RuntimeError("Set size changed during iteration")
+        if not SimpleSet_Next(self.set, &self.pos, &key):
+            self.set = None
+            raise StopIteration
+        # we found something
+        the_key = <object>key # INCREF
+        self.len = self.len - 1
+        return the_key
+
+    def __length_hint__(self):
+        if self.set is not None and self._used == self.set._used:
+            return self.len
+        return 0
+    
+
+
+cdef api SimpleSet SimpleSet_New():
+    """Create a new SimpleSet object."""
+    return SimpleSet()
+
+
+cdef SimpleSet _check_self(object self):
+    """Check that the parameter is not None.
+
+    Pyrex/Cython will do type checking, but only to ensure that an object is
+    either the right type or None. You can say "object foo not None" for pure
+    python functions, but not for C functions.
+    So this is just a helper for all the apis that need to do the check.
+    """
+    cdef SimpleSet true_self
+    if self is None:
+        raise TypeError('self must not be None')
+    true_self = self
+    return true_self
+
+
+cdef PyObject **_lookup(SimpleSet self, object key) except NULL:
+    """Find the slot where 'key' would fit.
+
+    This is the same as a dicts 'lookup' function.
+
+    :param key: An object we are looking up
+    :param hash: The hash for key
+    :return: The location in self.table where key should be put.
+        location == NULL is an exception, but (*location) == NULL just
+        indicates the slot is empty and can be used.
+    """
+    # This uses Quadratic Probing:
+    #  http://en.wikipedia.org/wiki/Quadratic_probing
+    # with c1 = c2 = 1/2
+    # This leads to probe locations at:
+    #  h0 = hash(k1)
+    #  h1 = h0 + 1
+    #  h2 = h0 + 3 = h1 + 1 + 1
+    #  h3 = h0 + 6 = h2 + 1 + 2
+    #  h4 = h0 + 10 = h2 + 1 + 3
+    # Note that all of these are '& mask', but that is computed *after* the
+    # offset.
+    # This differs from the algorithm used by Set and Dict. Which, effectively,
+    # use double-hashing, and a step size that starts large, but dwindles to
+    # stepping one-by-one.
+    # This gives more 'locality' in that if you have a collision at offset X,
+    # the first fallback is X+1, which is fast to check. However, that means
+    # that an object w/ hash X+1 will also check there, and then X+2 next.
+    # However, for objects with differing hashes, their chains are different.
+    # The former checks X, X+1, X+3, ... the latter checks X+1, X+2, X+4, ...
+    # So different hashes diverge quickly.
+    # A bigger problem is that we *only* ever use the lowest bits of the hash
+    # So all integers (x + SIZE*N) will resolve into the same bucket, and all
+    # use the same collision resolution. We may want to try to find a way to
+    # incorporate the upper bits of the hash with quadratic probing. (For
+    # example, X, X+1, X+3+some_upper_bits, X+6+more_upper_bits, etc.)
+    cdef size_t i, n_lookup
+    cdef Py_ssize_t mask
+    cdef long key_hash
+    cdef PyObject **table, **slot, *cur, **free_slot, *py_key
+
+    py_key = <PyObject *>key
+    # Note: avoid using hash(obj) because of a bug w/ pyrex 0.9.8.5 and 64-bit
+    #       (it treats hash() as returning an 'int' rather than a 'long')
+    key_hash = PyObject_Hash(py_key)
+    i = <size_t>key_hash
+    mask = self._mask
+    table = self._table
+    free_slot = NULL
+    for n_lookup from 0 <= n_lookup <= <size_t>mask: # Don't loop forever
+        slot = &table[i & mask]
+        cur = slot[0]
+        if cur == NULL:
+            # Found a blank spot
+            if free_slot != NULL:
+                # Did we find an earlier _dummy entry?
+                return free_slot
+            else:
+                return slot
+        if cur == py_key:
+            # Found an exact pointer to the key
+            return slot
+        if cur == _dummy:
+            if free_slot == NULL:
+                free_slot = slot
+        elif _is_equal(py_key, key_hash, cur):
+            # Both py_key and cur belong in this slot, return it
+            return slot
+        i = i + 1 + n_lookup
+    raise AssertionError('should never get here')
+
+
+cdef api PyObject **_SimpleSet_Lookup(object self, object key) except NULL:
+    """Find the slot where 'key' would fit.
+
+    This is the same as a dicts 'lookup' function. This is a private
+    api because mutating what you get without maintaing the other invariants
+    is a 'bad thing'.
+
+    :param key: An object we are looking up
+    :param hash: The hash for key
+    :return: The location in self._table where key should be put
+        should never be NULL, but may reference a NULL (PyObject*)
+    """
+    return _lookup(_check_self(self), key)
+
+
+cdef api object SimpleSet_Add(object self, object key):
+    """Add a key to the SimpleSet (set).
+
+    :param self: The SimpleSet to add the key to.
+    :param key: The key to be added. If the key is already present,
+        self will not be modified
+    :return: The current key stored at the location defined by 'key'.
+        This may be the same object, or it may be an equivalent object.
+        (consider dict.setdefault(key, key))
+    """
+    return _check_self(self)._add(key)
+    
+
+cdef api int SimpleSet_Contains(object self, object key) except -1:
+    """Is key present in self?"""
+    return (key in _check_self(self))
+
+
+cdef api int SimpleSet_Discard(object self, object key) except -1:
+    """Remove the object referenced at location 'key'.
+
+    :param self: The SimpleSet being modified
+    :param key: The key we are checking on
+    :return: 1 if there was an object present, 0 if there was not, and -1 on
+        error.
+    """
+    return _check_self(self)._discard(key)
+
+
+cdef api PyObject *SimpleSet_Get(SimpleSet self, object key) except? NULL:
+    """Get a pointer to the object present at location 'key'.
+
+    This returns an object which is equal to key which was previously added to
+    self. This returns a borrowed reference, as it may also return NULL if no
+    value is present at that location.
+
+    :param key: The value we are looking for
+    :return: The object present at that location
+    """
+    return _check_self(self)._get(key)
+
+
+cdef api Py_ssize_t SimpleSet_Size(object self) except -1:
+    """Get the number of active entries in 'self'"""
+    return _check_self(self)._used
+
+
+cdef api int SimpleSet_Next(object self, Py_ssize_t *pos, PyObject **key):
+    """Walk over items in a SimpleSet.
+
+    :param pos: should be initialized to 0 by the caller, and will be updated
+        by this function
+    :param key: Will return a borrowed reference to key
+    :return: 0 if nothing left, 1 if we are returning a new value
+    """
+    cdef Py_ssize_t i, mask
+    cdef SimpleSet true_self
+    cdef PyObject **table
+    true_self = _check_self(self)
+    i = pos[0]
+    if (i < 0):
+        return 0
+    mask = true_self._mask
+    table= true_self._table
+    while (i <= mask and (table[i] == NULL or table[i] == _dummy)):
+        i = i + 1
+    pos[0] = i + 1
+    if (i > mask):
+        return 0 # All done
+    if (key != NULL):
+        key[0] = table[i]
+    return 1
+
+
+cdef int SimpleSet_traverse(SimpleSet self, visitproc visit, void *arg):
+    """This is an implementation of 'tp_traverse' that hits the whole table.
+
+    Cython/Pyrex don't seem to let you define a tp_traverse, and they only
+    define one for you if you have an 'object' attribute. Since they don't
+    support C arrays of objects, we access the PyObject * directly.
+    """
+    cdef Py_ssize_t pos
+    cdef PyObject *next_key
+    cdef int ret
+
+    pos = 0
+    while SimpleSet_Next(self, &pos, &next_key):
+        ret = visit(next_key, arg)
+        if ret:
+            return ret
+    return 0
+
+# It is a little bit ugly to do this, but it works, and means that Meliae can
+# dump the total memory consumed by all child objects.
+(<PyTypeObject *>SimpleSet).tp_traverse = <traverseproc>SimpleSet_traverse
 
=== added file 'bzrlib/_static_tuple_c.c'
--- bzrlib/_static_tuple_c.c	1970-01-01 00:00:00 +0000
+++ bzrlib/_static_tuple_c.c	2009-11-03 09:40:55 +0000
@@ -0,0 +1,941 @@
+/* Copyright (C) 2009 Canonical Ltd
+ * 
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/* Must be defined before importing _static_tuple_c.h so that we get the right
+ * linkage.
+ */
+#define STATIC_TUPLE_MODULE
+
+#include <Python.h>
+#include "python-compat.h"
+
+#include "_static_tuple_c.h"
+#include "_export_c_api.h"
+
+/* Pyrex 0.9.6.4 exports _simple_set_pyx_api as
+ * import__simple_set_pyx(), while Pyrex 0.9.8.5 and Cython 0.11.3 export them
+ * as import_bzrlib___simple_set_pyx(). As such, we just #define one to be
+ * equivalent to the other in our internal code.
+ */
+#define import__simple_set_pyx import_bzrlib___simple_set_pyx
+#include "_simple_set_pyx_api.h"
+
+#if defined(__GNUC__)
+#   define inline __inline__
+#elif defined(_MSC_VER)
+#   define inline __inline
+#else
+#   define inline
+#endif
+
+
+/* The one and only StaticTuple with no values */
+static StaticTuple *_empty_tuple = NULL;
+static PyObject *_interned_tuples = NULL;
+
+
+static inline int
+_StaticTuple_is_interned(StaticTuple *self)
+{
+    return self->flags & STATIC_TUPLE_INTERNED_FLAG;
+}
+
+
+
+static PyObject *
+StaticTuple_as_tuple(StaticTuple *self)
+{
+    PyObject *tpl = NULL, *obj = NULL;
+    int i, len;
+
+    len = self->size;
+    tpl = PyTuple_New(len);
+    if (!tpl) {
+        /* Malloc failure */
+        return NULL;
+    }
+    for (i = 0; i < len; ++i) {
+        obj = (PyObject *)self->items[i];
+        Py_INCREF(obj);
+        PyTuple_SET_ITEM(tpl, i, obj);
+    }
+    return tpl;
+}
+
+
+static char StaticTuple_as_tuple_doc[] = "as_tuple() => tuple";
+
+static StaticTuple *
+StaticTuple_Intern(StaticTuple *self)
+{
+    PyObject *canonical_tuple = NULL;
+
+    if (_interned_tuples == NULL || _StaticTuple_is_interned(self)) {
+        Py_INCREF(self);
+        return self;
+    }
+    /* SimpleSet_Add returns whatever object is present at self
+     * or the new object if it needs to add it.
+     */
+    canonical_tuple = SimpleSet_Add(_interned_tuples, (PyObject *)self);
+    if (!canonical_tuple) {
+        // Some sort of exception, propogate it.
+        return NULL;
+    }
+    if (canonical_tuple != (PyObject *)self) {
+        // There was already a tuple with that value
+        return (StaticTuple *)canonical_tuple;
+    }
+    self->flags |= STATIC_TUPLE_INTERNED_FLAG;
+    // The two references in the dict do not count, so that the StaticTuple
+    // object does not become immortal just because it was interned.
+    Py_REFCNT(self) -= 1;
+    return self;
+}
+
+static char StaticTuple_Intern_doc[] = "intern() => unique StaticTuple\n"
+    "Return a 'canonical' StaticTuple object.\n"
+    "Similar to intern() for strings, this makes sure there\n"
+    "is only one StaticTuple object for a given value\n."
+    "Common usage is:\n"
+    "  key = StaticTuple('foo', 'bar').intern()\n";
+
+
+static void
+StaticTuple_dealloc(StaticTuple *self)
+{
+    int i, len;
+
+    if (_StaticTuple_is_interned(self)) {
+        /* revive dead object temporarily for Discard */
+        Py_REFCNT(self) = 2;
+        if (SimpleSet_Discard(_interned_tuples, (PyObject*)self) != 1)
+            Py_FatalError("deletion of interned StaticTuple failed");
+        self->flags &= ~STATIC_TUPLE_INTERNED_FLAG;
+    }
+    len = self->size;
+    for (i = 0; i < len; ++i) {
+        Py_XDECREF(self->items[i]);
+    }
+    Py_TYPE(self)->tp_free((PyObject *)self);
+}
+
+
+/* Similar to PyTuple_New() */
+static StaticTuple *
+StaticTuple_New(Py_ssize_t size)
+{
+    StaticTuple *stuple;
+    if (size < 0) {
+        PyErr_BadInternalCall();
+        return NULL;
+    }
+
+    if (size < 0 || size > 255) {
+        /* Too big or too small */
+        PyErr_SetString(PyExc_ValueError, "StaticTuple(...)"
+            " takes from 0 to 255 items");
+        return NULL;
+    }
+    if (size == 0 && _empty_tuple != NULL) {
+        Py_INCREF(_empty_tuple);
+        return _empty_tuple;
+    }
+    /* Note that we use PyObject_NewVar because we want to allocate a variable
+     * width entry. However we *aren't* truly a PyVarObject because we don't
+     * use a long for ob_size. Instead we use a plain 'size' that is an int,
+     * and will be overloaded with flags in the future.
+     * As such we do the alloc, and then have to clean up anything it does
+     * incorrectly.
+     */
+    stuple = PyObject_NewVar(StaticTuple, &StaticTuple_Type, size);
+    if (stuple == NULL) {
+        return NULL;
+    }
+    stuple->size = size;
+    stuple->flags = 0;
+    stuple->_unused0 = 0;
+    stuple->_unused1 = 0;
+    if (size > 0) {
+        memset(stuple->items, 0, sizeof(PyObject *) * size);
+    }
+#if STATIC_TUPLE_HAS_HASH
+    stuple->hash = -1;
+#endif
+    return stuple;
+}
+
+
+static StaticTuple *
+StaticTuple_FromSequence(PyObject *sequence)
+{
+    StaticTuple *new = NULL;
+    PyObject *as_tuple = NULL;
+    PyObject *item;
+    Py_ssize_t i, size;
+
+    if (StaticTuple_CheckExact(sequence)) {
+        Py_INCREF(sequence);
+        return (StaticTuple *)sequence;
+    }
+    if (!PySequence_Check(sequence)) {
+        as_tuple = PySequence_Tuple(sequence);
+        if (as_tuple == NULL)
+            goto done;
+        sequence = as_tuple;
+    }
+    size = PySequence_Size(sequence);
+    if (size == -1) {
+        goto done;
+    }
+    new = StaticTuple_New(size);
+    if (new == NULL) {
+        goto done;
+    }
+    for (i = 0; i < size; ++i) {
+        // This returns a new reference, which we then 'steal' with 
+        // StaticTuple_SET_ITEM
+        item = PySequence_GetItem(sequence, i);
+        if (item == NULL) {
+            Py_DECREF(new);
+            new = NULL;
+            goto done;
+        }
+        StaticTuple_SET_ITEM(new, i, item);
+    }
+done:
+    Py_XDECREF(as_tuple);
+    return (StaticTuple *)new;
+}
+
+static StaticTuple *
+StaticTuple_from_sequence(PyObject *self, PyObject *args, PyObject *kwargs)
+{
+    PyObject *sequence;
+    if (!PyArg_ParseTuple(args, "O", &sequence))
+        return NULL;
+    return StaticTuple_FromSequence(sequence);
+}
+
+
+/* Check that all items we point to are 'valid' */
+static int
+StaticTuple_check_items(StaticTuple *self)
+{
+    int i;
+    PyObject *obj;
+
+    for (i = 0; i < self->size; ++i) {
+        obj = self->items[i];
+        if (obj == NULL) {
+            PyErr_SetString(PyExc_RuntimeError, "StaticTuple(...)"
+                " should not have a NULL entry.");
+            return 0;
+        }
+        if (PyString_CheckExact(obj)
+            || StaticTuple_CheckExact(obj)
+            || obj == Py_None
+            || PyBool_Check(obj)
+            || PyInt_CheckExact(obj)
+            || PyLong_CheckExact(obj)
+            || PyFloat_CheckExact(obj)
+            || PyUnicode_CheckExact(obj)
+            ) continue;
+        PyErr_Format(PyExc_TypeError, "StaticTuple(...)"
+            " requires that all items are one of"
+            " str, StaticTuple, None, bool, int, long, float, or unicode"
+            " not %s.", Py_TYPE(obj)->tp_name);
+        return 0;
+    }
+    return 1;
+}
+
+static PyObject *
+StaticTuple_new_constructor(PyTypeObject *type, PyObject *args, PyObject *kwds)
+{
+    StaticTuple *self;
+    PyObject *obj = NULL;
+    Py_ssize_t i, len = 0;
+
+    if (type != &StaticTuple_Type) {
+        PyErr_SetString(PyExc_TypeError, "we only support creating StaticTuple");
+        return NULL;
+    }
+    if (!PyTuple_CheckExact(args)) {
+        PyErr_SetString(PyExc_TypeError, "args must be a tuple");
+        return NULL;
+    }
+    len = PyTuple_GET_SIZE(args);
+    self = (StaticTuple *)StaticTuple_New(len);
+    if (self == NULL) {
+        return NULL;
+    }
+    for (i = 0; i < len; ++i) {
+        obj = PyTuple_GET_ITEM(args, i);
+        Py_INCREF(obj);
+        self->items[i] = obj;
+    }
+    if (!StaticTuple_check_items(self)) {
+        type->tp_dealloc((PyObject *)self);
+        return NULL;
+    }
+    return (PyObject *)self;
+}
+
+static PyObject *
+StaticTuple_repr(StaticTuple *self)
+{
+    PyObject *as_tuple, *tuple_repr, *result;
+
+    as_tuple = StaticTuple_as_tuple(self);
+    if (as_tuple == NULL) {
+        return NULL;
+    }
+    tuple_repr = PyObject_Repr(as_tuple);
+    Py_DECREF(as_tuple);
+    if (tuple_repr == NULL) {
+        return NULL;
+    }
+    result = PyString_FromFormat("StaticTuple%s",
+                                 PyString_AsString(tuple_repr));
+    return result;
+}
+
+static long
+StaticTuple_hash(StaticTuple *self)
+{
+    /* adapted from tuplehash(), is the specific hash value considered
+     * 'stable'?
+     */
+    register long x, y;
+    Py_ssize_t len = self->size;
+    PyObject **p;
+    long mult = 1000003L;
+
+#if STATIC_TUPLE_HAS_HASH
+    if (self->hash != -1) {
+        return self->hash;
+    }
+#endif
+    x = 0x345678L;
+    p = self->items;
+    // TODO: We could set specific flags if we know that, for example, all the
+    //       items are strings. I haven't seen a real-world benefit to that
+    //       yet, though.
+    while (--len >= 0) {
+        y = PyObject_Hash(*p++);
+        if (y == -1) /* failure */
+            return -1;
+        x = (x ^ y) * mult;
+        /* the cast might truncate len; that doesn't change hash stability */
+        mult += (long)(82520L + len + len);
+    }
+    x += 97531L;
+    if (x == -1)
+        x = -2;
+#if STATIC_TUPLE_HAS_HASH
+    self->hash = x;
+#endif
+    return x;
+}
+
+static PyObject *
+StaticTuple_richcompare_to_tuple(StaticTuple *v, PyObject *wt, int op)
+{
+    PyObject *vt;
+    PyObject *result = NULL;
+    
+    vt = StaticTuple_as_tuple((StaticTuple *)v);
+    if (vt == NULL) {
+        goto done;
+    }
+    if (!PyTuple_Check(wt)) {
+        PyErr_BadInternalCall();
+        goto done;
+    }
+    /* Now we have 2 tuples to compare, do it */
+    result = PyTuple_Type.tp_richcompare(vt, wt, op);
+done:
+    Py_XDECREF(vt);
+    return result;
+}
+
+/** Compare two objects to determine if they are equivalent.
+ * The basic flow is as follows
+ *  1) First make sure that both objects are StaticTuple instances. If they
+ *     aren't then cast self to a tuple, and have the tuple do the comparison.
+ *  2) Special case comparison to Py_None, because it happens to occur fairly
+ *     often in the test suite.
+ *  3) Special case when v and w are the same pointer. As we know the answer to
+ *     all queries without walking individual items.
+ *  4) For all operations, we then walk the items to find the first paired
+ *     items that are not equal.
+ *  5) If all items found are equal, we then check the length of self and
+ *     other to determine equality.
+ *  6) If an item differs, then we apply "op" to those last two items. (eg.
+ *     StaticTuple(A, B) > StaticTuple(A, C) iff B > C)
+ */
+
+static PyObject *
+StaticTuple_richcompare(PyObject *v, PyObject *w, int op)
+{
+    StaticTuple *v_st, *w_st;
+    Py_ssize_t vlen, wlen, min_len, i;
+    PyObject *v_obj, *w_obj;
+    richcmpfunc string_richcompare;
+
+    if (!StaticTuple_CheckExact(v)) {
+        /* This has never triggered, according to python-dev it seems this
+         * might trigger if '__op__' is defined but '__rop__' is not, sort of
+         * case. Such as "None == StaticTuple()"
+         */
+        fprintf(stderr, "self is not StaticTuple\n");
+        Py_INCREF(Py_NotImplemented);
+        return Py_NotImplemented;
+    }
+    v_st = (StaticTuple *)v;
+    if (StaticTuple_CheckExact(w)) {
+        /* The most common case */
+        w_st = (StaticTuple*)w;
+    } else if (PyTuple_Check(w)) {
+        /* One of v or w is a tuple, so we go the 'slow' route and cast up to
+         * tuples to compare.
+         */
+        /* TODO: This seems to be triggering more than I thought it would...
+         *       We probably want to optimize comparing self to other when
+         *       other is a tuple.
+         */
+        return StaticTuple_richcompare_to_tuple(v_st, w, op);
+    } else if (w == Py_None) {
+        // None is always less than the object
+        switch (op) {
+        case Py_NE:case Py_GT:case Py_GE:
+            Py_INCREF(Py_True);
+            return Py_True;
+        case Py_EQ:case Py_LT:case Py_LE:
+            Py_INCREF(Py_False);
+            return Py_False;
+    default: // Should never happen
+        return Py_NotImplemented;
+        }
+    } else {
+        /* We don't special case this comparison, we just let python handle
+         * it.
+         */
+         Py_INCREF(Py_NotImplemented);
+         return Py_NotImplemented;
+    }
+    /* Now we know that we have 2 StaticTuple objects, so let's compare them.
+     * This code is inspired from tuplerichcompare, except we know our
+     * objects are limited in scope, so we can inline some comparisons.
+     */
+    if (v == w) {
+        /* Identical pointers, we can shortcut this easily. */
+        switch (op) {
+        case Py_EQ:case Py_LE:case Py_GE:
+            Py_INCREF(Py_True);
+            return Py_True;
+        case Py_NE:case Py_LT:case Py_GT:
+            Py_INCREF(Py_False);
+            return Py_False;
+        }
+    }
+    if (op == Py_EQ
+        && _StaticTuple_is_interned(v_st)
+        && _StaticTuple_is_interned(w_st))
+    {
+        /* If both objects are interned, we know they are different if the
+         * pointer is not the same, which would have been handled by the
+         * previous if. No need to compare the entries.
+         */
+        Py_INCREF(Py_False);
+        return Py_False;
+    }
+
+    /* The only time we are likely to compare items of different lengths is in
+     * something like the interned_keys set. However, the hash is good enough
+     * that it is rare. Note that 'tuple_richcompare' also does not compare
+     * lengths here.
+     */
+    vlen = v_st->size;
+    wlen = w_st->size;
+    min_len = (vlen < wlen) ? vlen : wlen;
+    string_richcompare = PyString_Type.tp_richcompare;
+    for (i = 0; i < min_len; i++) {
+        PyObject *result = NULL;
+        v_obj = StaticTuple_GET_ITEM(v_st, i);
+        w_obj = StaticTuple_GET_ITEM(w_st, i);
+        if (v_obj == w_obj) {
+            /* Shortcut case, these must be identical */
+            continue;
+        }
+        if (PyString_CheckExact(v_obj) && PyString_CheckExact(w_obj)) {
+            result = string_richcompare(v_obj, w_obj, Py_EQ);
+        } else if (StaticTuple_CheckExact(v_obj) &&
+                   StaticTuple_CheckExact(w_obj))
+        {
+            /* Both are StaticTuple types, so recurse */
+            result = StaticTuple_richcompare(v_obj, w_obj, Py_EQ);
+        } else {
+            /* Fall back to generic richcompare */
+            result = PyObject_RichCompare(v_obj, w_obj, Py_EQ);
+        }
+        if (result == NULL) {
+            return NULL; /* There seems to be an error */
+        }
+        if (result == Py_False) {
+            // This entry is not identical, Shortcut for Py_EQ
+            if (op == Py_EQ) {
+                return result;
+            }
+            Py_DECREF(result);
+            break;
+        }
+        if (result != Py_True) {
+            /* We don't know *what* richcompare is returning, but it
+             * isn't something we recognize
+             */
+            PyErr_BadInternalCall();
+            Py_DECREF(result);
+            return NULL;
+        }
+        Py_DECREF(result);
+    }
+    if (i >= min_len) {
+        /* We walked off one of the lists, but everything compared equal so
+         * far. Just compare the size.
+         */
+        int cmp;
+        PyObject *res;
+        switch (op) {
+        case Py_LT: cmp = vlen <  wlen; break;
+        case Py_LE: cmp = vlen <= wlen; break;
+        case Py_EQ: cmp = vlen == wlen; break;
+        case Py_NE: cmp = vlen != wlen; break;
+        case Py_GT: cmp = vlen >  wlen; break;
+        case Py_GE: cmp = vlen >= wlen; break;
+        default: return NULL; /* cannot happen */
+        }
+        if (cmp)
+            res = Py_True;
+        else
+            res = Py_False;
+        Py_INCREF(res);
+        return res;
+    }
+    /* The last item differs, shortcut the Py_NE case */
+    if (op == Py_NE) {
+        Py_INCREF(Py_True);
+        return Py_True;
+    }
+    /* It is some other comparison, go ahead and do the real check. */
+    if (PyString_CheckExact(v_obj) && PyString_CheckExact(w_obj))
+    {
+        return string_richcompare(v_obj, w_obj, op);
+    } else if (StaticTuple_CheckExact(v_obj) &&
+               StaticTuple_CheckExact(w_obj))
+    {
+        /* Both are StaticTuple types, so recurse */
+        return StaticTuple_richcompare(v_obj, w_obj, op);
+    } else {
+        return PyObject_RichCompare(v_obj, w_obj, op);
+    }
+}
+
+
+static Py_ssize_t
+StaticTuple_length(StaticTuple *self)
+{
+    return self->size;
+}
+
+
+static PyObject *
+StaticTuple__is_interned(StaticTuple *self)
+{
+    if (_StaticTuple_is_interned(self)) {
+        Py_INCREF(Py_True);
+        return Py_True;
+    }
+    Py_INCREF(Py_False);
+    return Py_False;
+}
+
+static char StaticTuple__is_interned_doc[] = "_is_interned() => True/False\n"
+    "Check to see if this tuple has been interned.\n";
+
+
+static PyObject *
+StaticTuple_reduce(StaticTuple *self)
+{
+    PyObject *result = NULL, *as_tuple = NULL;
+
+    result = PyTuple_New(2);
+    if (!result) {
+        return NULL;
+    }
+    as_tuple = StaticTuple_as_tuple(self);
+    if (as_tuple == NULL) {
+        Py_DECREF(result);
+        return NULL;
+    }
+    Py_INCREF(&StaticTuple_Type);
+    PyTuple_SET_ITEM(result, 0, (PyObject *)&StaticTuple_Type);
+    PyTuple_SET_ITEM(result, 1, as_tuple);
+    return result;
+}
+
+static char StaticTuple_reduce_doc[] = "__reduce__() => tuple\n";
+
+
+static PyObject *
+StaticTuple_add(PyObject *v, PyObject *w)
+{
+    Py_ssize_t i, len_v, len_w;
+    PyObject *item;
+    StaticTuple *result;
+     /* StaticTuples and plain tuples may be added (concatenated) to
+      * StaticTuples.
+      */
+    if (StaticTuple_CheckExact(v)) {
+        len_v = ((StaticTuple*)v)->size;
+    } else if (PyTuple_Check(v)) {
+        len_v = PyTuple_GET_SIZE(v);
+    } else {
+        Py_INCREF(Py_NotImplemented);
+        return Py_NotImplemented;
+    }
+    if (StaticTuple_CheckExact(w)) {
+        len_w = ((StaticTuple*)w)->size;
+    } else if (PyTuple_Check(w)) {
+        len_w = PyTuple_GET_SIZE(w);
+    } else {
+        Py_INCREF(Py_NotImplemented);
+        return Py_NotImplemented;
+    }
+    result = StaticTuple_New(len_v + len_w);
+    if (result == NULL)
+        return NULL;
+    for (i = 0; i < len_v; ++i) {
+        // This returns a new reference, which we then 'steal' with 
+        // StaticTuple_SET_ITEM
+        item = PySequence_GetItem(v, i);
+        if (item == NULL) {
+            Py_DECREF(result);
+            return NULL;
+        }
+        StaticTuple_SET_ITEM(result, i, item);
+    }
+    for (i = 0; i < len_w; ++i) {
+        item = PySequence_GetItem(w, i);
+        if (item == NULL) {
+            Py_DECREF(result);
+            return NULL;
+        }
+        StaticTuple_SET_ITEM(result, i+len_v, item);
+    }
+    if (!StaticTuple_check_items(result)) {
+        Py_DECREF(result);
+        return NULL;
+    }
+    return (PyObject *)result;
+}
+
+static PyObject *
+StaticTuple_item(StaticTuple *self, Py_ssize_t offset)
+{