Merge lp:~eric97/bzr/dump-btree-api into lp:bzr

Proposed by Eric Siegerman
Status: Work in progress
Proposed branch: lp:~eric97/bzr/dump-btree-api
Merge into: lp:bzr
Diff against target: 410 lines (+202/-139)
4 files modified
bzrlib/btree_index.py (+50/-0)
bzrlib/builtins.py (+4/-56)
bzrlib/tests/blackbox/test_dump_btree.py (+35/-83)
bzrlib/tests/test_btree_index.py (+113/-0)
To merge this branch: bzr merge lp:~eric97/bzr/dump-btree-api
Reviewer Review Type Date Requested Status
Jelmer Vernooij (community) code Approve
Vincent Ladeuil Approve
Review via email: mp+50107@code.launchpad.net

Commit message

Create a public API for generating (quasi-)text dumps of B-Tree index files, by turning two private methods in builtins.py into public methods on bzrlib.btree_index.BTreeGraphIndex.

Description of the change

A review comment for another MP of mine (https://code.launchpad.net/~eric97/bzr/regenerate-pack-names) asked that my tests use API calls instead of run_bzr('dump-btree') to dump B-Tree index files. The problem is, there is no API for that; the implementation of "bzr dump-btree" lives directly in builtins, as private methods of cmd_dump_btree.

This MP moves those methods to bzrlib.btree_index.BTreeGraphIndex, and makes them public. It is a pure refactoring; it is intended that there be no change in functionality.

To post a comment you must log in.
Revision history for this message
Vincent Ladeuil (vila) wrote :

Nicely done !
Just one tweak from me but good to land otherwise:

151 + test_raw_output_3nodes = (
152 + 'Root node:\n'
153 + 'B+Tree Graph Index 2\n'

You can just use:

test_raw_output_3nodes = '''Root node:
B+Tree Graph Index 2
...
'''

Or even:

test_raw_output_3nodes = '''\
Root node:
B+Tree Graph Index 2
...
'''

Easier to copy/paste (all rules have exceptions, copy/paste is bad, but copying an expected result from a failed test is way faster than typing it from scratch).

Nice to see tests going from blackbox to unit !

I let our PP do the second review ;)

review: Approve
Revision history for this message
Eric Siegerman (eric97) wrote :

On Thu, 2011-02-17 at 09:46 +0000, Vincent Ladeuil wrote:
> Review: Approve
> Nicely done !

Thanks.

> You can just use:
>
> test_raw_output_3nodes = '''Root node:
> B+Tree Graph Index 2
> ...
> '''

That won't match, so it requires a change -- either:
    test_raw_output_0nodes = re.sub("(?m)^ *", "",
        '''Root node:
        B+Tree Graph Index 2
        ...
        ''')
or:
    test_raw_output_0nodes = \
'''Root node:
B+Tree Graph Index 2
...
'''

Is it still an improvement? If so, which is preferable?

> (all rules have exceptions, copy/paste is bad, but copying an expected
result from a failed test is way faster than typing it from scratch).

I don't understand: why is one easier to copy than the other?

As for whether copying is bad, well, those data definitions, and most of
the method that reads them, are already duplicated between the two test
files. I didn't like doing that, but liked even less to reach across
from one to the other:
    self.assertEquals(some_other_module.SomeClass.expected_value,
actual_value)
Plus, for the module that does that, it would put the test data far from
the tests themselves, thus awkward to look up.

  - Eric

lp:~eric97/bzr/dump-btree-api updated
5672. By Eric Siegerman <email address hidden>

Remove another blackbox test that's now redundant.

Revision history for this message
Vincent Ladeuil (vila) wrote :

> On Thu, 2011-02-17 at 09:46 +0000, Vincent Ladeuil wrote:
> > Review: Approve
> > Nicely done !
>
> Thanks.
>
> > You can just use:
> >
> > test_raw_output_3nodes = '''Root node:
> > B+Tree Graph Index 2
> > ...
> > '''
>
> That won't match, so it requires a change -- either:
> test_raw_output_0nodes = re.sub("(?m)^ *", "",
> '''Root node:
> B+Tree Graph Index 2
> ...
> ''')

I meant: use a triple quoting string and don't indent the lines in it so the string matches the output of the command without adding all the quotes and the '\n's.

> or:
> test_raw_output_0nodes = \
> '''Root node:
> B+Tree Graph Index 2
> ...
> '''
>

Same as my second example, don't worry, it's just a matter of taste, *I* dislike the '= \' and prefer '="""\" instead, no big deal, anyway I misread it as being defined at the top level where no leading spaces won't break the reading. poolie disliked embedding such strings in classes anyway, so forget about it.

> I don't understand: why is one easier to copy than the other?

Both of my examples are as easy to copy (the second one slightly more) and your second example is easy as well. The point is to avoid the additional quotes and '\n's.

>
> As for whether copying is bad, well, those data definitions, and most of
> the method that reads them, are already duplicated between the two test
> files. I didn't like doing that, but liked even less to reach across
> from one to the other:
> self.assertEquals(some_other_module.SomeClass.expected_value,
> actual_value)
> Plus, for the module that does that, it would put the test data far from
> the tests themselves, thus awkward to look up.

Right, the counter-argument is that you'll need to update both files if the output format change. There is no perfect answer to this problem when the tests can't be in the same file. In the end, it's a matter of taste and pragmatism.

Revision history for this message
Eric Siegerman (eric97) wrote :

On Thu, 2011-02-17 at 15:47 +0000, Vincent Ladeuil wrote:
> [...] I misread it as being defined at the top level where no
> leading spaces won't break the reading.

Then I'd prefer to leave it as is, if it's OK with you. That's
how it was in the original, anyway...

  - Eric

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Seems reasonable to me, submitting to PQM...

review: Approve (code)
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

sent to pqm by email

Revision history for this message
Eric Siegerman (eric97) wrote :

Jelmer:

Did this go to PQM yet? If not, good! Please don't (re)send it yet; I've found a bug. It won't break anything existing, but still, it'd be nice to clean it up before the MP lands.

Thanks much,
  - Eric

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Hi Eric,

On Fri, 2011-02-18 at 17:58 +0000, Eric Siegerman wrote:
> Jelmer:
>
> Did this go to PQM yet? If not, good! Please don't (re)send it yet; I've found a bug. It won't break anything existing, but still, it'd be nice to clean it up before the MP lands.
It's been sent off to PQM but failed (don't recall the reason but I can
look it up).

I'll set the MP back to Work In Progress for the moment so we don't
accidentally submit it again. Just let us know when it's ready again. :)

Cheers,

Jelmer

Revision history for this message
Eric Siegerman (eric97) wrote :

On Fri, 2011-02-18 at 18:25 +0000, Jelmer Vernooij wrote:
> It's been sent off to PQM but failed (don't recall the reason but I
can
> look it up).

That'd be helpful.

Is the status of PQM jobs available to non-core folks so we don't
have to pester you guys? I found the queue page at
http://pqm.bazaar-vcs.org/, but it doesn't list completed jobs,
nor offer a link to a page that does. All I knew from Launchpad
and email was that it had been sent, and then no further status,
just silence. Or have I been looking in the wrong place?

> I'll set the MP back to Work In Progress for the moment so we don't
> accidentally submit it again. Just let us know when it's ready
again. :)

OK; will do.

Thanks,
  - Eric

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Fri, 2011-02-18 at 19:02 +0000, Eric Siegerman wrote:
> On Fri, 2011-02-18 at 18:25 +0000, Jelmer Vernooij wrote:
> > It's been sent off to PQM but failed (don't recall the reason but I
> can
> > look it up).
> That'd be helpful.
>
> Is the status of PQM jobs available to non-core folks so we don't
> have to pester you guys? I found the queue page at
> http://pqm.bazaar-vcs.org/, but it doesn't list completed jobs,
> nor offer a link to a page that does. All I knew from Launchpad
> and email was that it had been sent, and then no further status,
> just silence. Or have I been looking in the wrong place?
It's not available in a private place either :) PQM sends the output of
each job to the submitter of that job, in this case myself. I'll
forward that email to you.

> > I'll set the MP back to Work In Progress for the moment so we don't
> > accidentally submit it again. Just let us know when it's ready
> again. :)
> OK; will do.
Thanks!

Cheers,

Jelmer

Revision history for this message
Vincent Ladeuil (vila) wrote :

<snip/>
    > Is the status of PQM jobs available to non-core folks so we don't
    > have to pester you guys? I found the queue page at
    > http://pqm.bazaar-vcs.org/, but it doesn't list completed jobs,

That's because you don't refresh it often enough[1], if you are very very
lucky you may able to catch the number of failures and if you are even
more lucky you can get the name of the failing test(s).

    > nor offer a link to a page that does.

None exists.

    > All I knew from Launchpad and email was that it had been sent, and
    > then no further status,

...

    > just silence.

Yeah, the painful silence usually means something went wrong :-(

This is certainly one of the worst feedback you could get for a test
failure...

First steps of defense:

- run the full test suite locally,

- run make check | subunit2pyunit locally,

- run make check with python2.4 locally

If they all pass, bangs your head on the desktop (find a soft desktop if
you repeat the later too often) and nag the patch pilot :-/

If you've reached a high enough pain threshold, you can file a bug.

[1]: That's a painful joke if you didn't notice :)

Unmerged revisions

5672. By Eric Siegerman <email address hidden>

Remove another blackbox test that's now redundant.

5671. By Eric Siegerman <email address hidden>

Factor test data out of the blackbox tests too.

5670. By Eric Siegerman <email address hidden>

Don't need empty-btree blackbox tests, now that we have them as unit tests.

5669. By Eric Siegerman <email address hidden>

A little code-formatting cleanup.

5668. By Eric Siegerman <email address hidden>

Factor out the test data.

5667. By Eric Siegerman <email address hidden>

Adapt the new test_btree_index btree-dumping tests to use the new API
instead of calling run_bzr().

5666. By Eric Siegerman <email address hidden>

Delete a redundant test.

There's no need to test over multiple transports that the methods can
cope with a no-leaf-nodes B-Tree; local transport will suffice.

5665. By Eric Siegerman <email address hidden>

Add docstrings.

5664. By Eric Siegerman <email address hidden>

Move dump_raw_bytes() and dump_entries() from cmd_dump_btree to btree_index,
and adapt them to their new location.

5663. By Eric Siegerman <email address hidden>

Copy all tests verbatim from blackbox/test_dump_btree.py to test_btree_index.py.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'bzrlib/btree_index.py'
--- bzrlib/btree_index.py 2010-08-06 15:38:24 +0000
+++ bzrlib/btree_index.py 2011-02-17 14:57:46 +0000
@@ -1584,6 +1584,56 @@
1584 for node in self._read_nodes(range(start_node, node_end)):1584 for node in self._read_nodes(range(start_node, node_end)):
1585 pass1585 pass
15861586
1587 def dump_raw_bytes(self, out_file):
1588 """Produce a raw dump of this B-Tree.
1589
1590 Dump the header and the pages. The pages are decompressed, but
1591 otherwise uninterpreted.
1592
1593 :param out_file: File-like object to write the dump to.
1594 """
1595 # We need to parse at least the root node.
1596 # This is because the first page of every row starts with an
1597 # uncompressed header.
1598 bytes = self._transport.get_bytes(self._name)
1599 for page_idx, page_start in enumerate(xrange(0, len(bytes),
1600 _PAGE_SIZE)):
1601 page_end = min(page_start + _PAGE_SIZE, len(bytes))
1602 page_bytes = bytes[page_start:page_end]
1603 if page_idx == 0:
1604 out_file.write('Root node:\n')
1605 header_end, data = self._parse_header_from_bytes(page_bytes)
1606 out_file.write(page_bytes[:header_end])
1607 page_bytes = data
1608 out_file.write('\nPage %d\n' % (page_idx,))
1609 if len(page_bytes) == 0:
1610 out_file.write('(empty)\n');
1611 else:
1612 decomp_bytes = zlib.decompress(page_bytes)
1613 out_file.write(decomp_bytes)
1614 out_file.write('\n')
1615
1616 def dump_entries(self, out_file):
1617 """Produce a formatted dump of this B-Tree.
1618
1619 Dump the leaf nodes, formatted as nested tuples. The header is
1620 not dumped, so if the tree contains no leaf nodes, this method
1621 produces no output.
1622
1623 :param out_file: File-like object to write the dump to.
1624 """
1625 for node in self.iter_all_entries():
1626 # Node is made up of:
1627 # (index, key, value, [references])
1628 try:
1629 refs = node[3]
1630 except IndexError:
1631 refs_as_tuples = None
1632 else:
1633 refs_as_tuples = static_tuple.as_tuples(refs)
1634 as_tuple = (tuple(node[1]), node[2], refs_as_tuples)
1635 out_file.write('%s\n' % (as_tuple,))
1636
15871637
1588_gcchk_factory = _LeafNode1638_gcchk_factory = _LeafNode
15891639
15901640
=== modified file 'bzrlib/builtins.py'
--- bzrlib/builtins.py 2011-02-09 17:10:05 +0000
+++ bzrlib/builtins.py 2011-02-17 14:57:46 +0000
@@ -383,63 +383,11 @@
383 def run(self, path, raw=False):383 def run(self, path, raw=False):
384 dirname, basename = osutils.split(path)384 dirname, basename = osutils.split(path)
385 t = transport.get_transport(dirname)385 t = transport.get_transport(dirname)
386 btree = btree_index.BTreeGraphIndex(t, basename, None)
386 if raw:387 if raw:
387 self._dump_raw_bytes(t, basename)388 btree.dump_raw_bytes(self.outf)
388 else:389 else:
389 self._dump_entries(t, basename)390 btree.dump_entries(self.outf)
390
391 def _get_index_and_bytes(self, trans, basename):
392 """Create a BTreeGraphIndex and raw bytes."""
393 bt = btree_index.BTreeGraphIndex(trans, basename, None)
394 bytes = trans.get_bytes(basename)
395 bt._file = cStringIO.StringIO(bytes)
396 bt._size = len(bytes)
397 return bt, bytes
398
399 def _dump_raw_bytes(self, trans, basename):
400 import zlib
401
402 # We need to parse at least the root node.
403 # This is because the first page of every row starts with an
404 # uncompressed header.
405 bt, bytes = self._get_index_and_bytes(trans, basename)
406 for page_idx, page_start in enumerate(xrange(0, len(bytes),
407 btree_index._PAGE_SIZE)):
408 page_end = min(page_start + btree_index._PAGE_SIZE, len(bytes))
409 page_bytes = bytes[page_start:page_end]
410 if page_idx == 0:
411 self.outf.write('Root node:\n')
412 header_end, data = bt._parse_header_from_bytes(page_bytes)
413 self.outf.write(page_bytes[:header_end])
414 page_bytes = data
415 self.outf.write('\nPage %d\n' % (page_idx,))
416 if len(page_bytes) == 0:
417 self.outf.write('(empty)\n');
418 else:
419 decomp_bytes = zlib.decompress(page_bytes)
420 self.outf.write(decomp_bytes)
421 self.outf.write('\n')
422
423 def _dump_entries(self, trans, basename):
424 try:
425 st = trans.stat(basename)
426 except errors.TransportNotPossible:
427 # We can't stat, so we'll fake it because we have to do the 'get()'
428 # anyway.
429 bt, _ = self._get_index_and_bytes(trans, basename)
430 else:
431 bt = btree_index.BTreeGraphIndex(trans, basename, st.st_size)
432 for node in bt.iter_all_entries():
433 # Node is made up of:
434 # (index, key, value, [references])
435 try:
436 refs = node[3]
437 except IndexError:
438 refs_as_tuples = None
439 else:
440 refs_as_tuples = static_tuple.as_tuples(refs)
441 as_tuple = (tuple(node[1]), node[2], refs_as_tuples)
442 self.outf.write('%s\n' % (as_tuple,))
443391
444392
445class cmd_remove_tree(Command):393class cmd_remove_tree(Command):
446394
=== modified file 'bzrlib/tests/blackbox/test_dump_btree.py'
--- bzrlib/tests/blackbox/test_dump_btree.py 2011-02-08 23:06:34 +0000
+++ bzrlib/tests/blackbox/test_dump_btree.py 2011-02-17 14:57:46 +0000
@@ -28,12 +28,36 @@
2828
29class TestDumpBtree(tests.TestCaseWithTransport):29class TestDumpBtree(tests.TestCaseWithTransport):
3030
31 def create_sample_btree_index(self):31 # Each node is represented as the *string value* that
32 # BTreeGraphIndex.dump_entries() should produce for it.
33 test_data_3nodes = (
34 "(('test', 'key1'), 'value', ((('ref', 'entry'),),))\n",
35 "(('test', 'key2'), 'value2', ((('ref', 'entry2'),),))\n",
36 "(('test2', 'key3'), 'value3', ((('ref', 'entry3'),),))\n",
37 )
38 test_input_3nodes = [eval(node) for node in test_data_3nodes]
39 test_output_3nodes = "".join(test_data_3nodes)
40 test_raw_output_3nodes = (
41 'Root node:\n'
42 'B+Tree Graph Index 2\n'
43 'node_ref_lists=1\n'
44 'key_elements=2\n'
45 'len=3\n'
46 'row_lengths=1\n'
47 '\n'
48 'Page 0\n'
49 'type=leaf\n'
50 'test\0key1\0ref\0entry\0value\n'
51 'test\0key2\0ref\0entry2\0value2\n'
52 'test2\0key3\0ref\0entry3\0value3\n'
53 '\n'
54 )
55
56 def create_sample_btree_index(self, node_data):
32 builder = btree_index.BTreeBuilder(57 builder = btree_index.BTreeBuilder(
33 reference_lists=1, key_elements=2)58 reference_lists=1, key_elements=2)
34 builder.add_node(('test', 'key1'), 'value', ((('ref', 'entry'),),))59 for node in node_data:
35 builder.add_node(('test', 'key2'), 'value2', ((('ref', 'entry2'),),))60 builder.add_node(*node)
36 builder.add_node(('test2', 'key3'), 'value3', ((('ref', 'entry3'),),))
37 out_f = builder.finish()61 out_f = builder.finish()
38 try:62 try:
39 self.build_tree_contents([('test.btree', out_f.read())])63 self.build_tree_contents([('test.btree', out_f.read())])
@@ -41,90 +65,18 @@
41 out_f.close()65 out_f.close()
4266
43 def test_dump_btree_smoke(self):67 def test_dump_btree_smoke(self):
44 self.create_sample_btree_index()68 self.create_sample_btree_index(self.test_input_3nodes)
45 out, err = self.run_bzr('dump-btree test.btree')69 out, err = self.run_bzr('dump-btree test.btree')
46 self.assertEqualDiff(70 self.assertEqualDiff(self.test_output_3nodes, out)
47 "(('test', 'key1'), 'value', ((('ref', 'entry'),),))\n"
48 "(('test', 'key2'), 'value2', ((('ref', 'entry2'),),))\n"
49 "(('test2', 'key3'), 'value3', ((('ref', 'entry3'),),))\n",
50 out)
5171
52 def test_dump_btree_http_smoke(self):72 def test_dump_btree_http_smoke(self):
53 self.transport_readonly_server = http_server.HttpServer73 self.transport_readonly_server = http_server.HttpServer
54 self.create_sample_btree_index()74 self.create_sample_btree_index(self.test_input_3nodes)
55 url = self.get_readonly_url('test.btree')75 url = self.get_readonly_url('test.btree')
56 out, err = self.run_bzr(['dump-btree', url])76 out, err = self.run_bzr(['dump-btree', url])
57 self.assertEqualDiff(77 self.assertEqualDiff(self.test_output_3nodes, out)
58 "(('test', 'key1'), 'value', ((('ref', 'entry'),),))\n"
59 "(('test', 'key2'), 'value2', ((('ref', 'entry2'),),))\n"
60 "(('test2', 'key3'), 'value3', ((('ref', 'entry3'),),))\n",
61 out)
6278
63 def test_dump_btree_raw_smoke(self):79 def test_dump_btree_raw_smoke(self):
64 self.create_sample_btree_index()80 self.create_sample_btree_index(self.test_input_3nodes)
65 out, err = self.run_bzr('dump-btree test.btree --raw')81 out, err = self.run_bzr('dump-btree test.btree --raw')
66 self.assertEqualDiff(82 self.assertEqualDiff(self.test_raw_output_3nodes, out)
67 'Root node:\n'
68 'B+Tree Graph Index 2\n'
69 'node_ref_lists=1\n'
70 'key_elements=2\n'
71 'len=3\n'
72 'row_lengths=1\n'
73 '\n'
74 'Page 0\n'
75 'type=leaf\n'
76 'test\0key1\0ref\0entry\0value\n'
77 'test\0key2\0ref\0entry2\0value2\n'
78 'test2\0key3\0ref\0entry3\0value3\n'
79 '\n',
80 out)
81
82 def test_dump_btree_no_refs_smoke(self):
83 # A BTree index with no ref lists (such as *.cix) can be dumped without
84 # errors.
85 builder = btree_index.BTreeBuilder(
86 reference_lists=0, key_elements=2)
87 builder.add_node(('test', 'key1'), 'value')
88 out_f = builder.finish()
89 try:
90 self.build_tree_contents([('test.btree', out_f.read())])
91 finally:
92 out_f.close()
93 out, err = self.run_bzr('dump-btree test.btree')
94
95 def create_sample_empty_btree_index(self):
96 builder = btree_index.BTreeBuilder(
97 reference_lists=1, key_elements=2)
98 out_f = builder.finish()
99 try:
100 self.build_tree_contents([('test.btree', out_f.read())])
101 finally:
102 out_f.close()
103
104 def test_dump_empty_btree_smoke(self):
105 self.create_sample_empty_btree_index()
106 out, err = self.run_bzr('dump-btree test.btree')
107 self.assertEqualDiff("", out)
108
109 def test_dump_empty_btree_http_smoke(self):
110 self.transport_readonly_server = http_server.HttpServer
111 self.create_sample_empty_btree_index()
112 url = self.get_readonly_url('test.btree')
113 out, err = self.run_bzr(['dump-btree', url])
114 self.assertEqualDiff("", out)
115
116 def test_dump_empty_btree_raw_smoke(self):
117 self.create_sample_empty_btree_index()
118 out, err = self.run_bzr('dump-btree test.btree --raw')
119 self.assertEqualDiff(
120 'Root node:\n'
121 'B+Tree Graph Index 2\n'
122 'node_ref_lists=1\n'
123 'key_elements=2\n'
124 'len=0\n'
125 'row_lengths=\n'
126 '\n'
127 'Page 0\n'
128 '(empty)\n',
129 out)
130
13183
=== modified file 'bzrlib/tests/test_btree_index.py'
--- bzrlib/tests/test_btree_index.py 2011-01-27 14:27:18 +0000
+++ bzrlib/tests/test_btree_index.py 2011-02-17 14:57:46 +0000
@@ -17,6 +17,7 @@
1717
18"""Tests for btree indices."""18"""Tests for btree indices."""
1919
20import cStringIO
20import pprint21import pprint
21import zlib22import zlib
2223
@@ -31,6 +32,7 @@
31 )32 )
32from bzrlib.tests import (33from bzrlib.tests import (
33 TestCaseWithTransport,34 TestCaseWithTransport,
35 http_server,
34 scenarios,36 scenarios,
35 )37 )
3638
@@ -1522,3 +1524,114 @@
1522 self.set_cached_offsets(index, [0, 1, 2, 3, 4, 5, 6, 7, 100])1524 self.set_cached_offsets(index, [0, 1, 2, 3, 4, 5, 6, 7, 100])
1523 self.assertExpandOffsets([102, 103, 104, 105, 106, 107, 108], index,1525 self.assertExpandOffsets([102, 103, 104, 105, 106, 107, 108], index,
1524 [105])1526 [105])
1527
1528
1529class TestDumpBtree(tests.TestCaseWithTransport):
1530
1531 # Each node is represented as the *string value* that
1532 # BTreeGraphIndex.dump_entries() should produce for it.
1533 test_data_3nodes = (
1534 "(('test', 'key1'), 'value', ((('ref', 'entry'),),))\n",
1535 "(('test', 'key2'), 'value2', ((('ref', 'entry2'),),))\n",
1536 "(('test2', 'key3'), 'value3', ((('ref', 'entry3'),),))\n",
1537 )
1538 test_input_3nodes = [eval(node) for node in test_data_3nodes]
1539 test_output_3nodes = "".join(test_data_3nodes)
1540 test_raw_output_3nodes = (
1541 'Root node:\n'
1542 'B+Tree Graph Index 2\n'
1543 'node_ref_lists=1\n'
1544 'key_elements=2\n'
1545 'len=3\n'
1546 'row_lengths=1\n'
1547 '\n'
1548 'Page 0\n'
1549 'type=leaf\n'
1550 'test\0key1\0ref\0entry\0value\n'
1551 'test\0key2\0ref\0entry2\0value2\n'
1552 'test2\0key3\0ref\0entry3\0value3\n'
1553 '\n'
1554 )
1555
1556 def create_sample_btree_and_outf(self, node_data):
1557 """Create a simple B-Tree with the given content.
1558
1559 As a convenience, also creates an empty StringIO, which the caller can
1560 pass to the method under test as the destination for the dump.
1561
1562 :param node_data: An iterable of 3-tuples (key, value, refs), one tuple
1563 per leaf node in the desired tree,
1564 :return: (btree, outf_for_dump)
1565 """
1566 builder = btree_index.BTreeBuilder(
1567 reference_lists=1, key_elements=2)
1568 for node in node_data:
1569 builder.add_node(*node)
1570 out_f = builder.finish()
1571 try:
1572 self.build_tree_contents([('test.btree', out_f.read())])
1573 finally:
1574 out_f.close()
1575 bt = btree_index.BTreeGraphIndex(self.get_transport(''),
1576 'test.btree', None)
1577 outf_for_dump = cStringIO.StringIO()
1578 return bt, outf_for_dump
1579
1580 def test_dump_btree_smoke(self):
1581 bt, outf = self.create_sample_btree_and_outf(self.test_input_3nodes)
1582 bt.dump_entries(outf)
1583 self.assertEqualDiff(self.test_output_3nodes, outf.getvalue())
1584
1585 def test_dump_btree_http_smoke(self):
1586 self.transport_readonly_server = http_server.HttpServer
1587 _, outf = self.create_sample_btree_and_outf(self.test_input_3nodes)
1588 url = self.get_readonly_url()
1589 t = transport.get_transport(url)
1590 bt = btree_index.BTreeGraphIndex(t, 'test.btree', None)
1591 bt.dump_entries(outf)
1592 self.assertEqualDiff(self.test_output_3nodes, outf.getvalue())
1593
1594 def test_dump_btree_raw_smoke(self):
1595 bt, outf = self.create_sample_btree_and_outf(self.test_input_3nodes)
1596 bt.dump_raw_bytes(outf)
1597 self.assertEqualDiff(self.test_raw_output_3nodes, outf.getvalue())
1598
1599 def test_dump_btree_no_refs_smoke(self):
1600 # A BTree index with no ref lists (such as *.cix) can be dumped without
1601 # errors.
1602 builder = btree_index.BTreeBuilder(reference_lists=0, key_elements=2)
1603 builder.add_node(('test', 'key1'), 'value')
1604 out_f = builder.finish()
1605 try:
1606 self.build_tree_contents([('test.btree', out_f.read())])
1607 finally:
1608 out_f.close()
1609 bt = btree_index.BTreeGraphIndex(self.get_transport(''),
1610 'test.btree', None)
1611 outf = cStringIO.StringIO()
1612 bt.dump_entries(outf)
1613
1614 test_input_0nodes = ()
1615 test_output_0nodes = ""
1616 test_raw_output_0nodes = (
1617 'Root node:\n'
1618 'B+Tree Graph Index 2\n'
1619 'node_ref_lists=1\n'
1620 'key_elements=2\n'
1621 'len=0\n'
1622 'row_lengths=\n'
1623 '\n'
1624 'Page 0\n'
1625 '(empty)\n'
1626 )
1627
1628 def test_dump_empty_btree_smoke(self):
1629 bt, outf = self.create_sample_btree_and_outf(self.test_input_0nodes)
1630 bt.dump_entries(outf)
1631 self.assertEqualDiff(self.test_output_0nodes, outf.getvalue())
1632
1633 def test_dump_empty_btree_raw_smoke(self):
1634 bt, outf = self.create_sample_btree_and_outf(self.test_input_0nodes)
1635 bt.dump_raw_bytes(outf)
1636 self.assertEqualDiff(self.test_raw_output_0nodes, outf.getvalue())
1637