Merge lp:~adeuring/lazr.batchnavigator/slicing-error-for-too-short-last-backwards-batch into lp:lazr.batchnavigator
Status: | Merged |
---|---|
Approved by: | Graham Binns |
Approved revision: | 48 |
Merged at revision: | 48 |
Proposed branch: | lp:~adeuring/lazr.batchnavigator/slicing-error-for-too-short-last-backwards-batch |
Merge into: | lp:lazr.batchnavigator |
Diff against target: |
324 lines (+234/-9) 3 files modified
src/lazr/batchnavigator/tests/test_z3batching.py (+135/-1) src/lazr/batchnavigator/z3batching/batch.py (+83/-1) src/lazr/batchnavigator/z3batching/interfaces.py (+16/-7) |
To merge this branch: | bzr merge lp:~adeuring/lazr.batchnavigator/slicing-error-for-too-short-last-backwards-batch |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Graham Binns (community) | code | Approve | |
Review via email: mp+72745@code.launchpad.net |
Description of the change
The problem:
- When _Batch.
result set (line 235 of batch.py) to get the data
of a backwards batch:
sliced = self.range_
- when the result of this call has less elements than
needed
the method called
self.
in order to retrieve more elements of a result set. This
works fine, if self.range_memo, i.e., the endpoint memo
value used to retrieve the first, too small, chunk of data,
can be used as an endpoint value to retrieve a chunk of data
that follows the already existing chunk.
This works for the ListRangeFactory, which uses the regular
Python slicing protocol: The expression s[a, b] is that part
of the sequence s which starts at index a and ends before
index b.
In this case, self.range_memo is basically used like 'b' above
in
sliced = self.range_
and it is used as 'a' in
self.
But Launchpad's class StormRangeFactory works slightly
different: Its method getEndpointmemos() returns the
values of the columns used for sorting: The two values
returned by this method should be used in a query like
"return N rows from the result set where the sort columns
have values (smaller than|larger than) the endpoint memo
value".
If we assume a result set like
[1, 2, 3, 4, 5]
and the bachwards batch was retrieved for the memo value 3
(factory.
[1, 2]
The second call (factory.
would return
[4, 5]
The core of the fix: sliced_list() must explicitly retrieve
the endpoint memo values for the already retrieved chunk of
data. That's just these lines of the diff:
+ partial = _PartialBatch(
+ extra_memo = (
+ self.range_
extra = self.range_
- self.range_memo, forwards=True)
+ extra_memo[1], forwards=True)
The problem with these innocent lines: A method
IRangeFactory.
parameters from batch...
So we need to build again an object which implements the
full IBatch interface (at least formally -- the methods
prevBatch() and nextBatch() are irrelevant, hopefully also
in the future...) so that any range factory has access
to whatever batching related attribute or method it wants...
This "partial batch" (is there a better name?) is implemented
by, well, class _PartialBatch.
I also added the methods __getitem__, sliced_list, trueSize
to IBatch, because these sliced_list is explicitly used by
StormRangeFactory to retrieve results. And ListRangeFactory
wants access to trueSize.
I first reproduced and then the problem of the "missing
element" in a backwards batch with the new method
test_last_
which needs a special IRangeFactory, which in turn needed
a bit of testing (test_PartialBa
That's roughly how a six lines diff exploded into 320 lines ;)
turtles all the way down huh.
review: approve