Merge lp:~pbeaman/akiban-persistit/release-3.1.7-prep into lp:akiban-persistit

Proposed by Peter Beaman
Status: Merged
Approved by: Peter Beaman
Approved revision: 368
Merged at revision: 365
Proposed branch: lp:~pbeaman/akiban-persistit/release-3.1.7-prep
Merge into: lp:akiban-persistit
Diff against target: 775 lines (+368/-209)
11 files modified
doc/BugList (+145/-0)
doc/Configuration.rst (+20/-3)
doc/ReleaseNotes.rst (+63/-188)
doc/build/src/SphinxDocPrep.java (+127/-3)
src/main/java/com/persistit/IntegrityCheck.java (+2/-2)
src/main/java/com/persistit/JournalManager.java (+0/-1)
src/main/java/com/persistit/JournalManagerBench.java (+4/-4)
src/main/java/com/persistit/Persistit.java (+1/-1)
src/test/java/com/persistit/Bug1041293Test.java (+1/-1)
src/test/java/com/persistit/TrackingFileChannel.java (+3/-4)
src/test/java/com/persistit/stress/unit/CommitBench.java (+2/-2)
To merge this branch: bzr merge lp:~pbeaman/akiban-persistit/release-3.1.7-prep
Reviewer Review Type Date Requested Status
Peter Beaman Approve
Review via email: mp+123836@code.launchpad.net

Description of the change

Updated release notes:
- reformatted to be more concise and readable
- added new bugs and changes for version 3.1.7.
- added code in documentation preparation classes to create the bug list table

Applied code-cleanup rules (mostly adding "final" to a number of declarations) for consistency.

LBJ: NO_BUILD

To post a comment you must log in.
367. By Peter Beaman

Typo in SphinxDocPrep.java

368. By Peter Beaman

Add the BugList file

Revision history for this message
Peter Beaman (pbeaman) wrote :

Self-approving because there are no code changes (other than cosmetic changes) and Nathan is unavailable to review.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'doc/BugList'
2--- doc/BugList 1970-01-01 00:00:00 +0000
3+++ doc/BugList 2012-09-11 21:38:18 +0000
4@@ -0,0 +1,145 @@
5+1044513 : 3.1.7
6+
7+`AssertionError in IntegrityCheckTest`
8+
9+A bug in unit test code caused spurious assertion errors.
10+
11+1044397 : 3.1.7
12+
13+`IntegrityCheck throws ArrayIndexOutOfBoundsException`
14+
15+IntegrityCheck and the CLI ``pview`` command were not immune to
16+certain inconsistent byte patterns in a corrupt page, making them
17+inadequate for diagnosing a severely corrupted database.
18+
19+1041293 : 3.1.7
20+
21+`Mismatched buffer pool sizes and volume page size results in NPE`
22+
23+Attempting to open a Volume with a valid page size S resulted in a
24+NullPointerException if there was no buffer pool having page size
25+S specified. This same condition how throws an IllegalStateException.
26+
27+1043536 : 3.1.7
28+
29+`Build fails: persistit-coverage hangs with deadlock`
30+
31+A bug in a unit test left a deadlock condition under certain race
32+conditions. Speed variations caused by clover instrumentation
33+triggered frequent occurrences.
34+
35+1036422 : 3.1.6
36+
37+`CommitPolicy HARD can soak CPU`
38+
39+With CommitPolicy HARD we observed a CPU-soaking loop in the commit
40+logic.
41+
42+1032701 : 3.1.5
43+
44+`Interrupt causes Thread to exit without releasing claims`
45+
46+If a thread was interrupted at an inopportune time (for example,
47+by query cancellation in Akiban Server), a page in the buffer pool
48+could be left in a locked state blocking all further progress. The
49+only remedy was to stop and restart the JVM.
50+
51+1028016 : 3.1.4
52+
53+`Old Journal Files Not Being Deleted`
54+
55+If a volume was removed from the configuration and Persistit was
56+then restarted, the associated journal files would not be removed
57+due to internal safety checks. To support intentional removal of a
58+volume, a new configuration option was added.
59+
60+See the ``Detecting and Ignoring Missing Volumes`` feature description
61+for more details.
62+
63+
64+1024857,1026207 : 3.1.4
65+
66+`Class Index Updates Causing Write-Write Dependencies`
67+
68+Custom classes that are serialized into keys or values through a
69+custom KeyCoder or ValueCoder are given a unique identifier. The
70+identifier was determined within an application transaction the
71+first time a class was written and stored in an internal tree. This
72+would cause seemingly spurious aborts if multiple application threads
73+simultaneously attempted to store the first instance of a given
74+class. The identifier is now committed within an independent
75+transaction.
76+
77+1028050,1028134 : 3.1.4
78+
79+`Accumulator Memory Usage and Transaction Step Policy`
80+
81+Each individual change to an Accumulator was previously stored as
82+an individual Delta object. For long running transactions that
83+heavily utilized Accumulators, this would cause excessive memory
84+usage. These unique instances are now eliminated in most scenarios,
85+resulting in stable memory utilization in all but pathological
86+cases.
87+
88+Additionally, the Accumulator handling of the Transaction step value
89+was inconsistent with how it was treated through an Exchange. Now,
90+both classes allow a transaction at a given step N to see any update
91+created by that transaction at a step less N.
92+
93+
94+1005206 : 3.1.2
95+
96+`Infinite Loop When Repacking Buffer`
97+
98+This could occur if a buffer required restructuring during pruning
99+of a long value that was previously stored under a transaction.
100+Upon the next save of this buffer to disk (e.g. shutdown), an
101+infinite loop would occur.
102+
103+
104+1010079 : 3.1.2
105+
106+`Corruption Exceptions During Various Operations`
107+
108+NOTE: Only the message indicates a database corruption. The data
109+volume is actually correct and intact.
110+
111+This could occur if pruning a buffer containing a long record
112+previously stored under a transaction required removal of keys and
113+then that buffer was reused without further modification. A parallel
114+structure associated with the every Buffer, the FastIndex, was not
115+maintained during this operation.
116+
117+1012856 : 3.1.2
118+
119+`Slow Accumulator Operations`
120+
121+If a thread starting a new transaction was interrupted during the
122+call to begin(), there was a chance for an internal object to wind
123+up in an invalid state. This invalid state caused no visible
124+consequences other than slower than expected Accumulator actions
125+if this had occurred many times.
126+
127+
128+1017957 : 3.1.2
129+
130+`B+Tree Corruption in Stress Test`
131+
132+An extremely rare combination of events corrupted memory structures
133+causing an incorrect key-pointer pair to be inserted while deleting
134+records in a key range. The bug was detected in an 8-hour stress
135+test run.
136+
137+1018526 : 3.1.2
138+
139+`Very long recovery time due to many temporary tree IT records`
140+
141+Every Tree created in a temporary volume was being assigned a tree
142+handle recorded permanently in the journal. In one case the result
143+was a journal containing millions of IT (Identify Tree) records,
144+and these caused normal recovery to take a very long time. The fix
145+keeps temporary trees out of the journal and removes IT records
146+which may have been added previously.
147+
148+
149+
150
151=== modified file 'doc/Configuration.rst'
152--- doc/Configuration.rst 2012-08-23 21:10:56 +0000
153+++ doc/Configuration.rst 2012-09-11 21:38:18 +0000
154@@ -185,7 +185,24 @@
155
156 The journal is written by appending records to the end of the highest-numbered file. Read operations occur while copying page images from the journal to their home volume files. While copying, Persistit attempts to perform large sequential read operations from the journal. Read operations also occur at random when Persistit needs to reload the image of a previously evicted page.
157
158-Because of these characteristics a modern SSD (solid disk drive) is ideally suited for maintaining the journal. If no SSD is available in the server, placing the journal on a different physical disk drive than the volume file(s) can significantly improve performance.
159+Because of these characteristics a modern enterprise SSD (solid disk drive) is ideally suited for maintaining the journal. If no SSD is available in the server, placing the journal on a different physical disk drive than the volume file(s) can significantly improve performance.
160+
161+However, beware of consumer-grade SSDs. Some may cache writes and record them on non-volatile storage much later. In a power failure event, committed transactions recorded only in the write cache can be lost. The durability of committed transactions can only be guaranteed if the hardware stack cooperates in making sure all writes are durable.
162+
163+Buffer Pool Preload
164+-------------------
165+
166+Persistit includes an optional facility to record periodically an inventory of pages in the buffer pool. When Persistit starts up it attempts to re-read the same pages that were previously in the buffer pool so that performance after restart on a similar workload (e.g., queries that exhibit a moderate or strong degree of locality-of-reference) will run at full speed. This facility can be especially important in large buffer pool configurations in which under normal workloads a server may otherwise require minutes to hours to reach optimal speed due to a large number of random reads. Re-reading the inventory at startup can be much faster since pages are read in approximate physical order.
167+
168+This facility is controlled by two configuration parameters:
169+
170+- ``com.persistit.Configuration#setBufferInventoryEnabled`` controls whether Persistit records the inventory. If enabled, the recording happens once per Checkpoint and once at normal shutdown. The inventory is stored in the system volume.
171+
172+- ``com.persistit.Configuration#setBufferPreloadEnabled`` controls whether Persistit attempts to preload the buffer pool during startup. If there is a recorded inventory Persistit attempts to re-read the pages that were previously present; otherwise it silently continues the startup.
173+
174+If upon restart the buffer pool has become smaller so that the inventory is larger than the current buffer pool, Persistit only loads as many pages as there currently are buffers.
175+
176+Although by default these configuration properties are false, we recommend enabling them for production servers because the inventory process takes very little time, and buffer preload can restore the buffer pool to a useful working set orders of magnitude faster than warming it up through normal load.
177
178 Other Configuration Parameters
179 ------------------------------
180@@ -232,14 +249,14 @@
181 only when no adapter has been installed.
182
183 ``bufferinventory``: (``com.persistit.Configuration#setBufferInventoryEnabled``), True or False (default).
184- If true, Persistit periodically records an inventory of all the buffes in the buffers pools to the System Volume. The inventory
185+ If true, Persistit periodically records an inventory of all the pages in the buffers pools to the System Volume. The inventory
186 enables Persistit to preload the buffer pools then next time it starts up with approximately the same pages that were present
187 before shutdown. To enable buffer preloading, the bufferpreload property must also be true.
188
189 ``bufferpreload``: (``com.persistit.Configuration#setBufferPreloadEnabled``), True or False (default).
190 If true, and if a buffer pool inventory was previously recorded, Persistit attempts to "warm up" the buffer pool
191 by preloading pages that were present in the buffer pool when Persistit last shut down. This may allow a freshly started
192- Persistit instance to begin service a workload similar to what it had previously been processing without incurring the
193+ Persistit instance to begin servicing a workload similar to what it had previously been handling without incurring the
194 cost of many random disk reads to load pages.
195
196
197
198=== modified file 'doc/ReleaseNotes.rst'
199--- doc/ReleaseNotes.rst 2012-08-24 18:16:21 +0000
200+++ doc/ReleaseNotes.rst 2012-09-11 21:38:18 +0000
201@@ -44,207 +44,81 @@
202 |
203 |
204
205-
206-************************************
207-3.1.6
208-************************************
209-
210-Release Date
211-============
212-August 24, 2012
213-
214-Overview
215-========
216-This version of Persistit fixes one bug.
217-
218-Fixed Issues
219-============
220-
221-https://bugs.launchpad.net/akiban-persistit/+bug/1036422
222-
223-With CommitPolicy HARD we observed a CPU-soaking loop in the commit logic.
224-
225-************************************
226-3.1.5
227-************************************
228-
229-Release Date
230-============
231-August 6, 2012
232-
233-Overview
234-========
235-This version of Persistit changes the pom.xml to a form compatible with the Maven central repository and includes one bug fix.
236-
237-Fixed Issue
238-===========
239-
240-https://launchpad.net/akiban-persistit/+bug/1032701
241-
242-If a thread was interrupted at an inopportune time (for example, by query cancellation in Akiban Server), a page in the buffer pool could be left in a locked state blocking all further progress. The only remedy was to stop and restart the JVM.
243-
244-|
245-|
246-
247-************************************
248-3.1.4
249-************************************
250-
251-Release Date
252-============
253-August 3, 2012
254-
255-Overview
256-========
257-This version of Persistit changes the licensing to Eclipse Public License (EPL) and includes minor enhancements and bug fixes.
258-
259-New Features
260-============
261-
262-Detection and Ignoring of Missing Volumes
263------------------------------------------
264-
265-Warnings are now issues when a previously existing volume is missing, and an option to allow a system to continue by ignoring pages and transactions in the journal from missing volumes.
266-
267-Every time Persistit writes a modified page to disk, it does so first to the journal. During recovery processing, the page images from the journal are analyzed and reinserted into volumes in such a way that all B+Trees are restored to a consistent state. The issue addressed in this change is how Persistit behaves during recovery if it discovers that a volume referred to by a page in the journal no longer exists.
268-
269-Recognizing that under some circumstances an administrator may indeed wish to remove a volume from an existing Database, this change provides a configurable switch to optionally allow pages from missing volumes to be skipped (with logged warning messages) during recovery processing. The switch can be enabled by setting the configuration parameter ignoreMissingVolumes to true, see com.persistit.Configuration#setIgnoreMissingViolumes.
270-
271-
272-Reduce KeyCoder Serialized Object Size
273---------------------------------------
274+Version History
275+===============
276+
277++---------+--------------------+--------------------------------------------------------------------------+
278+| Version | Release Date | Summary |
279++=========+====================+==========================================================================+
280+| 3.1.7 | September 11, 2012 | Fix several bugs, add buffer pool preload ("warm-up"), |
281+| | | reformat code base |
282++---------+--------------------+--------------------------------------------------------------------------+
283+| 3.1.6 | August 24, 2012 | Fix bug 1036422 |
284++---------+--------------------+--------------------------------------------------------------------------+
285+| 3.1.5 | August 6, 2012 | Fix bug 1032701, modify pom.xml for Eclipse Juno |
286++---------+--------------------+--------------------------------------------------------------------------+
287+| 3.1.4 | August 3, 2012 | License changed to Eclipse Public License, various other enhancements |
288++---------+--------------------+--------------------------------------------------------------------------+
289+| 3.1.2 | July 13, 2012 | Fix several bugs |
290++---------+--------------------+--------------------------------------------------------------------------+
291+| 3.1.1 | May 31, 2012 | First open source release of Akiban Persistit |
292++---------+--------------------+--------------------------------------------------------------------------+
293+
294+
295+Resolved Issues
296+===============
297+
298+{{bug-list}}
299+
300+Changes and New Features
301+========================
302+
303+Persistit 3.1.7 - Code Base Reformatted
304+---------------------------------------
305+
306+To simplify diffs and improve legibility, the entire code base was reformatted and "cleaned up" by the Eclipse code formatting tool. The maven build now automatically formats all
307+source to ensure coherent diffs in the future. The settings for formatting and code style cleanup by Eclipse are found in the ``src/etc`` directory.
308+
309+Persistit 3.1.7 - Buffer Pool Preload
310+-------------------------------------
311+
312+On a server with a large buffer pool (many gigabytes), a Persistit instance can run for a long time before the buffer pool becomes populated with a suitable working set of database pages. Until then performance is degraded due to a potentially large number of random reads. For a production server the result may be poor performance for minutes to hours after restart.
313+
314+The preload facility periodically records an inventory of the pages currently in the buffer pool(s) and optionally reloads the same set of pages when Persistit is restarted. During the preload process Persistit attempts to read pages in approximate file-address order to accelerate reads. In one of our experiments Persistit preloads a buffer pool with over 800,000 16Kbyte buffers in about a minute, which is orders of magnitude faster than the same process would take with reads performed incrementally at random.
315+
316+Two new configuration properties com.persistit.Configuration#setBufferInventoryEnabled and com.persistit#setBufferPreloadEnabled control this behavior. These settings are turned off by default in version 3.1.7.
317+
318+Persistit 3.1.4 - Detecting and Ignoring Missing Volumes
319+--------------------------------------------------------
320+
321+Every time Persistit writes a modified page to disk, it does so first to the journal. During recovery processing, the page images from the journal are analyzed and reinserted into volumes in such a way that all B+Trees are restored to a consistent state. The issue addressed in this change is how Persistit behaves during recovery if it discovers that a volume referred to by a page in the journal no longer exists.
322+
323+Recognizing that under some circumstances an administrator may indeed wish to remove a volume from an existing Database, this change provides a configurable switch to optionally allow pages from missing volumes to be skipped (with logged warning messages) during recovery processing. The switch can be enabled by setting the configuration parameter com.persistit.Configuration#setIgnoreMissingViolumes to true.
324+
325+
326+Persistit 3.1.4 - Reduce KeyCoder Serialized Object Size
327+--------------------------------------------------------
328
329 .. note::
330 Any Database containing objects serialized by a custom KeyCoder from a previous version of Persistit is incompatible with this change
331
332 Minimize the per-instance overhead for application objects written into Persistit Keys by reducing the size of the internal identifier.
333
334-Persistit has rich support for serializing standard Java primitive and object types into a Key. Additionally, the KeyCoder class allows for any application level object to also be appended to a Key right next to any other type. This is tagged internally with per-class handles. This change lowers the initial offset to, in many cases, halve the serialized size.
335+Persistit has rich support for serializing standard Java primitive and object types into a Key. Additionally, the KeyCoder class allows for any application level object to also be appended to a Key right next to any other type. This is tagged internally with per-class handles. This change lowers the initial offset to reduce and in many cases halve the serialized size.
336
337-Maven POM Changes For Eclipse Juno
338-----------------------------------
339+Persistit 3.1.4 - Maven POM Changes For Eclipse Juno
340+----------------------------------------------------
341
342 The latest version of Eclipse, code named Juno, features a wide array of changes, including a new release of the m2eclipse plugin. In an effort to make getting started with Persistit as easy as possible, we have included the required m2e configuration sections in our pom.
343
344 Please contact Akiban if you have encounter any issues getting up and running with Persistit.
345
346-Fixed Issues
347-============
348-
349-Old Journal Files Not Being Deleted
350------------------------------------
351-
352-https://launchpad.net/akiban-persistit/+bug/1028016
353-
354-If a volume was removed from the configuration and Persistit was restarted the associated journal files would not be removed due to internal safety checks. In the event that the missing volume is intended behavior, a new configuration option was added. See the ``Detection and Ignoring of Missing Volumes`` feature description for more details.
355-
356-Class Index Updates Causing Write-Write Dependencies
357-----------------------------------------------------
358-
359-https://launchpad.net/akiban-persistit/+bug/1024857
360-
361-https://launchpad.net/akiban-persistit/+bug/1026207
362-
363-Custom classes that are serialized into keys or values, through a custom KeyCoder or ValueCoder, are given a unique identifier. The identifier is determined transactionally the first time a class is written and stored in an internal tree. This would cause seemingly spurious aborts if more than one application threads simultaneously attempt to store the first instance of a given class.
364-
365-Accumulator Memory Usage and Transaction Step Policy
366-----------------------------------------------------
367-
368-https://launchpad.net/akiban-persistit/+bug/1028050
369-
370-https://launchpad.net/akiban-persistit/+bug/1028134
371-
372-Changes to Accumulators are stored were previously stored as individual Delta objects. For long running transactions that heavily utilized Accumulators, this would cause excessive memory usage. These unique instances are no eliminated in most scenarios, resulting in no memory growth in all but pathological cases.
373-
374-Additionally, the Accumulator handling of the Transaction step value was inconsistent with how it was treated through an Exchange. Now, both classes allow a given step x to see any change that occurred at a step less than or equal to itself.
375-
376-
377-Known Issues
378-============
379-As described in the *3.1.1 Known Issues*.
380-
381-|
382-|
383-
384-************************************
385-3.1.2
386-************************************
387-
388-Release Date
389-============
390-July 13, 2012
391-
392-Overview
393-========
394-This is a bug fix release of the Persistit project (https://launchpad.net/akiban-persistit).
395-
396-Fixed Issues
397-============
398-
399-Infinite Loop When Repacking Buffer
400------------------------------------
401-
402-https://bugs.launchpad.net/bugs/1005206
403-
404-This was introduced late into the 3.1.1 development cycle. This could occur if a buffer required restructuring during pruning of a long value that was previously stored under a transaction. Upon the next save of this buffer to disk (e.g. shutdown), an infinite loop would occur.
405-
406-Corruption Exceptions During Various Operations
407------------------------------------------------
408-
409-https://bugs.launchpad.net/bugs/1010079
410-
411-.. note::
412- Only the message indicates a database corruption. The data volume is actually correct and intact.
413-
414-This was introduced late into the 3.1.1 development cycle. This could occur if pruning a buffer containing a long record previously stored under a transaction required removal of keys and then that buffer was reused without further modification. A parallel structure associated with the every ``Buffer``, the ``FastIndex``, was not maintained during this operation.
415-
416-Slow Accumulator Operations
417----------------------------
418-
419-https://bugs.launchpad.net/bugs/1012859
420-
421-This bug preexisted but was unknown to the 3.1.1 release. If a thread starting a new transaction was interrupted during the call to ``begin()``, there was a chance for an internal object to wind up in an invalid state. This invalid state caused no visible consequences other than slower than expected ``Accumulator`` actions if this had occurred many times.
422-
423-B+Tree Corruption in Stress Test
424---------------------------------
425-
426-https://bugs.launchpad.net/akiban-persistit/+bug/1017957
427-
428-This bug preexisted but was unknown to the 3.1.1 release. An extremely rare combination of events corrupted memory structures causing an incorrect key-pointer pair to be inserted while deleting records in a key range. The bug was detected in an 8-hour stress test run.
429-
430-
431-Slow Recovery Due to Temporary Tree IT Records
432-----------------------------------------------
433-
434-https://bugs.launchpad.net/akiban-persistit/+bug/1018526
435-
436-This bug preexisted but was unknown to the 3.1.1 release. Every Tree created in a temporary volume was being assigned a tree handle recorded permanently in the journal. In one case the result was a journal containing millions of IT (Identify Tree) records, and these caused normal recovery to take a very long time. The fix keeps temporary trees out of the journal and removes IT records which may have been added previously.
437-
438-
439-Asserts Added to Check for Correct Exchange Thread Behavior
440------------------------------------------------------------
441+Persistit 3.1.2 - Asserts Added to Check for Correct Exchange Thread Behavior
442+-----------------------------------------------------------------------------
443
444 A bug in the Akiban Server code caused an Exchange to be used concurrently by two Threads, causing serious and seemingly unrelated failures in Persistit including instances of IllegalMonitorException and IllegalStateException. To guard against future occurrences, asserts were added to catch such concurrent use by multiple threads. Applications should be tested with asserts enabled to verify correct thread usage.
445-
446-
447-Known Issues
448-============
449-As described in the *3.1.1 Known Issues*.
450-
451-|
452-|
453-
454-************************************
455-3.1.1
456-************************************
457-
458-Release Date
459-============
460-May 31, 2012
461-
462-Overview
463-========
464-This is the first open source release of the Persistit project (https://launchpad.net/akiban-persistit).
465+
466+
467
468 Known Issues
469 ============
470@@ -283,7 +157,8 @@
471 https://bugs.launchpad.net/akiban-persistit/+bug/1000331
472
473 Multi-version values are not always pruned properly causing volume growth. The number of MVV records and their overhead size can be obtaining by running the IntegrityCheck task.
474+
475 * Workaround 1: Run the IntegrityCheck task (CLI command icheck) with the -P option which will prune the MVVs. This will remove obsolete MVV instances and in many cases free up pages in which new data can be stored. However, it will not reduce the actual size of the volume file.
476
477-* Workaround 2: To reduce the size of the volume you can use the CLI commands save and load to offload and then reload the data into a newly created volume file. See http://www.akiban.com/ak-docs/admin/persistit/Management.html#management for more information about these operations.
478+* Workaround 2: To reduce the size of the volume you can use the CLI commands ``save`` and ``load`` to reload the data into a newly created volume file. See http://www.akiban.com/ak-docs/admin/persistit/Management.html#management for more information about these operations.
479
480
481=== modified file 'doc/build/src/SphinxDocPrep.java'
482--- doc/build/src/SphinxDocPrep.java 2012-06-11 21:25:37 +0000
483+++ doc/build/src/SphinxDocPrep.java 2012-09-11 21:38:18 +0000
484@@ -2,7 +2,10 @@
485 import java.io.File;
486 import java.io.FileReader;
487 import java.io.FileWriter;
488+import java.io.IOException;
489 import java.io.PrintWriter;
490+import java.util.ArrayList;
491+import java.util.List;
492 import java.util.SortedMap;
493 import java.util.regex.Matcher;
494 import java.util.regex.Pattern;
495@@ -68,7 +71,10 @@
496 }
497
498 private void processLine(final String line) throws Exception {
499- if (line.contains(".. code-block:") || line.endsWith("::")) {
500+ if (line.contains("{{bug-list}}")) {
501+ prepareBugList();
502+ return;
503+ } else if (line.contains(".. code-block:") || line.endsWith("::")) {
504 block = BlockState.WAIT_FIRST_BLANK_LINE;
505 } else if (line.isEmpty()) {
506 switch (block) {
507@@ -134,14 +140,132 @@
508 text = text.replace("com.persistit.", "");
509 text = text.replace("java.lang.", "");
510 text = text.replace("java.util.", "");
511- replacement= "`" + text + " <" + url + ">`_";
512+ replacement = "`" + text + " <" + url + ">`_";
513 }
514 }
515
516 matcher.appendReplacement(sb, Matcher.quoteReplacement(replacement));
517 }
518
519+ private void prepareBugList() throws IOException {
520+ List<String[]> rows = new ArrayList<String[]>();
521+ rows.add(new String[] { "Bug Reference", "Fixed in|Version", "Summary" });
522+ BufferedReader reader = null;
523+ try {
524+ String urls = "";
525+ String version = "";
526+ reader = new BufferedReader(new FileReader("../BugList"));
527+ StringBuilder sb = new StringBuilder();
528+ String line = null;
529+ while ((line = reader.readLine()) != null) {
530+ if (line.isEmpty()) {
531+ if (sb.length() > 0) {
532+ sb.append('|');
533+ }
534+ } else if (Character.isDigit(line.charAt(0))) {
535+ if (!urls.isEmpty()) {
536+ rows.add(new String[] { urls, version, sb.toString() });
537+ urls = "";
538+ version = "";
539+ sb.setLength(0);
540+ }
541+ String[] split = line.split("\\:");
542+ for (final String bug : split[0].trim().split(",")) {
543+ if (urls.length() > 0) {
544+ urls += '|';
545+ }
546+ urls = urls + "https://launchpad.net/bugs/" + bug;
547+ }
548+ version = split[1].trim();
549+ } else {
550+ sb.append(line.trim());
551+ if (sb.length() > 0) {
552+ sb.append('|');
553+ }
554+ }
555+ }
556+ if (!urls.isEmpty()) {
557+ rows.add(new String[] { urls, version, sb.toString() });
558+ }
559+
560+ int[] maxWidth = new int[3];
561+ for (final String[] row : rows) {
562+ for (int i = 0; i < 3; i++) {
563+ for (String s : row[i].split("\\|")) {
564+ maxWidth[i] = Math.max(s.length(), maxWidth[i]);
565+ }
566+ }
567+ }
568+
569+ bugTableLine(sb, false, maxWidth);
570+ for (int l = 0;; l++) {
571+ if (bugTableText(sb, rows.get(0), maxWidth, l)) {
572+ break;
573+ }
574+ }
575+ bugTableLine(sb, true, maxWidth);
576+ for (int i = 1; i < rows.size(); i++) {
577+ String[] text = rows.get(i);
578+ for (int l = 0;; l++) {
579+ if (bugTableText(sb, text, maxWidth, l)) {
580+ break;
581+ }
582+ }
583+ bugTableLine(sb, false, maxWidth);
584+ }
585+ } catch (IOException e) {
586+ System.out.println(e + " while trying to read BugList");
587+ } finally {
588+ if (reader != null) {
589+ reader.close();
590+ }
591+ }
592+ }
593+
594+ private void bugTableLine(StringBuilder sb, boolean dline, int[] width) {
595+ sb.setLength(0);
596+ sb.append('+');
597+ for (int j = 0; j < 3; j++) {
598+ for (int i = 0; i < width[j] + 2; i++) {
599+ sb.append(dline ? '=' : '-');
600+ }
601+ sb.append('+');
602+ }
603+ writer.println(sb);
604+ writer.flush();
605+ }
606+
607+ private boolean bugTableText(StringBuilder sb, String[] text, int[] width, int line) {
608+ String[] s = new String[3];
609+ boolean done = true;
610+ for (int j = 0; j < 3; j++) {
611+ String[] split = text[j].split("\\|");
612+ if (split.length > line) {
613+ done = false;
614+ s[j] = split[line];
615+ }
616+ }
617+ if (!done) {
618+ sb.setLength(0);
619+ sb.append("|");
620+ for (int j = 0; j < 3; j++) {
621+ if (s[j] == null) {
622+ s[j] = "";
623+ }
624+ sb.append(' ');
625+ sb.append(s[j]);
626+ for (int i = s[j].length() + 1; i < width[j] + 2; i++) {
627+ sb.append(' ');
628+ }
629+ sb.append('|');
630+ }
631+ writer.println(sb);
632+ }
633+ writer.flush();
634+ return done;
635+ }
636+
637 public static void main(final String[] args) throws Exception {
638- new SphinxDocPrep().prepare(args);
639+ new SphinxDocPrep().prepare(args);
640 }
641 }
642
643=== modified file 'src/main/java/com/persistit/IntegrityCheck.java'
644--- src/main/java/com/persistit/IntegrityCheck.java 2012-08-31 14:06:47 +0000
645+++ src/main/java/com/persistit/IntegrityCheck.java 2012-09-11 21:38:18 +0000
646@@ -1158,7 +1158,7 @@
647 }
648 }
649 }
650- } catch (Exception e) {
651+ } catch (final Exception e) {
652 addFault(e.toString(), page, level, 0);
653 }
654 return true;
655@@ -1213,7 +1213,7 @@
656
657 fromPage = longPage;
658 longPage = longBuffer.getRightSibling();
659- } catch (Exception e) {
660+ } catch (final Exception e) {
661 addFault(e.toString() + " while verifying long record page " + longPage, page, 0, foundAt);
662 break;
663 } finally {
664
665=== modified file 'src/main/java/com/persistit/JournalManager.java'
666--- src/main/java/com/persistit/JournalManager.java 2012-08-28 16:15:27 +0000
667+++ src/main/java/com/persistit/JournalManager.java 2012-09-11 21:38:18 +0000
668@@ -29,7 +29,6 @@
669 import java.util.Collections;
670 import java.util.Comparator;
671 import java.util.HashMap;
672-import java.util.HashSet;
673 import java.util.Iterator;
674 import java.util.List;
675 import java.util.Map;
676
677=== modified file 'src/main/java/com/persistit/JournalManagerBench.java'
678--- src/main/java/com/persistit/JournalManagerBench.java 2012-08-20 19:24:58 +0000
679+++ src/main/java/com/persistit/JournalManagerBench.java 2012-09-11 21:38:18 +0000
680@@ -123,7 +123,7 @@
681 }
682 if (remaining > buffer.capacity()) {
683 remaining = buffer.capacity();
684- long unaligned = fc.size() % 16384;
685+ final long unaligned = fc.size() % 16384;
686 if (unaligned > 0) {
687 remaining = remaining - (16384 - unaligned);
688 }
689@@ -139,12 +139,12 @@
690
691 private void doOneCycle(final long time, final int align, final long extension, final int size) throws Exception {
692 // Make a fake transaction record
693- String header = String.format("\nsize=%06d count=%06d time=%012d\n", size, count, time);
694- byte[] b = header.getBytes();
695+ final String header = String.format("\nsize=%06d count=%06d time=%012d\n", size, count, time);
696+ final byte[] b = header.getBytes();
697 System.arraycopy(b, 0, bytes, 0, b.length);
698
699 // Add the record, possibly offset to maintaining alignment
700- int toRewrite = (int) (currentAddress - writeAddress);
701+ final int toRewrite = (int) (currentAddress - writeAddress);
702 buffer.position(toRewrite);
703 buffer.put(bytes, 0, size);
704 boolean extended = false;
705
706=== modified file 'src/main/java/com/persistit/Persistit.java'
707--- src/main/java/com/persistit/Persistit.java 2012-08-31 20:20:13 +0000
708+++ src/main/java/com/persistit/Persistit.java 2012-09-11 21:38:18 +0000
709@@ -725,7 +725,7 @@
710 for (final BufferPool pool : _bufferPoolTable.values()) {
711 try {
712 pool.recordBufferInventory(timestamp);
713- } catch (PersistitException e) {
714+ } catch (final PersistitException e) {
715 getLogBase().bufferInventoryException.log(e);
716 }
717 }
718
719=== modified file 'src/test/java/com/persistit/Bug1041293Test.java'
720--- src/test/java/com/persistit/Bug1041293Test.java 2012-09-04 21:01:23 +0000
721+++ src/test/java/com/persistit/Bug1041293Test.java 2012-09-11 21:38:18 +0000
722@@ -34,7 +34,7 @@
723 final Configuration config = _persistit.getConfiguration();
724 final VolumeSpecification vspec = config.volumeSpecification("${datapath}/test,pageSize:2048,create,"
725 + "initialPages:100,extensionPages:100,maximumPages:25000");
726- Volume volume = _persistit.loadVolume(vspec);
727+ final Volume volume = _persistit.loadVolume(vspec);
728 volume.open(_persistit);
729 }
730
731
732=== modified file 'src/test/java/com/persistit/TrackingFileChannel.java'
733--- src/test/java/com/persistit/TrackingFileChannel.java 2012-08-27 17:31:02 +0000
734+++ src/test/java/com/persistit/TrackingFileChannel.java 2012-09-11 21:38:18 +0000
735@@ -15,9 +15,8 @@
736
737 package com.persistit;
738
739-import static org.junit.Assert.assertEquals;
740 import static org.junit.Assert.assertTrue;
741-import static org.junit.Assert.fail;
742+
743 import java.io.IOException;
744 import java.nio.ByteBuffer;
745 import java.nio.MappedByteBuffer;
746@@ -161,9 +160,9 @@
747 return _readPositions;
748 }
749
750- public void assertOrdered(boolean read, boolean forward) {
751+ public void assertOrdered(final boolean read, final boolean forward) {
752 final List<Long> list = read ? _readPositions : _writePositions;
753- long previous = forward ? -1 : Long.MAX_VALUE;
754+ final long previous = forward ? -1 : Long.MAX_VALUE;
755 for (final Long position : list) {
756 if (forward) {
757 assertTrue("Position should be larger", position > previous);
758
759=== modified file 'src/test/java/com/persistit/stress/unit/CommitBench.java'
760--- src/test/java/com/persistit/stress/unit/CommitBench.java 2012-08-24 13:57:19 +0000
761+++ src/test/java/com/persistit/stress/unit/CommitBench.java 2012-09-11 21:38:18 +0000
762@@ -85,11 +85,11 @@
763 /*
764 * Damage the file so that there's no keystone checkpoint
765 */
766- RandomAccessFile raf = new RandomAccessFile(file, "rws");
767+ final RandomAccessFile raf = new RandomAccessFile(file, "rws");
768 raf.seek(0);
769 raf.write(new byte[256]);
770 raf.close();
771- } catch (IOException e) {
772+ } catch (final IOException e) {
773 throw new RuntimeException(e);
774 }
775 } else {

Subscribers

People subscribed via source and target branches