Merge lp:~pbeaman/akiban-persistit/better_4hour_tpcc_3 into lp:akiban-persistit

[Peter Beaman]

This branch solves several problems relating to Journal I/O, pruning and performance.
Specifically, when running the 4-hour TPCC test on a particular server, perf02, the "terminal" threads outrun the ability of the JOURNAL_COPIER thread to remove obsolete journal files. The result is an ever-growing collection of journal files that eventually overflow the disk. The distinguishing feature of perf02 is that it has 12 fast cores (x2 with hyper-threading) but just a single disk drive.

This branch keeps the number of journal files under control during the test (never exceeding 16) and allows the test to complete. The test arrives at a stable configuration after a couple of hours and there's no reason to believe it would fail if run for much longer periods of time. However, a longer test run has not yet been completed.

Major changes:

1) The Transaction#commit() method now includes a call to new method JournalManager#throttle() which imposes a voluntary delay if the JournalCopier#urgency() value becomes large. The delay adjusts upward when urgency becomes extreme.

2) The order in which commit() completes its actions has changed to:
   flush()
   notifyComplete();
   throttle();
   waitForDurability();

Further, durability is measured from a timestamp taken immediately after the call to flush().
It is safe to call notifyComplete() and allow other concurrent transactions to continue (thereby hopefully allowing more transactions to join a GROUP commit). It is safe because no other transaction released by notifyComplete() can commit and become durable unless this one is also durable.

3) Pruning is accelerated, preventing old aborted transactions from holding up the JOURNAL_COPIER cleanup process. There are two main changes to support faster pruning:

3a) Scheduling: the CLEANUP_MANAGER normally polls once per second, but when the queue becomes more than half full it now runs continuously until the queue is no longer half full.

3b) Prune on write: when a dirty page thats needs pruning is being written to disk, a new optional code path prunes it before writing it. This reduces the likelihood that a page that is read back in from disk will need to be pruned.
Other changes made while analyzing test runs include updates in the IOMeter log function, and small fixes in the CLI.

One source a many lines of diff output is that to enable a new test the PersistitUnitTestCase class was moved from com.persistit.unit to com.persistit. In a future proposal I intend to move all of the unit tests to the correct package (same package as the class they are testing) and get rid of TestShim.

This branch is the same as the previously proposed better_4hour_tpcc_2 plus changes suggested by review comments. That branch and its merge proposal were deleted because of persistent problems with the diff containing phantom conflicts (which were probably induced by a complicate history of pre-requisite changes). All of the branches on which this code previously depended have been merged, so this new proposal is simply a clean new merge against trunk.

For the record the following review comments were written for the former proposal and corresponding changes are incorporated in this proposal:

--------------------

LP still showing conflicts, but trying it locally works just fine. Maybe a resubmit with no pre-req? Either way, line references below are on the current version (with "conflicts").

The description makes sense and the change are consistent with it, I think. Delicate change area, but everything running successfully with this is good to hear.

Minor stuff:

Diff line 162, 180, and 888 has TODO about debugging. Maybe belongs in if(Debug.ENABLED)?

Diff line 967, can we put the unit in the variable name instead of the comment? Slightly clearer to me, but it's OK if not for you.

Diff line 1065, there is a helper in JounalManager for computing journal count, getJournalFileCount(). It returns 1 greater than here. Can/should we use that here to be consistent?

Diff line 1086, can we add a new variable for the DELAY / 4?

Diff line 1801, re-throw is commented out. Should it be removed or was that temporary?
PersistitUnitTestCase was deleted/added instead of being 'bzr mv'ed. A plugin that I've been using recently is http://doc.bazaar.canonical.com/plugins/en/automv-plugin.html, which will

[Nathan Williams (nwilliams)]

Thanks for the tweaks! Just a few more questions.

The MVV.verify() is now invoked inline to the call to Debug.$assert0.t(). assert0 is disabled by default so we expect it to get optimized away, but do we expect that to happy for verify() as well?

Diff line 420, the urgency call subtracts HALF_URGENT from journal file count. In the scenario where we go from 1-16 journal files, the urgency sequence would be: 1..5,0..10,10... right? Is that what we want?

It was like that in the last proposal, and I missed it, but the TransactionIndex polling was sped up by 5x. Is this required for passing on perf02? We are also adding throttling and pruning on write. Do we have any quantification for their individual contributions?

review: Needs Information

[Peter Beaman (pbeaman)]

Good questions.

To the best of my ability to tell from timing, Debug.$assert0.t(_anything_) gets optimized away with the Debug enabled flag is false. That is, unless _anything_ has a side-effect that could be used to detect that it has executed in which case of course the optimizer won't remove it. Therefore it's difficult to be certain.

Urgency calculation -- yes that looks wrong, will fix. I don't think the change will make much of a difference, but the intention is to provide a monotonically increasing function.

Yes, I increased the polling frequency to try to keep a tighter bound on the number of active transactions in the ATCache. The increase does not seem to have imposed any measurable increase in CPU consumption, and the benefit is that completed transactions can be recognized sooner as non-concurrent. I do not know if this one change is needed to help pass the test. What I have observed is that with the combination of changes, the CLEANUP_MANAGER now keeps up - with these changes I've never seen a refused CleanupManager.Action.

[Nathan Williams (nwilliams)]

Tweak looks good and other fixes responses make sense. So all good by me.

review: Approve

[Akiban Build User (build-akiban)]

There were 2 failures during build/test:

* job persistit-build failed at build number 413: http://172.16.20.104:8080/job/persistit-build/413/

* view must-pass failed: persistit-build is yellow

review: Needs Fixing

[Nathan Williams (nwilliams)]

Appears to be another instance of bug1019001 (intermittent failure). Going again.

[Timothy Wegner (timmwegner)]

Seems we should raise the priority of that bug? Or at the least
prioritize it :)

On Tue, Aug 28, 2012 at 5:31 PM, Nathan Williams <email address hidden>wrote:

> Appears to be another instance of bug1019001 (intermittent failure). Going
> again.
> --
>
> https://code.launchpad.net/~pbeaman/akiban-persistit/better_4hour_tpcc_3/+merge/121631
> Your team Akiban Technologies is subscribed to branch lp:akiban-persistit.
>

--
Akiban Technologies | Tim Wegner | VP of Engineering
skype: timmwegner | <email address hidden> | 508.735.6290

[Peter Beaman (pbeaman)]

Done. To High not Critical because so far we believe this is a test
problem no an underlying bug in the system. But we should look at it
soon to make sure that's correct.

On Tue, Aug 28, 2012 at 5:40 PM, Timothy Wegner <email address hidden> wrote:
> Seems we should raise the priority of that bug? Or at the least
> prioritize it :)
>
> On Tue, Aug 28, 2012 at 5:31 PM, Nathan Williams <email address hidden>wrote:
>
>> Appears to be another instance of bug1019001 (intermittent failure). Going
>> again.
>> --
>>
>> https://code.launchpad.net/~pbeaman/akiban-persistit/better_4hour_tpcc_3/+merge/121631
>> Your team Akiban Technologies is subscribed to branch lp:akiban-persistit.
>>
>
>
>
> --
> Akiban Technologies | Tim Wegner | VP of Engineering
> skype: timmwegner | <email address hidden> | 508.735.6290
>
> https://code.launchpad.net/~pbeaman/akiban-persistit/better_4hour_tpcc_3/+merge/121631
> Your team Akiban Technologies is subscribed to branch lp:akiban-persistit.

=== modified file 'src/main/java/com/persistit/Buffer.java'
--- src/main/java/com/persistit/Buffer.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/Buffer.java	2012-08-28 16:17:18 +0000
@@ -264,6 +264,9 @@
 
     private final static int BINARY_SEARCH_THRESHOLD = 6;
 
+    private final static int PRUNE_MVV_HELPER_CHANGED = 1;
+    private final static int PRUNE_MVV_HELPER_HAS_LONG = 2;
+
     abstract static class VerifyVisitor {
 
         protected void visitPage(final long timestamp, final Volume volume, final long page, final int type,
@@ -502,15 +505,23 @@
         Debug.$assert0.t(isMine());
         final long checkpointTimestamp = _persistit.getTimestampAllocator().getProposedCheckpointTimestamp();
         if (isDirty() && !isTemporary() && getTimestamp() < checkpointTimestamp && timestamp > checkpointTimestamp) {
-            writePage();
+            writePage(false);
             _pool.bumpForcedCheckpointWrites();
         }
     }
 
     void writePage() throws PersistitException {
+        writePage(_persistit.getJournalManager().isWritePagePruningEnabled());
+    }
+
+    private void writePage(final boolean prune) throws PersistitException {
+        assert isMine();
         _persistit.checkFatal();
         final Volume volume = getVolume();
         if (volume != null) {
+            if (prune) {
+                pruneMvvValues(null, false);
+            }
             clearSlack();
             save();
             _vol.getStorage().writePage(this);
@@ -1501,9 +1512,13 @@
                 Debug.$assert0.t(p + KEYBLOCK_LENGTH < _keyBlockEnd ? pointer > 0 : true);
                 putInt(newTail + TAILBLOCK_POINTER, pointer);
             } else {
-                final int storedLength = valueHelper.storeVersion(_bytes, newTail + _tailHeaderSize + klength, -1,
-                        _bytes.length); // TODO limit
-                incCountIfMvv(_bytes, newTail + _tailHeaderSize + klength, storedLength & MVV.STORE_LENGTH_MASK);
+                final int offset = newTail + _tailHeaderSize + klength;
+                final int storedLength = valueHelper.storeVersion(_bytes, offset, -1, _bytes.length); // TODO
+                // limit
+                incCountIfMvv(_bytes, offset, storedLength & MVV.STORE_LENGTH_MASK);
+
+                Debug.$assert0.t(MVV.verify(_persistit.getTransactionIndex(), _bytes, offset, storedLength
+                        & MVV.STORE_LENGTH_MASK));
             }
             //
             // Correct not to call getFastIndex()
@@ -1629,9 +1644,14 @@
             Debug.$assert0.t(p + KEYBLOCK_LENGTH < _keyBlockEnd ? pointer > 0 : pointer == -1);
             putInt(newTail + TAILBLOCK_POINTER, (int) pointer);
         } else {
-            final int storedLength = valueHelper.storeVersion(_bytes, newTail + _tailHeaderSize + klength, oldTailSize
-                    - _tailHeaderSize - klength, _bytes.length); // TODO - limit
-            isMVV = isValueMVV(_bytes, newTail + _tailHeaderSize + klength, storedLength & MVV.STORE_LENGTH_MASK);
+            final int offset = newTail + _tailHeaderSize + klength;
+            final int storedLength = valueHelper.storeVersion(_bytes, offset, oldTailSize - _tailHeaderSize - klength,
+                    _bytes.length); // TODO - limit
+            isMVV = isValueMVV(_bytes, offset, storedLength & MVV.STORE_LENGTH_MASK);
+
+            Debug.$assert0.t(MVV.verify(_persistit.getTransactionIndex(), _bytes, offset, storedLength
+                    & MVV.STORE_LENGTH_MASK));
+
         }
         if (!wasMVV && isMVV) {
             _mvvCount++;
@@ -3466,7 +3486,12 @@
                     if (visitor != null) {
                         visitor.visitDataRecord(key, p, tail, klength, offset, length, getBytes());
                     }
-                    Debug.$assert1.t(MVV.verify(_bytes, offset, length));
+
+                    if (!MVV.verify(_bytes, offset, length)) {
+                        throw new InvalidPageStructureException("invalid MVV record at offset/length=" + offset + "/"
+                                + length);
+                    }
+
                 }
 
                 if (_pool != null && getKeyCount() > _pool.getMaxKeys()) {
@@ -3546,63 +3571,15 @@
                 final long timestamp = _persistit.getTimestampAllocator().updateTimestamp();
                 _mvvCount = 0;
                 writePageOnCheckpoint(timestamp);
-                final List<PrunedVersion> prunedVersions = new ArrayList<PrunedVersion>();
-                for (int p = KEY_BLOCK_START; p < _keyBlockEnd; p += KEYBLOCK_LENGTH) {
-                    final int kbData = getInt(p);
-                    final int tail = decodeKeyBlockTail(kbData);
-                    final int tbData = getInt(tail);
-                    final int klength = decodeTailBlockKLength(tbData);
-                    final int oldTailSize = decodeTailBlockSize(tbData);
-                    final int offset = tail + _tailHeaderSize + klength;
-                    final int oldSize = oldTailSize - klength - _tailHeaderSize;
-
-                    if (oldSize > 0) {
-                        int valueByte = _bytes[offset] & 0xFF;
-                        if (isLongMVV(_bytes, offset, oldSize)) {
-                            /*
-                             * Can't prune in this pass because of long record
-                             * timestamp management. Simply remember that there
-                             * are long records to prune, then prune them in a
-                             * copy of the buffer.
-                             */
-                            hasLongMvvRecords = true;
-                        }
-                        if (valueByte == MVV.TYPE_MVV) {
-                            final int newSize = MVV.prune(_bytes, offset, oldSize, _persistit.getTransactionIndex(),
-                                    true, prunedVersions);
-                            if (newSize != oldSize) {
-                                changed = true;
-                                final int newTailSize = klength + newSize + _tailHeaderSize;
-                                final int oldNext = (tail + oldTailSize + ~TAILBLOCK_MASK) & TAILBLOCK_MASK;
-                                final int newNext = (tail + newTailSize + ~TAILBLOCK_MASK) & TAILBLOCK_MASK;
-                                if (newNext < oldNext) {
-                                    // Free the remainder of the old tail block
-                                    deallocTail(newNext, oldNext - newNext);
-                                } else {
-                                    Debug.$assert0.t(newNext == oldNext);
-                                }
-                                // Rewrite the tail block header
-                                putInt(tail, encodeTailBlock(newTailSize, klength));
-                                if (Debug.ENABLED) {
-                                    MVV.verify(_bytes, offset, newSize);
-                                }
-                            }
-                            valueByte = newSize > 0 ? _bytes[offset] & 0xFF : -1;
-                            incCountIfMvv(_bytes, offset, newSize);
-                        }
-
-                        if (pruneAntiValue(valueByte, p, tree)) {
-                            changed = true;
-                            p -= KEYBLOCK_LENGTH;
-                        }
-                    }
-                }
+                final int flags = pruneMvvValuesHelper(tree);
+                changed = (flags & PRUNE_MVV_HELPER_CHANGED) != 0;
+                hasLongMvvRecords = (flags & PRUNE_MVV_HELPER_HAS_LONG) != 0;
+
                 if (changed) {
                     setDirtyAtTimestamp(timestamp);
                 }
 
-                deallocatePrunedVersions(_persistit, _vol, prunedVersions);
-                prunedVersions.clear();
+                final List<PrunedVersion> prunedVersions = new ArrayList<PrunedVersion>();
 
                 if (pruneLongMVVs && hasLongMvvRecords) {
                     final List<PersistitException> deferredExceptions = new ArrayList<PersistitException>();
@@ -3645,6 +3622,63 @@
         return changed;
     }
 
+    private int pruneMvvValuesHelper(final Tree tree) throws PersistitException {
+        boolean changed = false;
+        boolean hasLongMvvRecords = false;
+        final List<PrunedVersion> prunedVersions = new ArrayList<PrunedVersion>();
+
+        for (int p = KEY_BLOCK_START; p < _keyBlockEnd; p += KEYBLOCK_LENGTH) {
+            final int kbData = getInt(p);
+            final int tail = decodeKeyBlockTail(kbData);
+            final int tbData = getInt(tail);
+            final int klength = decodeTailBlockKLength(tbData);
+            final int oldTailSize = decodeTailBlockSize(tbData);
+            final int offset = tail + _tailHeaderSize + klength;
+            final int oldSize = oldTailSize - klength - _tailHeaderSize;
+
+            if (oldSize > 0) {
+                int valueByte = _bytes[offset] & 0xFF;
+                if (isLongMVV(_bytes, offset, oldSize)) {
+                    /*
+                     * Can't prune in this pass because of long record timestamp
+                     * management. Simply remember that there are long records
+                     * to prune, then prune them in a copy of the buffer.
+                     */
+                    hasLongMvvRecords = true;
+                }
+                if (valueByte == MVV.TYPE_MVV) {
+                    final int newSize = MVV.prune(_bytes, offset, oldSize, _persistit.getTransactionIndex(), true,
+                            prunedVersions);
+                    if (newSize != oldSize) {
+                        changed = true;
+                        final int newTailSize = klength + newSize + _tailHeaderSize;
+                        final int oldNext = (tail + oldTailSize + ~TAILBLOCK_MASK) & TAILBLOCK_MASK;
+                        final int newNext = (tail + newTailSize + ~TAILBLOCK_MASK) & TAILBLOCK_MASK;
+                        if (newNext < oldNext) {
+                            // Free the remainder of the old tail block
+                            deallocTail(newNext, oldNext - newNext);
+                        } else {
+                            Debug.$assert0.t(newNext == oldNext);
+                        }
+                        // Rewrite the tail block header
+                        putInt(tail, encodeTailBlock(newTailSize, klength));
+                        Debug.$assert0.t(MVV.verify(_bytes, offset, newSize));
+                    }
+                    valueByte = newSize > 0 ? _bytes[offset] & 0xFF : -1;
+                    incCountIfMvv(_bytes, offset, newSize);
+                }
+
+                if (pruneAntiValue(valueByte, p, tree)) {
+                    changed = true;
+                    p -= KEYBLOCK_LENGTH;
+                }
+            }
+        }
+        deallocatePrunedVersions(_persistit, _vol, prunedVersions);
+        prunedVersions.clear();
+        return (changed ? PRUNE_MVV_HELPER_CHANGED : 0) | (hasLongMvvRecords ? PRUNE_MVV_HELPER_HAS_LONG : 0);
+    }
+
     /**
      * This method attempts to prune each LongRecord MVVs found in the page.
      * This method is performed on a _copy_ of the original Buffer being pruned;
@@ -3745,7 +3779,7 @@
         final byte[] rawBytes = value.getEncodedBytes();
         final int oldLongSize = value.getEncodedSize();
         // TODO - perhaps remove. Done as a precaution for now.
-        MVV.verify(rawBytes, 0, oldLongSize);
+        Debug.$assert0.t(MVV.verify(rawBytes, 0, oldLongSize));
         final List<PrunedVersion> provisionalPrunedVersions = new ArrayList<PrunedVersion>();
         final int newLongSize = MVV.prune(rawBytes, 0, oldLongSize, _persistit.getTransactionIndex(), true,
                 provisionalPrunedVersions);

=== modified file 'src/main/java/com/persistit/CLI.java'
--- src/main/java/com/persistit/CLI.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/CLI.java	2012-08-28 16:17:18 +0000
@@ -747,7 +747,7 @@
             }
         }
         if (sysvol != null) {
-            properties.put(Persistit.SYSTEM_VOLUME_PROPERTY, sysvol.getName());
+            persistit.getConfiguration().setSysVolume(sysvol.getName());
         }
 
         _persistit = persistit;

=== modified file 'src/main/java/com/persistit/CleanupManager.java'
--- src/main/java/com/persistit/CleanupManager.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/CleanupManager.java	2012-08-28 16:17:18 +0000
@@ -87,6 +87,7 @@
             _accepted.incrementAndGet();
         } else {
             _refused.incrementAndGet();
+            kick();
         }
         return accepted;
     }
@@ -127,6 +128,15 @@
     }
 
     @Override
+    public long getPollInterval() {
+        if (_cleanupActionQueue.size() < DEFAULT_QUEUE_SIZE / 2) {
+            return super.getPollInterval();
+        } else {
+            return 0;
+        }
+    }
+
+    @Override
     public void poll() throws Exception {
         _persistit.getIOMeter().poll();
         _persistit.cleanup();

=== modified file 'src/main/java/com/persistit/Exchange.java'
--- src/main/java/com/persistit/Exchange.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/Exchange.java	2012-08-28 16:17:18 +0000
@@ -1529,6 +1529,8 @@
                             storedLength &= MVV.STORE_LENGTH_MASK;
                             spareValue.setEncodedSize(storedLength);
 
+                            Debug.$assert0.t(MVV.verify(_persistit.getTransactionIndex(), spareBytes, 0, storedLength));
+
                             if (spareValue.getEncodedSize() > maxSimpleValueSize) {
                                 newLongRecordPointerMVV = getLongRecordHelper().storeLongRecord(spareValue,
                                         _transaction.isActive());
@@ -3698,6 +3700,8 @@
             buffer.nextKey(_spareKey2, buffer.toKeyBlock(0));
             _value.setPointerValue(page);
             _value.setPointerPageType(buffer.getPageType());
+            buffer.release();
+            buffer = null;
             storeInternal(_spareKey2, _value, level + 1, Exchange.StoreOptions.NONE);
             return true;
         } finally {

=== modified file 'src/main/java/com/persistit/IOMeter.java'
--- src/main/java/com/persistit/IOMeter.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/IOMeter.java	2012-08-28 16:17:18 +0000
@@ -61,15 +61,16 @@
 
     private final static int READ_PAGE_FROM_VOLUME = 1;
     private final static int READ_PAGE_FROM_JOURNAL = 2;
-    private final static int COPY_PAGE_TO_VOLUME = 3;
-    private final static int WRITE_PAGE_TO_JOURNAL = 4;
-    private final static int WRITE_TX_TO_JOURNAL = 5;
-    private final static int WRITE_OTHER_TO_JOURNAL = 6;
-    private final static int EVICT_PAGE_FROM_POOL = 7;
-    private final static int FLUSH_JOURNAL = 8;
-    private final static int GET_PAGE = 9;
+    private final static int COPY_PAGE_FROM_JOURNAL = 3;
+    private final static int COPY_PAGE_TO_VOLUME = 4;
+    private final static int WRITE_PAGE_TO_JOURNAL = 5;
+    private final static int WRITE_TX_TO_JOURNAL = 6;
+    private final static int WRITE_OTHER_TO_JOURNAL = 7;
+    private final static int EVICT_PAGE_FROM_POOL = 8;
+    private final static int FLUSH_JOURNAL = 9;
+    private final static int GET_PAGE = 10;
 
-    private final static int ITEM_COUNT = 10;
+    private final static int ITEM_COUNT = 11;
 
     private long _quiescentIOthreshold = DEFAULT_QUIESCENT_IO_THRESHOLD;
 
@@ -266,7 +267,7 @@
 
                 if (analyzePages
                         && (op == WRITE_PAGE_TO_JOURNAL || op == READ_PAGE_FROM_JOURNAL || op == READ_PAGE_FROM_VOLUME
-                                || op == COPY_PAGE_TO_VOLUME || op == EVICT_PAGE_FROM_POOL)) {
+                                || op == COPY_PAGE_FROM_JOURNAL || op == COPY_PAGE_TO_VOLUME || op == EVICT_PAGE_FROM_POOL)) {
                     final long handle = (volumeHandle << 48) + pageAddress;
                     List<Event> list = events.get(handle);
                     if (list == null) {
@@ -291,6 +292,12 @@
         }
     }
 
+    public void chargeCopyPageFromJournal(final Volume volume, final long pageAddress, final int size,
+            final long journalAddress, final int urgency) {
+        charge(size, COPY_PAGE_FROM_JOURNAL);
+        log(COPY_PAGE_FROM_JOURNAL, volume, pageAddress, size, journalAddress, 0);
+    }
+
     public void chargeCopyPageToVolume(final Volume volume, final long pageAddress, final int size,
             final long journalAddress, final int urgency) {
         charge(size, COPY_PAGE_TO_VOLUME);

=== modified file 'src/main/java/com/persistit/JournalManager.java'
--- src/main/java/com/persistit/JournalManager.java	2012-08-23 15:31:26 +0000
+++ src/main/java/com/persistit/JournalManager.java	2012-08-28 16:17:18 +0000
@@ -77,6 +77,8 @@
     final static int URGENT = 10;
     final static int ALMOST_URGENT = 8;
     final static int HALF_URGENT = 5;
+    final static int URGENT_COMMIT_DELAY_MILLIS = 50;
+    final static int GENTLE_COMMIT_DELAY_MILLIS = 12;
     private final static long NS_PER_MS = 1000000L;
     private final static int IO_MEASUREMENT_CYCLES = 8;
 
@@ -140,13 +142,13 @@
      * requires more space than is available in the current journal file, it
      * will advance to the start of the next journal file.
      */
-    private long _currentAddress;
+    private volatile long _currentAddress;
 
     /**
      * Smallest journal address at which a record still needed is located.
      * Initially zero, increases as journal files are consumed and deleted.
      */
-    private long _baseAddress;
+    private volatile long _baseAddress;
 
     private final Map<Long, FileChannel> _journalFileChannels = new HashMap<Long, FileChannel>();
 
@@ -191,6 +193,8 @@
 
     private final TransactionPlayerListener _listener = new ProactiveRollbackListener();
 
+    private final AtomicBoolean _writePagePruning = new AtomicBoolean(true);
+
     private final AtomicBoolean _rollbackPruning = new AtomicBoolean(true);
 
     /*
@@ -202,8 +206,6 @@
 
     private volatile int _copiesPerCycle = DEFAULT_COPIES_PER_CYCLE;
 
-    private volatile int _pageListSizeBase = DEFAULT_PAGE_MAP_SIZE_BASE;
-
     private volatile long _copierTimestampLimit = Long.MAX_VALUE;
 
     private volatile long _earliestCommittedTimestamp = Long.MAX_VALUE;
@@ -423,6 +425,11 @@
         _rollbackPruning.set(rollbackPruning);
     }
 
+    @Override
+    public void setWritePagePruningEnabled(final boolean writePruning) {
+        _writePagePruning.set(writePruning);
+    }
+
     public JournalManager(final Persistit persistit) {
         _persistit = persistit;
     }
@@ -443,6 +450,11 @@
     }
 
     @Override
+    public boolean isWritePagePruningEnabled() {
+        return _writePagePruning.get();
+    }
+
+    @Override
     public String getJournalFilePath() {
         return _journalFilePath;
     }
@@ -539,16 +551,31 @@
      * @return the JOURNAL_COPIER urgency on a scale of 0 to 10
      */
     @Override
-    public synchronized int urgency() {
+    public int urgency() {
         if (_copyFast.get()) {
             return URGENT;
         }
-        int urgency = _pageList.size() / _pageListSizeBase;
-        final int journalFileCount = (int) (_currentAddress / _blockSize - _baseAddress / _blockSize);
-        if (!_appendOnly.get() && journalFileCount > 1) {
-            urgency += journalFileCount - 1;
+        final int journalFileCount = getJournalFileCount();
+        return Math.min(URGENT, journalFileCount);
+    }
+
+    /**
+     * Introduce delay into an application thread when JOURNAL_COPIER thread is
+     * behind. The amount of delay depends on the value returned by
+     * {@link #urgency()}. When that value is {@value #URGENT} then the delay is
+     * {@value #URGENT_COMMIT_DELAY_MILLIS} milliseconds.
+     * 
+     * @throws PersistitInterruptedException
+     */
+    public void throttle() throws PersistitInterruptedException {
+        final int urgency = urgency();
+        if (!_appendOnly.get()) {
+            if (urgency == URGENT) {
+                Util.sleep(URGENT_COMMIT_DELAY_MILLIS);
+            } else if (urgency >= ALMOST_URGENT) {
+                Util.sleep(GENTLE_COMMIT_DELAY_MILLIS);
+            }
         }
-        return Math.min(urgency, URGENT);
     }
 
     int handleForVolume(final Volume volume) throws PersistitException {
@@ -981,6 +1008,14 @@
         //
         force();
         //
+        // Make sure all copied pages have been flushed to disk.
+        //
+        for (final Volume vol : _volumeToHandleMap.keySet()) {
+            if (vol.isOpened()) {
+                vol.getStorage().force();
+            }
+        }
+        //
         // Prepare room for CP.OVERHEAD bytes in the journal. If doing so
         // started a new journal file then there's no need to write another
         // CP record.
@@ -1743,6 +1778,9 @@
      * three commit modes: SOFT, HARD and GROUP. The two parameters represent
      * time intervals in milliseconds.
      * 
+     * @param flushedTimestamp
+     *            a timestamp taken after the transaction buffer belonging to
+     *            the current transaction has been flushed.
      * @param leadTime
      *            time interval in milliseconds by which to anticipate I/O
      *            completion; the method will return as soon as the I/O
@@ -1757,10 +1795,11 @@
      * @throws PersistitInterruptedException
      */
 
-    void waitForDurability(final long leadTime, final long stallTime) throws PersistitException {
+    void waitForDurability(final long flushedTimestamp, final long leadTime, final long stallTime)
+            throws PersistitException {
         final JournalFlusher flusher = _flusher;
         if (flusher != null) {
-            flusher.waitForDurability(leadTime, stallTime);
+            flusher.waitForDurability(flushedTimestamp, leadTime, stallTime);
         } else {
             throw new IllegalStateException("JOURNAL_FLUSHER is not running");
         }
@@ -2161,7 +2200,7 @@
             int divisor = 1;
 
             if (iom.recentCharge() < iom.getQuiescentIOthreshold()) {
-                divisor = URGENT - HALF_URGENT;
+                divisor = HALF_URGENT;
             } else if (urgency > HALF_URGENT) {
                 divisor = urgency - HALF_URGENT;
             }
@@ -2199,16 +2238,16 @@
 
         /**
          * General method used to wait for durability. {@See
-         * JournalManager#waitForDurability(long, long)}.
+         * JournalManager#waitForDurability(long, long, long)}.
          * 
          * @throws PersistitInterruptedException
          */
-        private void waitForDurability(final long leadTime, final long stallTime) throws PersistitException {
+        private void waitForDurability(final long flushedTimestamp, final long leadTime, final long stallTime)
+                throws PersistitException {
             /*
              * Commit is known durable once the JOURNAL_FLUSHER thread has
-             * posted a timestamp value to _endTimestamp larger than this.
+             * posted an _endTimestamp larger than flushedTimestamp.
              */
-            final long timestamp = _persistit.getTimestampAllocator().getCurrentTimestamp();
             final long now = System.nanoTime();
 
             while (true) {
@@ -2231,7 +2270,7 @@
                     endTimestamp = _endTimestamp;
                     startTime = _startTime;
                     endTime = _endTime;
-                    if (timestamp > startTimestamp && startTimestamp > endTimestamp) {
+                    if (flushedTimestamp > startTimestamp && startTimestamp > endTimestamp) {
                         estimatedRemainingIoNanos = Math.max(startTime + _expectedIoTime - now, 0);
                     }
                     if (startTimestamp == _startTimestamp && endTimestamp == _endTimestamp) {
@@ -2240,7 +2279,7 @@
                     Util.spinSleep();
                 }
 
-                if (endTimestamp > timestamp && startTimestamp > timestamp) {
+                if (endTimestamp > flushedTimestamp && startTimestamp > flushedTimestamp) {
                     /*
                      * Done - commit is fully durable
                      */
@@ -2256,7 +2295,7 @@
                 }
 
                 long estimatedNanosToFinish = Math.max(estimatedRemainingIoNanos, 0);
-                if (startTimestamp < timestamp) {
+                if (startTimestamp < flushedTimestamp) {
                     estimatedNanosToFinish += remainingSleepNanos + _expectedIoTime;
                 }
 
@@ -2285,7 +2324,7 @@
                             if (_lock.readLock().tryLock(NS_PER_MS, TimeUnit.NANOSECONDS)) {
                                 _lock.readLock().unlock();
                             }
-                        } catch (InterruptedException e) {
+                        } catch (final InterruptedException e) {
                             throw new PersistitInterruptedException(e);
                         }
                     }
@@ -2310,7 +2349,7 @@
                     _waitLoopsWithNoDelay.incrementAndGet();
                 }
             }
-            if (_lastExceptionTimestamp > timestamp) {
+            if (_lastExceptionTimestamp > flushedTimestamp) {
                 final Exception e = _lastException;
                 if (e instanceof PersistitException) {
                     throw (PersistitException) e;
@@ -2451,6 +2490,8 @@
                     continue;
                 }
                 pageAddress = readPageBufferFromJournal(stablePageNode, bb);
+                _persistit.getIOMeter().chargeCopyPageFromJournal(volumeRef, pageAddress, volume.getPageSize(),
+                        stablePageNode.getJournalAddress(), urgency());
             } catch (final PersistitException ioe) {
                 _persistit
                         .getAlertMonitor()
@@ -2478,8 +2519,6 @@
         Volume volume = null;
         int handle = -1;
 
-        final HashSet<Volume> volumes = new HashSet<Volume>();
-
         for (final Iterator<PageNode> iterator = list.iterator(); iterator.hasNext();) {
             if (_closed.get() && !_copyFast.get() || _appendOnly.get()) {
                 list.clear();
@@ -2535,15 +2574,11 @@
                 throw ioe;
             }
 
-            volumes.add(volume);
             _copiedPageCount++;
             _persistit.getIOMeter().chargeCopyPageToVolume(volume, pageAddress, volume.getPageSize(),
-                    pageNode.getJournalAddress(), _copyFast.get() ? URGENT : urgency());
+                    pageNode.getJournalAddress(), urgency());
         }
 
-        for (final Volume vol : volumes) {
-            vol.getStorage().force();
-        }
     }
 
     private void cleanupForCopy(final List<PageNode> list) throws PersistitException {
@@ -2590,6 +2625,7 @@
             // Detect first journal address holding a mapped page
             // required for recovery
             //
+
             for (final PageNode pageNode : _pageMap.values()) {
                 //
                 // If there are multiple versions, we need to keep
@@ -2607,9 +2643,9 @@
             //
             for (final Iterator<TransactionMapItem> iterator = _liveTransactionMap.values().iterator(); iterator
                     .hasNext();) {
-                final TransactionMapItem ts = iterator.next();
-                if (ts.getStartAddress() < recoveryBoundary) {
-                    recoveryBoundary = ts.getStartAddress();
+                final TransactionMapItem item = iterator.next();
+                if (item.getStartAddress() < recoveryBoundary) {
+                    recoveryBoundary = item.getStartAddress();
                 }
             }
 
@@ -2772,7 +2808,6 @@
 
         @Override
         public void store(final long address, final long timestamp, final Exchange exchange) throws PersistitException {
-            final TransactionStatus ts = _persistit.getTransactionIndex().getStatus(timestamp);
             exchange.prune();
         }
 
@@ -2840,6 +2875,7 @@
     /**
      * Extend ArrayList to export the removeRange method.
      */
+    @SuppressWarnings("serial")
     static class RangeRemovingArrayList<T> extends ArrayList<T> {
         @Override
         public void removeRange(final int fromIndex, final int toIndex) {

=== added file 'src/main/java/com/persistit/JournalManagerBench.java'
--- src/main/java/com/persistit/JournalManagerBench.java	1970-01-01 00:00:00 +0000
+++ src/main/java/com/persistit/JournalManagerBench.java	2012-08-28 16:17:18 +0000
@@ -0,0 +1,186 @@
+/**
+ * Copyright © 2012 Akiban Technologies, Inc.  All rights reserved.
+ * 
+ * This program and the accompanying materials are made available
+ * under the terms of the Eclipse Public License v1.0 which
+ * accompanies this distribution, and is available at
+ * http://www.eclipse.org/legal/epl-v10.html
+ * 
+ * This program may also be available under different license terms.
+ * For more information, see www.akiban.com or contact [e-mail].
+ * 
+ * Contributors:
+ * Akiban Technologies, Inc.
+ */
+
+package com.persistit;
+
+import java.io.File;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
+import com.persistit.util.ArgParser;
+
+/**
+ * Benchmark for primitive I/O simulating HARD (durable) commit. This code is
+ * intended to explore two elements:
+ * 
+ * (1) Pre-extending the journal file (so that FileChannel.force(false) usually
+ * does not need to write any metadata.
+ * 
+ * (2) Performing I/O in fixed-length blocks so that to write a some bytes the
+ * file system does not first need to read existing data from disk.
+ * 
+ * Parameters
+ * 
+ * align - smallest unit of I/O (default = 1) datapath - directory in which fake
+ * journal file will be written (default = /tmp/persistit_test_data) buffersize
+ * - emulated journal buffer size (default = 64M)
+ * 
+ * 
+ * @author peter
+ * 
+ */
+public class JournalManagerBench {
+
+    private final long NS_PER_S = 1000000000L;
+    private final byte[] NULLS = new byte[65536];
+
+    private final String[] ARG_TEMPLATE = new String[] { "duration|int:10:10:86400|Duration of test in seconds",
+            "policy|String:HARD|Commit policy: SOFT, HARD or GROUP",
+            "datapath|String:/tmp/persistit_test_data|Datapath property",
+            "buffersize|int:64:1:1024|Emulated journal buffer size in MBytes",
+            "extension|int:0:0:1024|MBytes by which to extend file when full",
+            "prealloc|int:0:0:1024|Preallocated file size in MBytes",
+            "align|int:1:1:65536|Blocking factor for I/O size",
+            "recsize|int:123:64:65536|Emulated transaction record size" };
+
+    final ByteBuffer buffer;
+    final ArgParser ap;
+
+    private File file;
+    private FileChannel fc;
+
+    private long writeAddress = 0;
+    private long currentAddress = 0;
+
+    long count = 0;
+    long minTime = Long.MAX_VALUE;
+    long maxTime = Long.MIN_VALUE;
+
+    final byte[] bytes = new byte[65536];
+
+    public static void main(final String[] args) throws Exception {
+        final JournalManagerBench jmb = new JournalManagerBench(args);
+        jmb.runTest();
+    }
+
+    private JournalManagerBench(final String[] args) throws Exception {
+        ap = new ArgParser("JournalManagerBench", args, ARG_TEMPLATE).strict();
+        buffer = ByteBuffer.allocate(ap.getIntValue("buffersize") * 1024 * 1024);
+    }
+
+    @SuppressWarnings("resource")
+    private void runTest() throws Exception {
+        file = new File(ap.getStringValue("datapath"), "JManBench_TestFile");
+        fc = new RandomAccessFile(file, "rw").getChannel();
+        preallocateFile(ap.getIntValue("prealloc") * 1024 * 1024);
+        for (int i = 0; i < bytes.length; i++) {
+            bytes[i] = (byte) ('-');
+        }
+        final int align = ap.getIntValue("align");
+        final long extension = ap.getIntValue("extension") * 1024 * 1024;
+
+        final long start = System.nanoTime();
+        final long expires = start + ap.getIntValue("duration") * NS_PER_S;
+        long now = System.nanoTime();
+
+        while (now < expires) {
+            doOneCycle(now - start, align, extension, 100);
+            final long then = System.nanoTime();
+            count++;
+            minTime = Math.min(minTime, then - now);
+            maxTime = Math.max(maxTime, then - now);
+            now = then;
+        }
+        final long elapsed = now - start;
+        System.out.printf("%,d commits took %,dms at a rate of %,d/second minimum time=%,dns maximumTime=%,dns\n",
+                count, elapsed, (count * NS_PER_S) / elapsed, minTime, maxTime);
+
+    }
+
+    private void preallocateFile(final long size) throws Exception {
+        if (size > 0 && fc.size() > size) {
+            System.out.printf("Truncating file %s from %,d to %,d\n", file, fc.size(), size);
+            fc.truncate(size);
+        } else if (fc.size() < size) {
+            System.out.printf("Preallocating file %s to size %,d ", file, size);
+            while (true) {
+                long remaining = size - fc.size();
+                if (remaining <= 0) {
+                    break;
+                }
+                if (remaining > buffer.capacity()) {
+                    remaining = buffer.capacity();
+                    long unaligned = fc.size() % 16384;
+                    if (unaligned > 0) {
+                        remaining = remaining - (16384 - unaligned);
+                    }
+                }
+                buffer.position(0).limit((int) remaining);
+                fc.write(buffer, fc.size());
+                System.out.print(".");
+            }
+            fc.force(true);
+            System.out.println("done");
+        }
+    }
+
+    private void doOneCycle(final long time, final int align, final long extension, final int size) throws Exception {
+        // Make a fake transaction record
+        String header = String.format("\nsize=%06d count=%06d time=%012d\n", size, count, time);
+        byte[] b = header.getBytes();
+        System.arraycopy(b, 0, bytes, 0, b.length);
+
+        // Add the record, possibly offset to maintaining alignment
+        int toRewrite = (int) (currentAddress - writeAddress);
+        buffer.position(toRewrite);
+        buffer.put(bytes, 0, size);
+        boolean extended = false;
+
+        int position = buffer.position();
+
+        // If extension is needed, add those bytes
+        long currentSize;
+        if (extension > 0 && writeAddress + buffer.position() > (currentSize = fc.size())) {
+            long newSize = currentSize + extension;
+            if (newSize - writeAddress > buffer.capacity()) {
+                newSize = writeAddress + buffer.capacity();
+                assert newSize > currentSize;
+            }
+            int add = (int) (newSize - writeAddress - buffer.position());
+            while (add > 0) {
+                buffer.put(NULLS, 0, Math.min(NULLS.length, add));
+                add -= NULLS.length;
+            }
+            extended = true;
+        }
+
+        // Write and force the buffer
+
+        buffer.flip();
+        fc.write(buffer, writeAddress);
+        fc.force(extended);
+
+        // Align the bytes to the beginning of the buffer as needed
+
+        currentAddress = writeAddress + position;
+        buffer.limit(position);
+        position = (position / align) * align;
+        buffer.position(position);
+        buffer.compact();
+        writeAddress += position;
+    }
+
+}

=== modified file 'src/main/java/com/persistit/JournalTool.java'
--- src/main/java/com/persistit/JournalTool.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/JournalTool.java	2012-08-28 16:17:18 +0000
@@ -688,7 +688,10 @@
         }
 
         protected void flush() {
-            write(sb.toString());
+            if (sb.length() > 0) {
+                write(sb.toString());
+                sb.setLength(0);
+            }
         }
 
         protected void write(final String msg) {
@@ -908,7 +911,6 @@
             long address = from + PM.OVERHEAD;
             int index = 0;
             int loaded = 0;
-            sb.setLength(0);
             long lastPage = Long.MAX_VALUE;
             int lastVolumeHandle = Integer.MAX_VALUE;
             for (int remaining = count; remaining > 0; remaining--) {
@@ -930,10 +932,7 @@
                 final long journalAddress = PM.getEntryJournalAddress(_readBuffer, index);
                 if (_selectedPages.isSelected(pageAddress) && _selectedTimestamps.isSelected(pageTimestamp)) {
                     if (pageAddress != lastPage || volumeHandle != lastVolumeHandle) {
-                        if (sb.length() > 0) {
-                            flush();
-                            sb.setLength(0);
-                        }
+                        flush();
                         lastPage = pageAddress;
                         lastVolumeHandle = volumeHandle;
                         appendf("-- %5d:%,12d: ", volumeHandle, pageAddress);
@@ -953,7 +952,6 @@
             long address = from + TM.OVERHEAD;
             int index = 0;
             int loaded = 0;
-            sb.setLength(0);
             for (int remaining = count; remaining > 0; remaining--) {
                 if (index == loaded) {
                     read(address, Math.min(_readBuffer.capacity(), remaining * TM.ENTRY_SIZE));
@@ -972,7 +970,6 @@
                 appendf("--  start %,12d  commit %,12d  @%,18d %s", startTimestamp, commitTimestamp, journalAddress,
                         isCommitted ? "committed" : "uncommitted");
                 flush();
-                sb.setLength(0);
                 index++;
             }
         }

=== modified file 'src/main/java/com/persistit/MVV.java'
--- src/main/java/com/persistit/MVV.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/MVV.java	2012-08-28 16:17:18 +0000
@@ -19,6 +19,7 @@
 import static com.persistit.Buffer.LONGREC_SIZE;
 import static com.persistit.Buffer.LONGREC_TYPE;
 import static com.persistit.TransactionIndex.vh2ts;
+import static com.persistit.TransactionStatus.ABORTED;
 import static com.persistit.TransactionStatus.UNCOMMITTED;
 
 import java.util.List;
@@ -307,7 +308,7 @@
          * simply replaced.
          */
         else {
-            Debug.$assert0.t(verify(target, targetOffset, targetLength));
+            assert verify(target, targetOffset, targetLength);
             /*
              * Search for the matching version.
              */
@@ -600,6 +601,37 @@
         return true;
     }
 
+    static boolean verify(final TransactionIndex ti, final byte[] bytes, final int offset, final int length) {
+        if (!isArrayMVV(bytes, offset, length)) {
+            /*
+             * Not an MVV
+             */
+            return true;
+        }
+        int from = offset + 1;
+        final long lastVersion = -1;
+        while (from < offset + length) {
+            final int vlength = getLength(bytes, from);
+            final long version = getVersion(bytes, from);
+            if (vlength < 0 || from + vlength + LENGTH_PER_VERSION > offset + length) {
+                return false;
+            }
+            if (version < lastVersion) {
+                try {
+                    final long lastVersionTc = ti.commitStatus(lastVersion, UNCOMMITTED, 0);
+                    assert lastVersionTc == ABORTED;
+                } catch (final InterruptedException e) {
+                    // ignore
+                } catch (final TimeoutException e) {
+                    // ignore
+                }
+            }
+            assert version != lastVersion;
+            from += vlength + LENGTH_PER_VERSION;
+        }
+        return true;
+    }
+
     /**
      * Search for a known version within a MVV array. If the version is found
      * within the array, copy the contents out to the target and return the

=== modified file 'src/main/java/com/persistit/Transaction.java'
--- src/main/java/com/persistit/Transaction.java	2012-08-23 15:24:42 +0000
+++ src/main/java/com/persistit/Transaction.java	2012-08-28 16:17:18 +0000
@@ -854,21 +854,27 @@
             sequence(COMMIT_FLUSH_A);
             _commitTimestamp = _persistit.getTimestampAllocator().updateTimestamp();
             sequence(COMMIT_FLUSH_C);
+            long flushedTimetimestamp = 0;
             boolean committed = false;
             try {
+
                 if (flushTransactionBuffer(false)) {
-                    _persistit.getJournalManager().waitForDurability(
-                            policy == CommitPolicy.SOFT ? _persistit.getTransactionCommitLeadTime() : 0,
-                            policy == CommitPolicy.GROUP ? _persistit.getTransactionCommitStallTime() : 0);
+                    flushedTimetimestamp = _persistit.getTimestampAllocator().getCurrentTimestamp();
                 }
                 committed = true;
             } finally {
-
                 _persistit.getTransactionIndex().notifyCompleted(_transactionStatus,
                         committed ? _commitTimestamp : TransactionStatus.ABORTED);
                 _commitCompleted = committed;
                 _rollbackPending = _rollbackCompleted = !committed;
             }
+
+            _persistit.getJournalManager().throttle();
+            if (flushedTimetimestamp != 0) {
+                _persistit.getJournalManager().waitForDurability(flushedTimetimestamp,
+                        policy == CommitPolicy.SOFT ? _persistit.getTransactionCommitLeadTime() : 0,
+                        policy == CommitPolicy.GROUP ? _persistit.getTransactionCommitStallTime() : 0);
+            }
         }
     }
 

=== modified file 'src/main/java/com/persistit/TransactionIndex.java'
--- src/main/java/com/persistit/TransactionIndex.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/TransactionIndex.java	2012-08-28 16:17:18 +0000
@@ -50,7 +50,7 @@
     /**
      * Interval in milliseconds for updating the active transaction cache
      */
-    final static long POLLING_TASK_INTERVAL = 50;
+    final static long POLLING_TASK_INTERVAL = 10;
 
     /**
      * Default threshold value for moving long-running transactions to the

=== modified file 'src/main/java/com/persistit/mxbeans/IOMeterMXBean.java'
--- src/main/java/com/persistit/mxbeans/IOMeterMXBean.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/mxbeans/IOMeterMXBean.java	2012-08-28 16:17:18 +0000
@@ -59,12 +59,12 @@
      * 
      */
     public final static String[] OPERATION_NAMES = { "Unknown", "Read page from Volume", "Read page from Journal",
-            "Copy page from journal to volume", "Write page from Journal", "Write Transaction to Journal", "Other",
-            "Evict page from pool", "Flush journal", "Get page" };
-
-    public final static String[] OPERATIONS = { "??", "RV", "RJ", "CC", "WJ", "TJ", "XX", "EV", "FJ", "GP" };
-
-    public final static String[] SUMMARY_ITEMS = { "CC", "RV", "RJ", "WJ", "EV", "FJ" };
+            "Copy page from journal", "Copy page to volume", "Write page from Journal", "Write Transaction to Journal",
+            "Other", "Evict page from pool", "Flush journal", "Get page" };
+
+    public final static String[] OPERATIONS = { "??", "RV", "RJ", "CJ", "CV", "WJ", "TJ", "XX", "EV", "FJ", "GP" };
+
+    public final static String[] SUMMARY_ITEMS = { "CJ", "CV", "RV", "RJ", "WJ", "EV", "FJ" };
 
     /**
      * @return the quiescentIOthreshold

=== modified file 'src/main/java/com/persistit/mxbeans/JournalManagerMXBean.java'
--- src/main/java/com/persistit/mxbeans/JournalManagerMXBean.java	2012-08-15 16:11:36 +0000
+++ src/main/java/com/persistit/mxbeans/JournalManagerMXBean.java	2012-08-28 16:17:18 +0000
@@ -207,6 +207,12 @@
     @Description("True to enable pruning of rolled-back transactions")
     boolean isRollbackPruningEnabled();
 
+    @Description("True to enable pruning when writing pages to journal")
+    void setWritePagePruningEnabled(boolean rollbackPruning);
+
+    @Description("True to enable pruning when writing pages to journal")
+    boolean isWritePagePruningEnabled();
+
     @Description("Degree of urgency for copying pages: 0-10")
     int urgency();
 

=== modified file 'src/test/java/com/persistit/AccumulatorMemoryTest.java'
--- src/test/java/com/persistit/AccumulatorMemoryTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/AccumulatorMemoryTest.java	2012-08-28 16:17:18 +0000
@@ -21,8 +21,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class AccumulatorMemoryTest extends PersistitUnitTestCase {
 
     @Test

=== modified file 'src/test/java/com/persistit/AccumulatorRecoveryTest.java'
--- src/test/java/com/persistit/AccumulatorRecoveryTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/AccumulatorRecoveryTest.java	2012-08-28 16:17:18 +0000
@@ -31,7 +31,6 @@
 import com.persistit.TransactionPlayer.TransactionPlayerListener;
 import com.persistit.exception.PersistitException;
 import com.persistit.exception.RollbackException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class AccumulatorRecoveryTest extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/AccumulatorTest.java'
--- src/test/java/com/persistit/AccumulatorTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/AccumulatorTest.java	2012-08-28 16:17:18 +0000
@@ -33,7 +33,6 @@
 import com.persistit.exception.PersistitException;
 import com.persistit.exception.TimeoutException;
 import com.persistit.unit.ConcurrentUtil;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class AccumulatorTest extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/AlertMonitorTest.java'
--- src/test/java/com/persistit/AlertMonitorTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/AlertMonitorTest.java	2012-08-28 16:17:18 +0000
@@ -31,7 +31,6 @@
 import com.persistit.logging.PersistitLevel;
 import com.persistit.logging.PersistitLogMessage.LogItem;
 import com.persistit.logging.PersistitLogger;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class AlertMonitorTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/BackupTaskTest.java'
--- src/test/java/com/persistit/BackupTaskTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/BackupTaskTest.java	2012-08-28 16:17:18 +0000
@@ -29,7 +29,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class BackupTaskTest extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/BufferMaxPack.java'
--- src/test/java/com/persistit/BufferMaxPack.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/BufferMaxPack.java	2012-08-28 16:17:18 +0000
@@ -20,7 +20,6 @@
 import org.junit.Test;
 
 import com.persistit.ValueHelper.RawValueWriter;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class BufferMaxPack extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/BufferPoolTest.java'
--- src/test/java/com/persistit/BufferPoolTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/BufferPoolTest.java	2012-08-28 16:17:18 +0000
@@ -26,7 +26,6 @@
 import org.junit.Test;
 
 import com.persistit.BufferPool.BufferHolder;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class BufferPoolTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/BufferTest.java'
--- src/test/java/com/persistit/BufferTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/BufferTest.java	2012-08-28 16:17:18 +0000
@@ -27,7 +27,6 @@
 import com.persistit.exception.RebalanceException;
 import com.persistit.policy.JoinPolicy;
 import com.persistit.policy.SplitPolicy;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class BufferTest extends PersistitUnitTestCase {
     int leftn;

=== modified file 'src/test/java/com/persistit/BufferTest2.java'
--- src/test/java/com/persistit/BufferTest2.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/BufferTest2.java	2012-08-28 16:17:18 +0000
@@ -33,7 +33,6 @@
 import com.persistit.exception.RebalanceException;
 import com.persistit.policy.JoinPolicy;
 import com.persistit.policy.SplitPolicy;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class BufferTest2 extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/Bug1003578Test.java'
--- src/test/java/com/persistit/Bug1003578Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug1003578Test.java	2012-08-28 16:17:18 +0000
@@ -27,7 +27,6 @@
 import org.junit.Test;
 
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 /**
  * https://bugs.launchpad.net/akiban-persistit/+bug/1003578

=== modified file 'src/test/java/com/persistit/Bug1017957Test.java'
--- src/test/java/com/persistit/Bug1017957Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug1017957Test.java	2012-08-28 16:17:18 +0000
@@ -23,7 +23,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 /**

=== modified file 'src/test/java/com/persistit/Bug1018526Test.java'
--- src/test/java/com/persistit/Bug1018526Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug1018526Test.java	2012-08-28 16:17:18 +0000
@@ -27,7 +27,6 @@
 import org.junit.Test;
 
 import com.persistit.JournalManager.TreeDescriptor;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class Bug1018526Test extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/Bug706132Test.java'
--- src/test/java/com/persistit/Bug706132Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug706132Test.java	2012-08-28 16:17:18 +0000
@@ -22,7 +22,6 @@
 import org.junit.Test;
 
 import com.persistit.policy.SplitPolicy;
-import com.persistit.unit.PersistitUnitTestCase;
 
 /*
  * Stress10 with 1K pages exhibited a failure in which a value was simply not

=== modified file 'src/test/java/com/persistit/Bug708592Test.java'
--- src/test/java/com/persistit/Bug708592Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug708592Test.java	2012-08-28 16:17:18 +0000
@@ -21,7 +21,6 @@
 import org.junit.Test;
 
 import com.persistit.policy.SplitPolicy;
-import com.persistit.unit.PersistitUnitTestCase;
 
 /*
  * Got this while loading sample data. Apparently the new PACK split policy 

=== modified file 'src/test/java/com/persistit/Bug739533Test.java'
--- src/test/java/com/persistit/Bug739533Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug739533Test.java	2012-08-28 16:17:18 +0000
@@ -17,8 +17,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class Bug739533Test extends PersistitUnitTestCase {
 
     private final String _volumeName = "persistit";

=== modified file 'src/test/java/com/persistit/Bug777918Test.java'
--- src/test/java/com/persistit/Bug777918Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug777918Test.java	2012-08-28 16:17:18 +0000
@@ -25,7 +25,6 @@
 import com.persistit.exception.MissingThreadException;
 import com.persistit.exception.PersistitException;
 import com.persistit.exception.TestException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 import com.persistit.util.Util;
 

=== modified file 'src/test/java/com/persistit/Bug790709Test.java'
--- src/test/java/com/persistit/Bug790709Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug790709Test.java	2012-08-28 16:17:18 +0000
@@ -20,8 +20,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 /**
  * Bug 790709 This happened on-site at XXX with the halo release. The server
  * process was running with assertions enabled. The DELETE statement that failed

=== modified file 'src/test/java/com/persistit/Bug870352Test.java'
--- src/test/java/com/persistit/Bug870352Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug870352Test.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class Bug870352Test extends PersistitUnitTestCase {
 
     @Test

=== modified file 'src/test/java/com/persistit/Bug877656Test.java'
--- src/test/java/com/persistit/Bug877656Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug877656Test.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class Bug877656Test extends PersistitUnitTestCase {
 
     @Test

=== modified file 'src/test/java/com/persistit/Bug882219Test.java'
--- src/test/java/com/persistit/Bug882219Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug882219Test.java	2012-08-28 16:17:18 +0000
@@ -26,7 +26,6 @@
 import com.persistit.Transaction.CommitPolicy;
 import com.persistit.exception.PersistitIOException;
 import com.persistit.exception.PersistitInterruptedException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 /**
  * Interrupting a thread while it is performing a read or write closes the

=== modified file 'src/test/java/com/persistit/Bug885477Test.java'
--- src/test/java/com/persistit/Bug885477Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug885477Test.java	2012-08-28 16:17:18 +0000
@@ -21,8 +21,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 /**
  * Suppose a persistit tree has keys (1, 10), (1, 20), (2, 30). If an exchange's
  * key is (1), then traverse(GTEQ, false) should traverse to the first record,

=== modified file 'src/test/java/com/persistit/Bug889850Test.java'
--- src/test/java/com/persistit/Bug889850Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug889850Test.java	2012-08-28 16:17:18 +0000
@@ -17,8 +17,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 /**
  * 
  * From akiban-server 0.81:

=== modified file 'src/test/java/com/persistit/Bug911849Test.java'
--- src/test/java/com/persistit/Bug911849Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug911849Test.java	2012-08-28 16:17:18 +0000
@@ -25,7 +25,6 @@
 import org.junit.Test;
 
 import com.persistit.exception.RollbackException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 /**
  * TableStatusRecoveryIT and RenameTableIT fail intermittently

=== modified file 'src/test/java/com/persistit/Bug912514Test.java'
--- src/test/java/com/persistit/Bug912514Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug912514Test.java	2012-08-28 16:17:18 +0000
@@ -20,8 +20,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 /**
  * https://bugs.launchpad.net/akiban-persistit/+bug/912514
  * 

=== modified file 'src/test/java/com/persistit/Bug915594Test.java'
--- src/test/java/com/persistit/Bug915594Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug915594Test.java	2012-08-28 16:17:18 +0000
@@ -21,8 +21,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 /**
  * I did an update that failed due to constraint violation and the transaction
  * was rolled back.

=== modified file 'src/test/java/com/persistit/Bug918909Test.java'
--- src/test/java/com/persistit/Bug918909Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug918909Test.java	2012-08-28 16:17:18 +0000
@@ -22,8 +22,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 /**
  * At a DP site experiencing very low insert rates the JournalManager was
  * failing to delete obsolete journal files. This was noticed just prior to the

=== modified file 'src/test/java/com/persistit/Bug920754Test.java'
--- src/test/java/com/persistit/Bug920754Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug920754Test.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class Bug920754Test extends PersistitUnitTestCase {
     /*
      * https://bugs.launchpad.net/akiban-persistit/+bug/920754

=== modified file 'src/test/java/com/persistit/Bug923790Test.java'
--- src/test/java/com/persistit/Bug923790Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug923790Test.java	2012-08-28 16:17:18 +0000
@@ -19,7 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.util.Util;
 
 /**

=== modified file 'src/test/java/com/persistit/Bug927701Test.java'
--- src/test/java/com/persistit/Bug927701Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug927701Test.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 /**
  * Failure detected during TPCC testing. Upon restarting server, the following
  * error was emitted:
@@ -54,7 +52,7 @@
     @Test
     public void testBug927701() throws Exception {
         final JournalManager jman = _persistit.getJournalManager();
-        _persistit.getCleanupManager().setMinimumPruningDelay(0);
+        disableBackgroundCleanup();
         jman.setCopierInterval(1000);
         final long blockSize = jman.getBlockSize();
         /*

=== modified file 'src/test/java/com/persistit/Bug932097Test.java'
--- src/test/java/com/persistit/Bug932097Test.java	2012-08-02 04:45:28 +0000
+++ src/test/java/com/persistit/Bug932097Test.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class Bug932097Test extends PersistitUnitTestCase {
 
     @Test

=== modified file 'src/test/java/com/persistit/Bug937877Test.java'
--- src/test/java/com/persistit/Bug937877Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug937877Test.java	2012-08-28 16:17:18 +0000
@@ -24,7 +24,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.util.ThreadSequencer;
 
 public class Bug937877Test extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/Bug942669Test.java'
--- src/test/java/com/persistit/Bug942669Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug942669Test.java	2012-08-28 16:17:18 +0000
@@ -26,8 +26,6 @@
 import org.junit.Ignore;
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class Bug942669Test extends PersistitUnitTestCase {
 
     // See https://bugs.launchpad.net/akiban-persistit/+bug/942669

=== modified file 'src/test/java/com/persistit/Bug947182Test.java'
--- src/test/java/com/persistit/Bug947182Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug947182Test.java	2012-08-28 16:17:18 +0000
@@ -38,7 +38,6 @@
 import org.junit.Test;
 
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class Bug947182Test extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/Bug974589Test.java'
--- src/test/java/com/persistit/Bug974589Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug974589Test.java	2012-08-28 16:17:18 +0000
@@ -22,7 +22,6 @@
 import org.junit.Test;
 
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class Bug974589Test extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/Bug980292Test.java'
--- src/test/java/com/persistit/Bug980292Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug980292Test.java	2012-08-28 16:17:18 +0000
@@ -22,7 +22,6 @@
 import org.junit.Test;
 
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class Bug980292Test extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/Bug989202Test.java'
--- src/test/java/com/persistit/Bug989202Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug989202Test.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 /**
  * Persistit rev: 284
  * 

=== modified file 'src/test/java/com/persistit/Bug992801Test.java'
--- src/test/java/com/persistit/Bug992801Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug992801Test.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class Bug992801Test extends PersistitUnitTestCase {
     /*
      * https://bugs.launchpad.net/akiban-persistit/+bug/992801

=== modified file 'src/test/java/com/persistit/Bug996241Test.java'
--- src/test/java/com/persistit/Bug996241Test.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/Bug996241Test.java	2012-08-28 16:17:18 +0000
@@ -20,7 +20,6 @@
 import org.junit.Test;
 
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class Bug996241Test extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/ClassIndexTest.java'
--- src/test/java/com/persistit/ClassIndexTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/ClassIndexTest.java	2012-08-28 16:17:18 +0000
@@ -31,7 +31,6 @@
 import org.junit.Test;
 
 import com.persistit.Transaction.CommitPolicy;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.util.Util;
 
 public class ClassIndexTest extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/CleanupManagerTest.java'
--- src/test/java/com/persistit/CleanupManagerTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/CleanupManagerTest.java	2012-08-28 16:17:18 +0000
@@ -24,7 +24,6 @@
 
 import com.persistit.CleanupManager.CleanupAction;
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class CleanupManagerTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/CommandLineTest.java'
--- src/test/java/com/persistit/CommandLineTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/CommandLineTest.java	2012-08-28 16:17:18 +0000
@@ -29,7 +29,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.util.Util;
 
 public class CommandLineTest extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/ConfigurationTest.java'
--- src/test/java/com/persistit/ConfigurationTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/ConfigurationTest.java	2012-08-28 16:17:18 +0000
@@ -33,7 +33,6 @@
 import org.junit.Test;
 
 import com.persistit.Configuration.BufferPoolConfiguration;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class ConfigurationTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/CorruptVolumeTest.java'
--- src/test/java/com/persistit/CorruptVolumeTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/CorruptVolumeTest.java	2012-08-28 16:17:18 +0000
@@ -23,7 +23,6 @@
 
 import com.persistit.exception.CorruptVolumeException;
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class CorruptVolumeTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/DumpTaskTest.java'
--- src/test/java/com/persistit/DumpTaskTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/DumpTaskTest.java	2012-08-28 16:17:18 +0000
@@ -32,7 +32,6 @@
 import com.persistit.JournalRecord.IV;
 import com.persistit.JournalRecord.PA;
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class DumpTaskTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/FastIndexTest.java'
--- src/test/java/com/persistit/FastIndexTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/FastIndexTest.java	2012-08-28 16:17:18 +0000
@@ -22,7 +22,6 @@
 import org.junit.Test;
 
 import com.persistit.ValueHelper.RawValueWriter;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class FastIndexTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/FatalErrorExceptionTest.java'
--- src/test/java/com/persistit/FatalErrorExceptionTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/FatalErrorExceptionTest.java	2012-08-28 16:17:18 +0000
@@ -20,7 +20,6 @@
 import org.junit.Test;
 
 import com.persistit.Persistit.FatalErrorException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class FatalErrorExceptionTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/IOFailureTest.java'
--- src/test/java/com/persistit/IOFailureTest.java	2012-08-23 17:52:56 +0000
+++ src/test/java/com/persistit/IOFailureTest.java	2012-08-28 16:17:18 +0000
@@ -34,7 +34,6 @@
 import com.persistit.exception.CorruptVolumeException;
 import com.persistit.exception.PersistitException;
 import com.persistit.exception.PersistitIOException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class IOFailureTest extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/IOMeterChargeBenchmark.java'
--- src/test/java/com/persistit/IOMeterChargeBenchmark.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/IOMeterChargeBenchmark.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class IOMeterChargeBenchmark extends PersistitUnitTestCase {
 
     private final static long SECOND = 1000000000;

=== modified file 'src/test/java/com/persistit/InsertSequenceTest.java'
--- src/test/java/com/persistit/InsertSequenceTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/InsertSequenceTest.java	2012-08-28 16:17:18 +0000
@@ -22,7 +22,6 @@
 import com.persistit.Exchange.Sequence;
 import com.persistit.exception.PersistitException;
 import com.persistit.policy.SplitPolicy;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class InsertSequenceTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/IntegrityCheckTest.java'
--- src/test/java/com/persistit/IntegrityCheckTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/IntegrityCheckTest.java	2012-08-28 16:17:18 +0000
@@ -25,7 +25,6 @@
 import org.junit.Test;
 
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class IntegrityCheckTest extends PersistitUnitTestCase {
 
@@ -55,7 +54,7 @@
     @Test
     public void testSimpleMvvTree() throws Exception {
         final Exchange ex = _persistit.getExchange(_volumeName, "mvv", true);
-        _persistit.getCleanupManager().setPollInterval(Integer.MAX_VALUE);
+        disableBackgroundCleanup();
 
         transactionalStore(ex);
 
@@ -76,7 +75,7 @@
     @Test
     public void testBrokenKeySequence() throws Exception {
         final Exchange ex = _persistit.getExchange(_volumeName, "mvv", true);
-        _persistit.getCleanupManager().setPollInterval(Integer.MAX_VALUE);
+        disableBackgroundCleanup();
 
         transactionalStore(ex);
 
@@ -90,7 +89,7 @@
     @Test
     public void testBrokenMVVs() throws Exception {
         final Exchange ex = _persistit.getExchange(_volumeName, "mvv", true);
-        _persistit.getCleanupManager().setPollInterval(Integer.MAX_VALUE);
+        disableBackgroundCleanup();
         transactionalStore(ex);
         corrupt2(ex);
         final IntegrityCheck icheck = icheck();
@@ -155,6 +154,7 @@
     @Test
     public void testPruneRemovesAbortedTransactionStatus() throws Exception {
         _persistit.getJournalManager().setRollbackPruningEnabled(false);
+        _persistit.getJournalManager().setWritePagePruningEnabled(false);
 
         for (int i = 0; i < 10; i++) {
             final Exchange ex = _persistit.getExchange(_volumeName, "mvv" + i, true);
@@ -179,9 +179,7 @@
         _persistit = new Persistit();
         _persistit.getRecoveryManager().setRecoveryDisabledForTestMode(true);
         _persistit.getJournalManager().setRollbackPruningEnabled(false);
-        _persistit.getCleanupManager().setMinimumPruningDelay(0); // no
-                                                                  // background
-                                                                  // pruningIn
+        disableBackgroundCleanup();
         _persistit.initialize(properties);
 
         for (int i = 0; i < 10; i++) {

=== modified file 'src/test/java/com/persistit/JournalManagerTest.java'
--- src/test/java/com/persistit/JournalManagerTest.java	2012-08-23 15:24:42 +0000
+++ src/test/java/com/persistit/JournalManagerTest.java	2012-08-28 16:17:18 +0000
@@ -47,7 +47,6 @@
 import com.persistit.TransactionPlayer.TransactionPlayerListener;
 import com.persistit.exception.PersistitException;
 import com.persistit.unit.ConcurrentUtil.ThrowingRunnable;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 import com.persistit.util.Util;
 
@@ -89,6 +88,7 @@
         jman.writeCheckpointToJournal(checkpoint1);
         final Exchange exchange = _persistit.getExchange(_volumeName, "JournalManagerTest1", false);
         volume.getTree("JournalManagerTest1", false).resetHandle();
+
         assertTrue(exchange.next(true));
         final long[] timestamps = new long[100];
 
@@ -299,6 +299,7 @@
     public void testRollback() throws Exception {
         // Allow test to control when pruning will happen
         _persistit.getJournalManager().setRollbackPruningEnabled(false);
+        _persistit.getJournalManager().setWritePagePruningEnabled(false);
         final Transaction txn = _persistit.getTransaction();
         for (int i = 0; i < 10; i++) {
             txn.begin();
@@ -349,6 +350,8 @@
     public void testRollbackLongRecords() throws Exception {
         // Allow test to control when pruning will happen
         _persistit.getJournalManager().setRollbackPruningEnabled(false);
+        _persistit.getJournalManager().setWritePagePruningEnabled(false);
+
         final Volume volume = _persistit.getVolume(_volumeName);
         final Transaction txn = _persistit.getTransaction();
         for (int i = 0; i < 10; i++) {

=== modified file 'src/test/java/com/persistit/KeyHistogramTest.java'
--- src/test/java/com/persistit/KeyHistogramTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/KeyHistogramTest.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class KeyHistogramTest extends PersistitUnitTestCase {
 
     private final String _volumeName = "persistit";

=== modified file 'src/test/java/com/persistit/MVCCPruneBufferTest.java'
--- src/test/java/com/persistit/MVCCPruneBufferTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/MVCCPruneBufferTest.java	2012-08-28 16:17:18 +0000
@@ -144,7 +144,7 @@
 
     @Test
     public void testPruneLongRecordsSimple() throws Exception {
-        _persistit.getCleanupManager().setPollInterval(-1);
+        disableBackgroundCleanup();
         trx1.begin();
         storeLongMVV(ex1, "x");
         trx1.commit();
@@ -156,7 +156,7 @@
 
     @Test
     public void testPruneLongRecordsWithRollback() throws Exception {
-        _persistit.getCleanupManager().setPollInterval(-1);
+        disableBackgroundCleanup();
         /*
          * Start a concurrent transaction to prevent pruning during the store
          * operations.
@@ -182,7 +182,7 @@
 
     @Test
     public void induceBug1006576() throws Exception {
-        _persistit.getCleanupManager().setPollInterval(-1);
+        disableBackgroundCleanup();
         trx1.begin();
         storeLongMVV(ex1, "x");
         storeLongMVV(ex1, "y");
@@ -203,7 +203,7 @@
 
     @Test
     public void induceBug1005206() throws Exception {
-        _persistit.getCleanupManager().setPollInterval(-1);
+        disableBackgroundCleanup();
         trx1.begin();
         storeLongMVV(ex1, "x");
 
@@ -297,6 +297,27 @@
         }
     }
 
+    @Test
+    public void testWritePagePrune() throws Exception {
+        final Transaction txn = _persistit.getTransaction();
+        try {
+            txn.begin();
+            storeNewVersion(1);
+            txn.commit();
+        } finally {
+            txn.end();
+        }
+        final Volume volume = _persistit.getVolume(TEST_VOLUME_NAME);
+        final Buffer buffer = volume.getPool().get(volume, 3, true, true);
+        assertTrue("Should have multiple MVV records", buffer.getMvvCount() > 2);
+        _persistit.getJournalManager().setWritePagePruningEnabled(true);
+        _persistit.getTransactionIndex().updateActiveTransactionCache();
+        buffer.setDirtyAtTimestamp(_persistit.getCurrentTimestamp());
+        buffer.writePage();
+        assertTrue("Should no more than one MVV record", buffer.getMvvCount() < 2);
+        buffer.release();
+    }
+
     private void storeNewVersion(final int cycle) throws Exception {
         final Exchange exchange = _persistit.getExchange(TEST_VOLUME_NAME,
                 String.format("%s%04d", TEST_TREE_NAME, cycle), true);

=== modified file 'src/test/java/com/persistit/MVCCTestBase.java'
--- src/test/java/com/persistit/MVCCTestBase.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/MVCCTestBase.java	2012-08-28 16:17:18 +0000
@@ -24,7 +24,6 @@
 
 import com.persistit.exception.PersistitException;
 import com.persistit.exception.PersistitInterruptedException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.util.Util;
 
 public abstract class MVCCTestBase extends PersistitUnitTestCase {

=== added file 'src/test/java/com/persistit/PersistitUnitTestCase.java'
--- src/test/java/com/persistit/PersistitUnitTestCase.java	1970-01-01 00:00:00 +0000
+++ src/test/java/com/persistit/PersistitUnitTestCase.java	2012-08-28 16:17:18 +0000
@@ -0,0 +1,136 @@
+/**
+ * Copyright © 2005-2012 Akiban Technologies, Inc.  All rights reserved.
+ * 
+ * This program and the accompanying materials are made available
+ * under the terms of the Eclipse Public License v1.0 which
+ * accompanies this distribution, and is available at
+ * http://www.eclipse.org/legal/epl-v10.html
+ * 
+ * This program may also be available under different license terms.
+ * For more information, see www.akiban.com or contact [e-mail].
+ * 
+ * Contributors:
+ * Akiban Technologies, Inc.
+ */
+
+package com.persistit;
+
+import java.lang.ref.WeakReference;
+import java.util.Map;
+import java.util.Properties;
+
+import org.junit.After;
+import org.junit.Before;
+
+import com.persistit.exception.PersistitException;
+import com.persistit.unit.UnitTestProperties;
+
+public abstract class PersistitUnitTestCase {
+
+    private final static long TEN_SECONDS = 10L * 1000L * 1000L * 1000L;
+
+    protected final static String RED_FOX = "The quick red fox jumped over the lazy brown dog.";
+
+    protected static String createString(final int exactLength) {
+        final StringBuilder sb = new StringBuilder(exactLength);
+        // Simple 0..9a..z string
+        for (int i = 0; i < 36; ++i) {
+            sb.append(Character.forDigit(i, 36));
+        }
+        final String numAndLetters = sb.toString();
+        while (sb.length() < exactLength) {
+            sb.append(numAndLetters);
+        }
+        return sb.toString().substring(0, exactLength);
+    }
+
+    protected Persistit _persistit = new Persistit();
+
+    protected Properties getProperties(final boolean cleanup) {
+        return UnitTestProperties.getProperties(cleanup);
+    }
+
+    @Before
+    public void setUp() throws Exception {
+        checkNoPersistitThreads();
+        _persistit.initialize(getProperties(true));
+    }
+
+    @After
+    public void tearDown() throws Exception {
+        final WeakReference<Persistit> ref = new WeakReference<Persistit>(_persistit);
+        _persistit.close(false);
+        _persistit = null;
+
+        if (!doesRefBecomeNull(ref)) {
+            System.out.println("Persistit has a leftover strong reference");
+        }
+        checkNoPersistitThreads();
+    }
+
+    public void runAllTests() throws Exception {
+
+    }
+
+    public void setPersistit(final Persistit persistit) {
+        _persistit = persistit;
+    }
+
+    protected void initAndRunTest() throws Exception {
+        setUp();
+        try {
+            runAllTests();
+        } catch (final Throwable t) {
+            t.printStackTrace();
+        } finally {
+            tearDown();
+        }
+    }
+
+    private final static String[] PERSISTIT_THREAD_NAMES = { "CHECKPOINT_WRITER", "JOURNAL_COPIER", "JOURNAL_FLUSHER",
+            "PAGE_WRITER", "TXN_UPDATE" };
+
+    protected boolean checkNoPersistitThreads() {
+        boolean alive = false;
+        final Map<Thread, StackTraceElement[]> map = Thread.getAllStackTraces();
+        for (final Thread t : map.keySet()) {
+            final String name = t.getName();
+            for (final String p : PERSISTIT_THREAD_NAMES) {
+                if (name.contains(p)) {
+                    alive = true;
+                    System.err.println("Thread " + t + " is still alive");
+                }
+            }
+        }
+        return alive;
+    }
+
+    protected void safeCrashAndRestoreProperties() throws PersistitException {
+        final Properties properties = _persistit.getProperties();
+        _persistit.flush();
+        _persistit.crash();
+        _persistit = new Persistit();
+        _persistit.initialize(properties);
+    }
+
+    protected void crashWithoutFlushAndRestoreProperties() throws PersistitException {
+        final Properties properties = _persistit.getProperties();
+        _persistit.crash();
+        _persistit = new Persistit();
+        _persistit.initialize(properties);
+    }
+
+    public static boolean doesRefBecomeNull(final WeakReference<?> ref) throws InterruptedException {
+        final long expires = System.nanoTime() + TEN_SECONDS;
+        while (ref.get() != null && System.nanoTime() < expires) {
+            System.gc();
+            Thread.sleep(10);
+        }
+        return ref.get() == null;
+    }
+
+    protected void disableBackgroundCleanup() {
+        _persistit.getCleanupManager().setPollInterval(-1);
+        _persistit.getJournalManager().setWritePagePruningEnabled(false);
+    }
+}

=== modified file 'src/test/java/com/persistit/RecoveryTest.java'
--- src/test/java/com/persistit/RecoveryTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/RecoveryTest.java	2012-08-28 16:17:18 +0000
@@ -37,7 +37,6 @@
 import com.persistit.exception.PersistitException;
 import com.persistit.exception.RollbackException;
 import com.persistit.exception.TransactionFailedException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class RecoveryTest extends PersistitUnitTestCase {
@@ -53,6 +52,8 @@
     public void setUp() throws Exception {
         super.setUp();
         _persistit.getJournalManager().setRollbackPruningEnabled(false);
+        _persistit.getJournalManager().setWritePagePruningEnabled(false);
+
     }
 
     @Override

=== modified file 'src/test/java/com/persistit/SplitPolicyTest.java'
--- src/test/java/com/persistit/SplitPolicyTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/SplitPolicyTest.java	2012-08-28 16:17:18 +0000
@@ -24,7 +24,6 @@
 
 import com.persistit.Exchange.Sequence;
 import com.persistit.policy.SplitPolicy;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class SplitPolicyTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/StatisticsTaskTest.java'
--- src/test/java/com/persistit/StatisticsTaskTest.java	2012-08-02 04:45:28 +0000
+++ src/test/java/com/persistit/StatisticsTaskTest.java	2012-08-28 16:17:18 +0000
@@ -27,7 +27,6 @@
 
 import com.persistit.StatisticsTask.Display;
 import com.persistit.StatisticsTask.Stat;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class StatisticsTaskTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/TransactionLifetimeTest.java'
--- src/test/java/com/persistit/TransactionLifetimeTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/TransactionLifetimeTest.java	2012-08-28 16:17:18 +0000
@@ -24,7 +24,6 @@
 
 import com.persistit.exception.PersistitException;
 import com.persistit.exception.TreeNotFoundException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 /**
  * Attempt to cover all cases from the pseudo graph below and ensure that the
@@ -63,6 +62,8 @@
         _persistit.getCheckpointManager().setCheckpointIntervalNanos(FIVE_MIN_NANOS);
         super.setUp();
         _persistit.getJournalManager().setRollbackPruningEnabled(false);
+        _persistit.getJournalManager().setWritePagePruningEnabled(false);
+
     }
 
     @Test
@@ -354,6 +355,8 @@
                     currentInLiveMap = false;
                     safeCrashAndRestoreProperties();
                     _persistit.getJournalManager().setRollbackPruningEnabled(false);
+                    _persistit.getJournalManager().setWritePagePruningEnabled(false);
+
                 } else {
                     fail("Unknown test node: " + curNode);
                 }

=== modified file 'src/test/java/com/persistit/TransactionTest2.java'
--- src/test/java/com/persistit/TransactionTest2.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/TransactionTest2.java	2012-08-28 16:17:18 +0000
@@ -33,7 +33,6 @@
 import com.persistit.exception.PersistitIOException;
 import com.persistit.exception.PersistitInterruptedException;
 import com.persistit.exception.RollbackException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 /**
  * Demonstrates the use of Persistit Transactions. This demo runs multiple

=== modified file 'src/test/java/com/persistit/TreeLifetimeTest.java'
--- src/test/java/com/persistit/TreeLifetimeTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/TreeLifetimeTest.java	2012-08-28 16:17:18 +0000
@@ -45,7 +45,6 @@
 import com.persistit.JournalManager.TreeDescriptor;
 import com.persistit.exception.PersistitException;
 import com.persistit.exception.TreeNotFoundException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class TreeLifetimeTest extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/TreeTest2.java'
--- src/test/java/com/persistit/TreeTest2.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/TreeTest2.java	2012-08-28 16:17:18 +0000
@@ -23,8 +23,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class TreeTest2 extends PersistitUnitTestCase {
 
     final static int COUNT = 10000;

=== modified file 'src/test/java/com/persistit/ValueTest7.java'
--- src/test/java/com/persistit/ValueTest7.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/ValueTest7.java	2012-08-28 16:17:18 +0000
@@ -23,7 +23,6 @@
 import org.junit.Test;
 
 import com.persistit.Value.Version;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class ValueTest7 extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/VolumeTest.java'
--- src/test/java/com/persistit/VolumeTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/VolumeTest.java	2012-08-28 16:17:18 +0000
@@ -32,7 +32,6 @@
 import com.persistit.exception.InUseException;
 import com.persistit.exception.InvalidVolumeSpecificationException;
 import com.persistit.exception.VolumeFullException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 
 public class VolumeTest extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/WarmupTest.java'
--- src/test/java/com/persistit/WarmupTest.java	2012-08-27 21:54:10 +0000
+++ src/test/java/com/persistit/WarmupTest.java	2012-08-28 16:17:18 +0000
@@ -22,8 +22,6 @@
 
 import org.junit.Test;
 
-import com.persistit.unit.PersistitUnitTestCase;
-
 public class WarmupTest extends PersistitUnitTestCase {
 
     @Override

=== modified file 'src/test/java/com/persistit/stress/unit/CommitBench.java'
--- src/test/java/com/persistit/stress/unit/CommitBench.java	2012-08-23 15:24:42 +0000
+++ src/test/java/com/persistit/stress/unit/CommitBench.java	2012-08-28 16:17:18 +0000
@@ -15,15 +15,19 @@
 
 package com.persistit.stress.unit;
 
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
 import java.util.Properties;
 import java.util.Random;
 import java.util.concurrent.atomic.AtomicInteger;
+import java.util.regex.Pattern;
 
 import com.persistit.Exchange;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Transaction;
 import com.persistit.Transaction.CommitPolicy;
 import com.persistit.exception.RollbackException;
-import com.persistit.unit.PersistitUnitTestCase;
 import com.persistit.unit.UnitTestProperties;
 import com.persistit.util.ArgParser;
 
@@ -38,6 +42,9 @@
  * 
  */
 public class CommitBench extends PersistitUnitTestCase {
+
+    final static Pattern PATH_PATTERN = Pattern.compile("(.+)\\.(\\d{12})");
+
     /*
      * Want about 1000 pages worth of data (no evictions). Each record is about
      * 50 bytes, so 1000 * 16384 * 60% / 50 ~= 200000.
@@ -47,19 +54,56 @@
     private final int RECORDS_PER_TXN = 10;
     private final String[] ARG_TEMPLATE = new String[] { "threads|int:1:1:1000|Number of threads",
             "duration|int:10:10:86400|Duration of test in seconds",
-            "policy|String:HARD|Commit policy: SOFT, HARD or GROUP", };
+            "policy|String:HARD|Commit policy: SOFT, HARD or GROUP", "datapath|String|Datapath property",
+            "_flag|P|Reuse journal file" };
 
     volatile long stopTime;
+    final ArgParser ap;
     AtomicInteger commitCount = new AtomicInteger();
     AtomicInteger rollbackCount = new AtomicInteger();
 
+    CommitBench(final String[] args) {
+        ap = new ArgParser("CommitBench", args, ARG_TEMPLATE).strict();
+    }
+
     @Override
     protected Properties getProperties(final boolean cleanup) {
-        return UnitTestProperties.getBiggerProperties(cleanup);
+        final Properties p = UnitTestProperties.getBiggerProperties(false);
+        if (ap.isSpecified("datapath")) {
+            p.setProperty("datapath", ap.getStringValue("datapath"));
+        }
+        /*
+         * Custom data directory cleanup - leaving the journal file behind if
+         */
+        final String path = p.getProperty("datapath");
+        final File dir = new File(path);
+        assert dir.isDirectory() : "Data path does not specify a directory: " + path;
+        final File[] files = dir.listFiles();
+        for (final File file : files) {
+            if (ap.isFlag('P') && PATH_PATTERN.matcher(file.getName()).matches()) {
+                try {
+                    /*
+                     * Damage the file so that there's no keystone checkpoint
+                     */
+                    RandomAccessFile raf = new RandomAccessFile(file, "rws");
+                    raf.seek(0);
+                    raf.write(new byte[256]);
+                    raf.close();
+                } catch (IOException e) {
+                    throw new RuntimeException(e);
+                }
+            } else {
+                if (file.isDirectory()) {
+                    UnitTestProperties.cleanUpDirectory(file);
+                } else {
+                    file.delete();
+                }
+            }
+        }
+        return p;
     }
 
-    public void bench(final String[] args) throws Exception {
-        final ArgParser ap = new ArgParser("CommitBench", args, ARG_TEMPLATE).strict();
+    public void bench() throws Exception {
         final int threadCount = ap.getIntValue("threads");
         final int duration = ap.getIntValue("duration");
         final String policy = ap.getStringValue("policy");
@@ -138,10 +182,10 @@
     }
 
     public static void main(final String[] args) throws Exception {
-        final CommitBench bench = new CommitBench();
+        final CommitBench bench = new CommitBench(args);
         try {
             bench.setUp();
-            bench.bench(args);
+            bench.bench();
         } finally {
             bench.tearDown();
         }

=== modified file 'src/test/java/com/persistit/stress/unit/TestSamePageOptimization.java'
--- src/test/java/com/persistit/stress/unit/TestSamePageOptimization.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/stress/unit/TestSamePageOptimization.java	2012-08-28 16:17:18 +0000
@@ -19,8 +19,8 @@
 
 import com.persistit.Exchange;
 import com.persistit.Key;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
-import com.persistit.unit.PersistitUnitTestCase;
 
 public class TestSamePageOptimization extends PersistitUnitTestCase {
     final int KEYS = 1000000;

=== modified file 'src/test/java/com/persistit/unit/BufferPoolMemConfigurationTest.java'
--- src/test/java/com/persistit/unit/BufferPoolMemConfigurationTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/BufferPoolMemConfigurationTest.java	2012-08-28 16:17:18 +0000
@@ -24,6 +24,7 @@
 import org.junit.Test;
 
 import com.persistit.Management.BufferPoolInfo;
+import com.persistit.PersistitUnitTestCase;
 
 public class BufferPoolMemConfigurationTest extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/unit/DefaultCoderManagerTest.java'
--- src/test/java/com/persistit/unit/DefaultCoderManagerTest.java	2012-08-02 04:45:28 +0000
+++ src/test/java/com/persistit/unit/DefaultCoderManagerTest.java	2012-08-28 16:17:18 +0000
@@ -25,6 +25,7 @@
 import org.junit.Test;
 
 import com.persistit.DefaultCoderManager;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
 
 public class DefaultCoderManagerTest extends PersistitUnitTestCase implements Serializable {

=== modified file 'src/test/java/com/persistit/unit/ExchangeTest.java'
--- src/test/java/com/persistit/unit/ExchangeTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/ExchangeTest.java	2012-08-28 16:17:18 +0000
@@ -28,6 +28,7 @@
 import com.persistit.Exchange;
 import com.persistit.Key;
 import com.persistit.KeyFilter;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Transaction;
 import com.persistit.Volume;
 import com.persistit.exception.ConversionException;

=== modified file 'src/test/java/com/persistit/unit/FileLockTest.java'
--- src/test/java/com/persistit/unit/FileLockTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/FileLockTest.java	2012-08-28 16:17:18 +0000
@@ -22,6 +22,7 @@
 import org.junit.Test;
 
 import com.persistit.Persistit;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
 
 /**

=== modified file 'src/test/java/com/persistit/unit/JoinTest1.java'
--- src/test/java/com/persistit/unit/JoinTest1.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/JoinTest1.java	2012-08-28 16:17:18 +0000
@@ -21,6 +21,7 @@
 
 import com.persistit.Exchange;
 import com.persistit.Key;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Value;
 import com.persistit.exception.PersistitException;
 

=== modified file 'src/test/java/com/persistit/unit/KeyCoderTest1.java'
--- src/test/java/com/persistit/unit/KeyCoderTest1.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/KeyCoderTest1.java	2012-08-28 16:17:18 +0000
@@ -27,6 +27,7 @@
 import org.junit.Test;
 
 import com.persistit.Key;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.TestShim;
 import com.persistit.encoding.CoderContext;
 import com.persistit.encoding.KeyCoder;

=== modified file 'src/test/java/com/persistit/unit/KeyFilterTest1.java'
--- src/test/java/com/persistit/unit/KeyFilterTest1.java	2012-08-02 04:45:28 +0000
+++ src/test/java/com/persistit/unit/KeyFilterTest1.java	2012-08-28 16:17:18 +0000
@@ -28,6 +28,7 @@
 import com.persistit.KeyFilter;
 import com.persistit.KeyState;
 import com.persistit.Persistit;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
 
 /**

=== modified file 'src/test/java/com/persistit/unit/KeyFilterTest2.java'
--- src/test/java/com/persistit/unit/KeyFilterTest2.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/KeyFilterTest2.java	2012-08-28 16:17:18 +0000
@@ -23,6 +23,7 @@
 import com.persistit.Exchange;
 import com.persistit.Key;
 import com.persistit.KeyFilter;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
 
 /**

=== modified file 'src/test/java/com/persistit/unit/KeyParserTest1.java'
--- src/test/java/com/persistit/unit/KeyParserTest1.java	2012-08-02 04:45:28 +0000
+++ src/test/java/com/persistit/unit/KeyParserTest1.java	2012-08-28 16:17:18 +0000
@@ -23,6 +23,7 @@
 import com.persistit.Key;
 import com.persistit.KeyParser;
 import com.persistit.Persistit;
+import com.persistit.PersistitUnitTestCase;
 
 /**
  * @version 1.0

=== modified file 'src/test/java/com/persistit/unit/KeyParserTest2.java'
--- src/test/java/com/persistit/unit/KeyParserTest2.java	2012-08-02 04:45:28 +0000
+++ src/test/java/com/persistit/unit/KeyParserTest2.java	2012-08-28 16:17:18 +0000
@@ -23,6 +23,7 @@
 import com.persistit.Key;
 import com.persistit.KeyFilter;
 import com.persistit.KeyParser;
+import com.persistit.PersistitUnitTestCase;
 
 public class KeyParserTest2 extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/unit/KeyTest1.java'
--- src/test/java/com/persistit/unit/KeyTest1.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/KeyTest1.java	2012-08-28 16:17:18 +0000
@@ -30,6 +30,7 @@
 import com.persistit.Buffer;
 import com.persistit.Key;
 import com.persistit.KeyState;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.TestShim;
 import com.persistit.exception.InvalidKeyException;
 import com.persistit.exception.MissingKeySegmentException;

=== modified file 'src/test/java/com/persistit/unit/Log4JLoggerTest.java'
--- src/test/java/com/persistit/unit/Log4JLoggerTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/Log4JLoggerTest.java	2012-08-28 16:17:18 +0000
@@ -20,6 +20,7 @@
 import org.junit.Test;
 
 import com.persistit.Persistit;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
 import com.persistit.logging.Log4JAdapter;
 

=== modified file 'src/test/java/com/persistit/unit/LongRecordTest1.java'
--- src/test/java/com/persistit/unit/LongRecordTest1.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/LongRecordTest1.java	2012-08-28 16:17:18 +0000
@@ -22,6 +22,7 @@
 
 import com.persistit.Exchange;
 import com.persistit.Key;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
 import com.persistit.util.Debug;
 import com.persistit.util.Util;

=== modified file 'src/test/java/com/persistit/unit/LotsaSmallKeys.java'
--- src/test/java/com/persistit/unit/LotsaSmallKeys.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/LotsaSmallKeys.java	2012-08-28 16:17:18 +0000
@@ -20,6 +20,7 @@
 import org.junit.Test;
 
 import com.persistit.Exchange;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
 import com.persistit.policy.SplitPolicy;
 

=== modified file 'src/test/java/com/persistit/unit/PersistitMapTest.java'
--- src/test/java/com/persistit/unit/PersistitMapTest.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/PersistitMapTest.java	2012-08-28 16:17:18 +0000
@@ -34,6 +34,7 @@
 import com.persistit.Exchange;
 import com.persistit.KeyFilter;
 import com.persistit.PersistitMap;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
 import com.persistit.util.Util;
 

=== removed file 'src/test/java/com/persistit/unit/PersistitUnitTestCase.java'
--- src/test/java/com/persistit/unit/PersistitUnitTestCase.java	2012-08-23 15:30:46 +0000
+++ src/test/java/com/persistit/unit/PersistitUnitTestCase.java	1970-01-01 00:00:00 +0000
@@ -1,133 +0,0 @@
-/**
- * Copyright © 2011-2012 Akiban Technologies, Inc.  All rights reserved.
- * 
- * This program and the accompanying materials are made available
- * under the terms of the Eclipse Public License v1.0 which
- * accompanies this distribution, and is available at
- * http://www.eclipse.org/legal/epl-v10.html
- * 
- * This program may also be available under different license terms.
- * For more information, see www.akiban.com or contact [e-mail].
- * 
- * Contributors:
- * Akiban Technologies, Inc.
- */
-
-package com.persistit.unit;
-
-import java.lang.ref.WeakReference;
-import java.util.Map;
-import java.util.Properties;
-
-import org.junit.After;
-import org.junit.Before;
-
-import com.persistit.Persistit;
-import com.persistit.exception.PersistitException;
-
-public abstract class PersistitUnitTestCase {
-
-    private final static long TEN_SECONDS = 10L * 1000L * 1000L * 1000L;
-
-    protected final static String RED_FOX = "The quick red fox jumped over the lazy brown dog.";
-
-    protected static String createString(final int exactLength) {
-        final StringBuilder sb = new StringBuilder(exactLength);
-        // Simple 0..9a..z string
-        for (int i = 0; i < 36; ++i) {
-            sb.append(Character.forDigit(i, 36));
-        }
-        final String numAndLetters = sb.toString();
-        while (sb.length() < exactLength) {
-            sb.append(numAndLetters);
-        }
-        return sb.toString().substring(0, exactLength);
-    }
-
-    protected Persistit _persistit = new Persistit();
-
-    protected Properties getProperties(final boolean cleanup) {
-        final Properties p = UnitTestProperties.getProperties(cleanup);
-        return p;
-    }
-
-    @Before
-    public void setUp() throws Exception {
-        checkNoPersistitThreads();
-
-        _persistit.initialize(getProperties(true));
-    }
-
-    @After
-    public void tearDown() throws Exception {
-        final WeakReference<Persistit> ref = new WeakReference<Persistit>(_persistit);
-        _persistit.close(false);
-        _persistit = null;
-
-        if (!doesRefBecomeNull(ref)) {
-            System.out.println("Persistit has a leftover strong reference");
-        }
-        checkNoPersistitThreads();
-    }
-
-    public void runAllTests() throws Exception {
-
-    }
-
-    public void setPersistit(final Persistit persistit) {
-        _persistit = persistit;
-    }
-
-    protected void initAndRunTest() throws Exception {
-        setUp();
-        try {
-            runAllTests();
-        } catch (final Throwable t) {
-            t.printStackTrace();
-        } finally {
-            tearDown();
-        }
-    }
-
-    private final static String[] PERSISTIT_THREAD_NAMES = { "CHECKPOINT_WRITER", "JOURNAL_COPIER", "JOURNAL_FLUSHER",
-            "PAGE_WRITER", "TXN_UPDATE" };
-
-    protected boolean checkNoPersistitThreads() {
-        boolean alive = false;
-        final Map<Thread, StackTraceElement[]> map = Thread.getAllStackTraces();
-        for (final Thread t : map.keySet()) {
-            final String name = t.getName();
-            for (final String p : PERSISTIT_THREAD_NAMES) {
-                if (name.contains(p)) {
-                    alive = true;
-                    System.err.println("Thread " + t + " is still alive");
-                }
-            }
-        }
-        return alive;
-    }
-
-    protected void safeCrashAndRestoreProperties() throws PersistitException {
-        final Properties properties = _persistit.getProperties();
-        _persistit.flush();
-        _persistit.crash();
-        _persistit = new Persistit();
-        _persistit.initialize(properties);
-    }
-
-    protected void crashWithoutFlushAndRestoreProperties() throws PersistitException {
-        final Properties properties = _persistit.getProperties();
-        _persistit.crash();
-        _persistit = new Persistit();
-        _persistit.initialize(properties);
-    }
-
-    public static boolean doesRefBecomeNull(final WeakReference<?> ref) throws InterruptedException {
-        final long expires = System.nanoTime() + TEN_SECONDS;
-        while (ref.get() != null && System.nanoTime() < expires) {
-            System.gc();
-            Thread.sleep(10);
-        }
-        return ref.get() == null;
-    }
-}

=== modified file 'src/test/java/com/persistit/unit/SaveLoadTest1.java'
--- src/test/java/com/persistit/unit/SaveLoadTest1.java	2012-08-02 04:45:28 +0000
+++ src/test/java/com/persistit/unit/SaveLoadTest1.java	2012-08-28 16:17:18 +0000
@@ -24,6 +24,7 @@
 import org.junit.Test;
 
 import com.persistit.PersistitMap;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.StreamLoader;
 import com.persistit.StreamSaver;
 import com.persistit.exception.PersistitException;

=== modified file 'src/test/java/com/persistit/unit/SimpleTest1.java'
--- src/test/java/com/persistit/unit/SimpleTest1.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/SimpleTest1.java	2012-08-28 16:17:18 +0000
@@ -25,6 +25,7 @@
 
 import com.persistit.Exchange;
 import com.persistit.Key;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Value;
 import com.persistit.exception.PersistitException;
 

=== modified file 'src/test/java/com/persistit/unit/TemporaryVolumeTest1.java'
--- src/test/java/com/persistit/unit/TemporaryVolumeTest1.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/TemporaryVolumeTest1.java	2012-08-28 16:17:18 +0000
@@ -28,6 +28,7 @@
 import com.persistit.Exchange;
 import com.persistit.Key;
 import com.persistit.Management;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Value;
 import com.persistit.Volume;
 import com.persistit.exception.PersistitException;

=== modified file 'src/test/java/com/persistit/unit/TransactionTest1.java'
--- src/test/java/com/persistit/unit/TransactionTest1.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/TransactionTest1.java	2012-08-28 16:17:18 +0000
@@ -28,6 +28,7 @@
 import com.persistit.Exchange;
 import com.persistit.Key;
 import com.persistit.KeyFilter;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Transaction;
 import com.persistit.Value;
 import com.persistit.exception.PersistitException;

=== modified file 'src/test/java/com/persistit/unit/TransactionTest3.java'
--- src/test/java/com/persistit/unit/TransactionTest3.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/TransactionTest3.java	2012-08-28 16:17:18 +0000
@@ -27,6 +27,7 @@
 
 import com.persistit.Exchange;
 import com.persistit.Key;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Transaction;
 import com.persistit.exception.PersistitException;
 

=== modified file 'src/test/java/com/persistit/unit/TreeTest1.java'
--- src/test/java/com/persistit/unit/TreeTest1.java	2012-08-02 04:45:28 +0000
+++ src/test/java/com/persistit/unit/TreeTest1.java	2012-08-28 16:17:18 +0000
@@ -23,6 +23,7 @@
 
 import com.persistit.Exchange;
 import com.persistit.Key;
+import com.persistit.PersistitUnitTestCase;
 
 public class TreeTest1 extends PersistitUnitTestCase {
 

=== modified file 'src/test/java/com/persistit/unit/ValueCoderTest1.java'
--- src/test/java/com/persistit/unit/ValueCoderTest1.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/ValueCoderTest1.java	2012-08-28 16:17:18 +0000
@@ -28,6 +28,7 @@
 import org.junit.Test;
 
 import com.persistit.Exchange;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Value;
 import com.persistit.encoding.CoderContext;
 import com.persistit.encoding.ValueCoder;

=== modified file 'src/test/java/com/persistit/unit/ValueCoderTest2.java'
--- src/test/java/com/persistit/unit/ValueCoderTest2.java	2012-08-02 04:45:28 +0000
+++ src/test/java/com/persistit/unit/ValueCoderTest2.java	2012-08-28 16:17:18 +0000
@@ -25,6 +25,7 @@
 
 import com.persistit.DefaultObjectCoder;
 import com.persistit.Exchange;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.exception.PersistitException;
 
 public class ValueCoderTest2 extends PersistitUnitTestCase {

=== modified file 'src/test/java/com/persistit/unit/ValueTest1.java'
--- src/test/java/com/persistit/unit/ValueTest1.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/ValueTest1.java	2012-08-28 16:17:18 +0000
@@ -28,6 +28,7 @@
 
 import org.junit.Test;
 
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Value;
 import com.persistit.util.Util;
 

=== modified file 'src/test/java/com/persistit/unit/ValueTest2.java'
--- src/test/java/com/persistit/unit/ValueTest2.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/ValueTest2.java	2012-08-28 16:17:18 +0000
@@ -30,6 +30,7 @@
 import org.junit.Test;
 
 import com.persistit.Exchange;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Value;
 import com.persistit.encoding.CollectionValueCoder;
 import com.persistit.exception.PersistitException;

=== modified file 'src/test/java/com/persistit/unit/ValueTest3.java'
--- src/test/java/com/persistit/unit/ValueTest3.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/ValueTest3.java	2012-08-28 16:17:18 +0000
@@ -34,6 +34,7 @@
 import com.persistit.DefaultValueCoder;
 import com.persistit.Exchange;
 import com.persistit.Persistit;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Value;
 import com.persistit.encoding.CoderContext;
 import com.persistit.encoding.CoderManager;

=== modified file 'src/test/java/com/persistit/unit/ValueTest4.java'
--- src/test/java/com/persistit/unit/ValueTest4.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/ValueTest4.java	2012-08-28 16:17:18 +0000
@@ -35,6 +35,7 @@
 import com.persistit.DefaultValueCoder;
 import com.persistit.Exchange;
 import com.persistit.Persistit;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.Value;
 import com.persistit.encoding.CoderContext;
 import com.persistit.encoding.CoderManager;

=== modified file 'src/test/java/com/persistit/unit/ValueTest5.java'
--- src/test/java/com/persistit/unit/ValueTest5.java	2012-08-15 16:11:36 +0000
+++ src/test/java/com/persistit/unit/ValueTest5.java	2012-08-28 16:17:18 +0000
@@ -22,6 +22,7 @@
 import org.junit.Test;
 
 import com.persistit.Exchange;
+import com.persistit.PersistitUnitTestCase;
 import com.persistit.encoding.CoderManager;
 import com.persistit.exception.PersistitException;