Merge lp:~pbeaman/akiban-persistit/transaction-tree-management into lp:akiban-persistit
- transaction-tree-management
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 430 |
Proposed branch: | lp:~pbeaman/akiban-persistit/transaction-tree-management |
Merge into: | lp:akiban-persistit |
Diff against target: |
3507 lines (+1893/-414) 37 files modified
pom.xml (+1/-1) src/main/java/com/persistit/Buffer.java (+1/-2) src/main/java/com/persistit/CheckpointManager.java (+1/-0) src/main/java/com/persistit/ClassIndex.java (+12/-1) src/main/java/com/persistit/Exchange.java (+60/-38) src/main/java/com/persistit/JournalManager.java (+5/-0) src/main/java/com/persistit/MVV.java (+1/-1) src/main/java/com/persistit/Persistit.java (+93/-43) src/main/java/com/persistit/RecoveryManager.java (+27/-0) src/main/java/com/persistit/SharedResource.java (+3/-3) src/main/java/com/persistit/TimelyResource.java (+505/-0) src/main/java/com/persistit/TransactionPlayer.java (+107/-68) src/main/java/com/persistit/Tree.java (+165/-46) src/main/java/com/persistit/ValueHelper.java (+12/-0) src/main/java/com/persistit/Version.java (+69/-0) src/main/java/com/persistit/Volume.java (+4/-1) src/main/java/com/persistit/VolumeStructure.java (+47/-54) src/main/java/com/persistit/logging/LogBase.java (+3/-0) src/main/java/com/persistit/util/SequencerConstants.java (+1/-12) src/test/java/com/persistit/AccumulatorRecoveryTest.java (+5/-0) src/test/java/com/persistit/AccumulatorTest.java (+1/-1) src/test/java/com/persistit/Bug1018526Test.java (+2/-2) src/test/java/com/persistit/Bug920754Test.java (+2/-1) src/test/java/com/persistit/Bug932097Test.java (+1/-1) src/test/java/com/persistit/CorruptVolumeTest.java (+2/-1) src/test/java/com/persistit/IOFailureTest.java (+4/-1) src/test/java/com/persistit/IntegrityCheckTest.java (+2/-0) src/test/java/com/persistit/JournalManagerTest.java (+12/-0) src/test/java/com/persistit/MVCCPruneBufferTest.java (+9/-4) src/test/java/com/persistit/PersistitUnitTestCase.java (+11/-0) src/test/java/com/persistit/RecoveryTest.java (+27/-19) src/test/java/com/persistit/SplitPolicyTest.java (+1/-1) src/test/java/com/persistit/TimelyResourceTest.java (+290/-0) src/test/java/com/persistit/TreeLifetimeTest.java (+26/-88) src/test/java/com/persistit/TreeTransactionalLifetimeTest.java (+277/-0) src/test/java/com/persistit/unit/ConcurrentUtil.java (+103/-24) src/test/java/com/persistit/unit/TransactionTest1.java (+1/-1) |
To merge this branch: | bzr merge lp:~pbeaman/akiban-persistit/transaction-tree-management |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Nathan Williams | Needs Fixing | ||
Review via email: mp+145415@code.launchpad.net |
Commit message
Description of the change
This branch makes the creation and removal of Tree instances transactional. That is, a Tree created within a transaction is not visible outside of that transaction until it commits (and then only to transactions that start after the creator's commit timestamp), and a Tree removed within a transaction remains visible to concurrent transactions; its removal is visible only to transactions that start after the removing transaction commits. Attempts by concurrent transactions to alter the existence of a Tree (either to create or remove it) result in write-write conflicts so that only one such transaction performing a tree creation or deletion can commit.
These changes will support DDL operations on Akiban Server. For example, a new index can be created inside of a transaction; the entire Tree becomes visible only when the transaction commits. Further, since the entire Tree would be removed in the event the transaction rolls back, the updates within the tree are made without writing MVVs that need to be pruned.
Similarly, the removal of an index by removing its Tree within a transaction will be visible only to transactions that start after the removal commits; concurrent transactions continue to see the tree.
To accommodate these behaviors, removing a tree within a transaction does not immediately add the tree's physical pages to the garbage chain. Instead, a version is recorded in memory, and when that version is eventually pruned the physical storage of the tree is removed.
The unit test TreeTransaction
The changes in the proposal are widespread because of many side effects. The following is a guide. I expect review questions and will expand upon areas that are unclear.
----
The largest body of new code is the TimelyResource class. It provides a general capability of maintaining a versioned history of a cached resource. It is intended as a mechanism within Persistit that can manage any kind of versioned object, where the object represents state created within a transaction. For example, a TimelyResource could be used to house versions of metadata (the AIS and/or schema) so that concurrent transactions can acquire and use the correct version. For this proposal, it is used to hold Tree.TreeVersion instances.
A TimelyResource manages Version instances, and TimelyResource is generic with respect to a particular Version implementation class. Each TimelyResource is added to a set maintained by the root Persistit instance, and these are polled periodically by the CleanupManager to perform pruning. If a Version maintained by a TimelyResource implements the subinterface PrunableVersion, then prune() method is called for any Version which has become obsolete according to our standard MVCC notion of liveness (i.e., no other active transaction needs the Version for a valid snapshot). The semantics of TimelyResource are documented in its JavaDoc, hopefully accurately and clearly.
Each Tree now uses one TimelyResource instance to manage its transactional state. A Tree created within a transaction has a Tree.TreeVersion instance within the TimelyResource. That version contains most of the state that was formerly held by the Tree object itself. Removing a Tree within a transaction adds a TreeVersion denoted the removed state; the actual reclamation of space held by that three is done when the versions are pruned.
Most of the accessors for state on the Tree object now delegate to the TreeVersion that is visible according to the currently active transaction.
There are some messy bits:
- The serialized state of a Tree represents its existence (which is transactional) and its physical B-Tree allocation (which is not). There are accommodations for this in the VolumeStructure class. We might have preferred a cleaner serialized representation, but the current version works, does not fundamentally limit the design, and is backward compatible. Another accommodation is that the update operations that change the serialized state of the Tree within the directory itself tree must not be reapplied during recovery. Instead, the state changes of the directory tree are implied by the operations that create and remove the trees themselves. Therefore the Transaction#store method explicitly prevents writing of certain directory tree updates even though they occur within the scope of a transaction.
- To make the new Exchange#lock method work properly, it was necessary to disable the transactional nature of tree creation/removal in the temporary volume used for locks. On the basis that this is likely the only temporary volume that will be shared among multiple threads, the current test creates and removes all trees in temporary volumes in a non-transactional (primordial) fashion. We could make this more precise, or we could simply define the transactional tree semantics as affecting only non-temporary volumes (which I think I prefer).
- The tree used by ClassIndex needs to be primordial to avoid unwanted write-write dependencies. Therefore there's an explicit call to create the tree during initialization.
- TreeStatistics are now maintained within a TreeVersion. Because the serialized form needs to be a snapshot view, these are now flushed within the checkpoint cycle only, and not periodically. As a consequence I believe the recovered tree statistics are always consistent with their state at checkpoint plus whatever updates are performed during recovery. Needless to say, the count of read operations performed after the keystone checkpoint is lost. This is not keeping me awake at night.
Because the semantic change here is pretty big, I am proposing a new major version increment to 3.3. Unlike other changes, this one does not merely add new methods; it subtly changes the fundamental behavior.
At the moment I have not run the server tests within this, nor have I run benchmarks to assess performance changes. These should be done before we approve the proposal.
- 416. By Peter Beaman
-
Skip directory tree updates in RecoveryManager
#DefaultRecover yListener rather than Transaction# store/# remove
Peter Beaman (pbeaman) wrote : | # |
Peter Beaman (pbeaman) wrote : | # |
Information only. Here's what we need to deal with in server:
Results :
Failed tests: deleteMissingRo
testYaml [test-bug1025059](com.akiban.
Tests in error:
dropGroupingF
setDataType_
setDataType_
dropColumn_
dropColumn_
dropPrimaryKe
dropPrimaryKe
addGroupingFo
dropGroupingF
setDataType_
dropColumn_
deleteMissing
testRepeated(
testMultipleU
Tests run: 2245, Failures: 2, Errors: 14, Skipped: 99
Nathan Williams (nwilliams) wrote : | # |
Very nice! The meat of the changes really aren't too complicated and everything seems to fit nicely.
The only code change I would suggest, ask about really, is the live-ness determination in TimelyResource#
I would also like to see a handful of tests around what is in the directory tree, shutdown and recovery where multiple versions were present, and the (lack of) MVVs in writes to trees created in the same transaction.
The rest is a smattering of feedback, ranging from small to nits.
TimelyResource
- JavaDoc has a number of invalid links. Lots of other docs like that too, but this is all new.
- Should getVersionCount() should be public? Fairly harmless, but I don't think we leak information like that from other places (Exchange, etc)
- Should delete() check for _first != null? Seems like deleting a resource with no versions should be exception or assert worthy.
Exchange
- Why was the assertCorrecThr
- The semantic change on checkThread (null-ing out of !set) is subtle. Perhaps renaming the method to checkAndSetThread or checkSetOrClear
- A couple places call throttle() and now have multiple conditions. A helper may be in order.
- The addition of 'spareValue.
Tree
- End of the new paragraph of JavaDoc has two open <code> tags instead of open and close.
- version() has a // TODO about throwing a RuntimeException
- There are now a number of places where isValid() and isDeleted() must be called together. Maybe a new helper? A more drastic change, but perhaps safer, would be to make it protected in SharedResource and not promote it to public in Tree.
VolumeStructure
- remoteTree() now only returns true (or throws)
TimelyResourceTest
- testAndPruneRes
- doConcurrentTra
- deleteResources() uses a raw assert instead of jUnits.
TreeTransaction
- simplePruning() has a 5s sleep and a commented out manual cleanup call. The latter seems cleaner and more deterministic, does it not work?
- I'd like to see some tests around the behavior regarding steps. I believe having trees obey it just like data makes the most sense, which is what it does now, but confirming it would be good.
- The TExec helper is duplicating part of the functionality in ConcurrentUtil and doesn't handle errors in a way that will fail the test.
- 417. By Peter Beaman
-
Modifications per review comments
- 418. By Peter Beaman
-
Source code cleanup
- 419. By Peter Beaman
-
Fix up documentation. Optimization: don't add new version to delete resource created with same version handle
- 420. By Peter Beaman
-
Add some crash and stop/start tests
- 421. By Peter Beaman
-
& -> &&
- 422. By Peter Beaman
-
assert -> assertTrue
Peter Beaman (pbeaman) wrote : | # |
Thanks. The updated version is changed as follows:
1. The only code change I would suggest, ask about really, is the live-ness determination in TimelyResource#
- No change yet. Will look at that again tomorrow.
2. I would also like to see a handful of tests around what is in the directory tree, shutdown and recovery where multiple versions were present, and the (lack of) MVVs in writes to trees created in the same transaction.
- I believe these cases are covered by new tests added to TreeTransaction
3. TimelyResource:
- JavaDoc fixed. I believe there are no broke links in the Javadoc - if you find some, please let me know.
- getVersionCount() - reduced to package private
- delete() - check for null and throw IllegalStateExc
Exchange
- Why was assertCorrectTh
- checkThread(
- throttle() - a new helper, as suggested..
- doMVCC && (spareValue.
Tree
- Javadoc issues: fixed
- version() now throws a subclass of RuntimeException and does not have a TODO
- isValue && !isDeleted() is now computed by helper isLive()
VolumeStructure
- removeTree() is now properly void
TimelyResourceTest
- Append withTransactions - done
- doConcurrentTra
- assert - fixed
TreeTransaction
- simplePruning() sleep - fixed
- I'd like to see new tests...
Peter Beaman (pbeaman) wrote : | # |
Note that Akiban Server requires changes to work with transactional trees. Branch lp:~pbeaman/akiban-server/try-transactional-tree contains more or less minimal changes necessary to complete tests. It does not make use of transactional trees to enhance DDL operations, however.
Nathan Williams (nwilliams) wrote : | # |
Thanks for all the tweaks. Aside from the (possibly unrealistic) desire to unify the live-ness determination, this all looks good to me.
Have we ran the long stress suite or tpcc against this? I assume we'll want to before putting into trunk.
- 423. By Peter Beaman
-
Merge from trunk with changes for lock pruning fix
- 424. By Peter Beaman
-
Synchronize volume open
Peter Beaman (pbeaman) wrote : | # |
Stress tests have run four successful nightly iterations.
- 425. By Peter Beaman
-
Merge from trunk
Preview Diff
1 | === modified file 'pom.xml' | |||
2 | --- pom.xml 2013-04-10 20:09:48 +0000 | |||
3 | +++ pom.xml 2013-04-29 21:38:25 +0000 | |||
4 | @@ -4,7 +4,7 @@ | |||
5 | 4 | 4 | ||
6 | 5 | <groupId>com.akiban</groupId> | 5 | <groupId>com.akiban</groupId> |
7 | 6 | <artifactId>akiban-persistit</artifactId> | 6 | <artifactId>akiban-persistit</artifactId> |
9 | 7 | <version>3.2.9-SNAPSHOT</version> | 7 | <version>3.3-SNAPSHOT</version> |
10 | 8 | <packaging>jar</packaging> | 8 | <packaging>jar</packaging> |
11 | 9 | 9 | ||
12 | 10 | <parent> | 10 | <parent> |
13 | 11 | 11 | ||
14 | === modified file 'src/main/java/com/persistit/Buffer.java' | |||
15 | --- src/main/java/com/persistit/Buffer.java 2013-03-06 16:20:57 +0000 | |||
16 | +++ src/main/java/com/persistit/Buffer.java 2013-04-29 21:38:25 +0000 | |||
17 | @@ -1424,7 +1424,6 @@ | |||
18 | 1424 | if (Debug.ENABLED) { | 1424 | if (Debug.ENABLED) { |
19 | 1425 | assertVerify(); | 1425 | assertVerify(); |
20 | 1426 | } | 1426 | } |
21 | 1427 | |||
22 | 1428 | final boolean exactMatch = (foundAt & EXACT_MASK) > 0; | 1427 | final boolean exactMatch = (foundAt & EXACT_MASK) > 0; |
23 | 1429 | final int p = foundAt & P_MASK; | 1428 | final int p = foundAt & P_MASK; |
24 | 1430 | 1429 | ||
25 | @@ -3869,7 +3868,7 @@ | |||
26 | 3869 | * @return a human-readable inventory of the contents of this buffer | 3868 | * @return a human-readable inventory of the contents of this buffer |
27 | 3870 | */ | 3869 | */ |
28 | 3871 | public String toStringDetail() { | 3870 | public String toStringDetail() { |
30 | 3872 | return toStringDetail(-1, 42, 42, 0, true); | 3871 | return toStringDetail(-1, 42, 82, 0, true); |
31 | 3873 | } | 3872 | } |
32 | 3874 | 3873 | ||
33 | 3875 | /** | 3874 | /** |
34 | 3876 | 3875 | ||
35 | === modified file 'src/main/java/com/persistit/CheckpointManager.java' | |||
36 | --- src/main/java/com/persistit/CheckpointManager.java 2013-02-04 16:32:35 +0000 | |||
37 | +++ src/main/java/com/persistit/CheckpointManager.java 2013-04-29 21:38:25 +0000 | |||
38 | @@ -245,6 +245,7 @@ | |||
39 | 245 | final List<Accumulator> accumulators = _persistit.takeCheckpointAccumulators(txn.getStartTimestamp()); | 245 | final List<Accumulator> accumulators = _persistit.takeCheckpointAccumulators(txn.getStartTimestamp()); |
40 | 246 | _persistit.getTransactionIndex().checkpointAccumulatorSnapshots(txn.getStartTimestamp(), accumulators); | 246 | _persistit.getTransactionIndex().checkpointAccumulatorSnapshots(txn.getStartTimestamp(), accumulators); |
41 | 247 | Accumulator.saveAccumulatorCheckpointValues(accumulators); | 247 | Accumulator.saveAccumulatorCheckpointValues(accumulators); |
42 | 248 | _persistit.flushStatistics(); | ||
43 | 248 | txn.commit(CommitPolicy.HARD); | 249 | txn.commit(CommitPolicy.HARD); |
44 | 249 | _currentCheckpoint = new Checkpoint(txn.getStartTimestamp(), System.currentTimeMillis()); | 250 | _currentCheckpoint = new Checkpoint(txn.getStartTimestamp(), System.currentTimeMillis()); |
45 | 250 | _outstandingCheckpoints.add(_currentCheckpoint); | 251 | _outstandingCheckpoints.add(_currentCheckpoint); |
46 | 251 | 252 | ||
47 | === modified file 'src/main/java/com/persistit/ClassIndex.java' | |||
48 | --- src/main/java/com/persistit/ClassIndex.java 2012-08-24 13:57:19 +0000 | |||
49 | +++ src/main/java/com/persistit/ClassIndex.java 2013-04-29 21:38:25 +0000 | |||
50 | @@ -94,11 +94,21 @@ | |||
51 | 94 | * | 94 | * |
52 | 95 | * @param persistit | 95 | * @param persistit |
53 | 96 | * Owning Persistit instance. | 96 | * Owning Persistit instance. |
54 | 97 | * @throws PersistitException | ||
55 | 97 | */ | 98 | */ |
56 | 98 | ClassIndex(final Persistit persistit) { | 99 | ClassIndex(final Persistit persistit) { |
57 | 99 | _persistit = persistit; | 100 | _persistit = persistit; |
58 | 100 | } | 101 | } |
59 | 101 | 102 | ||
60 | 103 | void initialize() throws PersistitException { | ||
61 | 104 | /* | ||
62 | 105 | * Called during Persistit initialization. This has the desired | ||
63 | 106 | * side-effect of the class index tree outside of a transaction so that | ||
64 | 107 | * its existence is primordial. | ||
65 | 108 | */ | ||
66 | 109 | getExchange(); | ||
67 | 110 | } | ||
68 | 111 | |||
69 | 102 | /** | 112 | /** |
70 | 103 | * @return Number of <code>ClassInfo</code> objects currently stored in this | 113 | * @return Number of <code>ClassInfo</code> objects currently stored in this |
71 | 104 | * ClassIndex. | 114 | * ClassIndex. |
72 | @@ -178,8 +188,9 @@ | |||
73 | 178 | } catch (final PersistitException pe) { | 188 | } catch (final PersistitException pe) { |
74 | 179 | throw new ConversionException(pe); | 189 | throw new ConversionException(pe); |
75 | 180 | } finally { | 190 | } finally { |
77 | 181 | if (ex != null) | 191 | if (ex != null) { |
78 | 182 | releaseExchange(ex); | 192 | releaseExchange(ex); |
79 | 193 | } | ||
80 | 183 | } | 194 | } |
81 | 184 | } | 195 | } |
82 | 185 | } | 196 | } |
83 | 186 | 197 | ||
84 | === modified file 'src/main/java/com/persistit/Exchange.java' | |||
85 | --- src/main/java/com/persistit/Exchange.java 2013-03-23 21:59:48 +0000 | |||
86 | +++ src/main/java/com/persistit/Exchange.java 2013-04-29 21:38:25 +0000 | |||
87 | @@ -413,7 +413,6 @@ | |||
88 | 413 | } | 413 | } |
89 | 414 | 414 | ||
90 | 415 | void init(final Volume volume, final String treeName, final boolean create) throws PersistitException { | 415 | void init(final Volume volume, final String treeName, final boolean create) throws PersistitException { |
91 | 416 | assertCorrectThread(true); | ||
92 | 417 | if (volume == null) { | 416 | if (volume == null) { |
93 | 418 | throw new NullPointerException(); | 417 | throw new NullPointerException(); |
94 | 419 | } | 418 | } |
95 | @@ -441,6 +440,7 @@ | |||
96 | 441 | _tree = tree; | 440 | _tree = tree; |
97 | 442 | _treeHolder = new ReentrantResourceHolder(_tree); | 441 | _treeHolder = new ReentrantResourceHolder(_tree); |
98 | 443 | _cachedTreeGeneration = -1; | 442 | _cachedTreeGeneration = -1; |
99 | 443 | _isDirectoryExchange = tree == _volume.getDirectoryTree(); | ||
100 | 444 | initCache(); | 444 | initCache(); |
101 | 445 | } | 445 | } |
102 | 446 | _splitPolicy = _persistit.getDefaultSplitPolicy(); | 446 | _splitPolicy = _persistit.getDefaultSplitPolicy(); |
103 | @@ -503,7 +503,7 @@ | |||
104 | 503 | 503 | ||
105 | 504 | private void checkLevelCache() throws PersistitException { | 504 | private void checkLevelCache() throws PersistitException { |
106 | 505 | 505 | ||
108 | 506 | if (!_tree.isValid()) { | 506 | if (!_tree.isLive()) { |
109 | 507 | if (_tree.getVolume().isTemporary()) { | 507 | if (_tree.getVolume().isTemporary()) { |
110 | 508 | _tree = _tree.getVolume().getTree(_tree.getName(), true); | 508 | _tree = _tree.getVolume().getTree(_tree.getName(), true); |
111 | 509 | _treeHolder = new ReentrantResourceHolder(_tree); | 509 | _treeHolder = new ReentrantResourceHolder(_tree); |
112 | @@ -1356,11 +1356,7 @@ | |||
113 | 1356 | if (!isDirectoryExchange()) { | 1356 | if (!isDirectoryExchange()) { |
114 | 1357 | _persistit.checkSuspended(); | 1357 | _persistit.checkSuspended(); |
115 | 1358 | } | 1358 | } |
121 | 1359 | if (!_ignoreTransactions && !_transaction.isActive()) { | 1359 | throttle(); |
117 | 1360 | _persistit.getJournalManager().throttle(); | ||
118 | 1361 | } | ||
119 | 1362 | // TODO: directoryExchange, and lots of tests, don't use transactions. | ||
120 | 1363 | // Skip MVCC for now. | ||
122 | 1364 | int options = StoreOptions.WAIT; | 1360 | int options = StoreOptions.WAIT; |
123 | 1365 | options |= (!_ignoreTransactions && _transaction.isActive()) ? StoreOptions.MVCC : 0; | 1361 | options |= (!_ignoreTransactions && _transaction.isActive()) ? StoreOptions.MVCC : 0; |
124 | 1366 | storeInternal(key, value, 0, options); | 1362 | storeInternal(key, value, 0, options); |
125 | @@ -1551,7 +1547,13 @@ | |||
126 | 1551 | } | 1547 | } |
127 | 1552 | } | 1548 | } |
128 | 1553 | 1549 | ||
130 | 1554 | if (doMVCC) { | 1550 | /* |
131 | 1551 | * If the Tree is private to an active transaction, and | ||
132 | 1552 | * if this is a virgin value, then we can store it | ||
133 | 1553 | * primordially because if the transaction rolls back, | ||
134 | 1554 | * the entire Tree will be removed. | ||
135 | 1555 | */ | ||
136 | 1556 | if (doMVCC && (_spareValue.isDefined() || !_tree.isTransactionPrivate(true))) { | ||
137 | 1555 | valueToStore = spareValue; | 1557 | valueToStore = spareValue; |
138 | 1556 | final int valueSize = value.getEncodedSize(); | 1558 | final int valueSize = value.getEncodedSize(); |
139 | 1557 | int retries = VERSIONS_OUT_OF_ORDER_RETRY_COUNT; | 1559 | int retries = VERSIONS_OUT_OF_ORDER_RETRY_COUNT; |
140 | @@ -3188,7 +3190,6 @@ | |||
141 | 3188 | * @throws PersistitException | 3190 | * @throws PersistitException |
142 | 3189 | */ | 3191 | */ |
143 | 3190 | public boolean hasChildren() throws PersistitException { | 3192 | public boolean hasChildren() throws PersistitException { |
144 | 3191 | assertCorrectThread(true); | ||
145 | 3192 | _key.copyTo(_spareKey2); | 3193 | _key.copyTo(_spareKey2); |
146 | 3193 | final int size = _key.getEncodedSize(); | 3194 | final int size = _key.getEncodedSize(); |
147 | 3194 | final boolean result = traverse(GT, true, 0, _key.getDepth() + 1, size, null); | 3195 | final boolean result = traverse(GT, true, 0, _key.getDepth() + 1, size, null); |
148 | @@ -3228,30 +3229,16 @@ | |||
149 | 3228 | */ | 3229 | */ |
150 | 3229 | public void removeTree() throws PersistitException { | 3230 | public void removeTree() throws PersistitException { |
151 | 3230 | assertCorrectThread(true); | 3231 | assertCorrectThread(true); |
152 | 3232 | _persistit.checkSuspended(); | ||
153 | 3231 | _persistit.checkClosed(); | 3233 | _persistit.checkClosed(); |
154 | 3232 | 3234 | ||
165 | 3233 | final long timestamp = _persistit.getCurrentTimestamp(); | 3235 | _volume.getStructure().removeTree(_tree); |
156 | 3234 | for (int i = 0; i < 100; i++) { | ||
157 | 3235 | _persistit.checkClosed(); | ||
158 | 3236 | _persistit.checkSuspended(); | ||
159 | 3237 | _persistit.getJournalManager().pruneObsoleteTransactions(); | ||
160 | 3238 | if (_persistit.getJournalManager().getEarliestAbortedTransactionTimestamp() > timestamp) { | ||
161 | 3239 | break; | ||
162 | 3240 | } | ||
163 | 3241 | Util.sleep(1000); | ||
164 | 3242 | } | ||
166 | 3243 | if (!_ignoreTransactions) { | 3236 | if (!_ignoreTransactions) { |
167 | 3237 | assert !isDirectoryExchange(); | ||
168 | 3244 | _transaction.removeTree(this); | 3238 | _transaction.removeTree(this); |
169 | 3245 | } | 3239 | } |
173 | 3246 | 3240 | _key.clear(); | |
171 | 3247 | clear(); | ||
172 | 3248 | |||
174 | 3249 | _value.clear(); | 3241 | _value.clear(); |
175 | 3250 | /* | ||
176 | 3251 | * Remove from directory tree. | ||
177 | 3252 | */ | ||
178 | 3253 | _volume.getStructure().removeTree(_tree); | ||
179 | 3254 | |||
180 | 3255 | initCache(); | 3242 | initCache(); |
181 | 3256 | } | 3243 | } |
182 | 3257 | 3244 | ||
183 | @@ -3410,9 +3397,8 @@ | |||
184 | 3410 | if (!isDirectoryExchange()) { | 3397 | if (!isDirectoryExchange()) { |
185 | 3411 | _persistit.checkSuspended(); | 3398 | _persistit.checkSuspended(); |
186 | 3412 | } | 3399 | } |
190 | 3413 | if (!_ignoreTransactions && !_transaction.isActive()) { | 3400 | |
191 | 3414 | _persistit.getJournalManager().throttle(); | 3401 | throttle(); |
189 | 3415 | } | ||
192 | 3416 | 3402 | ||
193 | 3417 | if (_ignoreTransactions || !_transaction.isActive()) { | 3403 | if (_ignoreTransactions || !_transaction.isActive()) { |
194 | 3418 | return raw_removeKeyRangeInternal(key1, key2, fetchFirst, false); | 3404 | return raw_removeKeyRangeInternal(key1, key2, fetchFirst, false); |
195 | @@ -3422,6 +3408,15 @@ | |||
196 | 3422 | 3408 | ||
197 | 3423 | _transaction.remove(this, key1, key2); | 3409 | _transaction.remove(this, key1, key2); |
198 | 3424 | 3410 | ||
199 | 3411 | /* | ||
200 | 3412 | * If the Tree was created within this transaction then we can just | ||
201 | 3413 | * range-delete the tree since it is not visible outside this | ||
202 | 3414 | * transaction. | ||
203 | 3415 | */ | ||
204 | 3416 | if (_tree.isTransactionPrivate(true)) { | ||
205 | 3417 | return raw_removeKeyRangeInternal(key1, key2, fetchFirst, false); | ||
206 | 3418 | } | ||
207 | 3419 | |||
208 | 3425 | checkLevelCache(); | 3420 | checkLevelCache(); |
209 | 3426 | 3421 | ||
210 | 3427 | _value.clear().putAntiValueMVV(); | 3422 | _value.clear().putAntiValueMVV(); |
211 | @@ -3994,7 +3989,7 @@ | |||
212 | 3994 | 3989 | ||
213 | 3995 | boolean prune(final Key key) throws PersistitException { | 3990 | boolean prune(final Key key) throws PersistitException { |
214 | 3996 | Buffer buffer = null; | 3991 | Buffer buffer = null; |
216 | 3997 | Debug.$assert1.t(_tree.isValid()); | 3992 | Debug.$assert1.t(_tree.isLive()); |
217 | 3998 | try { | 3993 | try { |
218 | 3999 | search(key, true); | 3994 | search(key, true); |
219 | 4000 | buffer = _levelCache[0]._buffer; | 3995 | buffer = _levelCache[0]._buffer; |
220 | @@ -4014,7 +4009,7 @@ | |||
221 | 4014 | Buffer buffer = null; | 4009 | Buffer buffer = null; |
222 | 4015 | boolean pruned = false; | 4010 | boolean pruned = false; |
223 | 4016 | 4011 | ||
225 | 4017 | Debug.$assert1.t(_tree.isValid()); | 4012 | Debug.$assert1.t(_tree.isLive()); |
226 | 4018 | try { | 4013 | try { |
227 | 4019 | search(key1, true); | 4014 | search(key1, true); |
228 | 4020 | buffer = _levelCache[0]._buffer; | 4015 | buffer = _levelCache[0]._buffer; |
229 | @@ -4132,13 +4127,30 @@ | |||
230 | 4132 | assert checkThread(set) : "Thread " + Thread.currentThread() + " must not use " + this + " owned by " + _thread; | 4127 | assert checkThread(set) : "Thread " + Thread.currentThread() + " must not use " + this + " owned by " + _thread; |
231 | 4133 | } | 4128 | } |
232 | 4134 | 4129 | ||
233 | 4130 | /** | ||
234 | 4131 | * Ensure the this Exchange is compatible with the current Thread; if a | ||
235 | 4132 | * Thread was previously assigned then this thread must be the same one. | ||
236 | 4133 | * | ||
237 | 4134 | * @param set | ||
238 | 4135 | * whether to assign the current thread | ||
239 | 4136 | * @return true if and only if there was no assigned Thread or the assigned | ||
240 | 4137 | * Thread is same as the current Thread. | ||
241 | 4138 | */ | ||
242 | 4135 | private boolean checkThread(final boolean set) { | 4139 | private boolean checkThread(final boolean set) { |
243 | 4136 | final Thread t = Thread.currentThread(); | 4140 | final Thread t = Thread.currentThread(); |
249 | 4137 | final boolean okay = _thread == null || _thread == t; | 4141 | if (_thread == t) { |
250 | 4138 | if (okay) { | 4142 | if (!set) { |
251 | 4139 | _thread = set ? t : null; | 4143 | _thread = null; |
252 | 4140 | } | 4144 | } |
253 | 4141 | return okay; | 4145 | return true; |
254 | 4146 | } | ||
255 | 4147 | if (_thread == null) { | ||
256 | 4148 | if (set) { | ||
257 | 4149 | _thread = t; | ||
258 | 4150 | } | ||
259 | 4151 | return true; | ||
260 | 4152 | } | ||
261 | 4153 | return false; | ||
262 | 4142 | } | 4154 | } |
263 | 4143 | 4155 | ||
264 | 4144 | /** | 4156 | /** |
265 | @@ -4187,7 +4199,6 @@ | |||
266 | 4187 | * <code>false</code>. | 4199 | * <code>false</code>. |
267 | 4188 | */ | 4200 | */ |
268 | 4189 | boolean isDirectoryExchange() { | 4201 | boolean isDirectoryExchange() { |
269 | 4190 | assertCorrectThread(true); | ||
270 | 4191 | return _isDirectoryExchange; | 4202 | return _isDirectoryExchange; |
271 | 4192 | } | 4203 | } |
272 | 4193 | 4204 | ||
273 | @@ -4426,4 +4437,15 @@ | |||
274 | 4426 | _ignoreMVCCFetch = savedIgnore; | 4437 | _ignoreMVCCFetch = savedIgnore; |
275 | 4427 | } | 4438 | } |
276 | 4428 | } | 4439 | } |
277 | 4440 | |||
278 | 4441 | private void throttle() throws PersistitInterruptedException { | ||
279 | 4442 | /* | ||
280 | 4443 | * Don't throttle operations on the directory tree since that makes some | ||
281 | 4444 | * unit tests very slow. This test is now necessary because a directory | ||
282 | 4445 | * tree update can now occur within the scope of a transaction. | ||
283 | 4446 | */ | ||
284 | 4447 | if (!_ignoreTransactions && !_transaction.isActive() && !isDirectoryExchange()) { | ||
285 | 4448 | _persistit.getJournalManager().throttle(); | ||
286 | 4449 | } | ||
287 | 4450 | } | ||
288 | 4429 | } | 4451 | } |
289 | 4430 | 4452 | ||
290 | === modified file 'src/main/java/com/persistit/JournalManager.java' | |||
291 | --- src/main/java/com/persistit/JournalManager.java 2013-02-15 15:39:42 +0000 | |||
292 | +++ src/main/java/com/persistit/JournalManager.java 2013-04-29 21:38:25 +0000 | |||
293 | @@ -2862,6 +2862,11 @@ | |||
294 | 2862 | return false; | 2862 | return false; |
295 | 2863 | } | 2863 | } |
296 | 2864 | 2864 | ||
297 | 2865 | @Override | ||
298 | 2866 | public boolean createTree(final long timestamp) throws PersistitException { | ||
299 | 2867 | return false; | ||
300 | 2868 | } | ||
301 | 2869 | |||
302 | 2865 | } | 2870 | } |
303 | 2866 | 2871 | ||
304 | 2867 | /** | 2872 | /** |
305 | 2868 | 2873 | ||
306 | === modified file 'src/main/java/com/persistit/MVV.java' | |||
307 | --- src/main/java/com/persistit/MVV.java 2012-10-03 14:43:25 +0000 | |||
308 | +++ src/main/java/com/persistit/MVV.java 2013-04-29 21:38:25 +0000 | |||
309 | @@ -451,7 +451,7 @@ | |||
310 | 451 | if (tc == UNCOMMITTED) { | 451 | if (tc == UNCOMMITTED) { |
311 | 452 | final long ts = vh2ts(versionHandle); | 452 | final long ts = vh2ts(versionHandle); |
312 | 453 | if (uncommittedTransactionTs != 0 && uncommittedTransactionTs != ts) { | 453 | if (uncommittedTransactionTs != 0 && uncommittedTransactionTs != ts) { |
314 | 454 | throw new CorruptValueException("Multiple uncommitted version"); | 454 | throw new CorruptValueException("Multiple uncommitted versions"); |
315 | 455 | } | 455 | } |
316 | 456 | uncommittedTransactionTs = ts; | 456 | uncommittedTransactionTs = ts; |
317 | 457 | mark(bytes, from); | 457 | mark(bytes, from); |
318 | 458 | 458 | ||
319 | === modified file 'src/main/java/com/persistit/Persistit.java' | |||
320 | --- src/main/java/com/persistit/Persistit.java 2013-03-20 16:04:52 +0000 | |||
321 | +++ src/main/java/com/persistit/Persistit.java 2013-04-29 21:38:25 +0000 | |||
322 | @@ -29,6 +29,7 @@ | |||
323 | 29 | import java.lang.management.MemoryMXBean; | 29 | import java.lang.management.MemoryMXBean; |
324 | 30 | import java.lang.management.MemoryUsage; | 30 | import java.lang.management.MemoryUsage; |
325 | 31 | import java.lang.ref.SoftReference; | 31 | import java.lang.ref.SoftReference; |
326 | 32 | import java.lang.ref.WeakReference; | ||
327 | 32 | import java.rmi.RemoteException; | 33 | import java.rmi.RemoteException; |
328 | 33 | import java.util.ArrayList; | 34 | import java.util.ArrayList; |
329 | 34 | import java.util.Collections; | 35 | import java.util.Collections; |
330 | @@ -264,6 +265,8 @@ | |||
331 | 264 | 265 | ||
332 | 265 | private final Set<AccumulatorRef> _accumulators = new HashSet<AccumulatorRef>(); | 266 | private final Set<AccumulatorRef> _accumulators = new HashSet<AccumulatorRef>(); |
333 | 266 | 267 | ||
334 | 268 | private final Set<WeakReference<TimelyResource<?>>> _timelyResourceSet = new HashSet<WeakReference<TimelyResource<?>>>(); | ||
335 | 269 | |||
336 | 267 | private final WeakHashMap<SessionId, CLI> _cliSessionMap = new WeakHashMap<SessionId, CLI>(); | 270 | private final WeakHashMap<SessionId, CLI> _cliSessionMap = new WeakHashMap<SessionId, CLI>(); |
337 | 268 | 271 | ||
338 | 269 | private boolean _readRetryEnabled; | 272 | private boolean _readRetryEnabled; |
339 | @@ -440,6 +443,7 @@ | |||
340 | 440 | startJournal(); | 443 | startJournal(); |
341 | 441 | startBufferPools(); | 444 | startBufferPools(); |
342 | 442 | preloadBufferPools(); | 445 | preloadBufferPools(); |
343 | 446 | initializeClassIndex(); | ||
344 | 443 | finishRecovery(); | 447 | finishRecovery(); |
345 | 444 | startTransactionIndexPollTask(); | 448 | startTransactionIndexPollTask(); |
346 | 445 | flush(); | 449 | flush(); |
347 | @@ -672,6 +676,10 @@ | |||
348 | 672 | _enableBufferInventory.set(_configuration.isBufferInventoryEnabled()); | 676 | _enableBufferInventory.set(_configuration.isBufferInventoryEnabled()); |
349 | 673 | } | 677 | } |
350 | 674 | 678 | ||
351 | 679 | private void initializeClassIndex() throws PersistitException { | ||
352 | 680 | _classIndex.initialize(); | ||
353 | 681 | } | ||
354 | 682 | |||
355 | 675 | void startCheckpointManager() { | 683 | void startCheckpointManager() { |
356 | 676 | _checkpointManager.start(); | 684 | _checkpointManager.start(); |
357 | 677 | } | 685 | } |
358 | @@ -793,17 +801,21 @@ | |||
359 | 793 | } | 801 | } |
360 | 794 | } | 802 | } |
361 | 795 | 803 | ||
367 | 796 | synchronized void addVolume(final Volume volume) throws VolumeAlreadyExistsException { | 804 | void addVolume(final Volume volume) throws VolumeAlreadyExistsException { |
368 | 797 | Volume otherVolume; | 805 | synchronized (_volumes) { |
369 | 798 | otherVolume = getVolume(volume.getName()); | 806 | Volume otherVolume; |
370 | 799 | if (otherVolume != null) { | 807 | otherVolume = getVolume(volume.getName()); |
371 | 800 | throw new VolumeAlreadyExistsException("Volume " + otherVolume); | 808 | if (otherVolume != null) { |
372 | 809 | throw new VolumeAlreadyExistsException("Volume " + otherVolume); | ||
373 | 810 | } | ||
374 | 811 | _volumes.add(volume); | ||
375 | 801 | } | 812 | } |
376 | 802 | _volumes.add(volume); | ||
377 | 803 | } | 813 | } |
378 | 804 | 814 | ||
381 | 805 | synchronized void removeVolume(final Volume volume) throws PersistitInterruptedException { | 815 | void removeVolume(final Volume volume) throws PersistitInterruptedException { |
382 | 806 | _volumes.remove(volume); | 816 | synchronized (_volumes) { |
383 | 817 | _volumes.remove(volume); | ||
384 | 818 | } | ||
385 | 807 | } | 819 | } |
386 | 808 | 820 | ||
387 | 809 | /** | 821 | /** |
388 | @@ -965,7 +977,6 @@ | |||
389 | 965 | } | 977 | } |
390 | 966 | List<Exchange> stack; | 978 | List<Exchange> stack; |
391 | 967 | final SessionId sessionId = getSessionId(); | 979 | final SessionId sessionId = getSessionId(); |
392 | 968 | |||
393 | 969 | synchronized (_exchangePoolMap) { | 980 | synchronized (_exchangePoolMap) { |
394 | 970 | stack = _exchangePoolMap.get(sessionId); | 981 | stack = _exchangePoolMap.get(sessionId); |
395 | 971 | if (stack == null) { | 982 | if (stack == null) { |
396 | @@ -986,7 +997,9 @@ | |||
397 | 986 | * @return the List | 997 | * @return the List |
398 | 987 | */ | 998 | */ |
399 | 988 | public List<Volume> getVolumes() { | 999 | public List<Volume> getVolumes() { |
401 | 989 | return new ArrayList<Volume>(_volumes); | 1000 | synchronized (_volumes) { |
402 | 1001 | return new ArrayList<Volume>(_volumes); | ||
403 | 1002 | } | ||
404 | 990 | } | 1003 | } |
405 | 991 | 1004 | ||
406 | 992 | /** | 1005 | /** |
407 | @@ -1000,9 +1013,10 @@ | |||
408 | 1000 | * @return the List | 1013 | * @return the List |
409 | 1001 | * @throws PersistitException | 1014 | * @throws PersistitException |
410 | 1002 | */ | 1015 | */ |
412 | 1003 | public synchronized List<Tree> getSelectedTrees(final TreeSelector selector) throws PersistitException { | 1016 | public List<Tree> getSelectedTrees(final TreeSelector selector) throws PersistitException { |
413 | 1004 | final List<Tree> list = new ArrayList<Tree>(); | 1017 | final List<Tree> list = new ArrayList<Tree>(); |
415 | 1005 | for (final Volume volume : _volumes) { | 1018 | final List<Volume> volumes = getVolumes(); |
416 | 1019 | for (final Volume volume : volumes) { | ||
417 | 1006 | if (selector.isSelected(volume)) { | 1020 | if (selector.isSelected(volume)) { |
418 | 1007 | if (selector.isVolumeOnlySelection(volume.getName())) { | 1021 | if (selector.isVolumeOnlySelection(volume.getName())) { |
419 | 1008 | list.add(volume.getDirectoryTree()); | 1022 | list.add(volume.getDirectoryTree()); |
420 | @@ -1239,10 +1253,10 @@ | |||
421 | 1239 | if (name == null) { | 1253 | if (name == null) { |
422 | 1240 | throw new NullPointerException("Null volume name"); | 1254 | throw new NullPointerException("Null volume name"); |
423 | 1241 | } | 1255 | } |
424 | 1256 | final List<Volume> volumes = getVolumes(); | ||
425 | 1242 | Volume result = null; | 1257 | Volume result = null; |
429 | 1243 | 1258 | for (int i = 0; i < volumes.size(); i++) { | |
430 | 1244 | for (int i = 0; i < _volumes.size(); i++) { | 1259 | final Volume vol = volumes.get(i); |
428 | 1245 | final Volume vol = _volumes.get(i); | ||
431 | 1246 | if (name.equals(vol.getName())) { | 1260 | if (name.equals(vol.getName())) { |
432 | 1247 | if (result == null) | 1261 | if (result == null) |
433 | 1248 | result = vol; | 1262 | result = vol; |
434 | @@ -1256,8 +1270,8 @@ | |||
435 | 1256 | } | 1270 | } |
436 | 1257 | 1271 | ||
437 | 1258 | final File file = new File(name).getAbsoluteFile(); | 1272 | final File file = new File(name).getAbsoluteFile(); |
440 | 1259 | for (int i = 0; i < _volumes.size(); i++) { | 1273 | for (int i = 0; i < volumes.size(); i++) { |
441 | 1260 | final Volume vol = _volumes.get(i); | 1274 | final Volume vol = volumes.get(i); |
442 | 1261 | if (file.equals(vol.getAbsoluteFile())) { | 1275 | if (file.equals(vol.getAbsoluteFile())) { |
443 | 1262 | if (result == null) | 1276 | if (result == null) |
444 | 1263 | result = vol; | 1277 | result = vol; |
445 | @@ -1477,16 +1491,17 @@ | |||
446 | 1477 | */ | 1491 | */ |
447 | 1478 | private Volume getSpecialVolume(final String propName, final String dflt) throws VolumeNotFoundException { | 1492 | private Volume getSpecialVolume(final String propName, final String dflt) throws VolumeNotFoundException { |
448 | 1479 | final String volumeName = _configuration.getSysVolume(); | 1493 | final String volumeName = _configuration.getSysVolume(); |
452 | 1480 | 1494 | synchronized (_volumes) { | |
450 | 1481 | Volume volume = getVolume(volumeName); | ||
451 | 1482 | if (volume == null) { | ||
453 | 1483 | if ((_volumes.size() == 1) && (volumeName.equals(dflt))) { | 1495 | if ((_volumes.size() == 1) && (volumeName.equals(dflt))) { |
457 | 1484 | volume = _volumes.get(0); | 1496 | return _volumes.get(0); |
455 | 1485 | } else { | ||
456 | 1486 | throw new VolumeNotFoundException(volumeName); | ||
458 | 1487 | } | 1497 | } |
459 | 1488 | } | 1498 | } |
461 | 1489 | return volume; | 1499 | final Volume volume = getVolume(volumeName); |
462 | 1500 | if (volume == null) { | ||
463 | 1501 | throw new VolumeNotFoundException(volumeName); | ||
464 | 1502 | } else { | ||
465 | 1503 | return volume; | ||
466 | 1504 | } | ||
467 | 1490 | } | 1505 | } |
468 | 1491 | 1506 | ||
469 | 1492 | /** | 1507 | /** |
470 | @@ -1512,13 +1527,8 @@ | |||
471 | 1512 | */ | 1527 | */ |
472 | 1513 | void cleanup() { | 1528 | void cleanup() { |
473 | 1514 | closeZombieTransactions(false); | 1529 | closeZombieTransactions(false); |
481 | 1515 | final List<Volume> volumes; | 1530 | _transactionIndex.updateActiveTransactionCache(); |
482 | 1516 | synchronized (this) { | 1531 | pruneTimelyResources(); |
476 | 1517 | volumes = new ArrayList<Volume>(_volumes); | ||
477 | 1518 | } | ||
478 | 1519 | for (final Volume volume : volumes) { | ||
479 | 1520 | volume.getStructure().flushStatistics(); | ||
480 | 1521 | } | ||
483 | 1522 | } | 1532 | } |
484 | 1523 | 1533 | ||
485 | 1524 | /** | 1534 | /** |
486 | @@ -1653,7 +1663,6 @@ | |||
487 | 1653 | 1663 | ||
488 | 1654 | if (flush) { | 1664 | if (flush) { |
489 | 1655 | for (final Volume volume : volumes) { | 1665 | for (final Volume volume : volumes) { |
490 | 1656 | volume.getStructure().flushStatistics(); | ||
491 | 1657 | volume.getStorage().flush(); | 1666 | volume.getStorage().flush(); |
492 | 1658 | } | 1667 | } |
493 | 1659 | } | 1668 | } |
494 | @@ -1755,7 +1764,7 @@ | |||
495 | 1755 | // the volume files - otherwise there will be left over channels | 1764 | // the volume files - otherwise there will be left over channels |
496 | 1756 | // and FileLocks that interfere with subsequent tests. | 1765 | // and FileLocks that interfere with subsequent tests. |
497 | 1757 | // | 1766 | // |
499 | 1758 | final List<Volume> volumes = new ArrayList<Volume>(_volumes); | 1767 | final List<Volume> volumes = getVolumes(); |
500 | 1759 | for (final Volume volume : volumes) { | 1768 | for (final Volume volume : volumes) { |
501 | 1760 | try { | 1769 | try { |
502 | 1761 | volume.getStorage().close(); | 1770 | volume.getStorage().close(); |
503 | @@ -1824,8 +1833,11 @@ | |||
504 | 1824 | synchronized (_accumulators) { | 1833 | synchronized (_accumulators) { |
505 | 1825 | _accumulators.clear(); | 1834 | _accumulators.clear(); |
506 | 1826 | } | 1835 | } |
508 | 1827 | synchronized (this) { | 1836 | synchronized (_volumes) { |
509 | 1828 | _volumes.clear(); | 1837 | _volumes.clear(); |
510 | 1838 | } | ||
511 | 1839 | |||
512 | 1840 | synchronized (this) { | ||
513 | 1829 | _alertMonitors.clear(); | 1841 | _alertMonitors.clear(); |
514 | 1830 | _bufferPoolTable.clear(); | 1842 | _bufferPoolTable.clear(); |
515 | 1831 | _intArrayThreadLocal.set(null); | 1843 | _intArrayThreadLocal.set(null); |
516 | @@ -1867,8 +1879,8 @@ | |||
517 | 1867 | if (_closed.get() || !_initialized.get()) { | 1879 | if (_closed.get() || !_initialized.get()) { |
518 | 1868 | return false; | 1880 | return false; |
519 | 1869 | } | 1881 | } |
522 | 1870 | for (final Volume volume : _volumes) { | 1882 | final List<Volume> volumes = getVolumes(); |
523 | 1871 | volume.getStructure().flushStatistics(); | 1883 | for (final Volume volume : volumes) { |
524 | 1872 | volume.getStorage().flush(); | 1884 | volume.getStorage().flush(); |
525 | 1873 | volume.getStorage().force(); | 1885 | volume.getStorage().force(); |
526 | 1874 | } | 1886 | } |
527 | @@ -1894,6 +1906,13 @@ | |||
528 | 1894 | } | 1906 | } |
529 | 1895 | } | 1907 | } |
530 | 1896 | 1908 | ||
531 | 1909 | void flushStatistics() throws PersistitException { | ||
532 | 1910 | final List<Volume> volumes = getVolumes(); | ||
533 | 1911 | for (final Volume volume : volumes) { | ||
534 | 1912 | volume.getStructure().flushStatistics(); | ||
535 | 1913 | } | ||
536 | 1914 | } | ||
537 | 1915 | |||
538 | 1897 | void waitForIOTaskStop(final IOTaskRunnable task) { | 1916 | void waitForIOTaskStop(final IOTaskRunnable task) { |
539 | 1898 | if (_beginCloseTime == 0) { | 1917 | if (_beginCloseTime == 0) { |
540 | 1899 | _beginCloseTime = System.nanoTime(); | 1918 | _beginCloseTime = System.nanoTime(); |
541 | @@ -1926,7 +1945,7 @@ | |||
542 | 1926 | if (_closed.get() || !_initialized.get()) { | 1945 | if (_closed.get() || !_initialized.get()) { |
543 | 1927 | return; | 1946 | return; |
544 | 1928 | } | 1947 | } |
546 | 1929 | final ArrayList<Volume> volumes = _volumes; | 1948 | final List<Volume> volumes = getVolumes(); |
547 | 1930 | 1949 | ||
548 | 1931 | for (int index = 0; index < volumes.size(); index++) { | 1950 | for (int index = 0; index < volumes.size(); index++) { |
549 | 1932 | final Volume volume = volumes.get(index); | 1951 | final Volume volume = volumes.get(index); |
550 | @@ -2236,10 +2255,6 @@ | |||
551 | 2236 | return _transactionIndex; | 2255 | return _transactionIndex; |
552 | 2237 | } | 2256 | } |
553 | 2238 | 2257 | ||
554 | 2239 | public long getCheckpointIntervalNanos() { | ||
555 | 2240 | return _checkpointManager.getCheckpointIntervalNanos(); | ||
556 | 2241 | } | ||
557 | 2242 | |||
558 | 2243 | /** | 2258 | /** |
559 | 2244 | * Replaces the current logger implementation. | 2259 | * Replaces the current logger implementation. |
560 | 2245 | * | 2260 | * |
561 | @@ -2286,8 +2301,9 @@ | |||
562 | 2286 | */ | 2301 | */ |
563 | 2287 | public void checkAllVolumes() throws PersistitException { | 2302 | public void checkAllVolumes() throws PersistitException { |
564 | 2288 | final IntegrityCheck icheck = new IntegrityCheck(this); | 2303 | final IntegrityCheck icheck = new IntegrityCheck(this); |
567 | 2289 | for (int index = 0; index < _volumes.size(); index++) { | 2304 | final List<Volume> volumes = getVolumes(); |
568 | 2290 | final Volume volume = _volumes.get(index); | 2305 | for (int index = 0; index < volumes.size(); index++) { |
569 | 2306 | final Volume volume = volumes.get(index); | ||
570 | 2291 | System.out.println("Checking " + volume + " "); | 2307 | System.out.println("Checking " + volume + " "); |
571 | 2292 | try { | 2308 | try { |
572 | 2293 | icheck.checkVolume(volume); | 2309 | icheck.checkVolume(volume); |
573 | @@ -2403,6 +2419,12 @@ | |||
574 | 2403 | _suspendUpdates.set(suspended); | 2419 | _suspendUpdates.set(suspended); |
575 | 2404 | } | 2420 | } |
576 | 2405 | 2421 | ||
577 | 2422 | void addTimelyResource(final TimelyResource<? extends Version> resource) { | ||
578 | 2423 | synchronized (_timelyResourceSet) { | ||
579 | 2424 | _timelyResourceSet.add(new WeakReference<TimelyResource<? extends Version>>(resource)); | ||
580 | 2425 | } | ||
581 | 2426 | } | ||
582 | 2427 | |||
583 | 2406 | void addAccumulator(final Accumulator accumulator) throws PersistitException { | 2428 | void addAccumulator(final Accumulator accumulator) throws PersistitException { |
584 | 2407 | int checkpointCount = 0; | 2429 | int checkpointCount = 0; |
585 | 2408 | synchronized (_accumulators) { | 2430 | synchronized (_accumulators) { |
586 | @@ -2462,6 +2484,34 @@ | |||
587 | 2462 | return result; | 2484 | return result; |
588 | 2463 | } | 2485 | } |
589 | 2464 | 2486 | ||
590 | 2487 | void pruneTimelyResources() { | ||
591 | 2488 | final List<TimelyResource<?>> resourcesToPrune = new ArrayList<TimelyResource<?>>(); | ||
592 | 2489 | synchronized (_timelyResourceSet) { | ||
593 | 2490 | for (final Iterator<WeakReference<TimelyResource<?>>> iter = _timelyResourceSet.iterator(); iter.hasNext();) { | ||
594 | 2491 | final WeakReference<TimelyResource<?>> ref = iter.next(); | ||
595 | 2492 | final TimelyResource<?> resource = ref.get(); | ||
596 | 2493 | if (resource != null) { | ||
597 | 2494 | resourcesToPrune.add(resource); | ||
598 | 2495 | } | ||
599 | 2496 | } | ||
600 | 2497 | } | ||
601 | 2498 | for (final TimelyResource<?> resource : resourcesToPrune) { | ||
602 | 2499 | try { | ||
603 | 2500 | resource.prune(); | ||
604 | 2501 | } catch (final PersistitException e) { | ||
605 | 2502 | _logBase.timelyResourcePruneException.log(e, resource); | ||
606 | 2503 | } | ||
607 | 2504 | } | ||
608 | 2505 | synchronized (_timelyResourceSet) { | ||
609 | 2506 | for (final Iterator<WeakReference<TimelyResource<?>>> iter = _timelyResourceSet.iterator(); iter.hasNext();) { | ||
610 | 2507 | final WeakReference<TimelyResource<?>> ref = iter.next(); | ||
611 | 2508 | if (ref.get() == null) { | ||
612 | 2509 | iter.remove(); | ||
613 | 2510 | } | ||
614 | 2511 | } | ||
615 | 2512 | } | ||
616 | 2513 | } | ||
617 | 2514 | |||
618 | 2465 | synchronized CLI getSessionCLI() { | 2515 | synchronized CLI getSessionCLI() { |
619 | 2466 | CLI cli = _cliSessionMap.get(getSessionId()); | 2516 | CLI cli = _cliSessionMap.get(getSessionId()); |
620 | 2467 | if (cli == null) { | 2517 | if (cli == null) { |
621 | 2468 | 2518 | ||
622 | === modified file 'src/main/java/com/persistit/RecoveryManager.java' | |||
623 | --- src/main/java/com/persistit/RecoveryManager.java 2013-02-15 15:39:42 +0000 | |||
624 | +++ src/main/java/com/persistit/RecoveryManager.java 2013-04-29 21:38:25 +0000 | |||
625 | @@ -224,14 +224,32 @@ | |||
626 | 224 | private final TransactionPlayer _player = new TransactionPlayer(new RecoveryTransactionPlayerSupport()); | 224 | private final TransactionPlayer _player = new TransactionPlayer(new RecoveryTransactionPlayerSupport()); |
627 | 225 | 225 | ||
628 | 226 | static class DefaultRecoveryListener implements TransactionPlayerListener { | 226 | static class DefaultRecoveryListener implements TransactionPlayerListener { |
629 | 227 | |||
630 | 227 | @Override | 228 | @Override |
631 | 228 | public void store(final long address, final long timestamp, final Exchange exchange) throws PersistitException { | 229 | public void store(final long address, final long timestamp, final Exchange exchange) throws PersistitException { |
632 | 230 | if (exchange.isDirectoryExchange() && exchange.getValue().isDefined() | ||
633 | 231 | && exchange.getValue().getTypeHandle() == Value.CLASS_TREE) { | ||
634 | 232 | /* | ||
635 | 233 | * Don't recover tree structure updates within transactions | ||
636 | 234 | * because the allocation of root pages is not transactional. | ||
637 | 235 | * The intent of the change is conveyed by the implicit creation | ||
638 | 236 | * of new trees and explicit remove tree records. | ||
639 | 237 | */ | ||
640 | 238 | return; | ||
641 | 239 | } | ||
642 | 229 | exchange.store(); | 240 | exchange.store(); |
643 | 230 | } | 241 | } |
644 | 231 | 242 | ||
645 | 232 | @Override | 243 | @Override |
646 | 233 | public void removeKeyRange(final long address, final long timestamp, final Exchange exchange, final Key from, | 244 | public void removeKeyRange(final long address, final long timestamp, final Exchange exchange, final Key from, |
647 | 234 | final Key to) throws PersistitException { | 245 | final Key to) throws PersistitException { |
648 | 246 | if (exchange.isDirectoryExchange()) { | ||
649 | 247 | /* | ||
650 | 248 | * Don't recover directory tree removes because they are implied | ||
651 | 249 | * by Remove Tree records in the journal. | ||
652 | 250 | */ | ||
653 | 251 | return; | ||
654 | 252 | } | ||
655 | 235 | exchange.raw_removeKeyRangeInternal(from, to, false, false); | 253 | exchange.raw_removeKeyRangeInternal(from, to, false, false); |
656 | 236 | } | 254 | } |
657 | 237 | 255 | ||
658 | @@ -275,6 +293,10 @@ | |||
659 | 275 | return true; | 293 | return true; |
660 | 276 | } | 294 | } |
661 | 277 | 295 | ||
662 | 296 | @Override | ||
663 | 297 | public boolean createTree(final long timestamp) throws PersistitException { | ||
664 | 298 | return true; | ||
665 | 299 | } | ||
666 | 278 | } | 300 | } |
667 | 279 | 301 | ||
668 | 280 | class DefaultRollbackListener implements TransactionPlayerListener { | 302 | class DefaultRollbackListener implements TransactionPlayerListener { |
669 | @@ -336,6 +358,11 @@ | |||
670 | 336 | return false; | 358 | return false; |
671 | 337 | } | 359 | } |
672 | 338 | 360 | ||
673 | 361 | @Override | ||
674 | 362 | public boolean createTree(final long timestamp) throws PersistitException { | ||
675 | 363 | return false; | ||
676 | 364 | } | ||
677 | 365 | |||
678 | 339 | } | 366 | } |
679 | 340 | 367 | ||
680 | 341 | private class RecoveryTransactionPlayerSupport implements TransactionPlayerSupport { | 368 | private class RecoveryTransactionPlayerSupport implements TransactionPlayerSupport { |
681 | 342 | 369 | ||
682 | === modified file 'src/main/java/com/persistit/SharedResource.java' | |||
683 | --- src/main/java/com/persistit/SharedResource.java 2013-04-10 20:09:48 +0000 | |||
684 | +++ src/main/java/com/persistit/SharedResource.java 2013-04-29 21:38:25 +0000 | |||
685 | @@ -38,7 +38,7 @@ | |||
686 | 38 | * | 38 | * |
687 | 39 | * @author peter | 39 | * @author peter |
688 | 40 | */ | 40 | */ |
690 | 41 | class SharedResource { | 41 | abstract class SharedResource { |
691 | 42 | 42 | ||
692 | 43 | /** | 43 | /** |
693 | 44 | * Default maximum time to wait for access to this resource. Methods throw | 44 | * Default maximum time to wait for access to this resource. Methods throw |
694 | @@ -260,10 +260,10 @@ | |||
695 | 260 | /** | 260 | /** |
696 | 261 | * A counter that increments every time the resource is changed. | 261 | * A counter that increments every time the resource is changed. |
697 | 262 | */ | 262 | */ |
699 | 263 | protected AtomicLong _generation = new AtomicLong(); | 263 | private final AtomicLong _generation = new AtomicLong(); |
700 | 264 | 264 | ||
701 | 265 | protected SharedResource(final Persistit persistit) { | 265 | protected SharedResource(final Persistit persistit) { |
703 | 266 | this._persistit = persistit; | 266 | _persistit = persistit; |
704 | 267 | } | 267 | } |
705 | 268 | 268 | ||
706 | 269 | public boolean isAvailable(final boolean writer) { | 269 | public boolean isAvailable(final boolean writer) { |
707 | 270 | 270 | ||
708 | === added file 'src/main/java/com/persistit/TimelyResource.java' | |||
709 | --- src/main/java/com/persistit/TimelyResource.java 1970-01-01 00:00:00 +0000 | |||
710 | +++ src/main/java/com/persistit/TimelyResource.java 2013-04-29 21:38:25 +0000 | |||
711 | @@ -0,0 +1,505 @@ | |||
712 | 1 | /** | ||
713 | 2 | * Copyright © 2012 Akiban Technologies, Inc. All rights reserved. | ||
714 | 3 | * | ||
715 | 4 | * This program and the accompanying materials are made available | ||
716 | 5 | * under the terms of the Eclipse Public License v1.0 which | ||
717 | 6 | * accompanies this distribution, and is available at | ||
718 | 7 | * http://www.eclipse.org/legal/epl-v10.html | ||
719 | 8 | * | ||
720 | 9 | * This program may also be available under different license terms. | ||
721 | 10 | * For more information, see www.akiban.com or contact licensing@akiban.com. | ||
722 | 11 | * | ||
723 | 12 | * Contributors: | ||
724 | 13 | * Akiban Technologies, Inc. | ||
725 | 14 | */ | ||
726 | 15 | |||
727 | 16 | package com.persistit; | ||
728 | 17 | |||
729 | 18 | import static com.persistit.TransactionIndex.tss2vh; | ||
730 | 19 | import static com.persistit.TransactionIndex.vh2step; | ||
731 | 20 | import static com.persistit.TransactionIndex.vh2ts; | ||
732 | 21 | import static com.persistit.TransactionStatus.ABORTED; | ||
733 | 22 | import static com.persistit.TransactionStatus.PRIMORDIAL; | ||
734 | 23 | import static com.persistit.TransactionStatus.TIMED_OUT; | ||
735 | 24 | import static com.persistit.TransactionStatus.UNCOMMITTED; | ||
736 | 25 | |||
737 | 26 | import java.util.ArrayList; | ||
738 | 27 | import java.util.List; | ||
739 | 28 | |||
740 | 29 | import com.persistit.Version.PrunableVersion; | ||
741 | 30 | import com.persistit.Version.VersionCreator; | ||
742 | 31 | import com.persistit.exception.PersistitException; | ||
743 | 32 | import com.persistit.exception.PersistitInterruptedException; | ||
744 | 33 | import com.persistit.exception.RollbackException; | ||
745 | 34 | import com.persistit.exception.TimeoutException; | ||
746 | 35 | import com.persistit.exception.WWRetryException; | ||
747 | 36 | |||
748 | 37 | /** | ||
749 | 38 | * <p> | ||
750 | 39 | * Transactionally manage multiple versions of a resource. For example, a | ||
751 | 40 | * resource which caches state created and either committed or rolled back by a | ||
752 | 41 | * transaction may be use a TimelyResource to hold its versions. Each version | ||
753 | 42 | * must implement the {@link Version} interface and may optionally implement | ||
754 | 43 | * {@link PrunableVersion} in which case its {@link PrunableVersion#prune()} | ||
755 | 44 | * method is called when <code>TimelyResource</code> detects that the version is | ||
756 | 45 | * obsolete. | ||
757 | 46 | * </p> | ||
758 | 47 | * <p> | ||
759 | 48 | * The method {@link #addVersion(Version, Transaction)} attempts to add a new | ||
760 | 49 | * version on behalf of the supplied transaction. | ||
761 | 50 | * </p> | ||
762 | 51 | * <p> | ||
763 | 52 | * The method {@link #getVersion()} returns the snapshot version associated with | ||
764 | 53 | * the transaction, in other words, an instance of a P which was either | ||
765 | 54 | * committed last before this transaction started, or which was created by this | ||
766 | 55 | * transaction. If there is no such version the method returns <code>null</code> | ||
767 | 56 | * . | ||
768 | 57 | * </p> | ||
769 | 58 | * <p> | ||
770 | 59 | * The method {@link #getVersion(com.persistit.Version.VersionCreator)} does the | ||
771 | 60 | * same thing, except that if there is no snapshot version the | ||
772 | 61 | * {@link VersionCreator#createVersion(TimelyResource)} method is called to | ||
773 | 62 | * construct a new version. | ||
774 | 63 | * </p> | ||
775 | 64 | * | ||
776 | 65 | * @author peter | ||
777 | 66 | * | ||
778 | 67 | * @param <V> | ||
779 | 68 | * specific type of {@link Version} this <code>TimelyResource</code> | ||
780 | 69 | * manages | ||
781 | 70 | */ | ||
782 | 71 | public class TimelyResource<V extends Version> { | ||
783 | 72 | |||
784 | 73 | private final Persistit _persistit; | ||
785 | 74 | private volatile Entry _first; | ||
786 | 75 | |||
787 | 76 | public TimelyResource(final Persistit persistit) { | ||
788 | 77 | _persistit = persistit; | ||
789 | 78 | _persistit.addTimelyResource(this); | ||
790 | 79 | } | ||
791 | 80 | |||
792 | 81 | private long tss2v(final Transaction txn) { | ||
793 | 82 | if (txn == null || !txn.isActive()) { | ||
794 | 83 | return tss2vh(_persistit.getTimestampAllocator().updateTimestamp(), 0); | ||
795 | 84 | } else { | ||
796 | 85 | return tss2vh(txn.getStartTimestamp(), txn.getStep()); | ||
797 | 86 | } | ||
798 | 87 | } | ||
799 | 88 | |||
800 | 89 | public synchronized void delete() throws RollbackException, PersistitException { | ||
801 | 90 | if (_first == null) { | ||
802 | 91 | throw new IllegalStateException("There is no resource to delete"); | ||
803 | 92 | } | ||
804 | 93 | if (!_first.isDeleted()) { | ||
805 | 94 | final Transaction txn = _persistit.getTransaction(); | ||
806 | 95 | final long version = tss2v(txn); | ||
807 | 96 | if (version == _first.getVersion()) { | ||
808 | 97 | _first.setDeleted(); | ||
809 | 98 | } else { | ||
810 | 99 | final V resource = _first.getResource(); | ||
811 | 100 | final Entry entry = new Entry(version, resource); | ||
812 | 101 | entry.setDeleted(); | ||
813 | 102 | addVersion(entry, txn); | ||
814 | 103 | } | ||
815 | 104 | } | ||
816 | 105 | } | ||
817 | 106 | |||
818 | 107 | public void addVersion(final V resource, final Transaction txn) throws PersistitInterruptedException, | ||
819 | 108 | RollbackException { | ||
820 | 109 | if (resource == null) { | ||
821 | 110 | throw new NullPointerException("Null resource"); | ||
822 | 111 | } | ||
823 | 112 | addVersion(new Entry(tss2v(txn), resource), txn); | ||
824 | 113 | } | ||
825 | 114 | |||
826 | 115 | public V getVersion() throws TimeoutException, PersistitInterruptedException { | ||
827 | 116 | final Entry first = _first; | ||
828 | 117 | if (first != null && first.getVersion() == PRIMORDIAL) { | ||
829 | 118 | return first.getResource(); | ||
830 | 119 | } | ||
831 | 120 | final Transaction txn = _persistit.getTransaction(); | ||
832 | 121 | return getVersion(tss2v(txn)); | ||
833 | 122 | } | ||
834 | 123 | |||
835 | 124 | public V getVersion(final VersionCreator<V> creator) throws PersistitException, RollbackException { | ||
836 | 125 | final Entry first = _first; | ||
837 | 126 | if (first != null && first.getVersion() == PRIMORDIAL) { | ||
838 | 127 | return first.getResource(); | ||
839 | 128 | } | ||
840 | 129 | final Transaction txn = _persistit.getTransaction(); | ||
841 | 130 | V version = getVersion(tss2v(txn)); | ||
842 | 131 | if (version == null) { | ||
843 | 132 | version = creator.createVersion(this); | ||
844 | 133 | addVersion(version, txn); | ||
845 | 134 | } | ||
846 | 135 | return version; | ||
847 | 136 | } | ||
848 | 137 | |||
849 | 138 | /** | ||
850 | 139 | * @return <code>true</code> if and only if this <code>TimelyResource</code> | ||
851 | 140 | * has no <code>Version</code> instances. | ||
852 | 141 | */ | ||
853 | 142 | public boolean isEmpty() throws TimeoutException, PersistitInterruptedException { | ||
854 | 143 | Entry first = _first; | ||
855 | 144 | if (first == null) { | ||
856 | 145 | return true; | ||
857 | 146 | } | ||
858 | 147 | if (first.getVersion() == PRIMORDIAL) { | ||
859 | 148 | return false; | ||
860 | 149 | } | ||
861 | 150 | first = getEntry(tss2v(_persistit.getTransaction())); | ||
862 | 151 | if (first == null) { | ||
863 | 152 | return true; | ||
864 | 153 | } | ||
865 | 154 | if (first.isDeleted()) { | ||
866 | 155 | return true; | ||
867 | 156 | } | ||
868 | 157 | return false; | ||
869 | 158 | } | ||
870 | 159 | |||
871 | 160 | /** | ||
872 | 161 | * | ||
873 | 162 | * @return Whether this resource exists only within the context of the | ||
874 | 163 | * current transaction. | ||
875 | 164 | * @throws TimeoutException | ||
876 | 165 | * @throws PersistitInterruptedException | ||
877 | 166 | */ | ||
878 | 167 | |||
879 | 168 | public boolean isTransactionPrivate(final boolean byStep) throws TimeoutException, PersistitInterruptedException { | ||
880 | 169 | Entry entry = _first; | ||
881 | 170 | if (entry != null && entry.getVersion() == PRIMORDIAL) { | ||
882 | 171 | return false; | ||
883 | 172 | } | ||
884 | 173 | final Transaction txn = _persistit.getTransaction(); | ||
885 | 174 | final long versionHandle = tss2v(txn); | ||
886 | 175 | entry = getEntry(versionHandle); | ||
887 | 176 | if (entry == null) { | ||
888 | 177 | return true; | ||
889 | 178 | } else { | ||
890 | 179 | if (byStep) { | ||
891 | 180 | return entry.getVersion() == versionHandle; | ||
892 | 181 | } else { | ||
893 | 182 | return vh2ts(entry.getVersion()) == vh2ts(versionHandle); | ||
894 | 183 | } | ||
895 | 184 | } | ||
896 | 185 | } | ||
897 | 186 | |||
898 | 187 | /** | ||
899 | 188 | * @return Count of versions currently being managed. | ||
900 | 189 | */ | ||
901 | 190 | int getVersionCount() { | ||
902 | 191 | int count = 0; | ||
903 | 192 | for (Entry e = _first; e != null; e = e._previous) { | ||
904 | 193 | count++; | ||
905 | 194 | } | ||
906 | 195 | return count; | ||
907 | 196 | } | ||
908 | 197 | |||
909 | 198 | @Override | ||
910 | 199 | public String toString() { | ||
911 | 200 | final StringBuilder sb = new StringBuilder("TimelyResource("); | ||
912 | 201 | boolean first = true; | ||
913 | 202 | for (Entry entry = _first; entry != null; entry = entry.getPrevious()) { | ||
914 | 203 | if (sb.length() > 1000) { | ||
915 | 204 | sb.append("..."); | ||
916 | 205 | break; | ||
917 | 206 | } | ||
918 | 207 | if (!first) { | ||
919 | 208 | sb.append(','); | ||
920 | 209 | } else { | ||
921 | 210 | first = false; | ||
922 | 211 | } | ||
923 | 212 | sb.append(entry); | ||
924 | 213 | } | ||
925 | 214 | sb.append(')'); | ||
926 | 215 | return sb.toString(); | ||
927 | 216 | } | ||
928 | 217 | |||
929 | 218 | synchronized void setPrimordial() { | ||
930 | 219 | if (_first != null && _first.getPrevious() == null) { | ||
931 | 220 | _first.setPrimordial(); | ||
932 | 221 | } else { | ||
933 | 222 | throw new IllegalStateException("Cannot be made primordial: " + this); | ||
934 | 223 | } | ||
935 | 224 | } | ||
936 | 225 | |||
937 | 226 | /** | ||
938 | 227 | * Remove all obsolete <code>Version</code> instances. For each instance | ||
939 | 228 | * that implements <code>PrunableVersion</code>, invoke its | ||
940 | 229 | * {@link PrunableVersion#prune()} method. | ||
941 | 230 | * | ||
942 | 231 | * @throws TimeoutException | ||
943 | 232 | * @throws PersistitException | ||
944 | 233 | */ | ||
945 | 234 | void prune() throws TimeoutException, PersistitException { | ||
946 | 235 | final List<Entry> entriesToPrune = new ArrayList<Entry>(); | ||
947 | 236 | PrunableVersion versionToVacate = null; | ||
948 | 237 | |||
949 | 238 | final TransactionIndex ti = _persistit.getTransactionIndex(); | ||
950 | 239 | |||
951 | 240 | synchronized (this) { | ||
952 | 241 | try { | ||
953 | 242 | |||
954 | 243 | Entry newer = null; | ||
955 | 244 | Entry latest = null; | ||
956 | 245 | boolean isPrimordial = true; | ||
957 | 246 | long lastCommit = UNCOMMITTED; | ||
958 | 247 | |||
959 | 248 | for (Entry entry = _first; entry != null; entry = entry.getPrevious()) { | ||
960 | 249 | boolean keepIt = false; | ||
961 | 250 | final long versionHandle = entry.getVersion(); | ||
962 | 251 | final long tc = ti.commitStatus(versionHandle, UNCOMMITTED, 0); | ||
963 | 252 | if (tc >= PRIMORDIAL) { | ||
964 | 253 | if (tc == UNCOMMITTED) { | ||
965 | 254 | keepIt = true; | ||
966 | 255 | isPrimordial = false; | ||
967 | 256 | } else { | ||
968 | 257 | final boolean hasConcurrent = ti.hasConcurrentTransaction(tc, lastCommit); | ||
969 | 258 | if (latest == null || hasConcurrent) { | ||
970 | 259 | keepIt = true; | ||
971 | 260 | if (latest == null) { | ||
972 | 261 | latest = entry; | ||
973 | 262 | } | ||
974 | 263 | } | ||
975 | 264 | if (keepIt && ti.hasConcurrentTransaction(0, tc)) { | ||
976 | 265 | isPrimordial = false; | ||
977 | 266 | } | ||
978 | 267 | } | ||
979 | 268 | lastCommit = tc; | ||
980 | 269 | } else { | ||
981 | 270 | assert tc == ABORTED; | ||
982 | 271 | } | ||
983 | 272 | if (keepIt) { | ||
984 | 273 | newer = entry; | ||
985 | 274 | } else { | ||
986 | 275 | if (tc == ABORTED ^ entry.isDeleted()) { | ||
987 | 276 | entriesToPrune.add(entry); | ||
988 | 277 | } | ||
989 | 278 | if (newer == null) { | ||
990 | 279 | _first = entry.getPrevious(); | ||
991 | 280 | } else { | ||
992 | 281 | newer.setPrevious(entry.getPrevious()); | ||
993 | 282 | } | ||
994 | 283 | } | ||
995 | 284 | } | ||
996 | 285 | if (isPrimordial && _first != null) { | ||
997 | 286 | assert _first.getPrevious() == null; | ||
998 | 287 | if (_first.isDeleted()) { | ||
999 | 288 | final V version = _first.getResource(); | ||
1000 | 289 | if (version instanceof PrunableVersion) { | ||
1001 | 290 | versionToVacate = (PrunableVersion) version; | ||
1002 | 291 | } | ||
1003 | 292 | entriesToPrune.add(_first); | ||
1004 | 293 | _first = null; | ||
1005 | 294 | } else { | ||
1006 | 295 | _first.setPrimordial(); | ||
1007 | 296 | } | ||
1008 | 297 | } | ||
1009 | 298 | } catch (final InterruptedException ie) { | ||
1010 | 299 | throw new PersistitInterruptedException(ie); | ||
1011 | 300 | } | ||
1012 | 301 | } | ||
1013 | 302 | for (final Entry e : entriesToPrune) { | ||
1014 | 303 | if (versionToVacate != null) { | ||
1015 | 304 | versionToVacate.vacate(); | ||
1016 | 305 | } | ||
1017 | 306 | e.prune(); | ||
1018 | 307 | } | ||
1019 | 308 | } | ||
1020 | 309 | |||
1021 | 310 | /** | ||
1022 | 311 | * Helper method that adds an Entry containing a Version and its timestamp | ||
1023 | 312 | * information. This method checks for write-write dependencies throws a | ||
1024 | 313 | * RollbackException if there is a conflict. | ||
1025 | 314 | * | ||
1026 | 315 | * @param entry | ||
1027 | 316 | * @param txn | ||
1028 | 317 | * @throws PersistitException | ||
1029 | 318 | * @throws RollbackException | ||
1030 | 319 | */ | ||
1031 | 320 | private void addVersion(final Entry entry, final Transaction txn) throws PersistitInterruptedException, | ||
1032 | 321 | RollbackException { | ||
1033 | 322 | final TransactionIndex ti = _persistit.getTransactionIndex(); | ||
1034 | 323 | while (true) { | ||
1035 | 324 | try { | ||
1036 | 325 | synchronized (this) { | ||
1037 | 326 | if (_first != null) { | ||
1038 | 327 | if (_first.getVersion() > entry.getVersion()) { | ||
1039 | 328 | /* | ||
1040 | 329 | * This thread lost a race to make the most recent | ||
1041 | 330 | * version | ||
1042 | 331 | */ | ||
1043 | 332 | throw new RollbackException(); | ||
1044 | 333 | } | ||
1045 | 334 | if (txn.isActive()) { | ||
1046 | 335 | for (Entry e = _first; e != null; e = e.getPrevious()) { | ||
1047 | 336 | final long version = e.getVersion(); | ||
1048 | 337 | final long depends = ti.wwDependency(version, txn.getTransactionStatus(), 0); | ||
1049 | 338 | if (depends == TIMED_OUT) { | ||
1050 | 339 | throw new WWRetryException(version); | ||
1051 | 340 | } | ||
1052 | 341 | if (depends != 0 && depends != ABORTED) { | ||
1053 | 342 | /* | ||
1054 | 343 | * version is from a concurrent transaction | ||
1055 | 344 | * that already committed or timed out | ||
1056 | 345 | * waiting to see. Either way, must abort. | ||
1057 | 346 | */ | ||
1058 | 347 | throw new RollbackException(); | ||
1059 | 348 | } | ||
1060 | 349 | } | ||
1061 | 350 | } | ||
1062 | 351 | } | ||
1063 | 352 | entry.setPrevious(_first); | ||
1064 | 353 | _first = entry; | ||
1065 | 354 | break; | ||
1066 | 355 | } | ||
1067 | 356 | } catch (final WWRetryException re) { | ||
1068 | 357 | try { | ||
1069 | 358 | final long depends = _persistit.getTransactionIndex().wwDependency(re.getVersionHandle(), | ||
1070 | 359 | txn.getTransactionStatus(), SharedResource.DEFAULT_MAX_WAIT_TIME); | ||
1071 | 360 | if (depends != 0 && depends != ABORTED) { | ||
1072 | 361 | /* | ||
1073 | 362 | * version is from concurrent txn that already committed | ||
1074 | 363 | * or timed out waiting to see. Either way, must abort. | ||
1075 | 364 | */ | ||
1076 | 365 | throw new RollbackException(); | ||
1077 | 366 | } | ||
1078 | 367 | } catch (final InterruptedException ie) { | ||
1079 | 368 | throw new PersistitInterruptedException(ie); | ||
1080 | 369 | } | ||
1081 | 370 | } catch (final InterruptedException ie) { | ||
1082 | 371 | throw new PersistitInterruptedException(ie); | ||
1083 | 372 | } | ||
1084 | 373 | } | ||
1085 | 374 | } | ||
1086 | 375 | |||
1087 | 376 | /** | ||
1088 | 377 | * Get the <code>Version</code> from the snapshot view specified by the | ||
1089 | 378 | * supplied version handle. | ||
1090 | 379 | * | ||
1091 | 380 | * @param version | ||
1092 | 381 | * versionHandle | ||
1093 | 382 | * @return <code>Version</code> for given version handle | ||
1094 | 383 | * @throws TimeoutException | ||
1095 | 384 | * @throws PersistitInterruptedException | ||
1096 | 385 | */ | ||
1097 | 386 | V getVersion(final long version) throws TimeoutException, PersistitInterruptedException { | ||
1098 | 387 | final Entry e = getEntry(version); | ||
1099 | 388 | return e == null ? null : e.getResource(); | ||
1100 | 389 | } | ||
1101 | 390 | |||
1102 | 391 | Entry getEntry(final long version) throws TimeoutException, PersistitInterruptedException { | ||
1103 | 392 | final TransactionIndex ti = _persistit.getTransactionIndex(); | ||
1104 | 393 | try { | ||
1105 | 394 | /* | ||
1106 | 395 | * Note: not necessary to synchronize here. A concurrent transaction | ||
1107 | 396 | * may modify _first, but this method does not need to see that | ||
1108 | 397 | * version since it has not been committed. Conversely, if there is | ||
1109 | 398 | * some transaction that committed before this transaction's start | ||
1110 | 399 | * timestamp, then there is a happened-before relationship due to | ||
1111 | 400 | * the synchronization in transaction registration; since _first is | ||
1112 | 401 | * volatile, we are guaranteed to see the modification made by the | ||
1113 | 402 | * committed transaction. | ||
1114 | 403 | */ | ||
1115 | 404 | for (Entry e = _first; e != null; e = e._previous) { | ||
1116 | 405 | final long commitTs = ti.commitStatus(e.getVersion(), vh2ts(version), vh2step(version)); | ||
1117 | 406 | if (commitTs >= 0 && commitTs != UNCOMMITTED) { | ||
1118 | 407 | if (e.isDeleted()) { | ||
1119 | 408 | return null; | ||
1120 | 409 | } | ||
1121 | 410 | return e; | ||
1122 | 411 | } | ||
1123 | 412 | } | ||
1124 | 413 | return null; | ||
1125 | 414 | } catch (final InterruptedException e) { | ||
1126 | 415 | throw new PersistitInterruptedException(e); | ||
1127 | 416 | } | ||
1128 | 417 | } | ||
1129 | 418 | |||
1130 | 419 | /** | ||
1131 | 420 | * <p> | ||
1132 | 421 | * Holder for one instance of an object, such as a {@link Tree}, whose | ||
1133 | 422 | * availability is governed by visibility within a transaction. | ||
1134 | 423 | * </p> | ||
1135 | 424 | * <p> | ||
1136 | 425 | * An Entry has a version handle just like a version in an {@link MVV}. It | ||
1137 | 426 | * is invisible to any transactions that started before the start timestamp, | ||
1138 | 427 | * in write-write conflict with any concurrent transaction, and available to | ||
1139 | 428 | * any transaction that starts after the version has committed. Visibility | ||
1140 | 429 | * within the transaction that creates the TimelyResourceEntry is determined | ||
1141 | 430 | * by the relative step numbers of the current transaction and the version | ||
1142 | 431 | * handle. If there exists a TimelyResource instance with start and commit | ||
1143 | 432 | * timestamps ts1, tc1 then any attempt to add another TimelyResource will | ||
1144 | 433 | * fail unless ts2 > tc1. | ||
1145 | 434 | * </p> | ||
1146 | 435 | * | ||
1147 | 436 | * @author peter | ||
1148 | 437 | */ | ||
1149 | 438 | |||
1150 | 439 | private class Entry { | ||
1151 | 440 | |||
1152 | 441 | private long _version; | ||
1153 | 442 | private V _resource; | ||
1154 | 443 | private volatile boolean _deleted; | ||
1155 | 444 | private volatile Entry _previous; | ||
1156 | 445 | |||
1157 | 446 | private Entry(final long versionHandle, final V resource) { | ||
1158 | 447 | _version = versionHandle; | ||
1159 | 448 | _resource = resource; | ||
1160 | 449 | } | ||
1161 | 450 | |||
1162 | 451 | private V getResource() { | ||
1163 | 452 | return _resource; | ||
1164 | 453 | } | ||
1165 | 454 | |||
1166 | 455 | private void setResource(final V resource) { | ||
1167 | 456 | _resource = resource; | ||
1168 | 457 | } | ||
1169 | 458 | |||
1170 | 459 | private Entry getPrevious() { | ||
1171 | 460 | return _previous; | ||
1172 | 461 | } | ||
1173 | 462 | |||
1174 | 463 | private void setPrevious(final Entry tr) { | ||
1175 | 464 | _previous = tr; | ||
1176 | 465 | } | ||
1177 | 466 | |||
1178 | 467 | private long getVersion() { | ||
1179 | 468 | return _version; | ||
1180 | 469 | } | ||
1181 | 470 | |||
1182 | 471 | private void setPrimordial() { | ||
1183 | 472 | _version = PRIMORDIAL; | ||
1184 | 473 | } | ||
1185 | 474 | |||
1186 | 475 | private void setDeleted() { | ||
1187 | 476 | _deleted = true; | ||
1188 | 477 | } | ||
1189 | 478 | |||
1190 | 479 | private boolean isDeleted() { | ||
1191 | 480 | return _deleted; | ||
1192 | 481 | } | ||
1193 | 482 | |||
1194 | 483 | private boolean prune() throws PersistitException { | ||
1195 | 484 | if (_resource instanceof PrunableVersion) { | ||
1196 | 485 | return ((PrunableVersion) _resource).prune(); | ||
1197 | 486 | } else { | ||
1198 | 487 | return true; | ||
1199 | 488 | } | ||
1200 | 489 | } | ||
1201 | 490 | |||
1202 | 491 | @Override | ||
1203 | 492 | public String toString() { | ||
1204 | 493 | String tcStatus; | ||
1205 | 494 | try { | ||
1206 | 495 | final long tc = _persistit.getTransactionIndex().commitStatus(_version, Long.MAX_VALUE, 0); | ||
1207 | 496 | tcStatus = TransactionStatus.tcString(tc); | ||
1208 | 497 | } catch (final Exception e) { | ||
1209 | 498 | tcStatus = e.toString(); | ||
1210 | 499 | } | ||
1211 | 500 | return String.format("(ts=%s tc=%s)->%s%s", TransactionStatus.versionString(_version), tcStatus, _resource, | ||
1212 | 501 | _previous != null ? "*" : ""); | ||
1213 | 502 | } | ||
1214 | 503 | } | ||
1215 | 504 | |||
1216 | 505 | } | ||
1217 | 0 | 506 | ||
1218 | === modified file 'src/main/java/com/persistit/TransactionPlayer.java' | |||
1219 | --- src/main/java/com/persistit/TransactionPlayer.java 2012-11-28 18:52:21 +0000 | |||
1220 | +++ src/main/java/com/persistit/TransactionPlayer.java 2013-04-29 21:38:25 +0000 | |||
1221 | @@ -15,11 +15,8 @@ | |||
1222 | 15 | 15 | ||
1223 | 16 | package com.persistit; | 16 | package com.persistit; |
1224 | 17 | 17 | ||
1230 | 18 | /** | 18 | import static com.persistit.TransactionIndex.ts2vh; |
1231 | 19 | * | 19 | |
1227 | 20 | * Read and apply transaction from the journal to the live database. To apply | ||
1228 | 21 | * a transaction, this class calls methods of a | ||
1229 | 22 | */ | ||
1232 | 23 | import java.nio.ByteBuffer; | 20 | import java.nio.ByteBuffer; |
1233 | 24 | import java.util.ArrayList; | 21 | import java.util.ArrayList; |
1234 | 25 | import java.util.List; | 22 | import java.util.List; |
1235 | @@ -39,6 +36,11 @@ | |||
1236 | 39 | import com.persistit.exception.PersistitException; | 36 | import com.persistit.exception.PersistitException; |
1237 | 40 | import com.persistit.exception.VolumeNotFoundException; | 37 | import com.persistit.exception.VolumeNotFoundException; |
1238 | 41 | 38 | ||
1239 | 39 | /** | ||
1240 | 40 | * Read and apply transaction from the journal to the live database. To apply a | ||
1241 | 41 | * transaction, this class calls methods of a TransactionPlayerListener. | ||
1242 | 42 | */ | ||
1243 | 43 | |||
1244 | 42 | class TransactionPlayer { | 44 | class TransactionPlayer { |
1245 | 43 | 45 | ||
1246 | 44 | private final AtomicLong appliedUpdates = new AtomicLong(); | 46 | private final AtomicLong appliedUpdates = new AtomicLong(); |
1247 | @@ -67,6 +69,7 @@ | |||
1248 | 67 | 69 | ||
1249 | 68 | boolean requiresLongRecordConversion(); | 70 | boolean requiresLongRecordConversion(); |
1250 | 69 | 71 | ||
1251 | 72 | boolean createTree(long timestamp) throws PersistitException; | ||
1252 | 70 | } | 73 | } |
1253 | 71 | 74 | ||
1254 | 72 | final TransactionPlayerSupport _support; | 75 | final TransactionPlayerSupport _support; |
1255 | @@ -153,93 +156,106 @@ | |||
1256 | 153 | case SR.TYPE: { | 156 | case SR.TYPE: { |
1257 | 154 | final int keySize = SR.getKeySize(bb); | 157 | final int keySize = SR.getKeySize(bb); |
1258 | 155 | final int treeHandle = SR.getTreeHandle(bb); | 158 | final int treeHandle = SR.getTreeHandle(bb); |
1273 | 156 | final Exchange exchange = getExchange(treeHandle, address, startTimestamp); | 159 | |
1274 | 157 | exchange.ignoreTransactions(); | 160 | final Exchange exchange = getExchange(treeHandle, address, startTimestamp, listener); |
1275 | 158 | final Key key = exchange.getKey(); | 161 | if (exchange != null) { |
1276 | 159 | final Value value = exchange.getValue(); | 162 | exchange.ignoreTransactions(); |
1277 | 160 | System.arraycopy(bb.array(), bb.position() + SR.OVERHEAD, key.getEncodedBytes(), 0, keySize); | 163 | final Key key = exchange.getKey(); |
1278 | 161 | key.setEncodedSize(keySize); | 164 | final Value value = exchange.getValue(); |
1279 | 162 | final int valueSize = innerSize - SR.OVERHEAD - keySize; | 165 | System.arraycopy(bb.array(), bb.position() + SR.OVERHEAD, key.getEncodedBytes(), 0, keySize); |
1280 | 163 | value.ensureFit(valueSize); | 166 | key.setEncodedSize(keySize); |
1281 | 164 | System.arraycopy(bb.array(), bb.position() + SR.OVERHEAD + keySize, value.getEncodedBytes(), 0, | 167 | final int valueSize = innerSize - SR.OVERHEAD - keySize; |
1282 | 165 | valueSize); | 168 | value.ensureFit(valueSize); |
1283 | 166 | value.setEncodedSize(valueSize); | 169 | System.arraycopy(bb.array(), bb.position() + SR.OVERHEAD + keySize, value.getEncodedBytes(), 0, |
1284 | 167 | 170 | valueSize); | |
1285 | 168 | if (value.getEncodedSize() >= Buffer.LONGREC_SIZE | 171 | value.setEncodedSize(valueSize); |
1286 | 169 | && (value.getEncodedBytes()[0] & 0xFF) == Buffer.LONGREC_TYPE) { | 172 | |
1287 | 173 | if (value.getEncodedSize() >= Buffer.LONGREC_SIZE | ||
1288 | 174 | && (value.getEncodedBytes()[0] & 0xFF) == Buffer.LONGREC_TYPE) { | ||
1289 | 175 | /* | ||
1290 | 176 | * convertToLongRecord will pollute the | ||
1291 | 177 | * getReadBuffer(). Therefore before calling it we | ||
1292 | 178 | * need to copy the TX record to a fresh ByteBuffer. | ||
1293 | 179 | */ | ||
1294 | 180 | if (bb == _support.getReadBuffer()) { | ||
1295 | 181 | end = recordSize - (position - start); | ||
1296 | 182 | bb = ByteBuffer.allocate(end); | ||
1297 | 183 | bb.put(_support.getReadBuffer().array(), position, end); | ||
1298 | 184 | bb.flip(); | ||
1299 | 185 | position = 0; | ||
1300 | 186 | } | ||
1301 | 187 | if (listener.requiresLongRecordConversion()) { | ||
1302 | 188 | _support.convertToLongRecord(value, treeHandle, address, commitTimestamp); | ||
1303 | 189 | } | ||
1304 | 190 | } | ||
1305 | 191 | |||
1306 | 192 | listener.store(address, startTimestamp, exchange); | ||
1307 | 170 | /* | 193 | /* |
1311 | 171 | * convertToLongRecord will pollute the getReadBuffer(). | 194 | * Don't keep exchanges with enlarged value - let them |
1312 | 172 | * Therefore before calling it we need to copy the TX | 195 | * be GC'd |
1310 | 173 | * record to a fresh ByteBuffer. | ||
1313 | 174 | */ | 196 | */ |
1323 | 175 | if (bb == _support.getReadBuffer()) { | 197 | if (exchange.getValue().getMaximumSize() < Value.DEFAULT_MAXIMUM_SIZE) { |
1324 | 176 | end = recordSize - (position - start); | 198 | releaseExchange(exchange); |
1316 | 177 | bb = ByteBuffer.allocate(end); | ||
1317 | 178 | bb.put(_support.getReadBuffer().array(), position, end); | ||
1318 | 179 | bb.flip(); | ||
1319 | 180 | position = 0; | ||
1320 | 181 | } | ||
1321 | 182 | if (listener.requiresLongRecordConversion()) { | ||
1322 | 183 | _support.convertToLongRecord(value, treeHandle, address, commitTimestamp); | ||
1325 | 184 | } | 199 | } |
1326 | 185 | } | 200 | } |
1327 | 186 | |||
1328 | 187 | listener.store(address, startTimestamp, exchange); | ||
1329 | 188 | appliedUpdates.incrementAndGet(); | 201 | appliedUpdates.incrementAndGet(); |
1330 | 189 | // Don't keep exchanges with enlarged value - let them be | ||
1331 | 190 | // GC'd | ||
1332 | 191 | if (exchange.getValue().getMaximumSize() < Value.DEFAULT_MAXIMUM_SIZE) { | ||
1333 | 192 | releaseExchange(exchange); | ||
1334 | 193 | } | ||
1335 | 194 | break; | 202 | break; |
1336 | 195 | } | 203 | } |
1337 | 196 | 204 | ||
1338 | 197 | case DR.TYPE: { | 205 | case DR.TYPE: { |
1339 | 198 | final int key1Size = DR.getKey1Size(bb); | 206 | final int key1Size = DR.getKey1Size(bb); |
1340 | 199 | final int elisionCount = DR.getKey2Elision(bb); | 207 | final int elisionCount = DR.getKey2Elision(bb); |
1353 | 200 | final Exchange exchange = getExchange(DR.getTreeHandle(bb), address, startTimestamp); | 208 | final Exchange exchange = getExchange(DR.getTreeHandle(bb), address, startTimestamp, listener); |
1354 | 201 | exchange.ignoreTransactions(); | 209 | if (exchange != null) { |
1355 | 202 | final Key key1 = exchange.getAuxiliaryKey3(); | 210 | exchange.ignoreTransactions(); |
1356 | 203 | final Key key2 = exchange.getAuxiliaryKey4(); | 211 | final Key key1 = exchange.getAuxiliaryKey3(); |
1357 | 204 | System.arraycopy(bb.array(), bb.position() + DR.OVERHEAD, key1.getEncodedBytes(), 0, key1Size); | 212 | final Key key2 = exchange.getAuxiliaryKey4(); |
1358 | 205 | key1.setEncodedSize(key1Size); | 213 | System.arraycopy(bb.array(), bb.position() + DR.OVERHEAD, key1.getEncodedBytes(), 0, key1Size); |
1359 | 206 | final int key2Size = innerSize - DR.OVERHEAD - key1Size; | 214 | key1.setEncodedSize(key1Size); |
1360 | 207 | System.arraycopy(key1.getEncodedBytes(), 0, key2.getEncodedBytes(), 0, elisionCount); | 215 | final int key2Size = innerSize - DR.OVERHEAD - key1Size; |
1361 | 208 | System.arraycopy(bb.array(), bb.position() + DR.OVERHEAD + key1Size, key2.getEncodedBytes(), | 216 | System.arraycopy(key1.getEncodedBytes(), 0, key2.getEncodedBytes(), 0, elisionCount); |
1362 | 209 | elisionCount, key2Size); | 217 | System.arraycopy(bb.array(), bb.position() + DR.OVERHEAD + key1Size, key2.getEncodedBytes(), |
1363 | 210 | key2.setEncodedSize(key2Size + elisionCount); | 218 | elisionCount, key2Size); |
1364 | 211 | listener.removeKeyRange(address, startTimestamp, exchange, key1, key2); | 219 | key2.setEncodedSize(key2Size + elisionCount); |
1365 | 220 | listener.removeKeyRange(address, startTimestamp, exchange, key1, key2); | ||
1366 | 221 | releaseExchange(exchange); | ||
1367 | 222 | } | ||
1368 | 212 | appliedUpdates.incrementAndGet(); | 223 | appliedUpdates.incrementAndGet(); |
1369 | 213 | releaseExchange(exchange); | ||
1370 | 214 | break; | 224 | break; |
1371 | 215 | } | 225 | } |
1372 | 216 | 226 | ||
1373 | 217 | case DT.TYPE: { | 227 | case DT.TYPE: { |
1376 | 218 | final Exchange exchange = getExchange(DT.getTreeHandle(bb), address, startTimestamp); | 228 | final Exchange exchange = getExchange(DT.getTreeHandle(bb), address, startTimestamp, listener); |
1377 | 219 | listener.removeTree(address, startTimestamp, exchange); | 229 | if (exchange != null) { |
1378 | 230 | listener.removeTree(address, startTimestamp, exchange); | ||
1379 | 231 | releaseExchange(exchange); | ||
1380 | 232 | } | ||
1381 | 220 | appliedUpdates.incrementAndGet(); | 233 | appliedUpdates.incrementAndGet(); |
1382 | 221 | releaseExchange(exchange); | ||
1383 | 222 | break; | 234 | break; |
1384 | 223 | } | 235 | } |
1385 | 224 | 236 | ||
1386 | 225 | case D0.TYPE: { | 237 | case D0.TYPE: { |
1397 | 226 | final Exchange exchange = getExchange(D0.getTreeHandle(bb), address, startTimestamp); | 238 | final Exchange exchange = getExchange(D0.getTreeHandle(bb), address, startTimestamp, listener); |
1398 | 227 | /* | 239 | if (exchange != null) { |
1399 | 228 | * Note that the commitTimestamp, not startTimestamp is | 240 | /* |
1400 | 229 | * passed to the delta method. The | 241 | * Note that the commitTimestamp, not startTimestamp is |
1401 | 230 | * Accumulator#updateBaseValue method needs the | 242 | * passed to the delta method. The |
1402 | 231 | * commitTimestamp. | 243 | * Accumulator#updateBaseValue method needs the |
1403 | 232 | */ | 244 | * commitTimestamp. |
1404 | 233 | listener.delta(address, commitTimestamp, exchange.getTree(), D0.getIndex(bb), | 245 | */ |
1405 | 234 | D0.getAccumulatorTypeOrdinal(bb), 1); | 246 | listener.delta(address, commitTimestamp, exchange.getTree(), D0.getIndex(bb), |
1406 | 235 | appliedUpdates.incrementAndGet(); | 247 | D0.getAccumulatorTypeOrdinal(bb), 1); |
1407 | 248 | appliedUpdates.incrementAndGet(); | ||
1408 | 249 | } | ||
1409 | 236 | break; | 250 | break; |
1410 | 237 | } | 251 | } |
1411 | 238 | 252 | ||
1412 | 239 | case D1.TYPE: { | 253 | case D1.TYPE: { |
1416 | 240 | final Exchange exchange = getExchange(D1.getTreeHandle(bb), address, startTimestamp); | 254 | final Exchange exchange = getExchange(D1.getTreeHandle(bb), address, startTimestamp, listener); |
1417 | 241 | listener.delta(address, startTimestamp, exchange.getTree(), D1.getIndex(bb), | 255 | if (exchange != null) { |
1418 | 242 | D1.getAccumulatorTypeOrdinal(bb), D1.getValue(bb)); | 256 | listener.delta(address, startTimestamp, exchange.getTree(), D1.getIndex(bb), |
1419 | 257 | D1.getAccumulatorTypeOrdinal(bb), D1.getValue(bb)); | ||
1420 | 258 | } | ||
1421 | 243 | appliedUpdates.incrementAndGet(); | 259 | appliedUpdates.incrementAndGet(); |
1422 | 244 | break; | 260 | break; |
1423 | 245 | } | 261 | } |
1424 | @@ -281,7 +297,26 @@ | |||
1425 | 281 | return String.format("JournalAddress %,d{%,d}", address, timestamp); | 297 | return String.format("JournalAddress %,d{%,d}", address, timestamp); |
1426 | 282 | } | 298 | } |
1427 | 283 | 299 | ||
1429 | 284 | private Exchange getExchange(final int treeHandle, final long from, final long timestamp) throws PersistitException { | 300 | /** |
1430 | 301 | * Returns an Exchange on which an operation can be applied or rolled back. | ||
1431 | 302 | * For a {@link TransactionPlayerListener} that performs roll backs, it is | ||
1432 | 303 | * important not to create a new tree when none exists. Therefore this | ||
1433 | 304 | * method may return <code>null</code> to indicate that no tree exists and | ||
1434 | 305 | * therefore the requested operation should be ignored. Whether to create a | ||
1435 | 306 | * new tree is determined by the | ||
1436 | 307 | * {@link TransactionPlayerListener#createTree(long)} method. | ||
1437 | 308 | * | ||
1438 | 309 | * @param treeHandle | ||
1439 | 310 | * @param from | ||
1440 | 311 | * @param timestamp | ||
1441 | 312 | * @param listener | ||
1442 | 313 | * @return the <code>Exchange</code> on which a recovery operation should be | ||
1443 | 314 | * applied, or <code>null</code> if there is no backing | ||
1444 | 315 | * <code>Tree</code>. | ||
1445 | 316 | * @throws PersistitException | ||
1446 | 317 | */ | ||
1447 | 318 | private Exchange getExchange(final int treeHandle, final long from, final long timestamp, | ||
1448 | 319 | final TransactionPlayerListener listener) throws PersistitException { | ||
1449 | 285 | final TreeDescriptor td = _support.getPersistit().getJournalManager().lookupTreeHandle(treeHandle); | 320 | final TreeDescriptor td = _support.getPersistit().getJournalManager().lookupTreeHandle(treeHandle); |
1450 | 286 | if (td == null) { | 321 | if (td == null) { |
1451 | 287 | throw new CorruptJournalException("Tree handle " + treeHandle + " is undefined at " | 322 | throw new CorruptJournalException("Tree handle " + treeHandle + " is undefined at " |
1452 | @@ -295,6 +330,10 @@ | |||
1453 | 295 | if (VolumeStructure.DIRECTORY_TREE_NAME.equals(td.getTreeName())) { | 330 | if (VolumeStructure.DIRECTORY_TREE_NAME.equals(td.getTreeName())) { |
1454 | 296 | return volume.getStructure().directoryExchange(); | 331 | return volume.getStructure().directoryExchange(); |
1455 | 297 | } else { | 332 | } else { |
1456 | 333 | final Tree tree = volume.getStructure().getTreeInternal(td.getTreeName()); | ||
1457 | 334 | if (!listener.createTree(timestamp) && (tree == null || !tree.hasVersion(ts2vh(timestamp)))) { | ||
1458 | 335 | return null; | ||
1459 | 336 | } | ||
1460 | 298 | final Exchange exchange = _support.getPersistit().getExchange(volume, td.getTreeName(), true); | 337 | final Exchange exchange = _support.getPersistit().getExchange(volume, td.getTreeName(), true); |
1461 | 299 | exchange.ignoreTransactions(); | 338 | exchange.ignoreTransactions(); |
1462 | 300 | return exchange; | 339 | return exchange; |
1463 | 301 | 340 | ||
1464 | === modified file 'src/main/java/com/persistit/Tree.java' | |||
1465 | --- src/main/java/com/persistit/Tree.java 2013-04-11 15:31:16 +0000 | |||
1466 | +++ src/main/java/com/persistit/Tree.java 2013-04-29 21:38:25 +0000 | |||
1467 | @@ -23,8 +23,13 @@ | |||
1468 | 23 | import com.persistit.Accumulator.MinAccumulator; | 23 | import com.persistit.Accumulator.MinAccumulator; |
1469 | 24 | import com.persistit.Accumulator.SeqAccumulator; | 24 | import com.persistit.Accumulator.SeqAccumulator; |
1470 | 25 | import com.persistit.Accumulator.SumAccumulator; | 25 | import com.persistit.Accumulator.SumAccumulator; |
1471 | 26 | import com.persistit.Version.PrunableVersion; | ||
1472 | 27 | import com.persistit.Version.VersionCreator; | ||
1473 | 26 | import com.persistit.exception.CorruptVolumeException; | 28 | import com.persistit.exception.CorruptVolumeException; |
1474 | 27 | import com.persistit.exception.PersistitException; | 29 | import com.persistit.exception.PersistitException; |
1475 | 30 | import com.persistit.exception.PersistitInterruptedException; | ||
1476 | 31 | import com.persistit.exception.RollbackException; | ||
1477 | 32 | import com.persistit.exception.TimeoutException; | ||
1478 | 28 | import com.persistit.util.Debug; | 33 | import com.persistit.util.Debug; |
1479 | 29 | import com.persistit.util.Util; | 34 | import com.persistit.util.Util; |
1480 | 30 | 35 | ||
1481 | @@ -36,6 +41,20 @@ | |||
1482 | 36 | * {@link Accumulator}s for a B-Tree. | 41 | * {@link Accumulator}s for a B-Tree. |
1483 | 37 | * </p> | 42 | * </p> |
1484 | 38 | * <p> | 43 | * <p> |
1485 | 44 | * As of Persistit 3.3, this class supports version within transactions. A new | ||
1486 | 45 | * <code>Tree</code> created within the cope of a {@link Transaction} is not | ||
1487 | 46 | * visible within the other transactions until it commits. Similarly, if a | ||
1488 | 47 | * <code>Tree</code> is removed within the scope of a transaction, other | ||
1489 | 48 | * transactions that started before the current transaction commits will | ||
1490 | 49 | * continue to be able to read and write the <code>Tree</code>. As a | ||
1491 | 50 | * side-effect, the physical storage for a <code>Tree</code> is not deallocated | ||
1492 | 51 | * until there are no remaining active transactions that started before the | ||
1493 | 52 | * commit timestamp of the current transaction. Concurrent transactions that | ||
1494 | 53 | * attempt to create or remove the same <code>Tree</code> instance are subject | ||
1495 | 54 | * to a a write-write dependency (see {@link Transaction}); all but one such | ||
1496 | 55 | * transaction must roll back. | ||
1497 | 56 | * </p> | ||
1498 | 57 | * <p> | ||
1499 | 39 | * <code>Tree</code> instances are created by | 58 | * <code>Tree</code> instances are created by |
1500 | 40 | * {@link Volume#getTree(String, boolean)}. If the <code>Volume</code> already | 59 | * {@link Volume#getTree(String, boolean)}. If the <code>Volume</code> already |
1501 | 41 | * has a B-Tree with the specified name, then the <code>Tree</code> object | 60 | * has a B-Tree with the specified name, then the <code>Tree</code> object |
1502 | @@ -67,15 +86,76 @@ | |||
1503 | 67 | 86 | ||
1504 | 68 | private final String _name; | 87 | private final String _name; |
1505 | 69 | private final Volume _volume; | 88 | private final Volume _volume; |
1506 | 70 | private volatile long _rootPageAddr; | ||
1507 | 71 | private volatile int _depth; | ||
1508 | 72 | private final AtomicLong _changeCount = new AtomicLong(0); | ||
1509 | 73 | private final AtomicReference<Object> _appCache = new AtomicReference<Object>(); | 89 | private final AtomicReference<Object> _appCache = new AtomicReference<Object>(); |
1510 | 74 | private final AtomicInteger _handle = new AtomicInteger(); | 90 | private final AtomicInteger _handle = new AtomicInteger(); |
1511 | 75 | 91 | ||
1515 | 76 | private final Accumulator[] _accumulators = new Accumulator[MAX_ACCUMULATOR_COUNT]; | 92 | private final TimelyResource<TreeVersion> _timelyResource; |
1516 | 77 | 93 | ||
1517 | 78 | private final TreeStatistics _treeStatistics = new TreeStatistics(); | 94 | private final VersionCreator<TreeVersion> _creator = new VersionCreator<TreeVersion>() { |
1518 | 95 | |||
1519 | 96 | @Override | ||
1520 | 97 | public TreeVersion createVersion(final TimelyResource<? extends TreeVersion> resource) | ||
1521 | 98 | throws PersistitException { | ||
1522 | 99 | return new TreeVersion(); | ||
1523 | 100 | } | ||
1524 | 101 | }; | ||
1525 | 102 | |||
1526 | 103 | class TreeVersion implements PrunableVersion { | ||
1527 | 104 | volatile long _rootPageAddr; | ||
1528 | 105 | volatile int _depth; | ||
1529 | 106 | volatile long _generation = _persistit.getTimestampAllocator().updateTimestamp(); | ||
1530 | 107 | final AtomicLong _changeCount = new AtomicLong(); | ||
1531 | 108 | volatile boolean _pruned; | ||
1532 | 109 | private final Accumulator[] _accumulators = new Accumulator[MAX_ACCUMULATOR_COUNT]; | ||
1533 | 110 | private final TreeStatistics _treeStatistics = new TreeStatistics(); | ||
1534 | 111 | |||
1535 | 112 | @Override | ||
1536 | 113 | public boolean prune() throws PersistitException { | ||
1537 | 114 | assert !_pruned; | ||
1538 | 115 | _volume.getStructure().deallocateTree(_rootPageAddr, _depth); | ||
1539 | 116 | discardAccumulators(); | ||
1540 | 117 | _pruned = true; | ||
1541 | 118 | _rootPageAddr = -1; | ||
1542 | 119 | return true; | ||
1543 | 120 | } | ||
1544 | 121 | |||
1545 | 122 | @Override | ||
1546 | 123 | public void vacate() { | ||
1547 | 124 | clearValid(); | ||
1548 | 125 | _volume.getStructure().removed(Tree.this); | ||
1549 | 126 | } | ||
1550 | 127 | |||
1551 | 128 | @Override | ||
1552 | 129 | public String toString() { | ||
1553 | 130 | return String.format("Tree(%d,%d)%s", _rootPageAddr, _depth, _pruned ? "#" : ""); | ||
1554 | 131 | } | ||
1555 | 132 | |||
1556 | 133 | /** | ||
1557 | 134 | * Forget about any instantiated accumulator and remove it from the | ||
1558 | 135 | * active list in Persistit. This should only be called in the during | ||
1559 | 136 | * the process of removing a tree. | ||
1560 | 137 | */ | ||
1561 | 138 | void discardAccumulators() { | ||
1562 | 139 | for (int i = 0; i < _accumulators.length; ++i) { | ||
1563 | 140 | if (_accumulators[i] != null) { | ||
1564 | 141 | _persistit.removeAccumulator(_accumulators[i]); | ||
1565 | 142 | _accumulators[i] = null; | ||
1566 | 143 | } | ||
1567 | 144 | } | ||
1568 | 145 | } | ||
1569 | 146 | } | ||
1570 | 147 | |||
1571 | 148 | /** | ||
1572 | 149 | * Unchecked wrapper for PersistitException thrown while trying to acquire a | ||
1573 | 150 | * TreeVersion. | ||
1574 | 151 | */ | ||
1575 | 152 | public static class TreeVersionException extends RuntimeException { | ||
1576 | 153 | private static final long serialVersionUID = -6372589972106489591L; | ||
1577 | 154 | |||
1578 | 155 | TreeVersionException(final Exception e) { | ||
1579 | 156 | super(e); | ||
1580 | 157 | } | ||
1581 | 158 | } | ||
1582 | 79 | 159 | ||
1583 | 80 | Tree(final Persistit persistit, final Volume volume, final String name) { | 160 | Tree(final Persistit persistit, final Volume volume, final String name) { |
1584 | 81 | super(persistit); | 161 | super(persistit); |
1585 | @@ -86,7 +166,35 @@ | |||
1586 | 86 | } | 166 | } |
1587 | 87 | _name = name; | 167 | _name = name; |
1588 | 88 | _volume = volume; | 168 | _volume = volume; |
1590 | 89 | _generation.set(1); | 169 | _timelyResource = new TimelyResource<TreeVersion>(persistit); |
1591 | 170 | } | ||
1592 | 171 | |||
1593 | 172 | TreeVersion version() { | ||
1594 | 173 | try { | ||
1595 | 174 | return _timelyResource.getVersion(_creator); | ||
1596 | 175 | } catch (final PersistitException e) { | ||
1597 | 176 | throw new TreeVersionException(e); | ||
1598 | 177 | } | ||
1599 | 178 | } | ||
1600 | 179 | |||
1601 | 180 | public boolean isDeleted() throws TimeoutException, PersistitInterruptedException { | ||
1602 | 181 | return _timelyResource.isEmpty(); | ||
1603 | 182 | } | ||
1604 | 183 | |||
1605 | 184 | boolean isLive() throws TimeoutException, PersistitInterruptedException { | ||
1606 | 185 | return isValid() && !isDeleted(); | ||
1607 | 186 | } | ||
1608 | 187 | |||
1609 | 188 | boolean isTransactionPrivate(final boolean byStep) throws TimeoutException, PersistitInterruptedException { | ||
1610 | 189 | return _timelyResource.isTransactionPrivate(byStep); | ||
1611 | 190 | } | ||
1612 | 191 | |||
1613 | 192 | boolean hasVersion(final long versionHandle) throws TimeoutException, PersistitInterruptedException { | ||
1614 | 193 | return _timelyResource.getVersion(versionHandle) != null; | ||
1615 | 194 | } | ||
1616 | 195 | |||
1617 | 196 | void delete() throws RollbackException, PersistitException { | ||
1618 | 197 | _timelyResource.delete(); | ||
1619 | 90 | } | 198 | } |
1620 | 91 | 199 | ||
1621 | 92 | /** | 200 | /** |
1622 | @@ -110,7 +218,9 @@ | |||
1623 | 110 | 218 | ||
1624 | 111 | @Override | 219 | @Override |
1625 | 112 | public boolean equals(final Object o) { | 220 | public boolean equals(final Object o) { |
1627 | 113 | if (o instanceof Tree) { | 221 | if (o == this) { |
1628 | 222 | return true; | ||
1629 | 223 | } else if (o instanceof Tree) { | ||
1630 | 114 | final Tree tree = (Tree) o; | 224 | final Tree tree = (Tree) o; |
1631 | 115 | return _name.equals(tree._name) && _volume.equals(tree.getVolume()); | 225 | return _name.equals(tree._name) && _volume.equals(tree.getVolume()); |
1632 | 116 | } else { | 226 | } else { |
1633 | @@ -126,20 +236,32 @@ | |||
1634 | 126 | * @return The page address | 236 | * @return The page address |
1635 | 127 | */ | 237 | */ |
1636 | 128 | public long getRootPageAddr() { | 238 | public long getRootPageAddr() { |
1638 | 129 | return _rootPageAddr; | 239 | final TreeVersion version = version(); |
1639 | 240 | return version._rootPageAddr; | ||
1640 | 130 | } | 241 | } |
1641 | 131 | 242 | ||
1642 | 132 | /** | 243 | /** |
1643 | 133 | * @return the number of levels of the <code>Tree</code>. | 244 | * @return the number of levels of the <code>Tree</code>. |
1644 | 134 | */ | 245 | */ |
1645 | 135 | public int getDepth() { | 246 | public int getDepth() { |
1647 | 136 | return _depth; | 247 | return version()._depth; |
1648 | 248 | } | ||
1649 | 249 | |||
1650 | 250 | @Override | ||
1651 | 251 | public long getGeneration() { | ||
1652 | 252 | return version()._generation; | ||
1653 | 253 | } | ||
1654 | 254 | |||
1655 | 255 | @Override | ||
1656 | 256 | void bumpGeneration() { | ||
1657 | 257 | version()._generation = _persistit.getTimestampAllocator().updateTimestamp(); | ||
1658 | 137 | } | 258 | } |
1659 | 138 | 259 | ||
1660 | 139 | void changeRootPageAddr(final long rootPageAddr, final int deltaDepth) throws PersistitException { | 260 | void changeRootPageAddr(final long rootPageAddr, final int deltaDepth) throws PersistitException { |
1661 | 140 | Debug.$assert0.t(isOwnedAsWriterByMe()); | 261 | Debug.$assert0.t(isOwnedAsWriterByMe()); |
1664 | 141 | _rootPageAddr = rootPageAddr; | 262 | final TreeVersion version = version(); |
1665 | 142 | _depth += deltaDepth; | 263 | version._rootPageAddr = rootPageAddr; |
1666 | 264 | version._depth += deltaDepth; | ||
1667 | 143 | } | 265 | } |
1668 | 144 | 266 | ||
1669 | 145 | void bumpChangeCount() { | 267 | void bumpChangeCount() { |
1670 | @@ -147,7 +269,7 @@ | |||
1671 | 147 | // Note: the changeCount only gets written when there's a structure | 269 | // Note: the changeCount only gets written when there's a structure |
1672 | 148 | // change in the tree that causes it to be committed. | 270 | // change in the tree that causes it to be committed. |
1673 | 149 | // | 271 | // |
1675 | 150 | _changeCount.incrementAndGet(); | 272 | version()._changeCount.incrementAndGet(); |
1676 | 151 | } | 273 | } |
1677 | 152 | 274 | ||
1678 | 153 | /** | 275 | /** |
1679 | @@ -155,7 +277,7 @@ | |||
1680 | 155 | * this tree; does not including replacement of an existing value | 277 | * this tree; does not including replacement of an existing value |
1681 | 156 | */ | 278 | */ |
1682 | 157 | long getChangeCount() { | 279 | long getChangeCount() { |
1684 | 158 | return _changeCount.get(); | 280 | return version()._changeCount.get(); |
1685 | 159 | } | 281 | } |
1686 | 160 | 282 | ||
1687 | 161 | /** | 283 | /** |
1688 | @@ -165,9 +287,10 @@ | |||
1689 | 165 | */ | 287 | */ |
1690 | 166 | int store(final byte[] bytes, final int index) { | 288 | int store(final byte[] bytes, final int index) { |
1691 | 167 | final byte[] nameBytes = Util.stringToBytes(_name); | 289 | final byte[] nameBytes = Util.stringToBytes(_name); |
1695 | 168 | Util.putLong(bytes, index, _rootPageAddr); | 290 | final TreeVersion version = version(); |
1696 | 169 | Util.putLong(bytes, index + 8, getChangeCount()); | 291 | Util.putLong(bytes, index, version._rootPageAddr); |
1697 | 170 | Util.putShort(bytes, index + 16, _depth); | 292 | Util.putLong(bytes, index + 8, version._changeCount.get()); |
1698 | 293 | Util.putShort(bytes, index + 16, version._depth); | ||
1699 | 171 | Util.putShort(bytes, index + 18, nameBytes.length); | 294 | Util.putShort(bytes, index + 18, nameBytes.length); |
1700 | 172 | Util.putBytes(bytes, index + 20, nameBytes); | 295 | Util.putBytes(bytes, index + 20, nameBytes); |
1701 | 173 | return 20 + nameBytes.length; | 296 | return 20 + nameBytes.length; |
1702 | @@ -187,9 +310,10 @@ | |||
1703 | 187 | if (!_name.equals(name)) { | 310 | if (!_name.equals(name)) { |
1704 | 188 | throw new IllegalStateException("Invalid tree name recorded: " + name + " for tree " + _name); | 311 | throw new IllegalStateException("Invalid tree name recorded: " + name + " for tree " + _name); |
1705 | 189 | } | 312 | } |
1709 | 190 | _rootPageAddr = Util.getLong(bytes, index); | 313 | final TreeVersion version = version(); |
1710 | 191 | _changeCount.set(Util.getLong(bytes, index + 8)); | 314 | version._rootPageAddr = Util.getLong(bytes, index); |
1711 | 192 | _depth = Util.getShort(bytes, index + 16); | 315 | version._changeCount.set(Util.getLong(bytes, index + 8)); |
1712 | 316 | version._depth = Util.getShort(bytes, index + 16); | ||
1713 | 193 | return length; | 317 | return length; |
1714 | 194 | } | 318 | } |
1715 | 195 | 319 | ||
1716 | @@ -200,7 +324,8 @@ | |||
1717 | 200 | * @throws PersistitException | 324 | * @throws PersistitException |
1718 | 201 | */ | 325 | */ |
1719 | 202 | void setRootPageAddress(final long rootPageAddr) throws PersistitException { | 326 | void setRootPageAddress(final long rootPageAddr) throws PersistitException { |
1721 | 203 | if (_rootPageAddr != rootPageAddr) { | 327 | final TreeVersion version = version(); |
1722 | 328 | if (version._rootPageAddr != rootPageAddr) { | ||
1723 | 204 | // Derive the index depth | 329 | // Derive the index depth |
1724 | 205 | Buffer buffer = null; | 330 | Buffer buffer = null; |
1725 | 206 | try { | 331 | try { |
1726 | @@ -210,8 +335,8 @@ | |||
1727 | 210 | throw new CorruptVolumeException(String.format("Tree root page %,d has invalid type %s", | 335 | throw new CorruptVolumeException(String.format("Tree root page %,d has invalid type %s", |
1728 | 211 | rootPageAddr, buffer.getPageTypeName())); | 336 | rootPageAddr, buffer.getPageTypeName())); |
1729 | 212 | } | 337 | } |
1732 | 213 | _rootPageAddr = rootPageAddr; | 338 | version._rootPageAddr = rootPageAddr; |
1733 | 214 | _depth = type - Buffer.PAGE_TYPE_DATA + 1; | 339 | version._depth = type - Buffer.PAGE_TYPE_DATA + 1; |
1734 | 215 | } finally { | 340 | } finally { |
1735 | 216 | if (buffer != null) { | 341 | if (buffer != null) { |
1736 | 217 | buffer.releaseTouched(); | 342 | buffer.releaseTouched(); |
1737 | @@ -226,10 +351,15 @@ | |||
1738 | 226 | * <code>Tree</code> to fail. | 351 | * <code>Tree</code> to fail. |
1739 | 227 | */ | 352 | */ |
1740 | 228 | void invalidate() { | 353 | void invalidate() { |
1741 | 354 | final TreeVersion version = version(); | ||
1742 | 229 | super.clearValid(); | 355 | super.clearValid(); |
1746 | 230 | _depth = -1; | 356 | version._depth = -1; |
1747 | 231 | _rootPageAddr = -1; | 357 | version._rootPageAddr = -1; |
1748 | 232 | _generation.set(-1); | 358 | version._generation = _persistit.getTimestampAllocator().updateTimestamp(); |
1749 | 359 | } | ||
1750 | 360 | |||
1751 | 361 | void setPrimordial() { | ||
1752 | 362 | _timelyResource.setPrimordial(); | ||
1753 | 233 | } | 363 | } |
1754 | 234 | 364 | ||
1755 | 235 | /** | 365 | /** |
1756 | @@ -238,7 +368,7 @@ | |||
1757 | 238 | * </code>Tree</code> | 368 | * </code>Tree</code> |
1758 | 239 | */ | 369 | */ |
1759 | 240 | public TreeStatistics getStatistics() { | 370 | public TreeStatistics getStatistics() { |
1761 | 241 | return _treeStatistics; | 371 | return version()._treeStatistics; |
1762 | 242 | } | 372 | } |
1763 | 243 | 373 | ||
1764 | 244 | /** | 374 | /** |
1765 | @@ -248,8 +378,9 @@ | |||
1766 | 248 | */ | 378 | */ |
1767 | 249 | @Override | 379 | @Override |
1768 | 250 | public String toString() { | 380 | public String toString() { |
1771 | 251 | return "<Tree " + _name + " in volume " + _volume.getName() + " rootPageAddr=" + _rootPageAddr + " depth=" | 381 | final TreeVersion version = version(); |
1772 | 252 | + _depth + " status=" + getStatusDisplayString() + ">"; | 382 | return "<Tree " + _name + " in volume " + _volume.getName() + " rootPageAddr=" + version._rootPageAddr |
1773 | 383 | + " depth=" + version._depth + " status=" + getStatusDisplayString() + ">"; | ||
1774 | 253 | } | 384 | } |
1775 | 254 | 385 | ||
1776 | 255 | /** | 386 | /** |
1777 | @@ -278,8 +409,8 @@ | |||
1778 | 278 | } | 409 | } |
1779 | 279 | 410 | ||
1780 | 280 | /** | 411 | /** |
1783 | 281 | * Set the tree handle. The tree must may not be a member of a temporary | 412 | * Assign and set the tree handle. The tree must may not be a member of a |
1784 | 282 | * volume. | 413 | * temporary volume. |
1785 | 283 | * | 414 | * |
1786 | 284 | * @throws PersistitException | 415 | * @throws PersistitException |
1787 | 285 | */ | 416 | */ |
1788 | @@ -397,7 +528,8 @@ | |||
1789 | 397 | if (index < 0 || index >= MAX_ACCUMULATOR_COUNT) { | 528 | if (index < 0 || index >= MAX_ACCUMULATOR_COUNT) { |
1790 | 398 | throw new IllegalArgumentException("Invalid accumulator index: " + index); | 529 | throw new IllegalArgumentException("Invalid accumulator index: " + index); |
1791 | 399 | } | 530 | } |
1793 | 400 | Accumulator accumulator = _accumulators[index]; | 531 | final TreeVersion version = version(); |
1794 | 532 | Accumulator accumulator = version._accumulators[index]; | ||
1795 | 401 | if (accumulator == null) { | 533 | if (accumulator == null) { |
1796 | 402 | final AccumulatorState saved = Accumulator.getAccumulatorState(this, index); | 534 | final AccumulatorState saved = Accumulator.getAccumulatorState(this, index); |
1797 | 403 | long savedValue = 0; | 535 | long savedValue = 0; |
1798 | @@ -411,7 +543,7 @@ | |||
1799 | 411 | savedValue = saved.getValue(); | 543 | savedValue = saved.getValue(); |
1800 | 412 | } | 544 | } |
1801 | 413 | accumulator = Accumulator.accumulator(type, this, index, savedValue, _persistit.getTransactionIndex()); | 545 | accumulator = Accumulator.accumulator(type, this, index, savedValue, _persistit.getTransactionIndex()); |
1803 | 414 | _accumulators[index] = accumulator; | 546 | version._accumulators[index] = accumulator; |
1804 | 415 | _persistit.addAccumulator(accumulator); | 547 | _persistit.addAccumulator(accumulator); |
1805 | 416 | } else if (accumulator.getType() != type) { | 548 | } else if (accumulator.getType() != type) { |
1806 | 417 | throw new IllegalStateException("Wrong type " + accumulator + " is not a " + type + " accumulator"); | 549 | throw new IllegalStateException("Wrong type " + accumulator + " is not a " + type + " accumulator"); |
1807 | @@ -442,17 +574,4 @@ | |||
1808 | 442 | _handle.set(0); | 574 | _handle.set(0); |
1809 | 443 | } | 575 | } |
1810 | 444 | 576 | ||
1811 | 445 | /** | ||
1812 | 446 | * Forget about any instantiated accumulator and remove it from the active | ||
1813 | 447 | * list in Persistit. This should only be called in the during the process | ||
1814 | 448 | * of removing a tree. | ||
1815 | 449 | */ | ||
1816 | 450 | void discardAccumulators() { | ||
1817 | 451 | for (int i = 0; i < _accumulators.length; ++i) { | ||
1818 | 452 | if (_accumulators[i] != null) { | ||
1819 | 453 | _persistit.removeAccumulator(_accumulators[i]); | ||
1820 | 454 | _accumulators[i] = null; | ||
1821 | 455 | } | ||
1822 | 456 | } | ||
1823 | 457 | } | ||
1824 | 458 | } | 577 | } |
1825 | 459 | 578 | ||
1826 | === modified file 'src/main/java/com/persistit/ValueHelper.java' | |||
1827 | --- src/main/java/com/persistit/ValueHelper.java 2012-08-24 13:57:19 +0000 | |||
1828 | +++ src/main/java/com/persistit/ValueHelper.java 2013-04-29 21:38:25 +0000 | |||
1829 | @@ -15,6 +15,8 @@ | |||
1830 | 15 | 15 | ||
1831 | 16 | package com.persistit; | 16 | package com.persistit; |
1832 | 17 | 17 | ||
1833 | 18 | import com.persistit.util.Util; | ||
1834 | 19 | |||
1835 | 18 | interface ValueHelper { | 20 | interface ValueHelper { |
1836 | 19 | 21 | ||
1837 | 20 | int requiredLength(final byte[] target, int targetOffset, int targetLength); | 22 | int requiredLength(final byte[] target, int targetOffset, int targetLength); |
1838 | @@ -102,6 +104,11 @@ | |||
1839 | 102 | public boolean isMVV() { | 104 | public boolean isMVV() { |
1840 | 103 | return false; | 105 | return false; |
1841 | 104 | } | 106 | } |
1842 | 107 | |||
1843 | 108 | @Override | ||
1844 | 109 | public String toString() { | ||
1845 | 110 | return _value != null ? Util.hexDump(_value.getEncodedBytes(), 0, _value.getEncodedSize()) : "null"; | ||
1846 | 111 | } | ||
1847 | 105 | }; | 112 | }; |
1848 | 106 | 113 | ||
1849 | 107 | static class MVVValueWriter implements ValueHelper { | 114 | static class MVVValueWriter implements ValueHelper { |
1850 | @@ -145,5 +152,10 @@ | |||
1851 | 145 | public boolean isMVV() { | 152 | public boolean isMVV() { |
1852 | 146 | return true; | 153 | return true; |
1853 | 147 | } | 154 | } |
1854 | 155 | |||
1855 | 156 | @Override | ||
1856 | 157 | public String toString() { | ||
1857 | 158 | return _value != null ? Util.hexDump(_value.getEncodedBytes(), 0, _value.getEncodedSize()) : "null"; | ||
1858 | 159 | } | ||
1859 | 148 | }; | 160 | }; |
1860 | 149 | } | 161 | } |
1861 | 150 | 162 | ||
1862 | === added file 'src/main/java/com/persistit/Version.java' | |||
1863 | --- src/main/java/com/persistit/Version.java 1970-01-01 00:00:00 +0000 | |||
1864 | +++ src/main/java/com/persistit/Version.java 2013-04-29 21:38:25 +0000 | |||
1865 | @@ -0,0 +1,69 @@ | |||
1866 | 1 | /** | ||
1867 | 2 | * Copyright © 2012 Akiban Technologies, Inc. All rights reserved. | ||
1868 | 3 | * | ||
1869 | 4 | * This program and the accompanying materials are made available | ||
1870 | 5 | * under the terms of the Eclipse Public License v1.0 which | ||
1871 | 6 | * accompanies this distribution, and is available at | ||
1872 | 7 | * http://www.eclipse.org/legal/epl-v10.html | ||
1873 | 8 | * | ||
1874 | 9 | * This program may also be available under different license terms. | ||
1875 | 10 | * For more information, see www.akiban.com or contact licensing@akiban.com. | ||
1876 | 11 | * | ||
1877 | 12 | * Contributors: | ||
1878 | 13 | * Akiban Technologies, Inc. | ||
1879 | 14 | */ | ||
1880 | 15 | |||
1881 | 16 | package com.persistit; | ||
1882 | 17 | |||
1883 | 18 | import com.persistit.exception.PersistitException; | ||
1884 | 19 | |||
1885 | 20 | /** | ||
1886 | 21 | * Marker interface implemented by objects managed within the | ||
1887 | 22 | * {@link TimelyResource} framework. | ||
1888 | 23 | * | ||
1889 | 24 | * @author peter | ||
1890 | 25 | */ | ||
1891 | 26 | interface Version { | ||
1892 | 27 | /** | ||
1893 | 28 | * Sub-interface describing <code>Version</code> types that require pruning | ||
1894 | 29 | * when obsolete. | ||
1895 | 30 | * | ||
1896 | 31 | * @author peter | ||
1897 | 32 | */ | ||
1898 | 33 | interface PrunableVersion extends Version { | ||
1899 | 34 | /** | ||
1900 | 35 | * Clean up any state held by this resource. For example, when a | ||
1901 | 36 | * {@link Tree} is pruned, all pages allocated to its content are | ||
1902 | 37 | * deallocated. This method is called when a newer | ||
1903 | 38 | * <code>TimelyResource</code> has been created and is visible to all | ||
1904 | 39 | * active transactions. | ||
1905 | 40 | * | ||
1906 | 41 | * @return <code>true</code> if all pruning work for this resource has | ||
1907 | 42 | * been completed, <code>false</code> if the prune method should | ||
1908 | 43 | * be called again later | ||
1909 | 44 | */ | ||
1910 | 45 | boolean prune() throws PersistitException; | ||
1911 | 46 | |||
1912 | 47 | /** | ||
1913 | 48 | * Called after the last known <code>Version</code> managed by a | ||
1914 | 49 | * <code>TimelyResource</code> has been pruned. | ||
1915 | 50 | * | ||
1916 | 51 | * @throws PersistitException | ||
1917 | 52 | */ | ||
1918 | 53 | void vacate() throws PersistitException; | ||
1919 | 54 | |||
1920 | 55 | } | ||
1921 | 56 | |||
1922 | 57 | /** | ||
1923 | 58 | * Interface for a factory that creates Versions of the specified type. | ||
1924 | 59 | * | ||
1925 | 60 | * @author peter | ||
1926 | 61 | * | ||
1927 | 62 | * @param <T> | ||
1928 | 63 | * @param <V> | ||
1929 | 64 | */ | ||
1930 | 65 | interface VersionCreator<V> { | ||
1931 | 66 | V createVersion(final TimelyResource<? extends V> resource) throws PersistitException; | ||
1932 | 67 | } | ||
1933 | 68 | |||
1934 | 69 | } | ||
1935 | 0 | \ No newline at end of file | 70 | \ No newline at end of file |
1936 | 1 | 71 | ||
1937 | === modified file 'src/main/java/com/persistit/Volume.java' | |||
1938 | --- src/main/java/com/persistit/Volume.java 2013-03-13 17:14:51 +0000 | |||
1939 | +++ src/main/java/com/persistit/Volume.java 2013-04-29 21:38:25 +0000 | |||
1940 | @@ -456,8 +456,11 @@ | |||
1941 | 456 | * | 456 | * |
1942 | 457 | * @throws PersistitException | 457 | * @throws PersistitException |
1943 | 458 | */ | 458 | */ |
1945 | 459 | void open(final Persistit persistit) throws PersistitException { | 459 | synchronized void open(final Persistit persistit) throws PersistitException { |
1946 | 460 | checkClosing(); | 460 | checkClosing(); |
1947 | 461 | if (isOpened()) { | ||
1948 | 462 | return; | ||
1949 | 463 | } | ||
1950 | 461 | if (_specification == null) { | 464 | if (_specification == null) { |
1951 | 462 | throw new IllegalStateException("Missing VolumeSpecification"); | 465 | throw new IllegalStateException("Missing VolumeSpecification"); |
1952 | 463 | } | 466 | } |
1953 | 464 | 467 | ||
1954 | === modified file 'src/main/java/com/persistit/VolumeStructure.java' | |||
1955 | --- src/main/java/com/persistit/VolumeStructure.java 2013-03-13 16:27:04 +0000 | |||
1956 | +++ src/main/java/com/persistit/VolumeStructure.java 2013-04-29 21:38:25 +0000 | |||
1957 | @@ -15,17 +15,12 @@ | |||
1958 | 15 | 15 | ||
1959 | 16 | package com.persistit; | 16 | package com.persistit; |
1960 | 17 | 17 | ||
1961 | 18 | import static com.persistit.util.SequencerConstants.TREE_CREATE_REMOVE_A; | ||
1962 | 19 | import static com.persistit.util.ThreadSequencer.sequence; | ||
1963 | 20 | |||
1964 | 21 | import java.lang.ref.WeakReference; | 18 | import java.lang.ref.WeakReference; |
1965 | 22 | import java.util.ArrayList; | 19 | import java.util.ArrayList; |
1966 | 23 | import java.util.HashMap; | 20 | import java.util.HashMap; |
1967 | 24 | import java.util.List; | 21 | import java.util.List; |
1968 | 25 | import java.util.Map; | 22 | import java.util.Map; |
1969 | 26 | 23 | ||
1970 | 27 | import com.persistit.AlertMonitor.AlertLevel; | ||
1971 | 28 | import com.persistit.AlertMonitor.Event; | ||
1972 | 29 | import com.persistit.exception.BufferSizeUnavailableException; | 24 | import com.persistit.exception.BufferSizeUnavailableException; |
1973 | 30 | import com.persistit.exception.CorruptVolumeException; | 25 | import com.persistit.exception.CorruptVolumeException; |
1974 | 31 | import com.persistit.exception.InUseException; | 26 | import com.persistit.exception.InUseException; |
1975 | @@ -131,7 +126,6 @@ | |||
1976 | 131 | 126 | ||
1977 | 132 | Exchange directoryExchange() throws BufferSizeUnavailableException { | 127 | Exchange directoryExchange() throws BufferSizeUnavailableException { |
1978 | 133 | final Exchange ex = new Exchange(_directoryTree); | 128 | final Exchange ex = new Exchange(_directoryTree); |
1979 | 134 | ex.ignoreTransactions(); | ||
1980 | 135 | return ex; | 129 | return ex; |
1981 | 136 | } | 130 | } |
1982 | 137 | 131 | ||
1983 | @@ -191,21 +185,30 @@ | |||
1984 | 191 | if (DIRECTORY_TREE_NAME.equals(name)) { | 185 | if (DIRECTORY_TREE_NAME.equals(name)) { |
1985 | 192 | throw new IllegalArgumentException("Tree name is reserved: " + name); | 186 | throw new IllegalArgumentException("Tree name is reserved: " + name); |
1986 | 193 | } | 187 | } |
1988 | 194 | Tree tree; | 188 | Tree tree = null; |
1989 | 195 | final WeakReference<Tree> treeRef = _treeNameHashMap.get(name); | 189 | final WeakReference<Tree> treeRef = _treeNameHashMap.get(name); |
1990 | 196 | if (treeRef != null) { | 190 | if (treeRef != null) { |
1991 | 197 | tree = treeRef.get(); | 191 | tree = treeRef.get(); |
1994 | 198 | if (tree != null && tree.isValid()) { | 192 | if (tree != null) { |
1995 | 199 | return tree; | 193 | if (tree.isLive()) { |
1996 | 194 | return tree; | ||
1997 | 195 | } else { | ||
1998 | 196 | if (!createIfNecessary) { | ||
1999 | 197 | return null; | ||
2000 | 198 | } | ||
2001 | 199 | } | ||
2002 | 200 | } | 200 | } |
2003 | 201 | } | 201 | } |
2004 | 202 | if (tree == null) { | ||
2005 | 203 | tree = new Tree(_persistit, _volume, name); | ||
2006 | 204 | } | ||
2007 | 202 | final Exchange ex = directoryExchange(); | 205 | final Exchange ex = directoryExchange(); |
2008 | 203 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_ROOT).append(name); | 206 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_ROOT).append(name); |
2009 | 204 | final Value value = ex.fetch().getValue(); | 207 | final Value value = ex.fetch().getValue(); |
2010 | 205 | tree = new Tree(_persistit, _volume, name); | ||
2011 | 206 | if (value.isDefined()) { | 208 | if (value.isDefined()) { |
2012 | 207 | value.get(tree); | 209 | value.get(tree); |
2013 | 208 | loadTreeStatistics(tree); | 210 | loadTreeStatistics(tree); |
2014 | 211 | tree.setPrimordial(); | ||
2015 | 209 | tree.setValid(); | 212 | tree.setValid(); |
2016 | 210 | } else if (createIfNecessary) { | 213 | } else if (createIfNecessary) { |
2017 | 211 | final long rootPageAddr = createTreeRoot(tree); | 214 | final long rootPageAddr = createTreeRoot(tree); |
2018 | @@ -216,10 +219,14 @@ | |||
2019 | 216 | } else { | 219 | } else { |
2020 | 217 | return null; | 220 | return null; |
2021 | 218 | } | 221 | } |
2022 | 222 | if (_volume.isTemporary() || _volume.isLockVolume()) { | ||
2023 | 223 | tree.setPrimordial(); | ||
2024 | 224 | } | ||
2025 | 219 | if (!_volume.isTemporary()) { | 225 | if (!_volume.isTemporary()) { |
2026 | 220 | tree.loadHandle(); | 226 | tree.loadHandle(); |
2027 | 221 | } | 227 | } |
2028 | 222 | _treeNameHashMap.put(name, new WeakReference<Tree>(tree)); | 228 | _treeNameHashMap.put(name, new WeakReference<Tree>(tree)); |
2029 | 229 | |||
2030 | 223 | return tree; | 230 | return tree; |
2031 | 224 | } | 231 | } |
2032 | 225 | 232 | ||
2033 | @@ -249,13 +256,16 @@ | |||
2034 | 249 | } | 256 | } |
2035 | 250 | } else { | 257 | } else { |
2036 | 251 | final Exchange ex = directoryExchange(); | 258 | final Exchange ex = directoryExchange(); |
2037 | 259 | if (!tree.isTransactionPrivate(false)) { | ||
2038 | 260 | ex.ignoreTransactions(); | ||
2039 | 261 | } | ||
2040 | 252 | ex.getValue().put(tree); | 262 | ex.getValue().put(tree); |
2041 | 253 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_ROOT).append(tree.getName()).store(); | 263 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_ROOT).append(tree.getName()).store(); |
2042 | 254 | } | 264 | } |
2043 | 255 | } | 265 | } |
2044 | 256 | 266 | ||
2045 | 257 | void storeTreeStatistics(final Tree tree) throws PersistitException { | 267 | void storeTreeStatistics(final Tree tree) throws PersistitException { |
2047 | 258 | if (tree.getStatistics().isDirty() && !DIRECTORY_TREE_NAME.equals(tree.getName())) { | 268 | if (tree.isLive() && tree.getStatistics().isDirty() && tree != _directoryTree) { |
2048 | 259 | final Exchange ex = directoryExchange(); | 269 | final Exchange ex = directoryExchange(); |
2049 | 260 | if (!ex.getVolume().isReadOnly()) { | 270 | if (!ex.getVolume().isReadOnly()) { |
2050 | 261 | ex.getValue().put(tree.getStatistics()); | 271 | ex.getValue().put(tree.getStatistics()); |
2051 | @@ -273,43 +283,30 @@ | |||
2052 | 273 | } | 283 | } |
2053 | 274 | } | 284 | } |
2054 | 275 | 285 | ||
2056 | 276 | boolean removeTree(final Tree tree) throws PersistitException { | 286 | void removeTree(final Tree tree) throws PersistitException { |
2057 | 277 | if (tree == _directoryTree) { | 287 | if (tree == _directoryTree) { |
2058 | 278 | throw new IllegalArgumentException("Can't delete the Directory tree"); | 288 | throw new IllegalArgumentException("Can't delete the Directory tree"); |
2059 | 279 | } | 289 | } |
2060 | 280 | _persistit.checkSuspended(); | ||
2061 | 281 | 290 | ||
2062 | 282 | if (!tree.claim(true)) { | 291 | if (!tree.claim(true)) { |
2063 | 283 | throw new InUseException("Unable to acquire writer claim on " + tree); | 292 | throw new InUseException("Unable to acquire writer claim on " + tree); |
2064 | 284 | } | 293 | } |
2065 | 285 | |||
2066 | 286 | final int treeDepth = tree.getDepth(); | ||
2067 | 287 | final long treeRootPage = tree.getRootPageAddr(); | ||
2068 | 288 | |||
2069 | 289 | try { | 294 | try { |
2084 | 290 | tree.discardAccumulators(); | 295 | final Exchange ex = directoryExchange(); |
2085 | 291 | 296 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_ROOT).append(tree.getName()).remove(Key.GTEQ); | |
2086 | 292 | synchronized (this) { | 297 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_STATS).append(tree.getName()).remove(Key.GTEQ); |
2087 | 293 | _treeNameHashMap.remove(tree.getName()); | 298 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_ACCUMULATOR).append(tree.getName()).remove(Key.GTEQ); |
2088 | 294 | tree.bumpGeneration(); | 299 | tree.delete(); |
2075 | 295 | tree.invalidate(); | ||
2076 | 296 | |||
2077 | 297 | tree.changeRootPageAddr(-1, 0); | ||
2078 | 298 | final Exchange ex = directoryExchange(); | ||
2079 | 299 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_ROOT).append(tree.getName()).remove(Key.GTEQ); | ||
2080 | 300 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_STATS).append(tree.getName()).remove(Key.GTEQ); | ||
2081 | 301 | ex.clear().append(DIRECTORY_TREE_NAME).append(TREE_ACCUMULATOR).append(tree.getName()).remove(Key.GTEQ); | ||
2082 | 302 | } | ||
2083 | 303 | sequence(TREE_CREATE_REMOVE_A); | ||
2089 | 304 | } finally { | 300 | } finally { |
2090 | 305 | tree.release(); | 301 | tree.release(); |
2091 | 306 | } | 302 | } |
2098 | 307 | 303 | } | |
2099 | 308 | // The Tree is now gone. The following deallocates the | 304 | |
2100 | 309 | // pages formerly associated with it. If this fails we'll be | 305 | synchronized void removed(final Tree tree) { |
2101 | 310 | // left with allocated pages that are not available on the garbage | 306 | _treeNameHashMap.remove(tree.getName()); |
2102 | 311 | // chain for reuse. | 307 | } |
2103 | 312 | 308 | ||
2104 | 309 | void deallocateTree(final long treeRootPage, final int treeDepth) throws PersistitException { | ||
2105 | 313 | int depth = treeDepth; | 310 | int depth = treeDepth; |
2106 | 314 | long page = treeRootPage; | 311 | long page = treeRootPage; |
2107 | 315 | while (page != -1) { | 312 | while (page != -1) { |
2108 | @@ -341,7 +338,6 @@ | |||
2109 | 341 | deallocateGarbageChain(deallocate, 0); | 338 | deallocateGarbageChain(deallocate, 0); |
2110 | 342 | } | 339 | } |
2111 | 343 | } | 340 | } |
2112 | 344 | return true; | ||
2113 | 345 | } | 341 | } |
2114 | 346 | 342 | ||
2115 | 347 | /** | 343 | /** |
2116 | @@ -362,25 +358,22 @@ | |||
2117 | 362 | /** | 358 | /** |
2118 | 363 | * Flush dirty {@link TreeStatistics} instances. Called periodically on the | 359 | * Flush dirty {@link TreeStatistics} instances. Called periodically on the |
2119 | 364 | * PAGE_WRITER thread from {@link Persistit#cleanup()}. | 360 | * PAGE_WRITER thread from {@link Persistit#cleanup()}. |
2120 | 361 | * | ||
2121 | 362 | * @throws PersistitException | ||
2122 | 365 | */ | 363 | */ |
2132 | 366 | void flushStatistics() { | 364 | void flushStatistics() throws PersistitException { |
2133 | 367 | try { | 365 | final List<Tree> trees = new ArrayList<Tree>(); |
2134 | 368 | final List<Tree> trees = new ArrayList<Tree>(); | 366 | synchronized (this) { |
2135 | 369 | synchronized (this) { | 367 | for (final WeakReference<Tree> ref : _treeNameHashMap.values()) { |
2136 | 370 | for (final WeakReference<Tree> ref : _treeNameHashMap.values()) { | 368 | final Tree tree = ref.get(); |
2137 | 371 | final Tree tree = ref.get(); | 369 | if (tree != null && tree != _directoryTree) { |
2138 | 372 | if (tree != null && tree != _directoryTree) { | 370 | trees.add(tree); |
2130 | 373 | trees.add(tree); | ||
2131 | 374 | } | ||
2139 | 375 | } | 371 | } |
2140 | 376 | } | 372 | } |
2148 | 377 | for (final Tree tree : trees) { | 373 | } |
2149 | 378 | storeTreeStatistics(tree); | 374 | |
2150 | 379 | } | 375 | for (final Tree tree : trees) { |
2151 | 380 | } catch (final Exception e) { | 376 | storeTreeStatistics(tree); |
2145 | 381 | _persistit.getAlertMonitor().post( | ||
2146 | 382 | new Event(AlertLevel.ERROR, _persistit.getLogBase().adminFlushException, e), | ||
2147 | 383 | AlertMonitor.FLUSH_STATISTICS_CATEGORY); | ||
2152 | 384 | } | 377 | } |
2153 | 385 | } | 378 | } |
2154 | 386 | 379 | ||
2155 | 387 | 380 | ||
2156 | === modified file 'src/main/java/com/persistit/logging/LogBase.java' | |||
2157 | --- src/main/java/com/persistit/logging/LogBase.java 2012-10-04 20:23:10 +0000 | |||
2158 | +++ src/main/java/com/persistit/logging/LogBase.java 2013-04-29 21:38:25 +0000 | |||
2159 | @@ -226,6 +226,9 @@ | |||
2160 | 226 | @Message("WARNING|%s while pruning transaction record %s") | 226 | @Message("WARNING|%s while pruning transaction record %s") |
2161 | 227 | public final LogItem pruneException = PersistitLogMessage.empty(); | 227 | public final LogItem pruneException = PersistitLogMessage.empty(); |
2162 | 228 | 228 | ||
2163 | 229 | @Message("WARNING|%s while pruning TimelyResource %s") | ||
2164 | 230 | public final LogItem timelyResourcePruneException = PersistitLogMessage.empty(); | ||
2165 | 231 | |||
2166 | 229 | @Message("WARNING|Transaction %s pruning incomplete at %s after rollback") | 232 | @Message("WARNING|Transaction %s pruning incomplete at %s after rollback") |
2167 | 230 | public final LogItem pruningIncomplete = PersistitLogMessage.empty(); | 233 | public final LogItem pruningIncomplete = PersistitLogMessage.empty(); |
2168 | 231 | 234 | ||
2169 | 232 | 235 | ||
2170 | === modified file 'src/main/java/com/persistit/util/SequencerConstants.java' | |||
2171 | --- src/main/java/com/persistit/util/SequencerConstants.java 2012-10-10 16:06:49 +0000 | |||
2172 | +++ src/main/java/com/persistit/util/SequencerConstants.java 2013-04-29 21:38:25 +0000 | |||
2173 | @@ -61,17 +61,6 @@ | |||
2174 | 61 | array(WRITE_WRITE_STORE_A, WRITE_WRITE_STORE_C) }; | 61 | array(WRITE_WRITE_STORE_A, WRITE_WRITE_STORE_C) }; |
2175 | 62 | 62 | ||
2176 | 63 | /* | 63 | /* |
2177 | 64 | * Used in testing sequencing between tree creation and removal | ||
2178 | 65 | */ | ||
2179 | 66 | int TREE_CREATE_REMOVE_A = allocate("TREE_CREATE_REMOVE_A"); | ||
2180 | 67 | int TREE_CREATE_REMOVE_B = allocate("TREE_CREATE_REMOVE_B"); | ||
2181 | 68 | int TREE_CREATE_REMOVE_C = allocate("TREE_CREATE_REMOVE_C"); | ||
2182 | 69 | |||
2183 | 70 | int[][] TREE_CREATE_REMOVE_SCHEDULE = new int[][] { array(TREE_CREATE_REMOVE_A, TREE_CREATE_REMOVE_B), | ||
2184 | 71 | array(TREE_CREATE_REMOVE_B), array(TREE_CREATE_REMOVE_A, TREE_CREATE_REMOVE_C), | ||
2185 | 72 | array(TREE_CREATE_REMOVE_A, TREE_CREATE_REMOVE_C) }; | ||
2186 | 73 | |||
2187 | 74 | /* | ||
2188 | 75 | * Used in testing sequencing between pageNode reading and invalidation in | 64 | * Used in testing sequencing between pageNode reading and invalidation in |
2189 | 76 | * JournalManager | 65 | * JournalManager |
2190 | 77 | */ | 66 | */ |
2191 | @@ -104,7 +93,7 @@ | |||
2192 | 104 | array(DEALLOCATE_CHAIN_A, DEALLOCATE_CHAIN_C) }; | 93 | array(DEALLOCATE_CHAIN_A, DEALLOCATE_CHAIN_C) }; |
2193 | 105 | 94 | ||
2194 | 106 | /* | 95 | /* |
2196 | 107 | * Used in testing delete/deallocate sequence in Bug1022567Test | 96 | * Used in testing delete/deallocate sequence in Bug1064565Test |
2197 | 108 | */ | 97 | */ |
2198 | 109 | int ACCUMULATOR_CHECKPOINT_A = allocate("ACCUMULATOR_CHECKPOINT_A"); | 98 | int ACCUMULATOR_CHECKPOINT_A = allocate("ACCUMULATOR_CHECKPOINT_A"); |
2199 | 110 | int ACCUMULATOR_CHECKPOINT_B = allocate("ACCUMULATOR_CHECKPOINT_B"); | 99 | int ACCUMULATOR_CHECKPOINT_B = allocate("ACCUMULATOR_CHECKPOINT_B"); |
2200 | 111 | 100 | ||
2201 | === modified file 'src/test/java/com/persistit/AccumulatorRecoveryTest.java' | |||
2202 | --- src/test/java/com/persistit/AccumulatorRecoveryTest.java 2013-04-10 20:09:48 +0000 | |||
2203 | +++ src/test/java/com/persistit/AccumulatorRecoveryTest.java 2013-04-29 21:38:25 +0000 | |||
2204 | @@ -167,6 +167,11 @@ | |||
2205 | 167 | return true; | 167 | return true; |
2206 | 168 | } | 168 | } |
2207 | 169 | 169 | ||
2208 | 170 | @Override | ||
2209 | 171 | public boolean createTree(final long timestamp) throws PersistitException { | ||
2210 | 172 | return true; | ||
2211 | 173 | } | ||
2212 | 174 | |||
2213 | 170 | }; | 175 | }; |
2214 | 171 | plan.applyAllRecoveredTransactions(commitListener, plan.getDefaultRollbackListener()); | 176 | plan.applyAllRecoveredTransactions(commitListener, plan.getDefaultRollbackListener()); |
2215 | 172 | assertEquals(15, recoveryTimestamps.size()); | 177 | assertEquals(15, recoveryTimestamps.size()); |
2216 | 173 | 178 | ||
2217 | === modified file 'src/test/java/com/persistit/AccumulatorTest.java' | |||
2218 | --- src/test/java/com/persistit/AccumulatorTest.java 2013-04-10 20:09:48 +0000 | |||
2219 | +++ src/test/java/com/persistit/AccumulatorTest.java 2013-04-29 21:38:25 +0000 | |||
2220 | @@ -375,7 +375,7 @@ | |||
2221 | 375 | */ | 375 | */ |
2222 | 376 | @Test | 376 | @Test |
2223 | 377 | public void testRecreateAccumulatorAfterCheckpoint() throws PersistitException { | 377 | public void testRecreateAccumulatorAfterCheckpoint() throws PersistitException { |
2225 | 378 | final int PASS_COUNT = 2; | 378 | final int PASS_COUNT = 5; |
2226 | 379 | final int ROW_COUNT = 10; | 379 | final int ROW_COUNT = 10; |
2227 | 380 | final int ACCUM_INDEX = 0; | 380 | final int ACCUM_INDEX = 0; |
2228 | 381 | final String TEST_VOLUME_NAME = UnitTestProperties.VOLUME_NAME; | 381 | final String TEST_VOLUME_NAME = UnitTestProperties.VOLUME_NAME; |
2229 | 382 | 382 | ||
2230 | === modified file 'src/test/java/com/persistit/Bug1018526Test.java' | |||
2231 | --- src/test/java/com/persistit/Bug1018526Test.java 2012-11-20 17:45:51 +0000 | |||
2232 | +++ src/test/java/com/persistit/Bug1018526Test.java 2013-04-29 21:38:25 +0000 | |||
2233 | @@ -92,7 +92,7 @@ | |||
2234 | 92 | final TreeDescriptor td = map.remove(handle); | 92 | final TreeDescriptor td = map.remove(handle); |
2235 | 93 | assertNotNull("Permanent Tree should be un the tree map", td); | 93 | assertNotNull("Permanent Tree should be un the tree map", td); |
2236 | 94 | } | 94 | } |
2239 | 95 | // expect 1: the directory tree | 95 | // expect 2: _directory and _classIndex |
2240 | 96 | assertEquals("Recovered tree map should contain only permanent trees", 1, map.size()); | 96 | assertEquals("Recovered tree map should contain only permanent trees", 2, map.size()); |
2241 | 97 | } | 97 | } |
2242 | 98 | } | 98 | } |
2243 | 99 | 99 | ||
2244 | === modified file 'src/test/java/com/persistit/Bug920754Test.java' | |||
2245 | --- src/test/java/com/persistit/Bug920754Test.java 2013-04-10 20:09:48 +0000 | |||
2246 | +++ src/test/java/com/persistit/Bug920754Test.java 2013-04-29 21:38:25 +0000 | |||
2247 | @@ -68,6 +68,7 @@ | |||
2248 | 68 | while (dir.next(true)) { | 68 | while (dir.next(true)) { |
2249 | 69 | keys++; | 69 | keys++; |
2250 | 70 | } | 70 | } |
2252 | 71 | assertEquals("There should be no remaining keys in the directory tree", 0, keys); | 71 | // _classIndex |
2253 | 72 | assertEquals("There should be one remaining key in the directory tree", 1, keys); | ||
2254 | 72 | } | 73 | } |
2255 | 73 | } | 74 | } |
2256 | 74 | 75 | ||
2257 | === modified file 'src/test/java/com/persistit/Bug932097Test.java' | |||
2258 | --- src/test/java/com/persistit/Bug932097Test.java 2012-11-25 20:14:58 +0000 | |||
2259 | +++ src/test/java/com/persistit/Bug932097Test.java 2013-04-29 21:38:25 +0000 | |||
2260 | @@ -21,6 +21,7 @@ | |||
2261 | 21 | 21 | ||
2262 | 22 | @Test | 22 | @Test |
2263 | 23 | public void testInjectedAbortTransactionStatus() throws Exception { | 23 | public void testInjectedAbortTransactionStatus() throws Exception { |
2264 | 24 | final Exchange ex = _persistit.getExchange("persistit", "test", true); | ||
2265 | 24 | /* | 25 | /* |
2266 | 25 | * Create a bunch of incomplete transactions | 26 | * Create a bunch of incomplete transactions |
2267 | 26 | */ | 27 | */ |
2268 | @@ -28,7 +29,6 @@ | |||
2269 | 28 | _persistit.setSessionId(new SessionId()); | 29 | _persistit.setSessionId(new SessionId()); |
2270 | 29 | final Transaction txn = _persistit.getTransaction(); | 30 | final Transaction txn = _persistit.getTransaction(); |
2271 | 30 | txn.begin(); | 31 | txn.begin(); |
2272 | 31 | final Exchange ex = _persistit.getExchange("persistit", "test", true); | ||
2273 | 32 | ex.getValue().put(RED_FOX); | 32 | ex.getValue().put(RED_FOX); |
2274 | 33 | txn.begin(); | 33 | txn.begin(); |
2275 | 34 | for (int k = 1; k < 10; k++) { | 34 | for (int k = 1; k < 10; k++) { |
2276 | 35 | 35 | ||
2277 | === modified file 'src/test/java/com/persistit/CorruptVolumeTest.java' | |||
2278 | --- src/test/java/com/persistit/CorruptVolumeTest.java 2012-08-24 13:57:19 +0000 | |||
2279 | +++ src/test/java/com/persistit/CorruptVolumeTest.java 2013-04-29 21:38:25 +0000 | |||
2280 | @@ -37,7 +37,8 @@ | |||
2281 | 37 | exchange.to(i).store(); | 37 | exchange.to(i).store(); |
2282 | 38 | } | 38 | } |
2283 | 39 | // Corrupt the volume by zonking the the index page | 39 | // Corrupt the volume by zonking the the index page |
2285 | 40 | final Buffer buffer = exchange.getBufferPool().get(exchange.getVolume(), 4, true, true); | 40 | final long pageAddr = exchange.fetchBufferCopy(1).getPageAddress(); |
2286 | 41 | final Buffer buffer = exchange.getBufferPool().get(exchange.getVolume(), pageAddr, true, true); | ||
2287 | 41 | Arrays.fill(buffer.getBytes(), 20, 200, (byte) 0); | 42 | Arrays.fill(buffer.getBytes(), 20, 200, (byte) 0); |
2288 | 42 | buffer.setDirtyAtTimestamp(_persistit.getTimestampAllocator().updateTimestamp()); | 43 | buffer.setDirtyAtTimestamp(_persistit.getTimestampAllocator().updateTimestamp()); |
2289 | 43 | buffer.releaseTouched(); | 44 | buffer.releaseTouched(); |
2290 | 44 | 45 | ||
2291 | === modified file 'src/test/java/com/persistit/IOFailureTest.java' | |||
2292 | --- src/test/java/com/persistit/IOFailureTest.java 2012-11-20 17:45:51 +0000 | |||
2293 | +++ src/test/java/com/persistit/IOFailureTest.java 2013-04-29 21:38:25 +0000 | |||
2294 | @@ -283,7 +283,6 @@ | |||
2295 | 283 | 283 | ||
2296 | 284 | @Test | 284 | @Test |
2297 | 285 | public void testJournalEOFonRecovery() throws Exception { | 285 | public void testJournalEOFonRecovery() throws Exception { |
2298 | 286 | final Properties properties = _persistit.getProperties(); | ||
2299 | 287 | final JournalManager jman = _persistit.getJournalManager(); | 286 | final JournalManager jman = _persistit.getJournalManager(); |
2300 | 288 | final Exchange exchange = _persistit.getExchange(_volumeName, "RecoveryTest", true); | 287 | final Exchange exchange = _persistit.getExchange(_volumeName, "RecoveryTest", true); |
2301 | 289 | exchange.getValue().put(RED_FOX); | 288 | exchange.getValue().put(RED_FOX); |
2302 | @@ -465,6 +464,10 @@ | |||
2303 | 465 | boolean done = false; | 464 | boolean done = false; |
2304 | 466 | while (System.currentTimeMillis() < expires) { | 465 | while (System.currentTimeMillis() < expires) { |
2305 | 467 | try { | 466 | try { |
2306 | 467 | /* | ||
2307 | 468 | * Needed to avoid leaving a dirty page during checkpoint | ||
2308 | 469 | */ | ||
2309 | 470 | _persistit.flushStatistics(); | ||
2310 | 468 | _persistit.copyBackPages(); | 471 | _persistit.copyBackPages(); |
2311 | 469 | done = true; | 472 | done = true; |
2312 | 470 | break; | 473 | break; |
2313 | 471 | 474 | ||
2314 | === modified file 'src/test/java/com/persistit/IntegrityCheckTest.java' | |||
2315 | --- src/test/java/com/persistit/IntegrityCheckTest.java 2012-11-20 17:45:51 +0000 | |||
2316 | +++ src/test/java/com/persistit/IntegrityCheckTest.java 2013-04-29 21:38:25 +0000 | |||
2317 | @@ -173,6 +173,8 @@ | |||
2318 | 173 | txn.end(); | 173 | txn.end(); |
2319 | 174 | } | 174 | } |
2320 | 175 | } | 175 | } |
2321 | 176 | _persistit.checkAllVolumes(); | ||
2322 | 177 | System.out.println(); | ||
2323 | 176 | final Configuration config = _persistit.getConfiguration(); | 178 | final Configuration config = _persistit.getConfiguration(); |
2324 | 177 | _persistit.crash(); | 179 | _persistit.crash(); |
2325 | 178 | _persistit = new Persistit(); | 180 | _persistit = new Persistit(); |
2326 | 179 | 181 | ||
2327 | === modified file 'src/test/java/com/persistit/JournalManagerTest.java' | |||
2328 | --- src/test/java/com/persistit/JournalManagerTest.java 2013-04-10 20:09:48 +0000 | |||
2329 | +++ src/test/java/com/persistit/JournalManagerTest.java 2013-04-29 21:38:25 +0000 | |||
2330 | @@ -59,6 +59,12 @@ | |||
2331 | 59 | 59 | ||
2332 | 60 | private final String _volumeName = "persistit"; | 60 | private final String _volumeName = "persistit"; |
2333 | 61 | 61 | ||
2334 | 62 | @Override | ||
2335 | 63 | public void setUp() throws Exception { | ||
2336 | 64 | super.setUp(); | ||
2337 | 65 | _persistit.getExchange(_volumeName, "JournalManagerTest1", true); | ||
2338 | 66 | } | ||
2339 | 67 | |||
2340 | 62 | @Test | 68 | @Test |
2341 | 63 | public void testJournalRecords() throws Exception { | 69 | public void testJournalRecords() throws Exception { |
2342 | 64 | store1(); | 70 | store1(); |
2343 | @@ -216,6 +222,11 @@ | |||
2344 | 216 | return true; | 222 | return true; |
2345 | 217 | } | 223 | } |
2346 | 218 | 224 | ||
2347 | 225 | @Override | ||
2348 | 226 | public boolean createTree(final long timestamp) throws PersistitException { | ||
2349 | 227 | return true; | ||
2350 | 228 | } | ||
2351 | 229 | |||
2352 | 219 | }; | 230 | }; |
2353 | 220 | rman.applyAllRecoveredTransactions(actor, rman.getDefaultRollbackListener()); | 231 | rman.applyAllRecoveredTransactions(actor, rman.getDefaultRollbackListener()); |
2354 | 221 | assertEquals(commitCount, recoveryTimestamps.size()); | 232 | assertEquals(commitCount, recoveryTimestamps.size()); |
2355 | @@ -298,6 +309,7 @@ | |||
2356 | 298 | // Allow test to control when pruning will happen | 309 | // Allow test to control when pruning will happen |
2357 | 299 | _persistit.getJournalManager().setRollbackPruningEnabled(false); | 310 | _persistit.getJournalManager().setRollbackPruningEnabled(false); |
2358 | 300 | _persistit.getJournalManager().setWritePagePruningEnabled(false); | 311 | _persistit.getJournalManager().setWritePagePruningEnabled(false); |
2359 | 312 | |||
2360 | 301 | final Transaction txn = _persistit.getTransaction(); | 313 | final Transaction txn = _persistit.getTransaction(); |
2361 | 302 | for (int i = 0; i < 10; i++) { | 314 | for (int i = 0; i < 10; i++) { |
2362 | 303 | txn.begin(); | 315 | txn.begin(); |
2363 | 304 | 316 | ||
2364 | === modified file 'src/test/java/com/persistit/MVCCPruneBufferTest.java' | |||
2365 | --- src/test/java/com/persistit/MVCCPruneBufferTest.java 2013-03-06 16:20:57 +0000 | |||
2366 | +++ src/test/java/com/persistit/MVCCPruneBufferTest.java 2013-04-29 21:38:25 +0000 | |||
2367 | @@ -320,6 +320,8 @@ | |||
2368 | 320 | 320 | ||
2369 | 321 | @Test | 321 | @Test |
2370 | 322 | public void testWritePagePrune() throws Exception { | 322 | public void testWritePagePrune() throws Exception { |
2371 | 323 | _persistit.getJournalManager().setWritePagePruningEnabled(false); | ||
2372 | 324 | final Exchange exchange = exchange(1); | ||
2373 | 323 | final Transaction txn = _persistit.getTransaction(); | 325 | final Transaction txn = _persistit.getTransaction(); |
2374 | 324 | try { | 326 | try { |
2375 | 325 | txn.begin(); | 327 | txn.begin(); |
2376 | @@ -328,8 +330,8 @@ | |||
2377 | 328 | } finally { | 330 | } finally { |
2378 | 329 | txn.end(); | 331 | txn.end(); |
2379 | 330 | } | 332 | } |
2382 | 331 | final Volume volume = _persistit.getVolume(TEST_VOLUME_NAME); | 333 | final long pageAddr = exchange.fetchBufferCopy(0).getPageAddress(); |
2383 | 332 | final Buffer buffer = volume.getPool().get(volume, 3, true, true); | 334 | final Buffer buffer = exchange.getBufferPool().get(exchange.getVolume(), pageAddr, true, true); |
2384 | 333 | assertTrue("Should have multiple MVV records", buffer.getMvvCount() > 2); | 335 | assertTrue("Should have multiple MVV records", buffer.getMvvCount() > 2); |
2385 | 334 | _persistit.getJournalManager().setWritePagePruningEnabled(true); | 336 | _persistit.getJournalManager().setWritePagePruningEnabled(true); |
2386 | 335 | _persistit.getTransactionIndex().updateActiveTransactionCache(); | 337 | _persistit.getTransactionIndex().updateActiveTransactionCache(); |
2387 | @@ -340,14 +342,17 @@ | |||
2388 | 340 | } | 342 | } |
2389 | 341 | 343 | ||
2390 | 342 | private void storeNewVersion(final int cycle) throws Exception { | 344 | private void storeNewVersion(final int cycle) throws Exception { |
2393 | 343 | final Exchange exchange = _persistit.getExchange(TEST_VOLUME_NAME, | 345 | final Exchange exchange = exchange(cycle); |
2392 | 344 | String.format("%s%04d", TEST_TREE_NAME, cycle), true); | ||
2394 | 345 | exchange.getValue().put(String.format("%s%04d", RED_FOX, cycle)); | 346 | exchange.getValue().put(String.format("%s%04d", RED_FOX, cycle)); |
2395 | 346 | for (int i = 1; i <= 100; i++) { | 347 | for (int i = 1; i <= 100; i++) { |
2396 | 347 | exchange.to(i).store(); | 348 | exchange.to(i).store(); |
2397 | 348 | } | 349 | } |
2398 | 349 | } | 350 | } |
2399 | 350 | 351 | ||
2400 | 352 | private Exchange exchange(final int cycle) throws PersistitException { | ||
2401 | 353 | return _persistit.getExchange(TEST_VOLUME_NAME, String.format("%s%04d", TEST_TREE_NAME, cycle), true); | ||
2402 | 354 | } | ||
2403 | 355 | |||
2404 | 351 | private void removeKeys(final int cycle) throws Exception { | 356 | private void removeKeys(final int cycle) throws Exception { |
2405 | 352 | final Exchange exchange = _persistit.getExchange(TEST_VOLUME_NAME, TEST_TREE_NAME, true); | 357 | final Exchange exchange = _persistit.getExchange(TEST_VOLUME_NAME, TEST_TREE_NAME, true); |
2406 | 353 | for (int i = (cycle % 2) + 1; i <= 100; i += 2) { | 358 | for (int i = (cycle % 2) + 1; i <= 100; i += 2) { |
2407 | 354 | 359 | ||
2408 | === modified file 'src/test/java/com/persistit/PersistitUnitTestCase.java' | |||
2409 | --- src/test/java/com/persistit/PersistitUnitTestCase.java 2012-11-20 18:18:02 +0000 | |||
2410 | +++ src/test/java/com/persistit/PersistitUnitTestCase.java 2013-04-29 21:38:25 +0000 | |||
2411 | @@ -136,4 +136,15 @@ | |||
2412 | 136 | _persistit.getCleanupManager().setPollInterval(-1); | 136 | _persistit.getCleanupManager().setPollInterval(-1); |
2413 | 137 | _persistit.getJournalManager().setWritePagePruningEnabled(false); | 137 | _persistit.getJournalManager().setWritePagePruningEnabled(false); |
2414 | 138 | } | 138 | } |
2415 | 139 | |||
2416 | 140 | protected void drainJournal() throws Exception { | ||
2417 | 141 | _persistit.flush(); | ||
2418 | 142 | /* | ||
2419 | 143 | * Causes all TreeStatistics to be marked clean so that the subsequent | ||
2420 | 144 | * checkpoint will not add another dirty page. | ||
2421 | 145 | */ | ||
2422 | 146 | _persistit.flushStatistics(); | ||
2423 | 147 | _persistit.checkpoint(); | ||
2424 | 148 | _persistit.getJournalManager().copyBack(); | ||
2425 | 149 | } | ||
2426 | 139 | } | 150 | } |
2427 | 140 | 151 | ||
2428 | === modified file 'src/test/java/com/persistit/RecoveryTest.java' | |||
2429 | --- src/test/java/com/persistit/RecoveryTest.java 2012-11-20 17:45:51 +0000 | |||
2430 | +++ src/test/java/com/persistit/RecoveryTest.java 2013-04-29 21:38:25 +0000 | |||
2431 | @@ -83,9 +83,7 @@ | |||
2432 | 83 | store1(); | 83 | store1(); |
2433 | 84 | JournalManager jman = _persistit.getJournalManager(); | 84 | JournalManager jman = _persistit.getJournalManager(); |
2434 | 85 | assertTrue(jman.getPageMapSize() > 0); | 85 | assertTrue(jman.getPageMapSize() > 0); |
2438 | 86 | _persistit.flush(); | 86 | drainJournal(); |
2436 | 87 | _persistit.checkpoint(); | ||
2437 | 88 | jman.copyBack(); | ||
2439 | 89 | assertEquals(0, jman.getPageMapSize()); | 87 | assertEquals(0, jman.getPageMapSize()); |
2440 | 90 | _persistit.close(); | 88 | _persistit.close(); |
2441 | 91 | _persistit = new Persistit(_config); | 89 | _persistit = new Persistit(_config); |
2442 | @@ -159,6 +157,11 @@ | |||
2443 | 159 | return true; | 157 | return true; |
2444 | 160 | } | 158 | } |
2445 | 161 | 159 | ||
2446 | 160 | @Override | ||
2447 | 161 | public boolean createTree(final long timestamp) throws PersistitException { | ||
2448 | 162 | return true; | ||
2449 | 163 | } | ||
2450 | 164 | |||
2451 | 162 | }; | 165 | }; |
2452 | 163 | plan.applyAllRecoveredTransactions(actor, plan.getDefaultRollbackListener()); | 166 | plan.applyAllRecoveredTransactions(actor, plan.getDefaultRollbackListener()); |
2453 | 164 | assertEquals(15, recoveryTimestamps.size()); | 167 | assertEquals(15, recoveryTimestamps.size()); |
2454 | @@ -193,8 +196,7 @@ | |||
2455 | 193 | store1(); | 196 | store1(); |
2456 | 194 | 197 | ||
2457 | 195 | jman.rollover(); | 198 | jman.rollover(); |
2460 | 196 | _persistit.checkpoint(); | 199 | drainJournal(); |
2459 | 197 | jman.copyBack(); | ||
2461 | 198 | assertEquals(0, jman.getPageMapSize()); | 200 | assertEquals(0, jman.getPageMapSize()); |
2462 | 199 | 201 | ||
2463 | 200 | final Transaction txn = _persistit.getTransaction(); | 202 | final Transaction txn = _persistit.getTransaction(); |
2464 | @@ -203,8 +205,10 @@ | |||
2465 | 203 | store1(); | 205 | store1(); |
2466 | 204 | txn.commit(); | 206 | txn.commit(); |
2467 | 205 | txn.end(); | 207 | txn.end(); |
2470 | 206 | // Flush an uncommitted version of this transaction - should | 208 | /* |
2471 | 207 | // prevent journal cleanup. | 209 | * Flush an uncommitted version of this transaction - should prevent |
2472 | 210 | * journal cleanup. | ||
2473 | 211 | */ | ||
2474 | 208 | txn.begin(); | 212 | txn.begin(); |
2475 | 209 | store0(); | 213 | store0(); |
2476 | 210 | txn.flushTransactionBuffer(true); | 214 | txn.flushTransactionBuffer(true); |
2477 | @@ -212,14 +216,13 @@ | |||
2478 | 212 | txn.end(); | 216 | txn.end(); |
2479 | 213 | 217 | ||
2480 | 214 | jman.rollover(); | 218 | jman.rollover(); |
2488 | 215 | _persistit.checkpoint(); | 219 | drainJournal(); |
2482 | 216 | jman.copyBack(); | ||
2483 | 217 | // | ||
2484 | 218 | // Because JournalManager thinks there's an open transaction | ||
2485 | 219 | // it should preserve the journal file containing the TX record | ||
2486 | 220 | // for the transaction. | ||
2487 | 221 | // | ||
2489 | 222 | assertEquals(0, jman.getPageMapSize()); | 220 | assertEquals(0, jman.getPageMapSize()); |
2490 | 221 | /* | ||
2491 | 222 | * Because JournalManager thinks there's an open transaction it should | ||
2492 | 223 | * preserve the journal file containing the TX record for the | ||
2493 | 224 | * transaction. | ||
2494 | 225 | */ | ||
2495 | 223 | assertTrue(jman.getBaseAddress() < jman.getCurrentAddress()); | 226 | assertTrue(jman.getBaseAddress() < jman.getCurrentAddress()); |
2496 | 224 | txn.begin(); | 227 | txn.begin(); |
2497 | 225 | store1(); | 228 | store1(); |
2498 | @@ -229,6 +232,7 @@ | |||
2499 | 229 | jman.unitTestClearTransactionMap(); | 232 | jman.unitTestClearTransactionMap(); |
2500 | 230 | 233 | ||
2501 | 231 | jman.rollover(); | 234 | jman.rollover(); |
2502 | 235 | _persistit.flushStatistics(); | ||
2503 | 232 | _persistit.checkpoint(); | 236 | _persistit.checkpoint(); |
2504 | 233 | /* | 237 | /* |
2505 | 234 | * The TI active transaction cache may be a bit out of date, which can | 238 | * The TI active transaction cache may be a bit out of date, which can |
2506 | @@ -238,8 +242,7 @@ | |||
2507 | 238 | * copier does the right thing. | 242 | * copier does the right thing. |
2508 | 239 | */ | 243 | */ |
2509 | 240 | _persistit.getTransactionIndex().updateActiveTransactionCache(); | 244 | _persistit.getTransactionIndex().updateActiveTransactionCache(); |
2512 | 241 | jman.copyBack(); | 245 | drainJournal(); |
2511 | 242 | |||
2513 | 243 | assertEquals(jman.getBaseAddress(), jman.getCurrentAddress()); | 246 | assertEquals(jman.getBaseAddress(), jman.getCurrentAddress()); |
2514 | 244 | assertEquals(0, jman.getPageMapSize()); | 247 | assertEquals(0, jman.getPageMapSize()); |
2515 | 245 | 248 | ||
2516 | @@ -381,9 +384,13 @@ | |||
2517 | 381 | assertTrue(_persistit.getJournalManager().getHandleCount() > updatedHandleValue); | 384 | assertTrue(_persistit.getJournalManager().getHandleCount() > updatedHandleValue); |
2518 | 382 | } | 385 | } |
2519 | 383 | 386 | ||
2520 | 387 | private final static int T1 = 1000; | ||
2521 | 388 | private final static int T2 = 2000; | ||
2522 | 389 | |||
2523 | 384 | @Test | 390 | @Test |
2524 | 385 | public void testIndexHoles() throws Exception { | 391 | public void testIndexHoles() throws Exception { |
2525 | 386 | _persistit.getJournalManager().setAppendOnly(true); | 392 | _persistit.getJournalManager().setAppendOnly(true); |
2526 | 393 | |||
2527 | 387 | final Transaction transaction = _persistit.getTransaction(); | 394 | final Transaction transaction = _persistit.getTransaction(); |
2528 | 388 | final StringBuilder sb = new StringBuilder(); | 395 | final StringBuilder sb = new StringBuilder(); |
2529 | 389 | while (sb.length() < 1000) { | 396 | while (sb.length() < 1000) { |
2530 | @@ -392,7 +399,7 @@ | |||
2531 | 392 | 399 | ||
2532 | 393 | final String s = sb.toString(); | 400 | final String s = sb.toString(); |
2533 | 394 | for (int cycle = 0; cycle < 2; cycle++) { | 401 | for (int cycle = 0; cycle < 2; cycle++) { |
2535 | 395 | for (int i = 1000; i < 2000; i++) { | 402 | for (int i = T1; i < T2; i++) { |
2536 | 396 | final Exchange exchange = _persistit.getExchange("persistit", "RecoveryTest" + i, true); | 403 | final Exchange exchange = _persistit.getExchange("persistit", "RecoveryTest" + i, true); |
2537 | 397 | transaction.begin(); | 404 | transaction.begin(); |
2538 | 398 | try { | 405 | try { |
2539 | @@ -407,7 +414,7 @@ | |||
2540 | 407 | } | 414 | } |
2541 | 408 | 415 | ||
2542 | 409 | for (int j = 0; j < 20; j++) { | 416 | for (int j = 0; j < 20; j++) { |
2544 | 410 | for (int i = 1000; i < 2000; i++) { | 417 | for (int i = T1; i < T2; i++) { |
2545 | 411 | final Exchange exchange = _persistit.getExchange("persistit", "RecoveryTest" + i, true); | 418 | final Exchange exchange = _persistit.getExchange("persistit", "RecoveryTest" + i, true); |
2546 | 412 | transaction.begin(); | 419 | transaction.begin(); |
2547 | 413 | try { | 420 | try { |
2548 | @@ -419,7 +426,7 @@ | |||
2549 | 419 | } | 426 | } |
2550 | 420 | } | 427 | } |
2551 | 421 | 428 | ||
2553 | 422 | for (int i = 1000; i < 2000; i += 2) { | 429 | for (int i = T1; i < T2; i += 2) { |
2554 | 423 | transaction.begin(); | 430 | transaction.begin(); |
2555 | 424 | try { | 431 | try { |
2556 | 425 | final Exchange exchange = _persistit.getExchange("persistit", "RecoveryTest" + i, true); | 432 | final Exchange exchange = _persistit.getExchange("persistit", "RecoveryTest" + i, true); |
2557 | @@ -430,6 +437,7 @@ | |||
2558 | 430 | } | 437 | } |
2559 | 431 | } | 438 | } |
2560 | 432 | } | 439 | } |
2561 | 440 | _persistit.checkAllVolumes(); | ||
2562 | 433 | _persistit.crash(); | 441 | _persistit.crash(); |
2563 | 434 | 442 | ||
2564 | 435 | _persistit = new Persistit(); | 443 | _persistit = new Persistit(); |
2565 | 436 | 444 | ||
2566 | === modified file 'src/test/java/com/persistit/SplitPolicyTest.java' | |||
2567 | --- src/test/java/com/persistit/SplitPolicyTest.java 2012-08-24 13:57:19 +0000 | |||
2568 | +++ src/test/java/com/persistit/SplitPolicyTest.java 2013-04-29 21:38:25 +0000 | |||
2569 | @@ -253,7 +253,7 @@ | |||
2570 | 253 | // | 253 | // |
2571 | 254 | // forward sequential | 254 | // forward sequential |
2572 | 255 | // | 255 | // |
2574 | 256 | for (long page = 2; page < 20; page++) { | 256 | for (long page = ex.clear().append(0).fetchBufferCopy(0).getPageAddress(); page < 20; page++) { |
2575 | 257 | final Buffer buffer = ex.getBufferPool().get(ex.getVolume(), page, false, true); | 257 | final Buffer buffer = ex.getBufferPool().get(ex.getVolume(), page, false, true); |
2576 | 258 | if (buffer.isDataPage()) { | 258 | if (buffer.isDataPage()) { |
2577 | 259 | final int available = buffer.getAvailableSize(); | 259 | final int available = buffer.getAvailableSize(); |
2578 | 260 | 260 | ||
2579 | === added file 'src/test/java/com/persistit/TimelyResourceTest.java' | |||
2580 | --- src/test/java/com/persistit/TimelyResourceTest.java 1970-01-01 00:00:00 +0000 | |||
2581 | +++ src/test/java/com/persistit/TimelyResourceTest.java 2013-04-29 21:38:25 +0000 | |||
2582 | @@ -0,0 +1,290 @@ | |||
2583 | 1 | /** | ||
2584 | 2 | * Copyright © 2011-2012 Akiban Technologies, Inc. All rights reserved. | ||
2585 | 3 | * | ||
2586 | 4 | * This program and the accompanying materials are made available | ||
2587 | 5 | * under the terms of the Eclipse Public License v1.0 which | ||
2588 | 6 | * accompanies this distribution, and is available at | ||
2589 | 7 | * http://www.eclipse.org/legal/epl-v10.html | ||
2590 | 8 | * | ||
2591 | 9 | * This program may also be available under different license terms. | ||
2592 | 10 | * For more information, see www.akiban.com or contact licensing@akiban.com. | ||
2593 | 11 | * | ||
2594 | 12 | * Contributors: | ||
2595 | 13 | * Akiban Technologies, Inc. | ||
2596 | 14 | */ | ||
2597 | 15 | |||
2598 | 16 | package com.persistit; | ||
2599 | 17 | |||
2600 | 18 | import static com.persistit.TransactionIndex.tss2vh; | ||
2601 | 19 | import static com.persistit.TransactionStatus.UNCOMMITTED; | ||
2602 | 20 | import static com.persistit.unit.ConcurrentUtil.assertSuccess; | ||
2603 | 21 | import static com.persistit.unit.ConcurrentUtil.createThread; | ||
2604 | 22 | import static com.persistit.unit.ConcurrentUtil.join; | ||
2605 | 23 | import static com.persistit.unit.ConcurrentUtil.start; | ||
2606 | 24 | import static com.persistit.util.Util.NS_PER_S; | ||
2607 | 25 | import static org.junit.Assert.assertEquals; | ||
2608 | 26 | import static org.junit.Assert.assertTrue; | ||
2609 | 27 | |||
2610 | 28 | import java.util.ArrayList; | ||
2611 | 29 | import java.util.Iterator; | ||
2612 | 30 | import java.util.List; | ||
2613 | 31 | import java.util.Random; | ||
2614 | 32 | import java.util.concurrent.Semaphore; | ||
2615 | 33 | import java.util.concurrent.atomic.AtomicInteger; | ||
2616 | 34 | |||
2617 | 35 | import org.junit.Test; | ||
2618 | 36 | |||
2619 | 37 | import com.persistit.Version.PrunableVersion; | ||
2620 | 38 | import com.persistit.Version.VersionCreator; | ||
2621 | 39 | import com.persistit.exception.PersistitException; | ||
2622 | 40 | import com.persistit.exception.RollbackException; | ||
2623 | 41 | import com.persistit.unit.ConcurrentUtil.ThrowingRunnable; | ||
2624 | 42 | import com.persistit.unit.ConcurrentUtil.UncaughtExceptionHandler; | ||
2625 | 43 | import com.persistit.util.Util; | ||
2626 | 44 | |||
2627 | 45 | public class TimelyResourceTest extends PersistitUnitTestCase { | ||
2628 | 46 | |||
2629 | 47 | private int _idCounter; | ||
2630 | 48 | |||
2631 | 49 | static class TestVersion implements PrunableVersion { | ||
2632 | 50 | final int _id; | ||
2633 | 51 | final TimelyResourceTest _test; | ||
2634 | 52 | final AtomicInteger _pruned = new AtomicInteger(); | ||
2635 | 53 | |||
2636 | 54 | TestVersion(final int id, final TimelyResourceTest test) { | ||
2637 | 55 | _id = id; | ||
2638 | 56 | _test = test; | ||
2639 | 57 | } | ||
2640 | 58 | |||
2641 | 59 | @Override | ||
2642 | 60 | public boolean prune() { | ||
2643 | 61 | _pruned.incrementAndGet(); | ||
2644 | 62 | return true; | ||
2645 | 63 | } | ||
2646 | 64 | |||
2647 | 65 | @Override | ||
2648 | 66 | public void vacate() { | ||
2649 | 67 | System.out.println("No more versions"); | ||
2650 | 68 | } | ||
2651 | 69 | |||
2652 | 70 | @Override | ||
2653 | 71 | public String toString() { | ||
2654 | 72 | return String.format("<%,d:%,d>", _id, _pruned.get()); | ||
2655 | 73 | } | ||
2656 | 74 | |||
2657 | 75 | TimelyResourceTest getContainer() { | ||
2658 | 76 | return _test; | ||
2659 | 77 | } | ||
2660 | 78 | } | ||
2661 | 79 | |||
2662 | 80 | @Test | ||
2663 | 81 | public void testAddAndPruneResources() throws Exception { | ||
2664 | 82 | testAddAndPruneResources1(false); | ||
2665 | 83 | testAddAndPruneResources1(true); | ||
2666 | 84 | } | ||
2667 | 85 | |||
2668 | 86 | private void testAddAndPruneResources1(final boolean withTransactions) throws Exception { | ||
2669 | 87 | final Transaction txn = _persistit.getTransaction(); | ||
2670 | 88 | final TimelyResource<TestVersion> tr = new TimelyResource<TestVersion>(_persistit); | ||
2671 | 89 | final long[] history = new long[5]; | ||
2672 | 90 | final TestVersion[] resources = new TestVersion[5]; | ||
2673 | 91 | for (int i = 0; i < 5; i++) { | ||
2674 | 92 | if (withTransactions) { | ||
2675 | 93 | txn.begin(); | ||
2676 | 94 | } | ||
2677 | 95 | final TestVersion resource = new TestVersion(i, this); | ||
2678 | 96 | resources[i] = resource; | ||
2679 | 97 | if (!tr.isEmpty()) { | ||
2680 | 98 | tr.delete(); | ||
2681 | 99 | } | ||
2682 | 100 | tr.addVersion(resource, txn); | ||
2683 | 101 | if (withTransactions) { | ||
2684 | 102 | txn.commit(); | ||
2685 | 103 | txn.end(); | ||
2686 | 104 | } | ||
2687 | 105 | history[i] = _persistit.getTimestampAllocator().updateTimestamp(); | ||
2688 | 106 | } | ||
2689 | 107 | assertEquals("Incorrect version count " + withTransactions, 9, tr.getVersionCount()); | ||
2690 | 108 | |||
2691 | 109 | for (int i = 0; i < 5; i++) { | ||
2692 | 110 | final TestVersion t = tr.getVersion(tss2vh(history[i], 0)); | ||
2693 | 111 | assertTrue("Missing version " + withTransactions, t != null); | ||
2694 | 112 | assertEquals("Wrong version " + withTransactions, i, t._id); | ||
2695 | 113 | } | ||
2696 | 114 | _persistit.getTransactionIndex().updateActiveTransactionCache(); | ||
2697 | 115 | tr.prune(); | ||
2698 | 116 | assertEquals("Should have one version left " + withTransactions, 1, tr.getVersionCount()); | ||
2699 | 117 | assertEquals("Wrong version " + withTransactions, 4, tr.getVersion(tss2vh(UNCOMMITTED, 0))._id); | ||
2700 | 118 | |||
2701 | 119 | tr.delete(); | ||
2702 | 120 | |||
2703 | 121 | assertEquals("Should have two versions left " + withTransactions, 2, tr.getVersionCount()); | ||
2704 | 122 | _persistit.getTransactionIndex().updateActiveTransactionCache(); | ||
2705 | 123 | tr.prune(); | ||
2706 | 124 | assertEquals("Should have no versions left " + withTransactions, 0, tr.getVersionCount()); | ||
2707 | 125 | |||
2708 | 126 | for (int i = 0; i < 5; i++) { | ||
2709 | 127 | assertEquals("Should have been pruned " + withTransactions, 1, resources[i]._pruned.get()); | ||
2710 | 128 | } | ||
2711 | 129 | } | ||
2712 | 130 | |||
2713 | 131 | @Test | ||
2714 | 132 | public void concurrentAddAndPruneResources() throws Exception { | ||
2715 | 133 | final TimelyResource<TestVersion> tr = new TimelyResource<TestVersion>(_persistit); | ||
2716 | 134 | final Random random = new Random(1); | ||
2717 | 135 | final long expires = System.nanoTime() + 10 * NS_PER_S; | ||
2718 | 136 | final AtomicInteger sequence = new AtomicInteger(); | ||
2719 | 137 | final AtomicInteger rollbackCount = new AtomicInteger(); | ||
2720 | 138 | final List<Thread> threads = new ArrayList<Thread>(); | ||
2721 | 139 | final UncaughtExceptionHandler handler = new UncaughtExceptionHandler(); | ||
2722 | 140 | int threadCounter = 0; | ||
2723 | 141 | while (System.nanoTime() < expires) { | ||
2724 | 142 | for (final Iterator<Thread> iter = threads.iterator(); iter.hasNext();) { | ||
2725 | 143 | if (!iter.next().isAlive()) { | ||
2726 | 144 | iter.remove(); | ||
2727 | 145 | } | ||
2728 | 146 | } | ||
2729 | 147 | while (threads.size() < 20) { | ||
2730 | 148 | final Thread t = createThread(String.format("Thread_%06d", ++threadCounter), new ThrowingRunnable() { | ||
2731 | 149 | @Override | ||
2732 | 150 | public void run() throws Exception { | ||
2733 | 151 | doConcurrentTransaction(tr, random, sequence, rollbackCount); | ||
2734 | 152 | } | ||
2735 | 153 | }); | ||
2736 | 154 | threads.add(t); | ||
2737 | 155 | start(handler, t); | ||
2738 | 156 | } | ||
2739 | 157 | Util.sleep(10); | ||
2740 | 158 | tr.prune(); | ||
2741 | 159 | } | ||
2742 | 160 | join(Long.MAX_VALUE, handler.getThrowableMap(), threads.toArray(new Thread[threads.size()])); | ||
2743 | 161 | assertSuccess(handler.getThrowableMap()); | ||
2744 | 162 | assertTrue("Every transaction rolled back", rollbackCount.get() < sequence.get()); | ||
2745 | 163 | System.out.printf("%,d entries, %,d rollbacks\n", sequence.get(), rollbackCount.get()); | ||
2746 | 164 | } | ||
2747 | 165 | |||
2748 | 166 | private void doConcurrentTransaction(final TimelyResource<TestVersion> tr, final Random random, | ||
2749 | 167 | final AtomicInteger sequence, final AtomicInteger rollbackCount) throws PersistitException { | ||
2750 | 168 | try { | ||
2751 | 169 | final Transaction txn = _persistit.getTransaction(); | ||
2752 | 170 | for (int i = 0; i < 25; i++) { | ||
2753 | 171 | txn.begin(); | ||
2754 | 172 | try { | ||
2755 | 173 | final int id = sequence.incrementAndGet(); | ||
2756 | 174 | tr.addVersion(new TestVersion(id, this), txn); | ||
2757 | 175 | final int delay = (1 << random.nextInt(3)); | ||
2758 | 176 | // Up to 7/1000 of a second | ||
2759 | 177 | Util.sleep(delay); | ||
2760 | 178 | final TestVersion mine = tr.getVersion(); | ||
2761 | 179 | assertEquals("Should not have been pruned yet", 0, mine._pruned.get()); | ||
2762 | 180 | assertEquals("Wrong resource", id, mine._id); | ||
2763 | 181 | if (random.nextInt(10) == 0) { | ||
2764 | 182 | txn.rollback(); | ||
2765 | 183 | } else { | ||
2766 | 184 | txn.commit(); | ||
2767 | 185 | } | ||
2768 | 186 | } catch (final RollbackException e) { | ||
2769 | 187 | txn.rollback(); | ||
2770 | 188 | rollbackCount.incrementAndGet(); | ||
2771 | 189 | } finally { | ||
2772 | 190 | txn.end(); | ||
2773 | 191 | } | ||
2774 | 192 | } | ||
2775 | 193 | } catch (final RollbackException e) { | ||
2776 | 194 | rollbackCount.incrementAndGet(); | ||
2777 | 195 | } | ||
2778 | 196 | } | ||
2779 | 197 | |||
2780 | 198 | @Test | ||
2781 | 199 | public void deleteResource() throws Exception { | ||
2782 | 200 | final TimelyResource<TestVersion> tr = new TimelyResource<TestVersion>(_persistit); | ||
2783 | 201 | _idCounter = 0; | ||
2784 | 202 | final VersionCreator<TestVersion> creator = new VersionCreator<TestVersion>() { | ||
2785 | 203 | |||
2786 | 204 | @Override | ||
2787 | 205 | public TestVersion createVersion(final TimelyResource<? extends TestVersion> resource) | ||
2788 | 206 | throws PersistitException { | ||
2789 | 207 | return new TestVersion(++_idCounter, TimelyResourceTest.this); | ||
2790 | 208 | } | ||
2791 | 209 | }; | ||
2792 | 210 | final Transaction txn1 = _persistit.getTransaction(); | ||
2793 | 211 | _persistit.setSessionId(new SessionId()); | ||
2794 | 212 | final Transaction txn2 = _persistit.getTransaction(); | ||
2795 | 213 | TestVersion v1; | ||
2796 | 214 | txn1.begin(); | ||
2797 | 215 | v1 = tr.getVersion(creator); | ||
2798 | 216 | assertTrue("Version ID mismatch", v1._id == _idCounter); | ||
2799 | 217 | txn2.begin(); | ||
2800 | 218 | txn1.incrementStep(); | ||
2801 | 219 | tr.delete(); | ||
2802 | 220 | tr.prune(); | ||
2803 | 221 | assertEquals("Should still be two versions", 2, tr.getVersionCount()); | ||
2804 | 222 | txn2.commit(); | ||
2805 | 223 | txn1.commit(); | ||
2806 | 224 | _persistit.getTransactionIndex().updateActiveTransactionCache(); | ||
2807 | 225 | tr.prune(); | ||
2808 | 226 | assertEquals("Should now have no versions", 0, tr.getVersionCount()); | ||
2809 | 227 | txn1.end(); | ||
2810 | 228 | txn2.end(); | ||
2811 | 229 | |||
2812 | 230 | } | ||
2813 | 231 | |||
2814 | 232 | @Test | ||
2815 | 233 | public void versions() throws Exception { | ||
2816 | 234 | final TimelyResource<TestVersion> tr = new TimelyResource<TestVersion>(_persistit); | ||
2817 | 235 | _idCounter = 0; | ||
2818 | 236 | final VersionCreator<TestVersion> creator = new VersionCreator<TestVersion>() { | ||
2819 | 237 | |||
2820 | 238 | @Override | ||
2821 | 239 | public TestVersion createVersion(final TimelyResource<? extends TestVersion> resource) | ||
2822 | 240 | throws PersistitException { | ||
2823 | 241 | return new TestVersion(++_idCounter, TimelyResourceTest.this); | ||
2824 | 242 | } | ||
2825 | 243 | }; | ||
2826 | 244 | final Semaphore semaphore1 = new Semaphore(0); | ||
2827 | 245 | final Transaction txn = _persistit.getTransaction(); | ||
2828 | 246 | final Thread[] threads = new Thread[10]; | ||
2829 | 247 | for (int i = 0; i < 10; i++) { | ||
2830 | 248 | try { | ||
2831 | 249 | txn.begin(); | ||
2832 | 250 | tr.addVersion(creator.createVersion(tr), txn); | ||
2833 | 251 | txn.commit(); | ||
2834 | 252 | } finally { | ||
2835 | 253 | txn.end(); | ||
2836 | 254 | } | ||
2837 | 255 | final Semaphore semaphore2 = new Semaphore(0); | ||
2838 | 256 | threads[i] = new Thread(new Runnable() { | ||
2839 | 257 | @Override | ||
2840 | 258 | public void run() { | ||
2841 | 259 | final Transaction txn = _persistit.getTransaction(); | ||
2842 | 260 | try { | ||
2843 | 261 | txn.begin(); | ||
2844 | 262 | final TestVersion t = tr.getVersion(); | ||
2845 | 263 | assertEquals(t._id, _idCounter); | ||
2846 | 264 | semaphore2.release(); | ||
2847 | 265 | semaphore1.acquire(); | ||
2848 | 266 | txn.commit(); | ||
2849 | 267 | } catch (final Exception e) { | ||
2850 | 268 | e.printStackTrace(); | ||
2851 | 269 | } finally { | ||
2852 | 270 | txn.end(); | ||
2853 | 271 | } | ||
2854 | 272 | } | ||
2855 | 273 | }); | ||
2856 | 274 | threads[i].start(); | ||
2857 | 275 | semaphore2.acquire(); | ||
2858 | 276 | } | ||
2859 | 277 | _persistit.getTransactionIndex().updateActiveTransactionCache(); | ||
2860 | 278 | tr.prune(); | ||
2861 | 279 | assertEquals(10, tr.getVersionCount()); | ||
2862 | 280 | semaphore1.release(10); | ||
2863 | 281 | for (final Thread thread : threads) { | ||
2864 | 282 | thread.join(); | ||
2865 | 283 | } | ||
2866 | 284 | _persistit.getTransactionIndex().updateActiveTransactionCache(); | ||
2867 | 285 | tr.prune(); | ||
2868 | 286 | assertEquals(1, tr.getVersionCount()); | ||
2869 | 287 | assertEquals("Surviving primordial version should be last one committed", 10, tr.getVersion(null)._id); | ||
2870 | 288 | } | ||
2871 | 289 | |||
2872 | 290 | } | ||
2873 | 0 | 291 | ||
2874 | === modified file 'src/test/java/com/persistit/TreeLifetimeTest.java' | |||
2875 | --- src/test/java/com/persistit/TreeLifetimeTest.java 2012-08-24 13:57:19 +0000 | |||
2876 | +++ src/test/java/com/persistit/TreeLifetimeTest.java 2013-04-29 21:38:25 +0000 | |||
2877 | @@ -15,20 +15,6 @@ | |||
2878 | 15 | 15 | ||
2879 | 16 | package com.persistit; | 16 | package com.persistit; |
2880 | 17 | 17 | ||
2881 | 18 | import static com.persistit.util.SequencerConstants.TREE_CREATE_REMOVE_A; | ||
2882 | 19 | import static com.persistit.util.SequencerConstants.TREE_CREATE_REMOVE_B; | ||
2883 | 20 | import static com.persistit.util.SequencerConstants.TREE_CREATE_REMOVE_C; | ||
2884 | 21 | import static com.persistit.util.SequencerConstants.TREE_CREATE_REMOVE_SCHEDULE; | ||
2885 | 22 | import static com.persistit.util.ThreadSequencer.addSchedules; | ||
2886 | 23 | import static com.persistit.util.ThreadSequencer.array; | ||
2887 | 24 | import static com.persistit.util.ThreadSequencer.describeHistory; | ||
2888 | 25 | import static com.persistit.util.ThreadSequencer.describePartialOrdering; | ||
2889 | 26 | import static com.persistit.util.ThreadSequencer.disableSequencer; | ||
2890 | 27 | import static com.persistit.util.ThreadSequencer.enableSequencer; | ||
2891 | 28 | import static com.persistit.util.ThreadSequencer.historyMeetsPartialOrdering; | ||
2892 | 29 | import static com.persistit.util.ThreadSequencer.out; | ||
2893 | 30 | import static com.persistit.util.ThreadSequencer.rawSequenceHistoryCopy; | ||
2894 | 31 | import static com.persistit.util.ThreadSequencer.sequence; | ||
2895 | 32 | import static org.junit.Assert.assertEquals; | 18 | import static org.junit.Assert.assertEquals; |
2896 | 33 | import static org.junit.Assert.assertFalse; | 19 | import static org.junit.Assert.assertFalse; |
2897 | 34 | import static org.junit.Assert.assertNotNull; | 20 | import static org.junit.Assert.assertNotNull; |
2898 | @@ -38,7 +24,6 @@ | |||
2899 | 38 | 24 | ||
2900 | 39 | import java.util.Arrays; | 25 | import java.util.Arrays; |
2901 | 40 | import java.util.List; | 26 | import java.util.List; |
2902 | 41 | import java.util.concurrent.ConcurrentLinkedQueue; | ||
2903 | 42 | 27 | ||
2904 | 43 | import org.junit.Test; | 28 | import org.junit.Test; |
2905 | 44 | 29 | ||
2906 | @@ -49,9 +34,6 @@ | |||
2907 | 49 | 34 | ||
2908 | 50 | public class TreeLifetimeTest extends PersistitUnitTestCase { | 35 | public class TreeLifetimeTest extends PersistitUnitTestCase { |
2909 | 51 | private static final String TREE_NAME = "tree_one"; | 36 | private static final String TREE_NAME = "tree_one"; |
2910 | 52 | final int A = TREE_CREATE_REMOVE_A; | ||
2911 | 53 | final int B = TREE_CREATE_REMOVE_B; | ||
2912 | 54 | final int C = TREE_CREATE_REMOVE_C; | ||
2913 | 55 | 37 | ||
2914 | 56 | private Volume getVolume() { | 38 | private Volume getVolume() { |
2915 | 57 | return _persistit.getVolume(UnitTestProperties.VOLUME_NAME); | 39 | return _persistit.getVolume(UnitTestProperties.VOLUME_NAME); |
2916 | @@ -62,11 +44,34 @@ | |||
2917 | 62 | } | 44 | } |
2918 | 63 | 45 | ||
2919 | 64 | @Test | 46 | @Test |
2921 | 65 | public void testRemovedTreeGoesToGarbageChain() throws PersistitException { | 47 | public void testRemovedTreeGoesToGarbageChainNoTxn() throws PersistitException { |
2922 | 48 | Exchange ex = getExchange(true); | ||
2923 | 49 | for (int i = 0; i < 5; ++i) { | ||
2924 | 50 | ex.clear().append(i).getValue().clear().put(i); | ||
2925 | 51 | ex.store(); | ||
2926 | 52 | } | ||
2927 | 53 | _persistit.releaseExchange(ex); | ||
2928 | 54 | ex = null; | ||
2929 | 55 | |||
2930 | 56 | ex = getExchange(false); | ||
2931 | 57 | final long treeRoot = ex.getTree().getRootPageAddr(); | ||
2932 | 58 | ex.removeTree(); | ||
2933 | 59 | _persistit.releaseExchange(ex); | ||
2934 | 60 | ex = null; | ||
2935 | 61 | _persistit.cleanup(); | ||
2936 | 62 | |||
2937 | 63 | final List<Long> garbage = getVolume().getStructure().getGarbageList(); | ||
2938 | 64 | assertTrue("Expected tree root <" + treeRoot + "> in garbage list <" + garbage.toString() + ">", | ||
2939 | 65 | garbage.contains(treeRoot)); | ||
2940 | 66 | } | ||
2941 | 67 | |||
2942 | 68 | @Test | ||
2943 | 69 | public void testRemovedTreeGoesToGarbageChainTxn() throws PersistitException { | ||
2944 | 66 | final Transaction txn = _persistit.getTransaction(); | 70 | final Transaction txn = _persistit.getTransaction(); |
2945 | 71 | Exchange ex; | ||
2946 | 67 | 72 | ||
2947 | 68 | txn.begin(); | 73 | txn.begin(); |
2949 | 69 | Exchange ex = getExchange(true); | 74 | ex = getExchange(true); |
2950 | 70 | for (int i = 0; i < 5; ++i) { | 75 | for (int i = 0; i < 5; ++i) { |
2951 | 71 | ex.clear().append(i).getValue().clear().put(i); | 76 | ex.clear().append(i).getValue().clear().put(i); |
2952 | 72 | ex.store(); | 77 | ex.store(); |
2953 | @@ -84,7 +89,7 @@ | |||
2954 | 84 | txn.end(); | 89 | txn.end(); |
2955 | 85 | _persistit.releaseExchange(ex); | 90 | _persistit.releaseExchange(ex); |
2956 | 86 | ex = null; | 91 | ex = null; |
2958 | 87 | 92 | _persistit.cleanup(); | |
2959 | 88 | final List<Long> garbage = getVolume().getStructure().getGarbageList(); | 93 | final List<Long> garbage = getVolume().getStructure().getGarbageList(); |
2960 | 89 | assertTrue("Expected tree root <" + treeRoot + "> in garbage list <" + garbage.toString() + ">", | 94 | assertTrue("Expected tree root <" + treeRoot + "> in garbage list <" + garbage.toString() + ">", |
2961 | 90 | garbage.contains(treeRoot)); | 95 | garbage.contains(treeRoot)); |
2962 | @@ -143,71 +148,4 @@ | |||
2963 | 143 | assertNull("Tree should not exist after cleanup action", getVolume().getTree(TREE_NAME, false)); | 148 | assertNull("Tree should not exist after cleanup action", getVolume().getTree(TREE_NAME, false)); |
2964 | 144 | } | 149 | } |
2965 | 145 | 150 | ||
2966 | 146 | @Test | ||
2967 | 147 | public void testReanimatedTreeCreateAndRemoveSynchronization() throws PersistitException, InterruptedException { | ||
2968 | 148 | enableSequencer(true); | ||
2969 | 149 | addSchedules(TREE_CREATE_REMOVE_SCHEDULE); | ||
2970 | 150 | |||
2971 | 151 | final ConcurrentLinkedQueue<Throwable> threadErrors = new ConcurrentLinkedQueue<Throwable>(); | ||
2972 | 152 | |||
2973 | 153 | final Exchange origEx = getExchange(true); | ||
2974 | 154 | for (int i = 0; i < 5; ++i) { | ||
2975 | 155 | origEx.clear().append(i).store(); | ||
2976 | 156 | } | ||
2977 | 157 | _persistit.releaseExchange(origEx); | ||
2978 | 158 | |||
2979 | 159 | final Thread thread1 = new Thread(new Runnable() { | ||
2980 | 160 | @Override | ||
2981 | 161 | public void run() { | ||
2982 | 162 | Exchange ex = null; | ||
2983 | 163 | try { | ||
2984 | 164 | ex = getExchange(false); | ||
2985 | 165 | ex.removeTree(); | ||
2986 | 166 | } catch (final Throwable t) { | ||
2987 | 167 | threadErrors.add(t); | ||
2988 | 168 | } | ||
2989 | 169 | if (ex != null) { | ||
2990 | 170 | _persistit.releaseExchange(ex); | ||
2991 | 171 | } | ||
2992 | 172 | } | ||
2993 | 173 | }); | ||
2994 | 174 | |||
2995 | 175 | final Thread thread2 = new Thread(new Runnable() { | ||
2996 | 176 | @Override | ||
2997 | 177 | public void run() { | ||
2998 | 178 | sequence(TREE_CREATE_REMOVE_B); | ||
2999 | 179 | Exchange ex = null; | ||
3000 | 180 | try { | ||
3001 | 181 | ex = getExchange(true); | ||
3002 | 182 | int count = 0; | ||
3003 | 183 | while (ex.next(true)) { | ||
3004 | 184 | ++count; | ||
3005 | 185 | } | ||
3006 | 186 | sequence(TREE_CREATE_REMOVE_C); | ||
3007 | 187 | assertEquals("New tree has zero keys in it", 0, count); | ||
3008 | 188 | } catch (final Throwable t) { | ||
3009 | 189 | threadErrors.add(t); | ||
3010 | 190 | } | ||
3011 | 191 | if (ex != null) { | ||
3012 | 192 | _persistit.releaseExchange(ex); | ||
3013 | 193 | } | ||
3014 | 194 | } | ||
3015 | 195 | }); | ||
3016 | 196 | |||
3017 | 197 | thread1.start(); | ||
3018 | 198 | thread2.start(); | ||
3019 | 199 | |||
3020 | 200 | thread1.join(); | ||
3021 | 201 | thread2.join(); | ||
3022 | 202 | |||
3023 | 203 | assertEquals("Threads had no exceptions", "[]", threadErrors.toString()); | ||
3024 | 204 | |||
3025 | 205 | final int[] actual = rawSequenceHistoryCopy(); | ||
3026 | 206 | final int[][] expectedSequence = { array(A, B), array(out(B)), array(C), array(out(A), out(C)) }; | ||
3027 | 207 | if (!historyMeetsPartialOrdering(actual, expectedSequence)) { | ||
3028 | 208 | assertEquals("Unexpected sequencing", describePartialOrdering(expectedSequence), describeHistory(actual)); | ||
3029 | 209 | } | ||
3030 | 210 | |||
3031 | 211 | disableSequencer(); | ||
3032 | 212 | } | ||
3033 | 213 | } | 151 | } |
3034 | 214 | 152 | ||
3035 | === added file 'src/test/java/com/persistit/TreeTransactionalLifetimeTest.java' | |||
3036 | --- src/test/java/com/persistit/TreeTransactionalLifetimeTest.java 1970-01-01 00:00:00 +0000 | |||
3037 | +++ src/test/java/com/persistit/TreeTransactionalLifetimeTest.java 2013-04-29 21:38:25 +0000 | |||
3038 | @@ -0,0 +1,277 @@ | |||
3039 | 1 | /** | ||
3040 | 2 | * Copyright © 2013 Akiban Technologies, Inc. All rights reserved. | ||
3041 | 3 | * | ||
3042 | 4 | * This program and the accompanying materials are made available | ||
3043 | 5 | * under the terms of the Eclipse Public License v1.0 which | ||
3044 | 6 | * accompanies this distribution, and is available at | ||
3045 | 7 | * http://www.eclipse.org/legal/epl-v10.html | ||
3046 | 8 | * | ||
3047 | 9 | * This program may also be available under different license terms. | ||
3048 | 10 | * For more information, see www.akiban.com or contact licensing@akiban.com. | ||
3049 | 11 | * | ||
3050 | 12 | * Contributors: | ||
3051 | 13 | * Akiban Technologies, Inc. | ||
3052 | 14 | */ | ||
3053 | 15 | |||
3054 | 16 | package com.persistit; | ||
3055 | 17 | |||
3056 | 18 | import static com.persistit.unit.ConcurrentUtil.assertSuccess; | ||
3057 | 19 | import static com.persistit.unit.ConcurrentUtil.createThread; | ||
3058 | 20 | import static com.persistit.unit.ConcurrentUtil.join; | ||
3059 | 21 | import static com.persistit.unit.ConcurrentUtil.start; | ||
3060 | 22 | import static org.junit.Assert.assertEquals; | ||
3061 | 23 | import static org.junit.Assert.assertNull; | ||
3062 | 24 | import static org.junit.Assert.assertTrue; | ||
3063 | 25 | |||
3064 | 26 | import java.util.Map; | ||
3065 | 27 | import java.util.concurrent.Semaphore; | ||
3066 | 28 | |||
3067 | 29 | import org.junit.Test; | ||
3068 | 30 | |||
3069 | 31 | import com.persistit.exception.PersistitException; | ||
3070 | 32 | import com.persistit.unit.ConcurrentUtil.ThrowingRunnable; | ||
3071 | 33 | |||
3072 | 34 | public class TreeTransactionalLifetimeTest extends PersistitUnitTestCase { | ||
3073 | 35 | final static int TIMEOUT_MS = 10000; | ||
3074 | 36 | |||
3075 | 37 | final Semaphore semA = new Semaphore(0); | ||
3076 | 38 | final Semaphore semB = new Semaphore(0); | ||
3077 | 39 | final Semaphore semT = new Semaphore(0); | ||
3078 | 40 | final Semaphore semU = new Semaphore(0); | ||
3079 | 41 | |||
3080 | 42 | /* | ||
3081 | 43 | * This class needs to be in com.persistit because of some package-private | ||
3082 | 44 | * methods used in controlling the test. | ||
3083 | 45 | */ | ||
3084 | 46 | |||
3085 | 47 | private Tree tree(final String name) throws PersistitException { | ||
3086 | 48 | return vstruc().getTree(name, false); | ||
3087 | 49 | } | ||
3088 | 50 | |||
3089 | 51 | private Exchange exchange(final String name) throws PersistitException { | ||
3090 | 52 | return _persistit.getExchange("persistit", name, true); | ||
3091 | 53 | } | ||
3092 | 54 | |||
3093 | 55 | private VolumeStructure vstruc() { | ||
3094 | 56 | return _persistit.getVolume("persistit").getStructure(); | ||
3095 | 57 | } | ||
3096 | 58 | |||
3097 | 59 | @Test | ||
3098 | 60 | public void simplePruning() throws Exception { | ||
3099 | 61 | |||
3100 | 62 | final Transaction txn = _persistit.getTransaction(); | ||
3101 | 63 | for (int i = 0; i < 2; i++) { | ||
3102 | 64 | txn.begin(); | ||
3103 | 65 | final Exchange ex = exchange("ttlt"); | ||
3104 | 66 | ex.getValue().put(RED_FOX); | ||
3105 | 67 | ex.to(1).store(); | ||
3106 | 68 | ex.to(2).store(); | ||
3107 | 69 | txn.rollback(); | ||
3108 | 70 | txn.end(); | ||
3109 | 71 | } | ||
3110 | 72 | _persistit.cleanup(); | ||
3111 | 73 | assertEquals("There should be no tree", null, tree("ttlt")); | ||
3112 | 74 | assertTrue(vstruc().getGarbageRoot() != 0); | ||
3113 | 75 | } | ||
3114 | 76 | |||
3115 | 77 | @Test | ||
3116 | 78 | public void createdTreeIsNotVisibleUntilCommit() throws Exception { | ||
3117 | 79 | final Thread t = createThread("ttlt", new TExec() { | ||
3118 | 80 | @Override | ||
3119 | 81 | void exec(final Transaction txn) throws Exception { | ||
3120 | 82 | final Exchange ex1 = exchange("ttlt"); | ||
3121 | 83 | ex1.getValue().put(RED_FOX); | ||
3122 | 84 | ex1.to(1).store(); | ||
3123 | 85 | semA.release(); | ||
3124 | 86 | semB.acquire(); | ||
3125 | 87 | txn.commit(); | ||
3126 | 88 | } | ||
3127 | 89 | }); | ||
3128 | 90 | final Map<Thread, Throwable> errors = start(t); | ||
3129 | 91 | semA.acquire(); | ||
3130 | 92 | assertEquals(null, _persistit.getVolume("persistit").getTree("ttlt", false)); | ||
3131 | 93 | semB.release(); | ||
3132 | 94 | join(TIMEOUT_MS, errors, t); | ||
3133 | 95 | assertSuccess(errors); | ||
3134 | 96 | final Exchange ex = exchange("ttlt"); | ||
3135 | 97 | assertTrue(ex.to(Key.BEFORE).next()); | ||
3136 | 98 | assertEquals(1, ex.getKey().decodeInt()); | ||
3137 | 99 | } | ||
3138 | 100 | |||
3139 | 101 | @Test | ||
3140 | 102 | public void removeTreeIsNotVisibleUntilCommit() throws Exception { | ||
3141 | 103 | final Exchange ex = exchange("ttlt"); | ||
3142 | 104 | ex.getValue().put(RED_FOX); | ||
3143 | 105 | ex.to(1).store(); | ||
3144 | 106 | final Thread t = createThread("ttlt", new TExec() { | ||
3145 | 107 | @Override | ||
3146 | 108 | void exec(final Transaction txn) throws Exception { | ||
3147 | 109 | final Exchange ex1 = exchange("ttlt"); | ||
3148 | 110 | ex1.removeTree(); | ||
3149 | 111 | semA.release(); | ||
3150 | 112 | semB.acquire(); | ||
3151 | 113 | txn.commit(); | ||
3152 | 114 | } | ||
3153 | 115 | }); | ||
3154 | 116 | final Map<Thread, Throwable> errors = start(t); | ||
3155 | 117 | semA.acquire(); | ||
3156 | 118 | assertEquals(ex.getTree(), ex.getVolume().getTree("ttlt", false)); | ||
3157 | 119 | semB.release(); | ||
3158 | 120 | join(TIMEOUT_MS, errors, t); | ||
3159 | 121 | assertNull(ex.getVolume().getTree("ttlt", false)); | ||
3160 | 122 | } | ||
3161 | 123 | |||
3162 | 124 | @Test | ||
3163 | 125 | public void removeCreateRemove() throws Exception { | ||
3164 | 126 | |||
3165 | 127 | final Exchange ex = exchange("ttlt"); | ||
3166 | 128 | ex.getValue().put(RED_FOX); | ||
3167 | 129 | ex.to(1).store(); | ||
3168 | 130 | final Thread t = createThread("ttlt", new TExec() { | ||
3169 | 131 | @Override | ||
3170 | 132 | void exec(final Transaction txn) throws Exception { | ||
3171 | 133 | final Exchange ex1 = exchange("ttlt"); | ||
3172 | 134 | ex1.removeTree(); | ||
3173 | 135 | final Exchange ex2 = exchange("ttlt"); | ||
3174 | 136 | ex2.getValue().put(RED_FOX); | ||
3175 | 137 | ex2.to(2).store(); | ||
3176 | 138 | ex2.removeTree(); | ||
3177 | 139 | final Exchange ex3 = exchange("ttlt"); | ||
3178 | 140 | ex3.getValue().put(RED_FOX); | ||
3179 | 141 | ex3.to(3).store(); | ||
3180 | 142 | final Exchange ex4 = exchange("ttlt"); | ||
3181 | 143 | ex4.to(Key.BEFORE); | ||
3182 | 144 | assertTrue(ex4.next()); | ||
3183 | 145 | assertEquals(3, ex4.getKey().decodeInt()); | ||
3184 | 146 | assertTrue(!ex4.next()); | ||
3185 | 147 | semA.release(); | ||
3186 | 148 | semB.acquire(); | ||
3187 | 149 | txn.rollback(); | ||
3188 | 150 | } | ||
3189 | 151 | }); | ||
3190 | 152 | final Map<Thread, Throwable> errors = start(t); | ||
3191 | 153 | semA.acquire(); | ||
3192 | 154 | assertEquals(ex.getTree(), ex.getVolume().getTree("ttlt", false)); | ||
3193 | 155 | semB.release(); | ||
3194 | 156 | join(TIMEOUT_MS, errors, t); | ||
3195 | 157 | _persistit.getTransactionIndex().updateActiveTransactionCache(); | ||
3196 | 158 | _persistit.pruneTimelyResources(); | ||
3197 | 159 | assertTrue(ex.to(Key.BEFORE).next()); | ||
3198 | 160 | assertEquals(1, ex.getKey().decodeInt()); | ||
3199 | 161 | assertTrue(!ex.next()); | ||
3200 | 162 | assertTrue(ex.getVolume().getStructure().getGarbageRoot() != 0); | ||
3201 | 163 | } | ||
3202 | 164 | |||
3203 | 165 | @Test | ||
3204 | 166 | public void createRemoveByStep() throws Exception { | ||
3205 | 167 | createRemoveByStepHelper("ttlt1", false, true, false, false, "0,1:,2:a=step2,3,4:b=step4", "0:b=step4"); | ||
3206 | 168 | createRemoveByStepHelper("ttlt2", false, false, false, false, "0,1:,2:a=step2,3,4:b=step4", "0"); | ||
3207 | 169 | createRemoveByStepHelper("ttlt3", true, true, false, false, "0:,1:,2:a=step2,3,4:b=step4", "0:b=step4"); | ||
3208 | 170 | createRemoveByStepHelper("ttlt4", true, false, false, false, "0:,1:,2:a=step2,3,4:b=step4", "0:"); | ||
3209 | 171 | |||
3210 | 172 | createRemoveByStepHelper("ttlt5", false, true, false, true, "0,1:,2:a=step2,3,4:b=step4", "0:b=step4"); | ||
3211 | 173 | createRemoveByStepHelper("ttlt6", false, false, false, true, "0,1:,2:a=step2,3,4:b=step4", "0"); | ||
3212 | 174 | createRemoveByStepHelper("ttlt7", true, true, false, true, "0:,1:,2:a=step2,3,4:b=step4", "0:b=step4"); | ||
3213 | 175 | createRemoveByStepHelper("ttlt8", true, false, false, true, "0:,1:,2:a=step2,3,4:b=step4", "0:"); | ||
3214 | 176 | |||
3215 | 177 | createRemoveByStepHelper("ttlt1cr", false, true, true, false, "0,1:,2:a=step2,3,4:b=step4", "0"); | ||
3216 | 178 | createRemoveByStepHelper("ttlt2cr", false, false, true, false, "0,1:,2:a=step2,3,4:b=step4", "0"); | ||
3217 | 179 | createRemoveByStepHelper("ttlt3cr", true, true, true, false, "0:,1:,2:a=step2,3,4:b=step4", "0:"); | ||
3218 | 180 | createRemoveByStepHelper("ttlt4cr", true, false, false, false, "0:,1:,2:a=step2,3,4:b=step4", "0:"); | ||
3219 | 181 | |||
3220 | 182 | createRemoveByStepHelper("ttlt5cr", false, true, true, true, "0,1:,2:a=step2,3,4:b=step4", "0"); | ||
3221 | 183 | createRemoveByStepHelper("ttlt6cr", false, false, true, true, "0,1:,2:a=step2,3,4:b=step4", "0"); | ||
3222 | 184 | createRemoveByStepHelper("ttlt7cr", true, true, true, true, "0:,1:,2:a=step2,3,4:b=step4", "0:"); | ||
3223 | 185 | createRemoveByStepHelper("ttlt8cr", true, false, true, true, "0:,1:,2:a=step2,3,4:b=step4", "0:"); | ||
3224 | 186 | } | ||
3225 | 187 | |||
3226 | 188 | private void createRemoveByStepHelper(final String treeName, final boolean primordial, final boolean commit, | ||
3227 | 189 | final boolean crash, final boolean restart, final String expected1, final String expected2) | ||
3228 | 190 | throws Exception { | ||
3229 | 191 | |||
3230 | 192 | final Transaction txn = _persistit.getTransaction(); | ||
3231 | 193 | final Volume volume = _persistit.getVolume("persistit"); | ||
3232 | 194 | if (primordial) { | ||
3233 | 195 | volume.getTree(treeName, true); | ||
3234 | 196 | } | ||
3235 | 197 | txn.begin(); | ||
3236 | 198 | try { | ||
3237 | 199 | txn.setStep(1); | ||
3238 | 200 | volume.getTree(treeName, true); | ||
3239 | 201 | |||
3240 | 202 | txn.setStep(2); | ||
3241 | 203 | final Exchange ex1 = exchange(treeName); | ||
3242 | 204 | ex1.getValue().put("step2"); | ||
3243 | 205 | ex1.to("a").store(); | ||
3244 | 206 | |||
3245 | 207 | txn.setStep(3); | ||
3246 | 208 | ex1.removeTree(); | ||
3247 | 209 | |||
3248 | 210 | txn.setStep(4); | ||
3249 | 211 | final Exchange ex2 = exchange(treeName); | ||
3250 | 212 | ex2.getValue().put("step4"); | ||
3251 | 213 | ex2.to("b").store(); | ||
3252 | 214 | |||
3253 | 215 | assertEquals("Expected contents at steps", expected1, computeCreateRemoveState(treeName, 5)); | ||
3254 | 216 | |||
3255 | 217 | if (crash) { | ||
3256 | 218 | _persistit.checkpoint(); | ||
3257 | 219 | _persistit.crash(); | ||
3258 | 220 | } else { | ||
3259 | 221 | if (commit) { | ||
3260 | 222 | txn.commit(); | ||
3261 | 223 | } else { | ||
3262 | 224 | txn.rollback(); | ||
3263 | 225 | } | ||
3264 | 226 | } | ||
3265 | 227 | } finally { | ||
3266 | 228 | if (!crash) { | ||
3267 | 229 | txn.end(); | ||
3268 | 230 | } | ||
3269 | 231 | } | ||
3270 | 232 | if (restart) { | ||
3271 | 233 | _persistit.close(); | ||
3272 | 234 | } | ||
3273 | 235 | if (crash || restart) { | ||
3274 | 236 | _persistit = new Persistit(_config); | ||
3275 | 237 | _persistit.initialize(); | ||
3276 | 238 | } | ||
3277 | 239 | assertEquals("Expected contents at steps", expected2, computeCreateRemoveState(treeName, 1)); | ||
3278 | 240 | } | ||
3279 | 241 | |||
3280 | 242 | private String computeCreateRemoveState(final String treeName, final int steps) throws PersistitException { | ||
3281 | 243 | final StringBuilder sb = new StringBuilder(); | ||
3282 | 244 | for (int step = 0; step < steps; step++) { | ||
3283 | 245 | _persistit.getTransaction().setStep(step); | ||
3284 | 246 | if (sb.length() > 0) { | ||
3285 | 247 | sb.append(","); | ||
3286 | 248 | } | ||
3287 | 249 | sb.append(step); | ||
3288 | 250 | if (_persistit.getVolume("persistit").getTree(treeName, false) != null) { | ||
3289 | 251 | sb.append(":"); | ||
3290 | 252 | final Exchange ex = exchange(treeName); | ||
3291 | 253 | ex.append(Key.BEFORE); | ||
3292 | 254 | while (ex.next()) { | ||
3293 | 255 | sb.append(ex.getKey().decodeString()).append("=").append(ex.getValue().getString()); | ||
3294 | 256 | } | ||
3295 | 257 | } | ||
3296 | 258 | } | ||
3297 | 259 | return sb.toString(); | ||
3298 | 260 | } | ||
3299 | 261 | |||
3300 | 262 | abstract class TExec extends ThrowingRunnable { | ||
3301 | 263 | |||
3302 | 264 | @Override | ||
3303 | 265 | public void run() throws Exception { | ||
3304 | 266 | final Transaction txn = _persistit.getTransaction(); | ||
3305 | 267 | txn.begin(); | ||
3306 | 268 | try { | ||
3307 | 269 | exec(txn); | ||
3308 | 270 | } finally { | ||
3309 | 271 | txn.end(); | ||
3310 | 272 | } | ||
3311 | 273 | } | ||
3312 | 274 | |||
3313 | 275 | abstract void exec(final Transaction txn) throws Exception; | ||
3314 | 276 | } | ||
3315 | 277 | } | ||
3316 | 0 | 278 | ||
3317 | === modified file 'src/test/java/com/persistit/unit/ConcurrentUtil.java' | |||
3318 | --- src/test/java/com/persistit/unit/ConcurrentUtil.java 2012-08-24 13:57:19 +0000 | |||
3319 | +++ src/test/java/com/persistit/unit/ConcurrentUtil.java 2013-04-29 21:38:25 +0000 | |||
3320 | @@ -21,11 +21,49 @@ | |||
3321 | 21 | import java.util.HashMap; | 21 | import java.util.HashMap; |
3322 | 22 | import java.util.Map; | 22 | import java.util.Map; |
3323 | 23 | 23 | ||
3324 | 24 | /** | ||
3325 | 25 | * Helper methods to create, start, join and check error status of test threads. | ||
3326 | 26 | * The key element is a Map<Thread, Throwable> the can be checked after threads | ||
3327 | 27 | * run for unhandled exceptions. The {@link #assertSuccess(Map)} method is | ||
3328 | 28 | * called in the main thread to aggregate and report all exceptions that | ||
3329 | 29 | * occurred in other threads. | ||
3330 | 30 | */ | ||
3331 | 24 | public class ConcurrentUtil { | 31 | public class ConcurrentUtil { |
3332 | 32 | |||
3333 | 33 | /** | ||
3334 | 34 | * An implementation of {@link Thread.UncaughtExceptionHandler} which | ||
3335 | 35 | * records any uncaught errors or exceptions in a map. A test case can pass | ||
3336 | 36 | * the map to the {@link ConcurrentUtil#assertSuccess(Map)} method verify | ||
3337 | 37 | * that no exceptions or errors were caught on test threads. | ||
3338 | 38 | * | ||
3339 | 39 | */ | ||
3340 | 40 | public static class UncaughtExceptionHandler implements Thread.UncaughtExceptionHandler { | ||
3341 | 41 | final Map<Thread, Throwable> throwableMap = Collections.synchronizedMap(new HashMap<Thread, Throwable>()); | ||
3342 | 42 | |||
3343 | 43 | @Override | ||
3344 | 44 | public void uncaughtException(final Thread t, final Throwable e) { | ||
3345 | 45 | throwableMap.put(t, e); | ||
3346 | 46 | } | ||
3347 | 47 | |||
3348 | 48 | public Map<Thread, Throwable> getThrowableMap() { | ||
3349 | 49 | return throwableMap; | ||
3350 | 50 | } | ||
3351 | 51 | } | ||
3352 | 52 | |||
3353 | 53 | /** | ||
3354 | 54 | * A version of Runnable in which the #run method throws Exception. | ||
3355 | 55 | */ | ||
3356 | 25 | public static abstract class ThrowingRunnable { | 56 | public static abstract class ThrowingRunnable { |
3357 | 26 | public abstract void run() throws Throwable; | 57 | public abstract void run() throws Throwable; |
3358 | 27 | } | 58 | } |
3359 | 28 | 59 | ||
3360 | 60 | /** | ||
3361 | 61 | * Create a named thread from a ThrowableRunnable. | ||
3362 | 62 | * | ||
3363 | 63 | * @param name | ||
3364 | 64 | * @param runnable | ||
3365 | 65 | * @return | ||
3366 | 66 | */ | ||
3367 | 29 | public static Thread createThread(final String name, final ThrowingRunnable runnable) { | 67 | public static Thread createThread(final String name, final ThrowingRunnable runnable) { |
3368 | 30 | return new Thread(new Runnable() { | 68 | return new Thread(new Runnable() { |
3369 | 31 | @Override | 69 | @Override |
3370 | @@ -40,34 +78,49 @@ | |||
3371 | 40 | } | 78 | } |
3372 | 41 | 79 | ||
3373 | 42 | /** | 80 | /** |
3377 | 43 | * Start and join on all given threads. Wait on each thread, individually, | 81 | * Start all given threads. Return a map on which unhandled exceptions will |
3378 | 44 | * for <code>timeout</code> milliseconds. The {@link Thread#join(long)} | 82 | * be reported. |
3376 | 45 | * method is used for this (<code>0</code> means indefinite). | ||
3379 | 46 | * | 83 | * |
3380 | 47 | * @param timeout | ||
3381 | 48 | * How long to join on each thread for. | ||
3382 | 49 | * @param threads | 84 | * @param threads |
3384 | 50 | * Threads to start and join. | 85 | * Threads to start. |
3385 | 51 | * | 86 | * |
3386 | 52 | * @return A map with an entry for each thread that had an unhandled | 87 | * @return A map with an entry for each thread that had an unhandled |
3387 | 53 | * exception or did not complete in the allotted time. This map will | 88 | * exception or did not complete in the allotted time. This map will |
3388 | 54 | * be empty if all threads completed successfully. | 89 | * be empty if all threads completed successfully. |
3389 | 55 | */ | 90 | */ |
3400 | 56 | public static Map<Thread, Throwable> startAndJoin(final long timeout, final Thread... threads) { | 91 | public static Map<Thread, Throwable> start(final Thread... threads) { |
3401 | 57 | final Map<Thread, Throwable> throwableMap = Collections.synchronizedMap(new HashMap<Thread, Throwable>()); | 92 | final UncaughtExceptionHandler handler = new UncaughtExceptionHandler(); |
3402 | 58 | 93 | start(handler, threads); | |
3403 | 59 | final Thread.UncaughtExceptionHandler handler = new Thread.UncaughtExceptionHandler() { | 94 | return handler.getThrowableMap(); |
3404 | 60 | @Override | 95 | } |
3405 | 61 | public void uncaughtException(final Thread t, final Throwable e) { | 96 | |
3406 | 62 | throwableMap.put(t, e); | 97 | /** |
3407 | 63 | } | 98 | * Start all given threads with the supplied UncaughtExceptionHandler. The |
3408 | 64 | }; | 99 | * handler will record any uncaught exceptions or errors in a map associated |
3409 | 65 | 100 | * with the handler. | |
3410 | 101 | * | ||
3411 | 102 | * @param handler | ||
3412 | 103 | * @param threads | ||
3413 | 104 | */ | ||
3414 | 105 | public static void start(final UncaughtExceptionHandler handler, final Thread... threads) { | ||
3415 | 66 | for (final Thread t : threads) { | 106 | for (final Thread t : threads) { |
3416 | 67 | t.setUncaughtExceptionHandler(handler); | 107 | t.setUncaughtExceptionHandler(handler); |
3417 | 68 | t.start(); | 108 | t.start(); |
3418 | 69 | } | 109 | } |
3419 | 70 | 110 | ||
3420 | 111 | } | ||
3421 | 112 | |||
3422 | 113 | /** | ||
3423 | 114 | * Wait on each thread, individually, for <code>timeout</code> milliseconds. | ||
3424 | 115 | * The {@link Thread#join(long)} method is used for this (<code>0</code> | ||
3425 | 116 | * means indefinite). Add an Exception to the error map for any thread that | ||
3426 | 117 | * did not end within its timeout. | ||
3427 | 118 | * | ||
3428 | 119 | * @param timeout | ||
3429 | 120 | * @param throwableMap | ||
3430 | 121 | * @param threads | ||
3431 | 122 | */ | ||
3432 | 123 | public static void join(final long timeout, final Map<Thread, Throwable> throwableMap, final Thread... threads) { | ||
3433 | 71 | for (final Thread t : threads) { | 124 | for (final Thread t : threads) { |
3434 | 72 | Throwable error = null; | 125 | Throwable error = null; |
3435 | 73 | try { | 126 | try { |
3436 | @@ -78,12 +131,42 @@ | |||
3437 | 78 | } catch (final InterruptedException e) { | 131 | } catch (final InterruptedException e) { |
3438 | 79 | error = e; | 132 | error = e; |
3439 | 80 | } | 133 | } |
3440 | 81 | |||
3441 | 82 | if (error != null) { | 134 | if (error != null) { |
3442 | 83 | throwableMap.put(t, error); | 135 | throwableMap.put(t, error); |
3443 | 84 | } | 136 | } |
3444 | 85 | } | 137 | } |
3446 | 86 | 138 | } | |
3447 | 139 | |||
3448 | 140 | /** | ||
3449 | 141 | * Assert that no thread had any unhandled exceptions or timeouts. | ||
3450 | 142 | * | ||
3451 | 143 | * @param throwableMap | ||
3452 | 144 | * map in which threads accumulated any unhandled Exceptions | ||
3453 | 145 | */ | ||
3454 | 146 | public static void assertSuccess(final Map<Thread, Throwable> throwableMap) { | ||
3455 | 147 | String description = ""; | ||
3456 | 148 | for (final Map.Entry<Thread, Throwable> entry : throwableMap.entrySet()) { | ||
3457 | 149 | description += " " + entry.getKey().getName() + "=" + entry.getValue().toString(); | ||
3458 | 150 | } | ||
3459 | 151 | assertEquals("All threads completed successfully", "{}", "{" + description + "}"); | ||
3460 | 152 | } | ||
3461 | 153 | |||
3462 | 154 | /** | ||
3463 | 155 | * Call {@link #start(Thread...)} for all threads and then | ||
3464 | 156 | * {@link #join(long, Map, Thread...)} for all threads. | ||
3465 | 157 | * | ||
3466 | 158 | * @param timeout | ||
3467 | 159 | * How long to join on each thread for. | ||
3468 | 160 | * @param threads | ||
3469 | 161 | * Threads to start and join. | ||
3470 | 162 | * | ||
3471 | 163 | * @return A map with an entry for each thread that had an unhandled | ||
3472 | 164 | * exception or did not complete in the allotted time. This map will | ||
3473 | 165 | * be empty if all threads completed successfully. | ||
3474 | 166 | */ | ||
3475 | 167 | public static Map<Thread, Throwable> startAndJoin(final long timeout, final Thread... threads) { | ||
3476 | 168 | final Map<Thread, Throwable> throwableMap = start(threads); | ||
3477 | 169 | join(timeout, throwableMap, threads); | ||
3478 | 87 | return throwableMap; | 170 | return throwableMap; |
3479 | 88 | } | 171 | } |
3480 | 89 | 172 | ||
3481 | @@ -98,11 +181,7 @@ | |||
3482 | 98 | * Threads to start and join. | 181 | * Threads to start and join. |
3483 | 99 | */ | 182 | */ |
3484 | 100 | public static void startAndJoinAssertSuccess(final long timeout, final Thread... threads) { | 183 | public static void startAndJoinAssertSuccess(final long timeout, final Thread... threads) { |
3491 | 101 | final Map<Thread, Throwable> errors = startAndJoin(timeout, threads); | 184 | final Map<Thread, Throwable> throwableMap = startAndJoin(timeout, threads); |
3492 | 102 | String description = ""; | 185 | assertSuccess(throwableMap); |
3487 | 103 | for (final Map.Entry<Thread, Throwable> entry : errors.entrySet()) { | ||
3488 | 104 | description += " " + entry.getKey().getName() + "=" + entry.getValue().toString(); | ||
3489 | 105 | } | ||
3490 | 106 | assertEquals("All threads completed successfully", "{}", "{" + description + "}"); | ||
3493 | 107 | } | 186 | } |
3494 | 108 | } | 187 | } |
3495 | 109 | 188 | ||
3496 | === modified file 'src/test/java/com/persistit/unit/TransactionTest1.java' | |||
3497 | --- src/test/java/com/persistit/unit/TransactionTest1.java 2012-08-24 13:57:19 +0000 | |||
3498 | +++ src/test/java/com/persistit/unit/TransactionTest1.java 2013-04-29 21:38:25 +0000 | |||
3499 | @@ -88,7 +88,7 @@ | |||
3500 | 88 | } finally { | 88 | } finally { |
3501 | 89 | txn.end(); | 89 | txn.end(); |
3502 | 90 | } | 90 | } |
3504 | 91 | assertTrue(!ex.getTree().isValid()); | 91 | assertTrue(ex.getTree().isDeleted()); |
3505 | 92 | try { | 92 | try { |
3506 | 93 | ex.clear().append("test1").hasChildren(); | 93 | ex.clear().append("test1").hasChildren(); |
3507 | 94 | fail("Should have thrown an exception"); | 94 | fail("Should have thrown an exception"); |
Updated to store but not recover directory tree updates in the journal. Needed because rollback needs the records in order to prune. Otherwise we get an assertion on a negative mvvCount.