Merge lp:~nwilliams/akiban-server/seq-accum-fix into lp:~akiban-technologies/akiban-server/trunk

[Nathan Williams]

Fix bugs related to sequence value handling.

This addresses all issues outlined in bug1167045 by changing the underlying sequence accumulator to SEQ and providing routines for manual upgrade paths.

Also take the strict approach to using Persistit 3.2.9 so there is a tiny amount of noise. Look at all but the last 2 commits to avoid it. As Peter mentioned in his proposal, AccumInfo and AccumulatorAdapter could use a refactor pass but this doesn't do that.

Diff ordered summary, ignoring 3.2.9 noise, below.

AISMerge/SchemaFactory
Turns out there was another subtle bug in sequence tree usage. If you dropped an sequence and recreated one with the same name, you would pick the tree back up because the tree name handling was being skipped. Go through the whole AISMerge flow to fix this.

Sequence
The interesting part of the change. Switch to SEQ for the accumulator type and always update by 1 (as is now exposed). Simpler over all and next/current share common code now.

AISBasedBuilder
Sequences created through the builder had a user specified initial value but unconditionally Long.MIN for the min. Use init for min for consistency (cycle is always false).

TableChangeValidator
Detect if the incoming table had an identity generation change. Will be useful when we hook up the ALTER, but just serves the new routine right now.

PSSM
Remove now dead code and create routines on start-up (see below).

SequenceFixUpRoutines
Two routines for fixing existing databases, seq_tree_reset and seq_identity_default_to_always. They are JavaDoc-ed, so I won't repeat that here.

TableDDL
Change SERIAL alias from BY DEFAULT to ALWAYS.

PostgresServerMiscYamlIT
The seq_tree_reset routine requires the global lock enabled for extra-safety. Turn it on for this IT only so it can be tested in yaml.

AlterTableBasicIT/SequenceDDLIT/test-seq-fixup-routines.yaml/test-sequences.yaml
Extended existing tests, added new for identified bugs and new functionality.

[Thomas Jones-Low (tjoneslo)]

After several "no wait, oh, ok" moments I will say this addresses all of the issues it presents.

review: Approve

[Akiban Build User (build-akiban)]

There were 2 failures during build/test:

* job server-build failed at build number 3954: http://172.16.20.104:8080/job/server-build/3954/

* view must-pass failed: server-build is red

review: Needs Fixing

[Nathan Williams (nwilliams)]

Thanks for the review Thomas. Persistit side isn't quite in yet.

[Mike McMahon (mmcm)]

How do we want to do backups? The answer doesn't have to be that today's DDL works, but akdump needs to do something. For instance, could we create the new table with bare BIGINT NOT NULL and then ALTER with GENERATED ALWAYS AS IDENTITY (START WITH) after loading data?

review: Needs Information

[Nathan Williams (nwilliams)]

That is probably the simplest. Should be just a tiny parser tweak to get hat ALTER accepted I think.

Postgres is similar. For SERIAL columns, they create the table as is, create and attach sequence, restore value with a routine, and then do the table inserts. They don't do IDENTITY so the BY DEFAULT vs ALWAYS isn't an issue. If you insert values manually into them, it will get a duplicate key when the sequence comes to it.

[Peter Beaman (pbeaman)]

All good. I looked closely at the new logic in the Sequence method; other
changes look plausible but I'm not familiar enough with AIS building and
merging to be sure. I believe the Persistit branch is now also good to go.

On Thu, Apr 11, 2013 at 11:07 AM, Nathan Williams <email address hidden>wrote:

> That is probably the simplest. Should be just a tiny parser tweak to get
> hat ALTER accepted I think.
>
> Postgres is similar. For SERIAL columns, they create the table as is,
> create and attach sequence, restore value with a routine, and then do the
> table inserts. They don't do IDENTITY so the BY DEFAULT vs ALWAYS isn't an
> issue. If you insert values manually into them, it will get a duplicate key
> when the sequence comes to it.
> --
>
> https://code.launchpad.net/~nwilliams/akiban-server/seq-accum-fix/+merge/158279
> Your team Akiban Technologies is subscribed to branch lp:akiban-server.
>

[Nathan Williams (nwilliams)]

Mike, did you want dump support incorporated into this branch or can we do that separately?

[Mike McMahon (mmcm)]

Let's do it separately. I have the parser about ready to go. It was 90% of the way there, but a LOOKAHEAD wasn't broad enough.

=== modified file 'pom.xml'
--- pom.xml	2013-04-05 15:56:11 +0000
+++ pom.xml	2013-04-11 06:17:20 +0000
@@ -100,7 +100,7 @@
         <dependency>
             <groupId>com.akiban</groupId>
             <artifactId>akiban-persistit</artifactId>
-            <version>3.2.8-SNAPSHOT</version>
+            <version>3.2.9-SNAPSHOT</version>
         </dependency>
         <dependency>
             <groupId>com.akiban</groupId>
 
=== modified file 'src/main/java/com/akiban/ais/model/AISMerge.java'
--- src/main/java/com/akiban/ais/model/AISMerge.java	2013-04-02 05:00:56 +0000
+++ src/main/java/com/akiban/ais/model/AISMerge.java	2013-04-11 06:17:20 +0000
@@ -55,7 +55,7 @@
  * frozen. If you pass a frozen AIS into the merge, the copy process unfreeze the copy.
  */
 public class AISMerge {
-    public enum MergeType { ADD_TABLE, MODIFY_TABLE, ADD_INDEX }
+    public enum MergeType { ADD_TABLE, MODIFY_TABLE, ADD_INDEX, OTHER }
 
     private static class JoinChange {
         public final Join join;
@@ -121,6 +121,10 @@
         return new AISMerge(generator, copyAISForAdd(sourceAIS), null, MergeType.ADD_INDEX, null, null);
     }
 
+    public static AISMerge newForOther(NameGenerator generator, AkibanInformationSchema sourceAIS) {
+        return new AISMerge(generator, copyAISForAdd(sourceAIS), null, MergeType.OTHER, null, null);
+    }
+
     private AISMerge(NameGenerator nameGenerator, AkibanInformationSchema targetAIS, UserTable sourceTable,
                      MergeType mergeType, List<JoinChange> changedJoins, Map<IndexName,IndexInfo> indexesToFix) {
         this.nameGenerator = nameGenerator;
@@ -644,14 +648,14 @@
         newAIS.addView(newView);
     }
     
-    public static AkibanInformationSchema mergeSequence (AkibanInformationSchema oldAIS,
-                                                            Sequence sequence)
+    public AkibanInformationSchema mergeSequence(Sequence sequence)
     {
-        AkibanInformationSchema newAIS = copyAISForAdd(oldAIS);
-        newAIS.addSequence(sequence);
-        newAIS.validate(AISValidations.LIVE_AIS_VALIDATIONS).throwIfNecessary();
-        newAIS.freeze();
-        return newAIS;
+        Sequence newSeq = Sequence.create(targetAIS, sequence);
+        newSeq.setTreeName(nameGenerator.generateSequenceTreeName(newSeq));
+        targetAIS.addSequence(newSeq);
+        targetAIS.validate(AISValidations.LIVE_AIS_VALIDATIONS).throwIfNecessary();
+        targetAIS.freeze();
+        return targetAIS;
     }    
 
     public static AkibanInformationSchema mergeRoutine(AkibanInformationSchema oldAIS,
 
=== modified file 'src/main/java/com/akiban/ais/model/Column.java'
--- src/main/java/com/akiban/ais/model/Column.java	2013-03-26 20:26:45 +0000
+++ src/main/java/com/akiban/ais/model/Column.java	2013-04-11 06:17:20 +0000
@@ -274,15 +274,20 @@
     }
 
     /**
-     * This is a three state boolean: 
-     * True: column created with GENERATED BY DEFAULT AS IDENTITY
-     * False: Column created with GENERATED ALWAYS AS IDENTITY
-     * Null: Not generated by identity column
-     * 
-     * NOTE: It is possible for the GetInitialAutoIncrement to be
+     * <p>
+     * This is a three state boolean:
+     * <ul>
+     *   <li><b>True</b>: column created with GENERATED BY DEFAULT AS IDENTITY</li>
+     *   <li><b>False</b>: Column created with GENERATED ALWAYS AS IDENTITY</li>
+     *   <li><b>null</b>: Not generated by identity column</li>
+     * </ul>
+     * </p>
+     * <p>
+     * <b>NOTE</b>: It is possible for the GetInitialAutoIncrement to be
      * not null and this to be null, as MySQL generated tables use
      *  auto-increment value, where as the Akiban SQL use the 
-     * identity generators. 
+     * identity generators.
+     * </p>
      */
     public final Boolean getDefaultIdentity() {
         return defaultIdentity;
 
=== modified file 'src/main/java/com/akiban/ais/model/Index.java'
--- src/main/java/com/akiban/ais/model/Index.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/ais/model/Index.java	2013-04-11 06:17:20 +0000
@@ -359,12 +359,11 @@
 
     // Unique, non-PK indexes store a "null separator value", making index rows unique that would otherwise
     // be considered duplicates due to nulls.
-    public long nextNullSeparatorValue(TreeService treeService)
+    public long nextNullSeparatorValue()
     {
         Tree tree = indexDef.getTreeCache().getTree();
-        AccumulatorAdapter accumulator =
-            new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.UNIQUE_ID, treeService, tree);
-        return accumulator.updateAndGet(1);
+        AccumulatorAdapter accumulator = new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.UNIQUE_ID, tree);
+        return accumulator.seqAllocate();
     }
 
     // akTypes, akCollators and tInstances provide type info for physical index rows.
 
=== modified file 'src/main/java/com/akiban/ais/model/Sequence.java'
--- src/main/java/com/akiban/ais/model/Sequence.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/ais/model/Sequence.java	2013-04-11 06:17:20 +0000
@@ -42,7 +42,13 @@
         ais.addSequence(sequence);
         return sequence; 
     }
-    
+
+    /** Create a copy of <code>seq</code>. Internal data (e.g. tree name) is not copied. */
+    public static Sequence create (AkibanInformationSchema ais, Sequence seq) {
+        return create(ais, seq.sequenceName.getSchemaName(), seq.sequenceName.getTableName(),
+                      seq.startsWith, seq.increment, seq.minValue, seq.maxValue, seq.cycle);
+    }
+
     protected Sequence (AkibanInformationSchema ais,
             String schemaName, 
             String sequenceName, 
@@ -55,13 +61,14 @@
         AISInvariants.checkNullName(schemaName, "Sequence", "schema name");
         AISInvariants.checkNullName(sequenceName, "Sequence", "table name");
         AISInvariants.checkDuplicateSequence(ais, schemaName, sequenceName);
-        
+
         this.sequenceName = new TableName (schemaName, sequenceName);
         this.startsWith = start;
         this.increment = increment;
         this.minValue = minValue;
         this.maxValue = maxValue;
         this.cycle = cycle;
+        this.range = maxValue - minValue + 1;
     }
     
     public final TableName getSequenceName() {
@@ -108,6 +115,7 @@
     private final long maxValue;
     private final boolean cycle;
 
+    private final long range;
     private AtomicReference<TreeCache> treeCache = new AtomicReference<>();
     
    
@@ -126,44 +134,42 @@
     public TreeCache getTreeCache() {
         return treeCache.get();
     }
-    
-    public long nextValue(TreeService treeService) throws PersistitException {
-        
-        Tree tree = getTreeCache().getTree();
-        AccumulatorAdapter accum = new AccumulatorAdapter (AccumInfo.AUTO_INC, treeService, tree);
-        long value = accum.updateAndGet(increment);
-        
-        if (value > maxValue && increment > 0) {
-            if (cycle) {
-                value = minValue;
-                accum.set(value);
-            } else {
+
+    public long nextValue() throws PersistitException {
+        // Note: Ever increasing, always incremented by 1, rollbacks will leave gaps. See bug1167045 for discussion.
+        AccumulatorAdapter accum = getAdapter();
+        long rawSequence = accum.seqAllocate();
+
+        long nextValue = notCycled(rawSequence);
+        if (nextValue > maxValue || nextValue < minValue) {
+            if(!cycle) {
                 throw new SequenceLimitExceededException(this);
             }
-        } else if (value < minValue && increment < 0) {
-            if (cycle) {
-                value = maxValue;
-                accum.set(value);
-            } else {
-                throw new SequenceLimitExceededException (this);
-            }
-        }
-        return value;
-    }
-
-    public long currentValue(TreeService treeService) throws PersistitException {
-        Tree tree = getTreeCache().getTree();
-        AccumulatorAdapter accum = new AccumulatorAdapter (AccumInfo.AUTO_INC, treeService, tree);
-        return accum.getSnapshot(AccumInfo.AUTO_INC, treeService, tree);
-    }
-    
-    public void setStartWithAccumulator(TreeService treeService) throws PersistitException {
-        Tree tree = getTreeCache().getTree();
-        AccumulatorAdapter accum = new AccumulatorAdapter (AccumInfo.AUTO_INC, treeService, tree);
-        // Set the starting value to startsWith - increment, 
-        // which will be, on first call to nextValue() be updated to the start value
-        // TODO: This can cause problems if startsWith is within increment of 
-        // Long.MaxValue or Long.MinValue. 
-        accum.set(startsWith - increment);
+            nextValue = cycled(nextValue);
+        }
+        return nextValue;
+    }
+
+    public long currentValue() throws PersistitException {
+        AccumulatorAdapter accum = getAdapter();
+        return cycled(notCycled(accum.getSnapshot()));
+    }
+
+    private AccumulatorAdapter getAdapter() throws PersistitException {
+        Tree tree = getTreeCache().getTree();
+        return new AccumulatorAdapter(AccumInfo.SEQUENCE, tree);
+    }
+
+    private long notCycled(long rawSequence) {
+        // -1 so first is startsWith, second is startsWith+inc, etc
+        return startsWith + ((rawSequence - 1) * increment);
+    }
+
+    private long cycled(long notCycled) {
+        long mod = (notCycled - minValue) % range;
+        if(mod < 0) {
+            mod += range;
+        }
+        return minValue + mod;
     }
 }
 
=== modified file 'src/main/java/com/akiban/ais/model/aisb2/AISBBasedBuilder.java'
--- src/main/java/com/akiban/ais/model/aisb2/AISBBasedBuilder.java	2013-04-01 18:36:36 +0000
+++ src/main/java/com/akiban/ais/model/aisb2/AISBBasedBuilder.java	2013-04-11 06:17:20 +0000
@@ -234,7 +234,7 @@
                 String sequenceName = "temp-seq-" + userTable + "-" + name;
                 long initValue = initialAutoInc.longValue();
                 aisb.sequence(schema, sequenceName, 
-                              initValue, 1L, Long.MIN_VALUE, Long.MAX_VALUE, 
+                              initValue, 1L, initValue, Long.MAX_VALUE,
                               false);
                 aisb.columnAsIdentity(schema, userTable, name, sequenceName, true);
                 aisb.akibanInformationSchema().
 
=== modified file 'src/main/java/com/akiban/ais/util/TableChangeValidator.java'
--- src/main/java/com/akiban/ais/util/TableChangeValidator.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/ais/util/TableChangeValidator.java	2013-04-11 06:17:20 +0000
@@ -563,6 +563,7 @@
         if((oldNull != newNull) ||
            !oldCol.getName().equals(newCol.getName()) ||
            !Objects.equal(oldCol.getDefaultValue(), newCol.getDefaultValue()) ||
+           !Objects.equal(oldCol.getDefaultIdentity(), newCol.getDefaultIdentity()) ||
            sequenceChanged(oldCol.getIdentityGenerator(), newCol.getIdentityGenerator())) {
           return ChangeLevel.METADATA;
         }
 
=== modified file 'src/main/java/com/akiban/qp/persistitadapter/OperatorStoreGIHandler.java'
--- src/main/java/com/akiban/qp/persistitadapter/OperatorStoreGIHandler.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/qp/persistitadapter/OperatorStoreGIHandler.java	2013-04-11 06:17:20 +0000
@@ -115,7 +115,7 @@
     private void storeExchange(GroupIndex groupIndex, Exchange exchange) {
         try {
             exchange.store();
-            AccumulatorAdapter.updateAndGet(AccumulatorAdapter.AccumInfo.ROW_COUNT, exchange, 1);
+            AccumulatorAdapter.sumAdd(AccumulatorAdapter.AccumInfo.ROW_COUNT, exchange, 1);
         } catch (PersistitException e) {
             throw new PersistitAdapterException(e);
         }
@@ -127,7 +127,7 @@
     private void removeExchange(GroupIndex groupIndex, Exchange exchange) {
         try {
             if (exchange.remove()) {
-                AccumulatorAdapter.updateAndGet(AccumulatorAdapter.AccumInfo.ROW_COUNT, exchange, -1);
+                AccumulatorAdapter.sumAdd(AccumulatorAdapter.AccumInfo.ROW_COUNT, exchange, -1);
             }
             else
                 UNNEEDED_DELETE_TAP.hit();
 
=== modified file 'src/main/java/com/akiban/qp/persistitadapter/PersistitAdapter.java'
--- src/main/java/com/akiban/qp/persistitadapter/PersistitAdapter.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/qp/persistitadapter/PersistitAdapter.java	2013-04-11 06:17:20 +0000
@@ -447,9 +447,9 @@
     private long sequenceValue (Sequence sequence, boolean getCurrentValue) {
         try {
             if (getCurrentValue) {
-                return sequence.currentValue(treeService);
+                return sequence.currentValue();
             } else {
-                return sequence.nextValue(treeService);
+                return sequence.nextValue();
             }
         } catch (PersistitException e) {
             rollbackIfNeeded(e);
 
=== modified file 'src/main/java/com/akiban/qp/persistitadapter/indexrow/PersistitIndexRowBuffer.java'
--- src/main/java/com/akiban/qp/persistitadapter/indexrow/PersistitIndexRowBuffer.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/qp/persistitadapter/indexrow/PersistitIndexRowBuffer.java	2013-04-11 06:17:20 +0000
@@ -201,7 +201,7 @@
                     hasNull = pKey.isNull();
                 }
                 if (hasNull) {
-                    nullSeparator = index.nextNullSeparatorValue(adapter.persistit().treeService());
+                    nullSeparator = index.nextNullSeparatorValue();
                 }
             }
             // else: We're creating an index row to update or delete. Don't need a new null separator value.
 
=== modified file 'src/main/java/com/akiban/server/AccumulatorAdapter.java'
--- src/main/java/com/akiban/server/AccumulatorAdapter.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/server/AccumulatorAdapter.java	2013-04-11 06:17:20 +0000
@@ -18,41 +18,41 @@
 package com.akiban.server;
 
 import com.akiban.server.error.PersistitAdapterException;
-import com.akiban.server.service.tree.TreeService;
 import com.persistit.Accumulator;
 import com.persistit.Exchange;
-import com.persistit.Transaction;
 import com.persistit.Tree;
 import com.persistit.exception.PersistitException;
 import com.persistit.exception.PersistitInterruptedException;
 
 public class AccumulatorAdapter {
 
-    public static long getSnapshot(AccumInfo accumInfo, TreeService treeService, Tree tree)
-    throws PersistitInterruptedException
-    {
+    public static long getSnapshot(AccumInfo accumInfo, Tree tree) throws PersistitInterruptedException {
         Accumulator accumulator = getAccumulator(accumInfo, tree);
-        Transaction txn = getCurrentTrx(treeService);
-        return accumulator.getSnapshotValue(txn);
+        return accumulator.getSnapshotValue();
     }
 
-    public static long updateAndGet(AccumInfo accumInfo, Exchange exchange, long value) {
-        Accumulator accumulator = getAccumulator(accumInfo, exchange.getTree());
-        return accumulator.update(value, exchange.getTransaction());
+    public static void sumAdd(AccumInfo accumInfo, Exchange exchange, long value) {
+        Accumulator.SumAccumulator sum = (Accumulator.SumAccumulator)getAccumulator(accumInfo, exchange.getTree());
+        sum.add(value);
     }
     
-    public static long getLiveValue(AccumInfo accumInfo, TreeService treeService, Tree tree)
-    {
+    public static long getLiveValue(AccumInfo accumInfo, Tree tree) {
         Accumulator accumulator = getAccumulator(accumInfo, tree);
         return accumulator.getLiveValue();
     }
 
     public long getSnapshot() throws PersistitInterruptedException {
-        return accumulator.getSnapshotValue(getCurrentTrx());
-    }
-
-    public long updateAndGet(long value) {
-        return accumulator.update(value, getCurrentTrx());
+        return accumulator.getSnapshotValue();
+    }
+
+    public void sumAdd(long value) {
+        Accumulator.SumAccumulator sum = (Accumulator.SumAccumulator)accumulator;
+        sum.add(value);
+    }
+
+    public long seqAllocate() {
+        Accumulator.SeqAccumulator seq = (Accumulator.SeqAccumulator)accumulator;
+        return seq.allocate();
     }
 
     public long getLiveValue() {
@@ -67,32 +67,29 @@
         long current = getSnapshot();
         if(evenIfLess || value > current) {
             long diff = value - current;
-            this.updateAndGet(diff);
+            this.sumAdd(diff);
         }
     }
 
-    public AccumulatorAdapter(AccumInfo accumInfo, TreeService treeService, Tree tree) {
-        this.treeService = treeService;
+    public AccumulatorAdapter(AccumInfo accumInfo, Tree tree) {
         this.accumulator = getAccumulator(accumInfo, tree);
     }
 
     private static Accumulator getAccumulator(AccumInfo accumInfo, Tree tree) {
         try {
-            return tree.getAccumulator(accumInfo.getType(), accumInfo.getIndex());
+            switch(accumInfo.type) {
+                case SUM: return tree.getSumAccumulator(accumInfo.getIndex());
+                case MAX: return tree.getMaxAccumulator(accumInfo.getIndex());
+                case MIN: return tree.getMinAccumulator(accumInfo.getIndex());
+                case SEQ: return tree.getSeqAccumulator(accumInfo.getIndex());
+                default:
+                    throw new IllegalStateException("Unknown accumulator type: " + accumInfo.type);
+            }
         } catch (PersistitException e) {
             throw new PersistitAdapterException(e);
         }
     }
 
-    private Transaction getCurrentTrx() {
-        return getCurrentTrx(treeService);
-    }
-
-    private static Transaction getCurrentTrx(TreeService treeService) {
-        return treeService.getDb().getTransaction();
-    }
-
-    private final TreeService treeService;
     private final Accumulator accumulator;
 
     /**
@@ -104,10 +101,16 @@
      * </p>
      */
     public static enum AccumInfo {
+        /** Ordinal value as used in the hkey. Write once. Attached to the PK tree. */
         ORDINAL(0, Accumulator.Type.SUM),
+        /** Size of a table or group index. Attached to PK tree and GI tree, respectively */
         ROW_COUNT(1, Accumulator.Type.SUM),
+        /** Source of values for hidden primary keys. Attached to the PK tree. */
         UNIQUE_ID(2, Accumulator.Type.SEQ),
+        /** Saves values from the MySQL adapter AUTO INCREMENT columns. Attached to the PK tree. */
         AUTO_INC(3, Accumulator.Type.SUM),
+        /** Source of values for SQL sequences. Attached to the sequence tree. */
+        SEQUENCE(4, Accumulator.Type.SEQ)
         ;
     
         AccumInfo(int index, Accumulator.Type type) {
 
=== modified file 'src/main/java/com/akiban/server/MemoryOnlyTableStatusCache.java'
--- src/main/java/com/akiban/server/MemoryOnlyTableStatusCache.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/server/MemoryOnlyTableStatusCache.java	2013-04-11 06:17:20 +0000
@@ -102,13 +102,6 @@
         }
 
         @Override
-        public synchronized void setUniqueId(long value) {
-            if (value < uniqueID)
-                throw new IllegalArgumentException("can't decrement uniqueID from " + uniqueID + " to " + value);
-            this.uniqueID = value;
-        }
-
-        @Override
         public long getApproximateUniqueID() {
             return getUniqueID();
         }
 
=== modified file 'src/main/java/com/akiban/server/PersistitAccumulatorTableStatusCache.java'
--- src/main/java/com/akiban/server/PersistitAccumulatorTableStatusCache.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/server/PersistitAccumulatorTableStatusCache.java	2013-04-11 06:17:20 +0000
@@ -18,7 +18,6 @@
 package com.akiban.server;
 
 import com.akiban.qp.memoryadapter.MemoryTableFactory;
-import com.akiban.server.error.AkibanInternalException;
 import com.akiban.server.error.PersistitAdapterException;
 import com.akiban.server.rowdata.IndexDef;
 import com.akiban.server.rowdata.RowDef;
@@ -138,27 +137,18 @@
         }
 
         @Override
-        public void setUniqueId(long value) {
-            try {
-                this.uniqueID.set(value);
-            } catch (PersistitInterruptedException e) {
-                throw new PersistitAdapterException(e);
-            }
-        }
-
-        @Override
         public int getTableID() {
             return expectedID;
         }
 
         @Override
         public void rowDeleted() {
-            rowCount.updateAndGet(-1);
+            rowCount.sumAdd(-1);
         }
 
         @Override
         public void rowsWritten(long count) {
-            rowCount.updateAndGet(count);
+            rowCount.sumAdd(count);
         }
 
         public void setOrdinal(int ordinal) throws PersistitInterruptedException {
@@ -167,7 +157,7 @@
 
         @Override
         public long createNewUniqueID() throws PersistitInterruptedException {
-            return uniqueID.updateAndGet(1);
+            return uniqueID.seqAllocate();
         }
 
         @Override
@@ -188,10 +178,10 @@
             } else {
                 checkExpectedRowDefID(expectedID, rowDef);
                 Tree tree = getTreeForRowDef(rowDef);
-                ordinal = new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.ORDINAL, treeService, tree);
-                rowCount = new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.ROW_COUNT, treeService, tree);
-                uniqueID = new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.UNIQUE_ID, treeService, tree);
-                autoIncrement = new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.AUTO_INC, treeService, tree);
+                ordinal = new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.ORDINAL, tree);
+                rowCount = new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.ROW_COUNT, tree);
+                uniqueID = new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.UNIQUE_ID, tree);
+                autoIncrement = new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.AUTO_INC, tree);
             }
         }
 
@@ -240,11 +230,6 @@
         }
 
         @Override
-        public void setUniqueId(long value) {
-            throw new UnsupportedOperationException();
-        }
-
-        @Override
         public long getApproximateUniqueID() {
             throw new UnsupportedOperationException();
         }
 
=== modified file 'src/main/java/com/akiban/server/TableStatus.java'
--- src/main/java/com/akiban/server/TableStatus.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/server/TableStatus.java	2013-04-11 06:17:20 +0000
@@ -79,6 +79,4 @@
     void setRowCount(long rowCount);
 
     long getApproximateUniqueID();
-
-    void setUniqueId(long value);
 }
 
=== modified file 'src/main/java/com/akiban/server/service/dxl/BasicDDLFunctions.java'
--- src/main/java/com/akiban/server/service/dxl/BasicDDLFunctions.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/server/service/dxl/BasicDDLFunctions.java	2013-04-11 06:17:20 +0000
@@ -1162,8 +1162,7 @@
                 else {
                     final Exchange ex = pStore.getExchange(session, index);
                     try {
-                        AccumulatorAdapter accum =
-                                new AccumulatorAdapter(AccumInfo.ROW_COUNT, treeService(), ex.getTree());
+                        AccumulatorAdapter accum = new AccumulatorAdapter(AccumInfo.ROW_COUNT, ex.getTree());
                         accum.set(actual);
                     }
                     finally {
 
=== modified file 'src/main/java/com/akiban/server/store/PersistitStore.java'
--- src/main/java/com/akiban/server/store/PersistitStore.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/server/store/PersistitStore.java	2013-04-11 06:17:20 +0000
@@ -610,7 +610,13 @@
                 }
                 for (Map.Entry<RowDef, AtomicLong> hiddenPkEntry : bulkload.hiddenPks.entrySet()) {
                     RowDef rowDef = hiddenPkEntry.getKey();
-                    rowDef.getTableStatus().setUniqueId(hiddenPkEntry.getValue().get());
+                    TableStatus status = rowDef.getTableStatus();
+                    long target = hiddenPkEntry.getValue().get();
+                    long diff = target - status.getUniqueID();
+                    while(diff > 0) {
+                        status.createNewUniqueID();
+                        --diff;
+                    }
                 }
             }
             finally {
@@ -930,7 +936,7 @@
             try {
                 iEx.removeAll();
                 if (index.isGroupIndex()) {
-                    new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.ROW_COUNT, treeService, iEx.getTree()).set(0);
+                    new AccumulatorAdapter(AccumulatorAdapter.AccumInfo.ROW_COUNT, iEx.getTree()).set(0);
                 }
             } catch (PersistitException e) {
                 throw new PersistitAdapterException(e);
 
=== modified file 'src/main/java/com/akiban/server/store/PersistitStoreSchemaManager.java'
--- src/main/java/com/akiban/server/store/PersistitStoreSchemaManager.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/server/store/PersistitStoreSchemaManager.java	2013-04-11 06:17:20 +0000
@@ -239,7 +239,6 @@
     private static final String DELAYED_TREE_KEY = "delayedTree";
 
     private static final int SCHEMA_GEN_ACCUM_INDEX = 0;
-    private static final Accumulator.Type SCHEMA_GEN_ACCUM_TYPE = Accumulator.Type.SEQ;
 
     // Changed from 1 to 2 due to incompatibility related to index row changes (see bug 985007)
     private static final int PROTOBUF_PSSM_VERSION = 2;
@@ -806,14 +805,9 @@
     @Override
     public void createSequence(Session session, Sequence sequence) {
         checkSequenceName(session, sequence.getSequenceName(), false);
-        AkibanInformationSchema newAIS = AISMerge.mergeSequence(this.getAis(session), sequence);
+        AISMerge merge = AISMerge.newForOther(nameGenerator, getAis(session));
+        AkibanInformationSchema newAIS = merge.mergeSequence(sequence);
         saveAISChangeWithRowDefs(session, newAIS, Collections.singleton(sequence.getSchemaName()));
-        try {
-            sequence.setStartWithAccumulator(treeService);
-        } catch (PersistitException e) {
-            LOG.error("Setting sequence starting value for sequence {} failed", sequence.getSequenceName().getDescription());
-            throw wrapPersistitException(session, e);
-        }
     }
     
     /** Drop the given sequence from the current AIS. */
@@ -1467,14 +1461,14 @@
         return !curVer.equals(tableVer);
     }
 
-    private Accumulator getGenerationAccumulator(Session session) throws PersistitException {
+    private Accumulator.SeqAccumulator getGenerationAccumulator(Session session) throws PersistitException {
         // treespace policy could split the _schema_ tree across volumes and give us multiple accumulators, which would
         // be very bad. Work around that with a fake/constant schema name. It isn't a problem if this somehow got changed
         // across a restart. Really, we want a constant, system-like volume to put this in.
         final String SCHEMA = "pssm";
         Exchange ex = schemaTreeExchange(session, SCHEMA);
         try {
-            return ex.getTree().getAccumulator(SCHEMA_GEN_ACCUM_TYPE, SCHEMA_GEN_ACCUM_INDEX);
+            return ex.getTree().getSeqAccumulator(SCHEMA_GEN_ACCUM_INDEX);
         } finally {
             treeService.releaseExchange(session, ex);
         }
@@ -1482,16 +1476,15 @@
 
     private long getGenerationSnapshot(Session session) {
         try {
-            return getGenerationAccumulator(session).getSnapshotValue(treeService.getDb().getTransaction());
+            return getGenerationAccumulator(session).getSnapshotValue();
         } catch(PersistitException e) {
             throw wrapPersistitException(session, e);
         }
     }
 
     private long getNextGeneration(Session session) {
-        final int ACCUM_UPDATE_VALUE = 1;   // irrelevant for SEQ types
         try {
-            return getGenerationAccumulator(session).update(ACCUM_UPDATE_VALUE, treeService.getDb().getTransaction());
+            return getGenerationAccumulator(session).allocate();
         } catch(PersistitException e) {
             throw wrapPersistitException(session, e);
         }
@@ -1538,14 +1531,6 @@
             mergedTable.setMemoryTableFactory(factory);
             unSavedAISChangeWithRowDefs(session, newAIS);
         }
-        try {
-            if (mergedTable.getIdentityColumn() != null) {
-                mergedTable.getIdentityColumn().getIdentityGenerator().setStartWithAccumulator(treeService);
-            }
-        } catch (PersistitException ex) {
-            LOG.error("Setting sequence starting value for table {} failed", mergedTable.getName().getDescription());
-            throw wrapPersistitException(session, ex);
-        }
         return newName;
     }
 
@@ -1690,8 +1675,28 @@
                 .colBigInt("map_size", false)
                 .colBigInt("outstanding_count", false)
                 .colBigInt("task_queue_size", false);
-        UserTable table = builder.ais().getUserTable(factory.getName());
+
+        final int IDENT_MAX = 128;
+        builder.defaultSchema(TableName.SYS_SCHEMA);
+        builder.procedure("seq_tree_reset")
+               .language("java", Routine.CallingConvention.JAVA)
+               .paramStringIn("seq_schema", IDENT_MAX)
+               .paramStringIn("seq_name", IDENT_MAX)
+               .paramLongIn("new_value")
+               .externalName(SequenceFixUpRoutines.class.getCanonicalName(), "seq_tree_reset");
+        builder.procedure("seq_identity_default_to_always")
+                .language("java", Routine.CallingConvention.JAVA)
+                .paramStringIn("schema", IDENT_MAX)
+                .paramStringIn("table", IDENT_MAX)
+                .paramStringIn("column", IDENT_MAX)
+                .externalName(SequenceFixUpRoutines.class.getCanonicalName(), "seq_identity_default_to_always");
+
+        AkibanInformationSchema ais = builder.ais();
+        UserTable table = ais.getUserTable(factory.getName());
         registerMemoryInformationSchemaTable(table, factory);
+        for(Routine routine : ais.getRoutines().values()) {
+            registerSystemRoutine(routine);
+        }
     }
 
 
 
=== added file 'src/main/java/com/akiban/server/store/SequenceFixUpRoutines.java'
--- src/main/java/com/akiban/server/store/SequenceFixUpRoutines.java	1970-01-01 00:00:00 +0000
+++ src/main/java/com/akiban/server/store/SequenceFixUpRoutines.java	2013-04-11 06:17:20 +0000
@@ -0,0 +1,139 @@
+/**
+ * Copyright (C) 2009-2013 Akiban Technologies, Inc.
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package com.akiban.server.store;
+
+import com.akiban.ais.AISCloner;
+import com.akiban.ais.model.AkibanInformationSchema;
+import com.akiban.ais.model.Column;
+import com.akiban.ais.model.Sequence;
+import com.akiban.ais.model.TableName;
+import com.akiban.ais.model.UserTable;
+import com.akiban.ais.protobuf.ProtobufWriter;
+import com.akiban.ais.util.TableChange;
+import com.akiban.server.error.NoSuchColumnException;
+import com.akiban.server.error.NoSuchSchemaException;
+import com.akiban.server.error.NoSuchSequenceException;
+import com.akiban.server.error.NoSuchTableException;
+import com.akiban.server.error.PersistitAdapterException;
+import com.akiban.server.service.dxl.DXLFunctionsHook;
+import com.akiban.server.service.dxl.DXLReadWriteLockHook;
+import com.akiban.server.service.session.Session;
+import com.akiban.server.service.tree.TreeService;
+import com.akiban.sql.server.ServerCallContextStack;
+import com.akiban.sql.server.ServerQueryContext;
+import com.akiban.sql.server.ServerSession;
+import com.persistit.exception.PersistitException;
+
+import java.util.Arrays;
+
+import static com.akiban.server.service.transaction.TransactionService.CloseableTransaction;
+
+/**
+ * <p>
+ *     Added in support of bug1167045. These routines allow for a manual fix of existing databases with sequences.
+ * </p>
+ * <p>
+ *     Prior to 1.6.1, SEQUENCEs were backed by a SUM accumulator and used in a non-safe way. Particularly, rollbacks
+ *     would cause value reuse upon restart. Post bug fix, they are backed by a SEQ under a different algorithm.
+ * </p>
+ * <p>
+ *     seq_tree_rest allows for an existing sequence to be cleared and pegged at a new value.
+ * </p>
+ * <p>
+ *     seq_identity_default_to_always allows for an IDENTITY column to be set to ALWAYS instead of DEFAULT, which
+ *     is the new default behavior for SERIAL columns.
+ * </p>
+ */
+public class SequenceFixUpRoutines {
+    private static final DXLFunctionsHook.DXLFunction LOCK_FUNC = DXLFunctionsHook.DXLFunction.UNSPECIFIED_DDL_WRITE;
+
+    private SequenceFixUpRoutines() {
+    }
+
+    public static void seq_tree_reset(String seqSchema, String seqName, long newValue) throws InterruptedException {
+        if(!DXLReadWriteLockHook.only().isDDLLockEnabled()) {
+            throw new IllegalStateException("Unsafe to use with global lock disabled");
+        }
+
+        final ServerQueryContext context = ServerCallContextStack.current().getContext();
+        final ServerSession server = context.getServer();
+        final Session session = server.getSession();
+        if(!server.getSecurityService().isAccessible(session, seqSchema)) {
+            throw new NoSuchSchemaException(seqSchema);
+        }
+
+        // Global lock
+        DXLReadWriteLockHook.only().lock(session, LOCK_FUNC, -1);
+        try {
+            // Find sequence
+            Sequence seq = server.getDXL().ddlFunctions().getAIS(session).getSequence(new TableName(seqSchema, seqName));
+            if(seq == null) {
+                throw new NoSuchSequenceException(seqSchema, seqName);
+            }
+            TreeService treeService = server.getTreeService();
+            // Drop tree
+            treeService.getExchange(session, seq).removeTree();
+            // Create tree
+            treeService.populateTreeCache(seq);
+            try(CloseableTransaction txn = server.getTransactionService().beginCloseableTransaction(session)) {
+                for(int i = 0; i < newValue; ++i) {
+                    seq.nextValue();
+                }
+                txn.commit();
+            }
+        } catch(PersistitException e) {
+            throw new PersistitAdapterException(e);
+        } finally {
+            // unlock
+            DXLReadWriteLockHook.only().unlock(server.getSession(), LOCK_FUNC);
+        }
+    }
+
+    public static void seq_identity_default_to_always(String schema, String table, String column) {
+        final ServerQueryContext context = ServerCallContextStack.current().getContext();
+        final ServerSession server = context.getServer();
+        final Session session = server.getSession();
+        if(!server.getSecurityService().isAccessible(session, schema)) {
+            throw new NoSuchSchemaException(schema);
+        }
+
+        TableName tableName = new TableName(schema, table);
+        AkibanInformationSchema ais = server.getAIS();
+        UserTable curTable = ais.getUserTable(tableName);
+        if(curTable == null) {
+            throw new NoSuchTableException(tableName);
+        }
+        Column curColumn = curTable.getColumn(column);
+        if(curColumn == null) {
+            throw new NoSuchColumnException(column);
+        }
+        if(curColumn.getDefaultIdentity() == null) {
+            throw new IllegalArgumentException("Column " + column + " is not an IDENTITY");
+        }
+
+        AkibanInformationSchema copy = AISCloner.clone(ais, new ProtobufWriter.SingleSchemaSelector(schema));
+        UserTable newTable = copy.getUserTable(schema, table);
+        newTable.getColumn(column).setDefaultIdentity(false);
+        server.getDXL().ddlFunctions().alterTable(
+                session, tableName, newTable,
+                Arrays.asList(TableChange.createModify(column, column)),
+                Arrays.<TableChange>asList(),
+                context
+        );
+    }
+}
 
=== modified file 'src/main/java/com/akiban/server/store/statistics/IndexStatisticsServiceImpl.java'
--- src/main/java/com/akiban/server/store/statistics/IndexStatisticsServiceImpl.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/server/store/statistics/IndexStatisticsServiceImpl.java	2013-04-11 06:17:20 +0000
@@ -151,7 +151,7 @@
         }
         final Exchange ex = store.getExchange(session, index);
         try {
-            return AccumulatorAdapter.getSnapshot(AccumulatorAdapter.AccumInfo.ROW_COUNT, treeService, ex.getTree());
+            return AccumulatorAdapter.getSnapshot(AccumulatorAdapter.AccumInfo.ROW_COUNT, ex.getTree());
         }
         finally {
             store.releaseExchange(session, ex);
@@ -165,7 +165,7 @@
         }
         final Exchange ex = store.getExchange(session, index);
         try {
-            return AccumulatorAdapter.getLiveValue(AccumulatorAdapter.AccumInfo.ROW_COUNT, treeService, ex.getTree());
+            return AccumulatorAdapter.getLiveValue(AccumulatorAdapter.AccumInfo.ROW_COUNT, ex.getTree());
         }
         finally {
             store.releaseExchange(session, ex);
 
=== modified file 'src/main/java/com/akiban/sql/aisddl/TableDDL.java'
--- src/main/java/com/akiban/sql/aisddl/TableDDL.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/sql/aisddl/TableDDL.java	2013-04-11 06:17:20 +0000
@@ -208,15 +208,14 @@
                            final String schemaName, final String tableName, int colpos) {
 
         // Special handling for the "SERIAL" column type -> which is transformed to 
-        // BIGINT NOT NULL GENERATED BY DEFAULT AS IDENTITY (START WITH 1, INCREMENT BY 1) UNIQUE
+        // BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1) UNIQUE
         if (cdn.getType().getTypeName().equals("serial")) {
             // BIGINT NOT NULL 
             DataTypeDescriptor bigint = new DataTypeDescriptor (TypeId.BIGINT_ID, false);
             addColumn (builder, schemaName, tableName, cdn.getColumnName(), colpos,
                     bigint, false, true, getColumnDefault(cdn));
-            // GENERATED BY DEFAULT AS IDENTITY 
-            setAutoIncrement (builder, schemaName, tableName, cdn.getColumnName(),
-                    true, 1, 1);
+            // GENERATED ALWAYS AS IDENTITY
+            setAutoIncrement (builder, schemaName, tableName, cdn.getColumnName(), false, 1, 1);
             // UNIQUE (KEY)
             String constraint = Index.UNIQUE_KEY_CONSTRAINT;
             builder.index(schemaName, tableName, cdn.getColumnName(), true, constraint);
 
=== modified file 'src/test/java/com/akiban/server/rowdata/SchemaFactory.java'
--- src/test/java/com/akiban/server/rowdata/SchemaFactory.java	2013-03-22 20:05:57 +0000
+++ src/test/java/com/akiban/server/rowdata/SchemaFactory.java	2013-04-11 06:17:20 +0000
@@ -192,7 +192,8 @@
         
         @Override
         public void createSequence(Session session, Sequence sequence) {
-            ais = AISMerge.mergeSequence(ais, sequence);
+            AISMerge merge = AISMerge.newForOther(new DefaultNameGenerator(ais), ais);
+            ais = merge.mergeSequence(sequence);
         }
 
         @Override
 
=== modified file 'src/test/java/com/akiban/server/test/it/dxl/AlterTableBasicIT.java'
--- src/test/java/com/akiban/server/test/it/dxl/AlterTableBasicIT.java	2013-03-22 20:05:57 +0000
+++ src/test/java/com/akiban/server/test/it/dxl/AlterTableBasicIT.java	2013-04-11 06:17:20 +0000
@@ -20,6 +20,7 @@
 import com.akiban.ais.AISCloner;
 import com.akiban.ais.model.AISBuilder;
 import com.akiban.ais.model.AkibanInformationSchema;
+import com.akiban.ais.model.Column;
 import com.akiban.ais.model.Index;
 import com.akiban.ais.model.Sequence;
 import com.akiban.ais.model.TableName;
@@ -1169,4 +1170,22 @@
     public void changeColumnNameInGI_I() {
         changeColumnNameInGI("i", "i1", "i1_new");
     }
+
+    @Test
+    public void alterColumnDefaultIdentity() {
+        final int id = createTable(SCHEMA, C_TABLE,
+                                   "id INT NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY");
+        Column column = getUserTable(id).getColumn("id");
+        assertEquals("identity is default", true, column.getDefaultIdentity());
+        Sequence seq = column.getIdentityGenerator();
+        assertNotNull("id column has sequence", seq);
+
+        AkibanInformationSchema copy = AISCloner.clone(ais());
+        copy.getUserTable(id).getColumn("id").setDefaultIdentity(false);
+        runAlter(ChangeLevel.METADATA, C_NAME, copy.getUserTable(id),
+                 Arrays.asList(TableChange.createModify("id", "id")), Arrays.<TableChange>asList());
+
+        Column newColumn = getUserTable(id).getColumn("id");
+        assertEquals("altered is always", false, newColumn.getDefaultIdentity());
+    }
 }
 
=== modified file 'src/test/java/com/akiban/server/test/it/keyupdate/FixCountStarIT.java'
--- src/test/java/com/akiban/server/test/it/keyupdate/FixCountStarIT.java	2013-03-22 20:05:57 +0000
+++ src/test/java/com/akiban/server/test/it/keyupdate/FixCountStarIT.java	2013-04-11 06:17:20 +0000
@@ -101,7 +101,7 @@
                     PersistitStore store = store().getPersistitStore();
                     final Exchange ex = store.getExchange(session(), index);
                     try {
-                        new AccumulatorAdapter(AccumInfo.ROW_COUNT, treeService(), ex.getTree()).set(newVal);
+                        new AccumulatorAdapter(AccumInfo.ROW_COUNT, ex.getTree()).set(newVal);
                     }
                     catch (PersistitInterruptedException e) {
                         throw new PersistitAdapterException(e);
 
=== modified file 'src/test/java/com/akiban/sql/aisddl/SequenceDDLIT.java'
--- src/test/java/com/akiban/sql/aisddl/SequenceDDLIT.java	2013-03-22 20:05:57 +0000
+++ src/test/java/com/akiban/sql/aisddl/SequenceDDLIT.java	2013-04-11 06:17:20 +0000
@@ -21,6 +21,7 @@
 import static org.junit.Assert.assertNotNull;
 import static org.junit.Assert.fail;
 
+import com.akiban.ais.model.Sequence;
 import org.junit.Test;
 
 import com.akiban.ais.model.TableName;
@@ -91,5 +92,56 @@
         executeDDL(sql);
         assertEquals(Collections.singletonList(MessageFormat.format(ErrorCode.NO_SUCH_SEQUENCE.getMessage(), "test", "not_exists")), getWarnings());
     }
-    
+
+    @Test
+    public void durableAfterRollbackAndRestart() throws Exception {
+        TableName seqName = new TableName("test", "s1");
+        String sql = "CREATE SEQUENCE "+seqName+" START WITH 1 INCREMENT BY 1";
+        executeDDL(sql);
+        Sequence s1 = ais().getSequence(seqName);
+        assertNotNull("s1", s1);
+
+        txnService().beginTransaction(session());
+        assertEquals("start val a", 0, s1.currentValue());
+        assertEquals("next val a", 1, s1.nextValue());
+        txnService().commitTransaction(session());
+
+        txnService().beginTransaction(session());
+        assertEquals("next val b", 2, s1.nextValue());
+        assertEquals("cur val b", 2, s1.currentValue());
+        txnService().rollbackTransactionIfOpen(session());
+
+        txnService().beginTransaction(session());
+        assertEquals("cur val c", 1, s1.currentValue());
+        // Expected gap, see nextValue() impl
+        assertEquals("next val c", 3, s1.nextValue());
+        txnService().commitTransaction(session());
+
+        safeRestartTestServices();
+
+        s1 = ais().getSequence(seqName);
+        txnService().beginTransaction(session());
+        assertEquals("cur val after restart", 3, s1.currentValue());
+        assertEquals("next val after restart", 4, s1.nextValue());
+        txnService().commitTransaction(session());
+    }
+
+    @Test
+    public void freshValueAfterDropAndRecreate() throws Exception {
+        final TableName seqName = new TableName("test", "s2");
+        final String create = "CREATE SEQUENCE "+seqName+" START WITH 1 INCREMENT BY 1";
+        final String drop = "DROP SEQUENCE "+seqName+" RESTRICT";
+        for(int i = 1; i <= 2; ++i) {
+            executeDDL(create);
+            Sequence s1 = ais().getSequence(seqName);
+            assertNotNull("s1, loop"+i, s1);
+
+            txnService().beginTransaction(session());
+            assertEquals("start val, loop"+i, 0, s1.currentValue());
+            assertEquals("next val, loop"+i, 1, s1.nextValue());
+            txnService().commitTransaction(session());
+
+            executeDDL(drop);
+        }
+    }
 }
 
=== modified file 'src/test/java/com/akiban/sql/aisddl/TableDDLIT.java'
--- src/test/java/com/akiban/sql/aisddl/TableDDLIT.java	2013-03-22 20:05:57 +0000
+++ src/test/java/com/akiban/sql/aisddl/TableDDLIT.java	2013-04-11 06:17:20 +0000
@@ -367,7 +367,7 @@
         assertEquals (1, column.getIdentityGenerator().getStartsWith());
         assertEquals (1, column.getIdentityGenerator().getIncrement());
         assertNotNull(column.getDefaultIdentity());
-        assertTrue(column.getDefaultIdentity().booleanValue());
+        assertFalse(column.getDefaultIdentity());
 
         assertNotNull (table.getIndex("c1"));
         Index index = table.getIndex("c1");
 
=== modified file 'src/test/java/com/akiban/sql/pg/PostgresServerMiscYamlIT.java'
--- src/test/java/com/akiban/sql/pg/PostgresServerMiscYamlIT.java	2013-03-22 20:05:57 +0000
+++ src/test/java/com/akiban/sql/pg/PostgresServerMiscYamlIT.java	2013-04-11 06:17:20 +0000
@@ -31,6 +31,8 @@
 import java.io.FileFilter;
 import java.util.ArrayList;
 import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
 import java.util.Map;
 import java.util.regex.Pattern;
 
@@ -82,7 +84,12 @@
 
     @Override
     protected Map<String, String> startupConfigProperties() {
-        return uniqueStartupConfigProperties(getClass());
+        // TODO: Remove whenever test-seq-fixup-routines.yaml no longer exists
+        Map<String, String> props = new HashMap();
+        props.put("akserver.dxl.use_global_lock", "true");
+        props.putAll(uniqueStartupConfigProperties(getClass()));
+        return props;
+        //return uniqueStartupConfigProperties(getClass());
     }
 
     @Test
 
=== added file 'src/test/resources/com/akiban/sql/pg/yaml/functional/test-seq-fixup-routines.yaml'
--- src/test/resources/com/akiban/sql/pg/yaml/functional/test-seq-fixup-routines.yaml	1970-01-01 00:00:00 +0000
+++ src/test/resources/com/akiban/sql/pg/yaml/functional/test-seq-fixup-routines.yaml	2013-04-11 06:17:20 +0000
@@ -0,0 +1,40 @@
+# Can go away whenever SequenceFixUpRoutines does
+
+# Requires global lock enabled, skip in FTS
+---
+- Properties: sys-aksql
+- suppressed: true
+
+# Re-set a sequence
+---
+- Statement: CREATE SEQUENCE s1 START WITH 1 INCREMENT BY 1
+---
+- Statement: SELECT NEXT VALUE FOR s1
+- output: [[1]]
+---
+- Statement: CALL sys.seq_tree_reset('test', 's1', 150)
+---
+- Statement: SELECT CURRENT VALUE FOR s1
+- output: [[150]]
+---
+- Statement: DROP SEQUENCE s1 RESTRICT
+
+# ALTER BY DEFAULT to ALWAYS
+---
+- Statement: CREATE TABLE t(id INT NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY (START WITH 5))
+---
+- Statement: SELECT identity_generation FROM information_schema.columns WHERE table_name='t' AND column_name='id'
+- output: [['BY DEFAULT']]
+---
+- Statement: CALL sys.seq_identity_default_to_always('test', 't', 'id')
+---
+- Statement: SELECT identity_generation FROM information_schema.columns WHERE table_name='t' AND column_name='id'
+- output: [['ALWAYS']]
+---
+# Expect value to be ignored due to ALWAYS
+- Statement: INSERT INTO t VALUES (100)
+---
+- Statement: SELECT * FROM t
+- output: [[5]]
+---
+- Statement: DROP TABLE t
 
=== modified file 'src/test/resources/com/akiban/sql/pg/yaml/functional/test-sequence.yaml'
--- src/test/resources/com/akiban/sql/pg/yaml/functional/test-sequence.yaml	2012-09-26 03:50:12 +0000
+++ src/test/resources/com/akiban/sql/pg/yaml/functional/test-sequence.yaml	2013-04-11 06:17:20 +0000
@@ -95,5 +95,53 @@
 ---
 - Statement: create sequence bad_sequence start with 1 increment by 1 minvalue 1 maxvalue 1 no cycle;
 - error: [50020]
+
+
+---
+- Statement: create sequence sequence5 start with 3 increment by 3 minvalue -5 maxvalue 5 cycle;
+---
+- Statement: select next value for sequence5
+- output: [[3]]
+---
+- Statement: select next value for sequence5
+- output: [[-5]]
+---
+- Statement: select next value for sequence5
+- output: [[-2]]
+---
+- Statement: select next value for sequence5
+- output: [[1]]
+---
+- Statement: select next value for sequence5
+- output: [[4]]
+---
+- Statement: select current value for sequence5
+- output: [[4]]
+---
+- Statement: drop sequence sequence5 restrict
+
+---
+- Statement: create sequence sequence6 start with 3 increment by -3 minvalue -5 maxvalue 5 cycle;
+---
+- Statement: select next value for sequence6
+- output: [[3]]
+---
+- Statement: select next value for sequence6
+- output: [[0]]
+---
+- Statement: select next value for sequence6
+- output: [[-3]]
+---
+- Statement: select next value for sequence6
+- output: [[5]]
+---
+- Statement: select next value for sequence6
+- output: [[2]]
+---
+- Statement: select current value for sequence6
+- output: [[2]]
+---
+- Statement: drop sequence sequence6 restrict
+
 ...