Merge lp:~pbeaman/akiban-persistit/improve-KeyCoder into lp:akiban-persistit

[Peter Beaman]

Reduces the size of serialized objects written by a KeyCoder.

This branch modifies the encoding generated by KeyCoder objects (such as CString in Akiban Server) to reduce the number of bytes required. There are two main changes:

- The encoding of handles is usually reduced to two bytes to one
- KeyCoder now has a new method isZeroByteFree().

If isZeroByteFree() returns true then Key#appendByKeyCoder does not modify the serialized form; otherwise it escapes bytes having values of 0 or 1.

These are small changes, but it is desirable to modify the encoding format to a more compact form now before a significant amount of data has accumulated with the old encoding format.

[Peter Beaman (pbeaman)]

The server branch lp:~pbeaman/akiban-server/improve-KeyCoder must be merged first.

[Nathan Williams (nwilliams)]

computeBogusHash() looks unused, but otherwise looks as described.

Will wait until server side goes into Approve.

review: Approve

[lp:~pbeaman/akiban-persistit/improve-KeyCoder]

342. By Peter Beaman on 2012-07-24

Remove computeBogusHash

[Peter Beaman (pbeaman)]

Upon further reflection I suggest that we should not approve this without a version bump to avoid inconveniencing the server team. Will propose that shortly.

review: Needs Fixing

[lp:~pbeaman/akiban-persistit/improve-KeyCoder]

343. By Peter Beaman on 2012-07-25

Bump version number

[Peter Beaman (pbeaman)]

Bumped version to 3.1.4. Approving now since all other changes were already approved.

review: Approve

=== modified file 'pom.xml'
--- pom.xml	2012-07-13 20:09:41 +0000
+++ pom.xml	2012-07-25 14:54:18 +0000
@@ -4,7 +4,7 @@
     
     <groupId>com.akiban</groupId>
     <artifactId>akiban-persistit</artifactId>
-    <version>3.1.3-SNAPSHOT</version>
+    <version>3.1.4-SNAPSHOT</version>
     <packaging>jar</packaging>
     
     <name>akiban-persistit</name>

=== modified file 'src/main/java/com/persistit/ClassIndex.java'
--- src/main/java/com/persistit/ClassIndex.java	2012-07-20 19:25:35 +0000
+++ src/main/java/com/persistit/ClassIndex.java	2012-07-25 14:54:18 +0000
@@ -58,6 +58,7 @@
  * @version 1.1
  */
 final class ClassIndex {
+    final static int HANDLE_BASE = 64;
     private final static int INITIAL_CAPACITY = 123;
 
     private final static String BY_HANDLE = "byHandle";
@@ -283,7 +284,7 @@
                         // Store a new ClassInfo record
                         //
                         ex.clear().append(NEXT_ID).fetch();
-                        handle = Math.max(_testIdFloor, value.isDefined() ? value.getInt() + 1 : 65);
+                        handle = Math.max(_testIdFloor, value.isDefined() ? value.getInt() : HANDLE_BASE) + 1;
                         value.clear().put(handle);
                         ex.store();
 

=== modified file 'src/main/java/com/persistit/DefaultObjectCoder.java'
--- src/main/java/com/persistit/DefaultObjectCoder.java	2012-05-25 18:50:59 +0000
+++ src/main/java/com/persistit/DefaultObjectCoder.java	2012-07-25 14:54:18 +0000
@@ -436,33 +436,7 @@
         return _keyBuilder;
     }
 
-    /**
-     * <p>
-     * Append a key segment derived from an object to a {@link Key}. This method
-     * will be called only if this <code>KeyCoder</code> has been registered
-     * with the current {@link CoderManager} to encode objects having the class
-     * of the supplied object.
-     * </p>
-     * <p>
-     * Upon completion of this method, the backing byte array of the
-     * <code>Key</code> and its size should be updated to reflect the appended
-     * key segment. Use the methods {@link Key#getEncodedBytes},
-     * {@link Key#getEncodedSize} and {@link Key#setEncodedSize} to manipulate
-     * the byte array directly. More commonly, the implementation of this method
-     * will simply append appropriate identifying fields within the object to
-     * the key.
-     * </p>
-     * 
-     * @param key
-     *            The {@link Key} to append to
-     * @param object
-     *            The object to encode
-     * @param context
-     *            An arbitrary object that can optionally be supplied by the
-     *            application to convey an application-specific context for the
-     *            operation. (See {@link CoderContext}.) The default value is
-     *            <code>null</code>.
-     */
+
     @Override
     public void appendKeySegment(Key key, Object object, CoderContext context) throws ConversionException {
         Accessor accessor = null;
@@ -479,38 +453,6 @@
         }
     }
 
-    /**
-     * <p>
-     * Decode a key segment as an Object value. This method is invoked when
-     * {@link Key#decode(Object)} attempts to decode a key segment that was
-     * previously append by the {@link #appendKeySegment} method. Thus the
-     * implementation of this method must precisely decode the same bytes that
-     * were generated by the companion <code>appendKeySegment</code> method of
-     * this <code>ValueCoder</code>.
-     * </p>
-     * <p>
-     * This method will be called only if this <code>KeyCoder</code> is
-     * registered with the current {@link CoderManager} to encode and decode
-     * objects of the class that is actually represented in the <code>Key</code>
-     * . The class with which the key was encoded is provided as an argument.
-     * This permits one <code>KeyCoder</code> to handle multiple classes because
-     * the implementation can dispatch to code appropriate for the particular
-     * supplied class. The implementation should construct and return an Object
-     * having the same class as the supplied class.
-     * </p>
-     * 
-     * @param key
-     *            The {@link Key} from which data should be decoded
-     * @param clazz
-     *            The expected <code>Class</code> of the returned object
-     * @param context
-     *            An arbitrary object that can optionally be supplied by the
-     *            application to convey an application-specific context for the
-     *            operation. (See {@link CoderContext}.) The default value is
-     *            <code>null</code>.
-     * @throws ConversionException
-     */
-
     @Override
     public Object decodeKeySegment(Key key, Class clazz, CoderContext context) throws ConversionException {
         if (clazz != getClientClass())
@@ -527,6 +469,12 @@
 
         return readResolve(instance);
     }
+    
+
+    @Override
+    public boolean isZeroByteFree() {
+        return false;
+    }
 
     /**
      * <p>

=== modified file 'src/main/java/com/persistit/Key.java'
--- src/main/java/com/persistit/Key.java	2012-05-25 18:50:59 +0000
+++ src/main/java/com/persistit/Key.java	2012-07-25 14:54:18 +0000
@@ -889,6 +889,7 @@
     private int _depth;
     private int _maxSize;
     private long _generation;
+    private boolean _inKeyCoder;
     private Persistit _persistit;
 
     /**
@@ -2673,10 +2674,10 @@
                 throw new ConversionException("Invalid String lead-in byte (" + type + ") at position " + _index
                         + " in key");
             }
-            int size = unquoteNulls(index + 1);
+            int size = unquoteNulls(index + 1, false);
             byte[] result = new byte[size];
             System.arraycopy(_bytes, index + 1, result, 0, size);
-            index += quoteNulls(index + 1, size) + 2;
+            index += quoteNulls(index + 1, size, false) + 2;
             return result;
         } finally {
             _index = index;
@@ -3119,7 +3120,7 @@
     }
 
     void testValidForAppend() {
-        if (_size == 0 || _size > 1 && _bytes[_size - 1] == 0)
+        if (_size == 0 || _size > 1 && _bytes[_size - 1] == 0 || _inKeyCoder)
             return;
 
         int b = _bytes[_size - 1] & 0xFF;
@@ -3282,27 +3283,25 @@
     private Key appendNull() {
         int size = _size;
         _bytes[size++] = TYPE_NULL;
-
         return endSegment(size);
     }
 
     private Key appendByKeyCoder(Object object, Class<?> cl, KeyCoder coder, CoderContext context) {
         int size = _size;
+        boolean saveInKeyCoder = _inKeyCoder;
         try {
             int handle = _persistit.getClassIndex().lookupByClass(cl).getHandle();
-
             _size += encodeHandle(handle);
             int begin = _size;
-            byte saveByte = _bytes[begin - 1];
-            _bytes[begin - 1] = 0;
+            _inKeyCoder = true;
             coder.appendKeySegment(this, object, context);
-            _bytes[begin - 1] = saveByte;
-            quoteNulls(begin, _size - begin);
+            quoteNulls(begin, _size - begin, coder.isZeroByteFree());
             endSegment(_size);
             size = _size;
             return this;
         } finally {
             _size = size;
+            _inKeyCoder = saveInKeyCoder;
         }
     }
 
@@ -3312,6 +3311,7 @@
         Class<?> clazz = Object.class;
         int segmentSize = 0;
         boolean unquoted = false;
+        boolean zeroByteFree = false;
         try {
             int handle = decodeHandle();
             clazz = _persistit.classForHandle(handle);
@@ -3322,10 +3322,11 @@
             if (coder == null) {
                 throw new ConversionException("No KeyCoder for class " + clazz.getName());
             }
-            segmentSize = unquoteNulls(index);
-            unquoted = true;
+            zeroByteFree = coder.isZeroByteFree();
+            segmentSize = unquoteNulls(index, zeroByteFree);
             size = _size;
             _size = index + segmentSize;
+            unquoted = true;
             if (target == null) {
                 return coder.decodeKeySegment(this, clazz, context);
             } else {
@@ -3339,7 +3340,7 @@
         } finally {
             _size = size;
             if (unquoted) {
-                index += quoteNulls(index, segmentSize);
+                index += quoteNulls(index, segmentSize, zeroByteFree);
                 index = decodeEnd(index);
             }
             _index = index;
@@ -3352,6 +3353,7 @@
         Class<?> clazz;
         int segmentSize = 0;
         boolean unquoted = false;
+        boolean zeroByteFree = false;
         try {
             int handle = decodeHandle();
             clazz = _persistit.classForHandle(handle);
@@ -3365,8 +3367,9 @@
                 Util.append(sb, clazz.getName());
                 Util.append(sb, ")");
                 coder = _persistit.lookupKeyCoder(clazz);
+                zeroByteFree = coder.isZeroByteFree();
             }
-            segmentSize = unquoteNulls(index);
+            segmentSize = unquoteNulls(index, zeroByteFree);
             size = _size;
             unquoted = true;
 
@@ -3385,7 +3388,7 @@
         } finally {
             _size = size;
             if (unquoted) {
-                index += quoteNulls(index, segmentSize);
+                index += quoteNulls(index, segmentSize, zeroByteFree);
                 index = decodeEnd(index);
             }
             _index = index;
@@ -3462,7 +3465,7 @@
         int keySize = _size;
         System.arraycopy(bytes, offset, _bytes, keySize, size);
         _size += size;
-        keySize += quoteNulls(keySize, size);
+        keySize += quoteNulls(keySize, size, false);
         return endSegment(keySize);
     }
 
@@ -3552,31 +3555,33 @@
     }
 
     private int encodeHandle(int handle) {
+        
+        int v = handle - ClassIndex.HANDLE_BASE;
         int size = _size;
 
-        if (handle < 0x00000007) {
-            _bytes[size++] = (byte) (TYPE_CODER1 | ((handle) & 0x7));
+        if (v < 0x00000007) {
+            _bytes[size++] = (byte) (TYPE_CODER1 | ((v) & 0x7));
             return 1;
         }
 
-        if (handle < 0x000001FF) {
-            _bytes[size++] = (byte) (TYPE_CODER2 | ((handle >>> 6) & 0x7));
-            _bytes[size++] = (byte) (0x80 | ((handle) & 0x3F));
+        if (v < 0x000001FF) {
+            _bytes[size++] = (byte) (TYPE_CODER2 | ((v >>> 6) & 0x7));
+            _bytes[size++] = (byte) (0x80 | ((v) & 0x3F));
             return 2;
         }
 
-        if (handle < 0x00007FFF) {
-            _bytes[size++] = (byte) (TYPE_CODER3 | ((handle >>> 12) & 0x7));
-            _bytes[size++] = (byte) (0xC0 | ((handle >>> 6) & 0x3F));
-            _bytes[size++] = (byte) (0x80 | ((handle) & 0x3F));
+        if (v < 0x00007FFF) {
+            _bytes[size++] = (byte) (TYPE_CODER3 | ((v >>> 12) & 0x7));
+            _bytes[size++] = (byte) (0xC0 | ((v >>> 6) & 0x3F));
+            _bytes[size++] = (byte) (0x80 | ((v) & 0x3F));
             return 3;
         } else {
-            _bytes[size++] = (byte) (TYPE_CODER6 | ((handle >>> 30) & 0x7));
-            _bytes[size++] = (byte) (0xC0 | ((handle >>> 24) & 0x3F));
-            _bytes[size++] = (byte) (0xC0 | ((handle >>> 18) & 0x3F));
-            _bytes[size++] = (byte) (0xC0 | ((handle >>> 12) & 0x3F));
-            _bytes[size++] = (byte) (0xC0 | ((handle >>> 6) & 0x3F));
-            _bytes[size++] = (byte) (0x80 | ((handle) & 0x3F));
+            _bytes[size++] = (byte) (TYPE_CODER6 | ((v >>> 30) & 0x7));
+            _bytes[size++] = (byte) (0xC0 | ((v >>> 24) & 0x3F));
+            _bytes[size++] = (byte) (0xC0 | ((v >>> 18) & 0x3F));
+            _bytes[size++] = (byte) (0xC0 | ((v >>> 12) & 0x3F));
+            _bytes[size++] = (byte) (0xC0 | ((v >>> 6) & 0x3F));
+            _bytes[size++] = (byte) (0x80 | ((v) & 0x3F));
             return 6;
         }
     }
@@ -3617,22 +3622,15 @@
 
         case TYPE_CODER2:
             v = _bytes[index++] & 0xFF;
-            if ((v & 0xC0) != 0xC0)
+            if ((v & 0x80) != 0x80)
                 break;
             result = result << 6 | (v & 0x3F);
 
             // Intentionally falls through
 
         case TYPE_CODER1:
-            v = _bytes[index++] & 0xFF;
-            if ((v & 0xC0) != 0x80)
-                break;
-            result = result << 6 | (v & 0x3F);
-
-            // Intentionally falls through
-
             _index = index;
-            return result;
+            return result + ClassIndex.HANDLE_BASE;
 
         default:
         }
@@ -3674,21 +3672,27 @@
      * (1, C) where C is 32 for NUL and 33 for SOH.
      * 
      * @param index
-     *            Offet to first byte to convert
+     *            Offset to first byte to convert
      * @param size
      *            Length of raw segment, in bytes.
      * @return Length of the quoted segment
      */
-    private int quoteNulls(int index, int size) {
+    private int quoteNulls(int index, int size, boolean zeroByteFree) {
         for (int i = index; i < index + size; i++) {
             int c = _bytes[i] & 0xFF;
-            if (c == 0 || c == 1) {
-                System.arraycopy(_bytes, i + 1, _bytes, i + 2, _size - i - 1);
-                _bytes[i] = 1;
-                _bytes[i + 1] = (byte) (c + 32);
-                _size++;
-                size++;
-                i++;
+            if (zeroByteFree) {
+                if (c == 0) {
+                    throw new ConversionException("NUL found in encoded Key");
+                }
+            } else {
+                if (c == 0 || c == 1) {
+                    System.arraycopy(_bytes, i + 1, _bytes, i + 2, _size - i - 1);
+                    _bytes[i] = 1;
+                    _bytes[i + 1] = (byte) (c + 32);
+                    _size++;
+                    size++;
+                    i++;
+                }
             }
         }
         return size;
@@ -3702,12 +3706,13 @@
      * @param index
      * @return The unquoted length of the array.
      */
-    private int unquoteNulls(int index) {
+    private int unquoteNulls(int index, boolean zeroByteFree) {
         for (int i = index; i < _size; i++) {
             int c = _bytes[i] & 0xFF;
-            if (c == 0)
+            if (c == 0) {
                 return i - index;
-            if (c == 1 && i + 1 < _size) {
+            }
+            if (!zeroByteFree && c == 1 && i + 1 < _size) {
                 c = _bytes[i + 1];
                 if (c == 32 || c == 33) {
                     _bytes[i] = (byte) (c - 32);
@@ -3997,4 +4002,3 @@
         bis._scale = -exp;
     }
 }
-

=== modified file 'src/main/java/com/persistit/encoding/KeyCoder.java'
--- src/main/java/com/persistit/encoding/KeyCoder.java	2012-05-25 18:50:59 +0000
+++ src/main/java/com/persistit/encoding/KeyCoder.java	2012-07-25 14:54:18 +0000
@@ -95,6 +95,11 @@
      * supplied class. The implementation should construct and return an Object
      * having the same class as the supplied class.
      * </p>
+     * <p>
+     * When this method is called the value {@link Key#getIndex()} will be the
+     * offset within the key of the first encoded byte. The key segment is
+     * zero-byte terminated.
+     * </p>
      * 
      * @param key
      *            The {@link Key} from which data should be decoded
@@ -109,4 +114,27 @@
      */
 
     public Object decodeKeySegment(Key key, Class<?> clazz, CoderContext context) throws ConversionException;
+
+    /**
+     * <p>
+     * Since a key segment is terminated by a zero byte value the encoded
+     * portion of the segment must not contain zeroes. This method should
+     * indicate whether the encoding implemented by this KeyCoder produces zero
+     * bytes.
+     * </p>
+     * <p>
+     * If this method returns <code>false</code> then Persistit will insert
+     * escape sequences into the encoding produced by
+     * {@link #appendKeySegment(Key, Object, CoderContext)} to a form with no
+     * zeroes.
+     * </p>
+     * <p>
+     * If this method returns <code>true</code> then Persistit will verify that
+     * there are no zero bytes and throw an ConversionException otherwise.
+     * 
+     * @return whether this KeyCoder produces encodings guaranteed not to
+     *         contain zero bytes
+     * @throws ConversionException
+     */
+    public boolean isZeroByteFree() throws ConversionException;
 }

=== modified file 'src/main/java/com/persistit/encoding/KeyDisplayer.java'
--- src/main/java/com/persistit/encoding/KeyDisplayer.java	2012-05-25 18:50:59 +0000
+++ src/main/java/com/persistit/encoding/KeyDisplayer.java	2012-07-25 14:54:18 +0000
@@ -51,6 +51,11 @@
      * call this method to decode a value that was <code>null</code> when
      * written because null values are handled by built-in encoding logic.
      * </p>
+     * <p>
+     * When this method is called the value {@link Key#getIndex()} will be the
+     * offset within the key of the first encoded byte. The key segment is
+     * zero-byte terminated.
+     * </p>
      * 
      * @param key
      *            The <code>Key</code> from which interior fields of the object

=== modified file 'src/main/java/com/persistit/encoding/KeyRenderer.java'
--- src/main/java/com/persistit/encoding/KeyRenderer.java	2012-05-25 18:50:59 +0000
+++ src/main/java/com/persistit/encoding/KeyRenderer.java	2012-07-25 14:54:18 +0000
@@ -50,6 +50,11 @@
      * call this method to decode a value that was <code>null</code> when
      * written because null values are handled by built-in encoding logic.
      * </p>
+     * <p>
+     * When this method is called the value {@link Key#getIndex()} will be the
+     * offset within the key of the first encoded byte. The key segment is
+     * zero-byte terminated.
+     * </p>
      * 
      * @param key
      *            The <code>Key</code> from which interior fields of the object

=== modified file 'src/main/java/com/persistit/exception/ConversionException.java'
--- src/main/java/com/persistit/exception/ConversionException.java	2012-05-25 18:50:59 +0000
+++ src/main/java/com/persistit/exception/ConversionException.java	2012-07-25 14:54:18 +0000
@@ -22,8 +22,8 @@
 
 /**
  * Thrown by encoding and decoding methods of the {@link com.persistit.Value}
- * class when conversion to or from an value's serialized form cannot be
- * completed.
+ * and {@link com.persistit.Key} classes when conversion to or from an value's
+ * serialized form cannot be completed.
  * 
  * @version 1.0
  */

=== modified file 'src/test/java/com/persistit/unit/KeyCoderTest1.java'
--- src/test/java/com/persistit/unit/KeyCoderTest1.java	2012-05-25 18:50:59 +0000
+++ src/test/java/com/persistit/unit/KeyCoderTest1.java	2012-07-25 14:54:18 +0000
@@ -21,7 +21,9 @@
 package com.persistit.unit;
 
 import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
 
+import java.io.Serializable;
 import java.net.MalformedURLException;
 import java.net.URL;
 
@@ -33,35 +35,34 @@
 import com.persistit.TestShim;
 import com.persistit.encoding.CoderContext;
 import com.persistit.encoding.KeyCoder;
+import com.persistit.encoding.KeyDisplayer;
 import com.persistit.encoding.KeyRenderer;
 import com.persistit.exception.ConversionException;
 import com.persistit.exception.PersistitException;
+import com.persistit.util.Util;
 
 public class KeyCoderTest1 extends PersistitUnitTestCase {
 
     @Test
-    public void test2() throws MalformedURLException {
-        System.out.print("test2 ");
-        final KeyCoder coder = new TestKeyRenderer();
+    public void normalKeyCoder() throws MalformedURLException {
+        final KeyCoder coder = new TestKeyRenderer1();
         _persistit.getCoderManager().registerKeyCoder(URL.class, coder);
-        Key key1;
-
-        key1 = new Key(_persistit);
+        Key key = new Key(_persistit);
         final URL url1 = new URL("http://w/z");
         final URL url2 = new URL("http://w:8080/z?userid=pb");
-        key1.clear();
-        key1.append("a");
-        key1.append(url1);
-        key1.append("b");
-        key1.append(url2);
-        key1.append("c");
+        key.clear();
+        key.append("a");
+        key.append(url1);
+        key.append("b");
+        key.append(url2);
+        key.append("c");
 
-        key1.reset();
-        final String a = key1.decodeString();
-        final Object obj1 = key1.decode();
-        final String b = key1.decodeString();
-        final Object obj2 = key1.decode();
-        final String c = key1.decodeString();
+        key.reset();
+        final String a = key.decodeString();
+        final Object obj1 = key.decode();
+        final String b = key.decodeString();
+        final Object obj2 = key.decode();
+        final String c = key.decodeString();
 
         assertEquals("a", a);
         assertEquals("b", b);
@@ -70,26 +71,73 @@
         assertEquals(url2.toString(), obj2.toString());
 
         final StringBuilder sb = new StringBuilder();
-        key1.indexTo(1);
-        key1.decode(sb);
+        key.indexTo(1);
+        key.decode(sb);
         assertEquals(url1.toString(), sb.toString());
 
         sb.setLength(0);
-        key1.indexTo(3);
-        key1.decode(sb);
+        key.indexTo(3);
+        key.decode(sb);
         assertEquals(url2.toString(), sb.toString());
 
-        key1.reset();
-        final String toString = key1.toString();
+        key.reset();
+        final String toString = key.toString();
         assertEquals(toString, "{\"a\",(java.net.URL){\"http\",\"w\",-1,\"/z\"},\"b\","
                 + "(java.net.URL){\"http\",\"w\",8080,\"/z?userid=pb\"},\"c\"}");
-        System.out.println("- done");
+    }
+    
+    @Test
+    public void zeroByteFreeKeyCoder() throws Exception {
+        final KeyCoder coder = new TestKeyRenderer2();
+        _persistit.getCoderManager().registerKeyCoder(WrappedString.class, coder);
+        Key key = new Key(_persistit);
+        key.append(1.2f);
+        key.append(new WrappedString(RED_FOX));
+        key.append("another key");
+        key.append(new WrappedString("abcde"));
+        key.reset();
+        assertEquals(Float.valueOf(1.2f),key.decode());
+        Object ws = key.decode();
+        assertTrue(ws instanceof WrappedString);
+        assertEquals(RED_FOX, ((WrappedString)ws)._value);
+        assertEquals("another key", key.decodeString());
+        key.decode(ws);
+        assertEquals("abcde", ((WrappedString)ws)._value);
+    }
+    
+    @Test
+    public void normalKeyCoderQuotesNulls() throws Exception {
+        final KeyCoder coder = new TestKeyRenderer3();
+        _persistit.getCoderManager().registerKeyCoder(WrappedLong.class, coder);
+        Key key = new Key(_persistit);
+        for (long value = 0; value < 32; value++) {
+            key.append(new WrappedLong(value));
+        }
+        key.reset();
+        for (int depth = 0; depth < 50; depth++) {
+            key.indexTo(depth);
+            if (key.getIndex() == key.getEncodedSize()) {
+                assertEquals(32, depth);
+                break;
+            }
+            long value = ((WrappedLong)key.decode())._value;
+            assertEquals(depth, value);
+        }
+    }
+    
+    @Test(expected = ConversionException.class)
+    public void zeroByteFreeKeyCoderThrows() throws Exception {
+        final KeyCoder coder = new TestKeyRenderer2();
+        _persistit.getCoderManager().registerKeyCoder(WrappedString.class, coder);
+        Key key = new Key(_persistit);
+        WrappedString ws = new WrappedString(new String(new byte[]{'a', 'b', 0, 'c'}));
+        key.append(ws);
     }
 
     @Test
-    public void testKeyHandleEncodeDecode() throws Exception {
-        final int[] handles = { 0, 0x1FE, 0x1FF, 0x200, 0x201, 0x7FFE, 0x7FFF, 0x8000, 0x8001, Integer.MAX_VALUE };
-        final KeyCoder coder = new TestKeyRenderer();
+    public void keyHandleEncodeDecode() throws Exception {
+        final int[] handles = { 64, 0x1FE, 0x1FF, 0x200, 0x201, 0x7FFE, 0x7FFF, 0x8000, 0x8001, Integer.MAX_VALUE };
+        final KeyCoder coder = new TestKeyRenderer1();
         final Key key1 = new Key(_persistit);
         final URL url1 = new URL("http://w/z");
         final URL url2 = new URL("http://w:8080/z?userid=pb");
@@ -115,7 +163,7 @@
 
         }
     }
-
+    
     private void registerCoderDefinedKeyHandle(final KeyCoder coder, final int handle) throws PersistitException {
         TestShim.clearAllClassIndexEntries(_persistit);
         TestShim.setClassIndexTestIdFloor(_persistit, handle);
@@ -130,7 +178,7 @@
         Assert.assertTrue(condition);
     }
 
-    public static class TestKeyRenderer implements KeyRenderer {
+    public static class TestKeyRenderer1 implements KeyRenderer {
 
         @Override
         public void appendKeySegment(final Key key, final Object object, final CoderContext context) {
@@ -142,7 +190,7 @@
         }
 
         @Override
-        public Object decodeKeySegment(final Key key, final Class cl, final CoderContext context) {
+        public Object decodeKeySegment(final Key key, final Class<?> cl, final CoderContext context) {
             final String protocol = key.decodeString();
             final String host = key.decodeString();
             final int port = key.decodeInt();
@@ -155,7 +203,7 @@
         }
 
         @Override
-        public void renderKeySegment(final Key key, final Object target, final Class cl, final CoderContext context) {
+        public void renderKeySegment(final Key key, final Object target, final Class<?> cl, final CoderContext context) {
             final StringBuilder sb = (StringBuilder) target;
             key.decodeString(sb);
             sb.append("://");
@@ -167,5 +215,103 @@
             }
             key.decodeString(sb);
         }
-    }
+        
+        @Override
+        public boolean isZeroByteFree() {
+            return false;
+        }
+    }
+    
+    @SuppressWarnings("serial")
+    public static class WrappedString implements Serializable {
+        String _value;
+        WrappedString(final String value) {
+            _value = value;
+        }
+    }
+    
+    @SuppressWarnings("serial")
+    public static class WrappedLong implements Serializable {
+        long _value;
+        WrappedLong(final long value) {
+            _value = value;
+        }
+    }
+    
+    
+    public static class TestKeyRenderer2 implements KeyRenderer {
+
+        @Override
+        public void appendKeySegment(final Key key, final Object object, final CoderContext context) {
+            final WrappedString cstring = (WrappedString) object;
+            byte[] bytes = cstring._value.getBytes();
+            int size = key.getEncodedSize();
+            System.arraycopy(bytes, 0, key.getEncodedBytes(), size, bytes.length);
+            key.setEncodedSize(size + bytes.length);
+        }
+
+        @Override
+        public Object decodeKeySegment(final Key key, final Class<?> cl, final CoderContext context) {
+            WrappedString target = new WrappedString(null);
+            renderKeySegment(key, target, cl, context);
+            return target;
+        }
+
+        @Override
+        public void renderKeySegment(final Key key, final Object target, final Class<?> cl, final CoderContext context) {
+            WrappedString cstring = (WrappedString)target;
+            int size = key.getEncodedSize();
+            int index = key.getIndex();
+            byte[] b = new byte[size - index];
+            System.arraycopy(key.getEncodedBytes(), index, b, 0, b.length);
+            cstring._value = new String(b);
+        }
+        
+        @Override
+        public boolean isZeroByteFree() {
+            return true;
+        }
+    }
+
+    public static class TestKeyRenderer3 implements KeyRenderer, KeyDisplayer {
+
+        @Override
+        public void appendKeySegment(final Key key, final Object object, final CoderContext context) {
+            final WrappedLong wlong = (WrappedLong)object;
+           int size = key.getEncodedSize();
+           Util.putLong(key.getEncodedBytes(), size, wlong._value);
+            key.setEncodedSize(size + 8);
+        }
+
+        @Override
+        public Object decodeKeySegment(final Key key, final Class<?> cl, final CoderContext context) {
+            WrappedLong target = new WrappedLong(Long.MIN_VALUE);
+            renderKeySegment(key, target, cl, context);
+            return target;
+        }
+
+        @Override
+        public void renderKeySegment(final Key key, final Object target, final Class<?> cl, final CoderContext context) {
+            WrappedLong wl = (WrappedLong)target;
+            int size = key.getEncodedSize();
+            int index = key.getIndex();
+            assertEquals(8, size - index);
+            wl._value = Util.getLong(key.getEncodedBytes(), index);
+            key.setIndex(index + 8);
+        }
+        
+        @Override
+        public boolean isZeroByteFree() {
+            return false;
+        }
+
+        @Override
+        public void displayKeySegment(Key key, Appendable target, Class<?> clazz, CoderContext context)
+                throws ConversionException {
+            WrappedLong wl = new WrappedLong(Long.MIN_VALUE);
+            renderKeySegment(key, wl, clazz, context);
+            Util.append(target, Long.toString(wl._value));
+        }
+    }
+
 }