Merge lp:~pbeaman/akiban-server/fix-direct-rest-bugs into lp:~akiban-technologies/akiban-server/trunk

[Peter Beaman]

Fix several Akiban Direct issues itemized in bug1168547 and bug1168962.

Also, add new test framework, parallel to RestServiceFilesIT called RestServiceScriptIT. Difference is that RestServiceScriptIT reads test steps from a file with suffix ".script" and executes them in sequence. This allows tests to build over state over multiple REST calls. Several test scripts to verify fixes for the itemized bugs are included in src/test/resource/com/akiban/direct. Javadoc in the test framework describes how to write a script. Note: I did not use YAML for expediency - if there is strong sentiment to avoid defining yet another way of specifying tests, I would be willing to redo the script elements as YAML documents; however, this will cost some time and effort.

Specific changes:

- Loop for transaction retry limited to 3 iterations before an internal server error (500) is returned.
- Exception text returned to REST client now includes detailed cause information when _register method fails.
- When a Direct REST request fails to match a defined Direct end-point, the error now returned is NOT_FOUND (404).
- End-point matching logic now applies content type filter in all cases and avoid throwing an NPE when a content type is not specified.
- Registering a library that has no _register method no longer causes other valid libraries to be ignored.
- Validation now enforces that an end-point specification must be complete. For example, a missing IN definition is now caught during the attempt to register the end-point rather than causing an NPE when the end-pint is invoked.
- All valid Java names are now permitted in the end-point metadata specification for field requiring a name, such as the function name.

[lp:~pbeaman/akiban-server/fix-direct-rest-bugs]

2637. By Peter Beaman on 2013-04-21

Add a LOG message when transaction retries more than 3 times

[Mike McMahon (mmcm)]

Looks okay but doesn't compile.

There is no org.apache.http.HttpStatus. httpcomponents core isn't in the pom. I think HttpServletResponse will give the necessary SC_ constants, if that's all that's needed.

RestServiceScriptsIT.java does not have a proper header.

review: Needs Fixing

[lp:~pbeaman/akiban-server/fix-direct-rest-bugs]

2638. By Peter Beaman on 2013-04-23

Merge from trunk, fix HttpStatus import and bad copyright notice

[Peter Beaman (pbeaman)]

Thanks. Fixed.

[Mike McMahon (mmcm)]

review: Approve

=== modified file 'src/main/java/com/akiban/direct/DirectException.java'
--- src/main/java/com/akiban/direct/DirectException.java	2013-03-22 20:05:57 +0000
+++ src/main/java/com/akiban/direct/DirectException.java	2013-04-23 01:21:27 +0000
@@ -26,8 +26,6 @@
  */
 public class DirectException extends RuntimeException {
 
-    private static final long serialVersionUID = 9217986178149864740L;
-
     public DirectException(final Exception e) {
         super(e);
     }

=== modified file 'src/main/java/com/akiban/rest/RestResponseBuilder.java'
--- src/main/java/com/akiban/rest/RestResponseBuilder.java	2013-04-04 16:07:24 +0000
+++ src/main/java/com/akiban/rest/RestResponseBuilder.java	2013-04-23 01:21:27 +0000
@@ -145,7 +145,9 @@
 
     public WebApplicationException wrapException(Throwable e) {
         final ErrorCode code;
-        if(e instanceof InvalidOperationException) {
+        if (e instanceof WebApplicationException) {
+            return (WebApplicationException)e;
+        } else if(e instanceof InvalidOperationException) {
             code = ((InvalidOperationException)e).getCode();
         } else if(e instanceof SQLException) {
             code = ErrorCode.valueOfCode(((SQLException)e).getSQLState());

=== modified file 'src/main/java/com/akiban/rest/resources/DefaultResource.java'
--- src/main/java/com/akiban/rest/resources/DefaultResource.java	2013-04-06 05:07:23 +0000
+++ src/main/java/com/akiban/rest/resources/DefaultResource.java	2013-04-23 01:21:27 +0000
@@ -67,7 +67,7 @@
     }
     
     
-    private Response buildResponse(HttpServletRequest request) {
+    static Response buildResponse(HttpServletRequest request) {
         String msg = String.format("API %s not supported", request.getRequestURI());
         return RestResponseBuilder
                 .forRequest(request)

=== modified file 'src/main/java/com/akiban/server/service/restdml/DirectServiceImpl.java'
--- src/main/java/com/akiban/server/service/restdml/DirectServiceImpl.java	2013-04-16 00:56:31 +0000
+++ src/main/java/com/akiban/server/service/restdml/DirectServiceImpl.java	2013-04-23 01:21:27 +0000
@@ -33,10 +33,11 @@
 import java.util.regex.Matcher;
 
 import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import javax.ws.rs.WebApplicationException;
 import javax.ws.rs.core.MediaType;
 import javax.ws.rs.core.MultivaluedMap;
 
-import com.fasterxml.jackson.core.JsonGenerator;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -52,7 +53,6 @@
 import com.akiban.rest.RestServiceImpl;
 import com.akiban.rest.resources.ResourceHelper;
 import com.akiban.server.error.ExternalRoutineInvocationException;
-import com.akiban.server.error.MalformedRequestException;
 import com.akiban.server.error.NoSuchRoutineException;
 import com.akiban.server.error.ScriptLibraryRegistrationException;
 import com.akiban.server.service.Service;
@@ -70,6 +70,7 @@
 import com.akiban.server.types3.TInstance;
 import com.akiban.sql.embedded.EmbeddedJDBCService;
 import com.akiban.sql.embedded.JDBCConnection;
+import com.fasterxml.jackson.core.JsonGenerator;
 import com.google.inject.Inject;
 import com.persistit.exception.RollbackException;
 
@@ -92,10 +93,7 @@
     private final static String IS_INOUT = "is_inout";
     private final static String IS_RESULT = "is_result";
 
-    private final static String COMMENT_ANNOTATION1 = "//##";
-    private final static String COMMENT_ANNOTATION2 = "##//";
-    private final static String ENDPOINT = "endpoint";
-
+    private final static int TRANSACTION_RETRY_COUNT = 3;
     private final static String DISTINGUISHED_REGISTRATION_METHOD_NAME = "_register";
 
     private final static String CREATE_PROCEDURE_FORMAT = "CREATE OR REPLACE PROCEDURE \"%s\".\"%s\" ()"
@@ -205,8 +203,12 @@
         AkibanInformationSchema ais = dxlService.ddlFunctions().getAIS(session);
         EndpointMap endpointMap = null;
 
-        if (functionsOnly) {
-            endpointMap = getEndpointMap(session);
+        try {
+            if (functionsOnly) {
+                endpointMap = getEndpointMap(session);
+            }
+        } catch (RegistrationException e) {
+            throw new ScriptLibraryRegistrationException(e);
         }
 
         if (module.isEmpty()) {
@@ -334,27 +336,30 @@
             final TableName procName, final String pathParams, final MultivaluedMap<String, String> queryParameters,
             final byte[] content, final MediaType[] responseType) throws Exception {
         try (JDBCConnection conn = jdbcConnection(request, procName.getSchemaName());) {
-
+            LOG.debug("Invoking {} {}", request.getMethod(), request.getRequestURI());
             conn.setAutoCommit(false);
 
             boolean completed = false;
-            boolean repeat = true;
+            int repeat = TRANSACTION_RETRY_COUNT;
 
-            while (repeat) {
+            while (--repeat >= 0) {
                 try {
                     Direct.enter(procName.getSchemaName(), dxlService.ddlFunctions().getAIS(conn.getSession()));
                     Direct.getContext().setConnection(conn);
-                    repeat = false;
                     conn.beginTransaction();
                     invokeRestFunction(writer, conn, method, procName, pathParams, queryParameters, content,
                             request.getContentType(), responseType);
                     conn.commitTransaction();
                     completed = true;
+                    return;
                 } catch (RollbackException e) {
-                    repeat = true;
-                } catch (Exception e) {
-                    e.printStackTrace();
-                    throw e;
+                    if (repeat == 0) {
+                        LOG.error("Transaction failed " + TRANSACTION_RETRY_COUNT + " times: "
+                                + request.getRequestURI());
+                        throw new WebApplicationException(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
+                    }
+                } catch (RegistrationException e) {
+                    throw new ScriptLibraryRegistrationException(e);
                 } finally {
                     try {
                         if (!completed) {
@@ -389,24 +394,22 @@
 
         EndpointMetadata md = selectEndpoint(list, pathParams, requestType, responseType, cache);
         if (md == null) {
-            // TODO - Is this the correct Exception? Is there a way to convey
-            // this without logged stack trace?
-            throw new MalformedRequestException("No matching endpoint");
+            throw new WebApplicationException(HttpServletResponse.SC_NOT_FOUND);
         }
 
         final Object[] args = createArgsArray(pathParams, queryParameters, content, cache, md);
 
-        final ScriptPool<ScriptLibrary> libraryPool = conn.getRoutineLoader()
-            .getScriptLibrary(conn.getSession(), new TableName(procName.getSchemaName(),
-                                                               md.routineName));
+        final ScriptPool<ScriptLibrary> libraryPool = conn.getRoutineLoader().getScriptLibrary(conn.getSession(),
+                new TableName(procName.getSchemaName(), md.routineName));
         final ScriptLibrary library = libraryPool.get();
         boolean success = false;
         Object result;
+
+        LOG.debug("Endpoint {}", md);
         try {
             result = library.invoke(md.function, args);
             success = true;
-        }
-        finally {
+        } finally {
             libraryPool.put(library, success);
         }
 
@@ -468,13 +471,13 @@
         EndpointMetadata md = null;
         if (list != null) {
             for (final EndpointMetadata candidate : list) {
+                if (requestType != null && candidate.expectedContentType != null
+                        && !requestType.startsWith(candidate.expectedContentType)) {
+                    continue;
+                }
                 if (candidate.pattern != null) {
                     Matcher matcher = candidate.getParamPathMatcher(cache, pathParams);
                     if (matcher.matches()) {
-                        if (responseType != null && candidate.expectedContentType != null
-                                && !requestType.startsWith(candidate.expectedContentType)) {
-                            continue;
-                        }
                         md = candidate;
                         break;
                     }
@@ -541,7 +544,8 @@
             for (final Routine routine : ais.getRoutines().values()) {
                 if (routine.getCallingConvention().equals(CallingConvention.SCRIPT_LIBRARY)
                         && routine.getDynamicResultSets() == 0 && routine.getParameters().isEmpty()) {
-                    final ScriptPool<ScriptLibrary> libraryPool = routineLoader.getScriptLibrary(session, routine.getName());
+                    final ScriptPool<ScriptLibrary> libraryPool = routineLoader.getScriptLibrary(session,
+                            routine.getName());
                     final ScriptLibrary library = libraryPool.get();
                     boolean success = false;
                     try {
@@ -549,6 +553,8 @@
                                 new Object[] { new RestFunctionRegistrar() {
                                     @Override
                                     public void register(String specification) throws Exception {
+                                        LOG.debug("Registering endpoint in routine {}: {}", routine.getName(),
+                                                specification);
                                         EndpointMap.this.register(routine.getName().getSchemaName(), routine.getName()
                                                 .getTableName(), specification);
                                     }
@@ -558,7 +564,7 @@
                         if (e.getCause() instanceof NoSuchMethodException) {
                             LOG.warn("Script library " + routine.getName() + " has no _register function");
                             success = true;
-                            return;
+                            continue;
                         }
                         Throwable previous = e;
                         Throwable current;
@@ -570,8 +576,10 @@
                         }
                         throw e;
                     } catch (RegistrationException e) {
+                        LOG.warn("Endpoint registration failure {} in routine {}", e, routine.getName());
                         throw e;
                     } catch (Exception e) {
+                        LOG.warn("Endpoint registration failure {} in routine {}", e, routine.getName());
                         throw new RegistrationException(e);
                     } finally {
                         libraryPool.put(library, success);
@@ -580,7 +588,6 @@
             }
         }
 
-
         void register(final String schema, final String routine, final String spec) throws Exception {
 
             try {
@@ -596,7 +603,8 @@
                     list.add(em);
                 }
             } catch (Exception e) {
-                throw new RegistrationException("Invalid function specification: " + spec, e);
+                String msg = e instanceof IllegalArgumentException ? e.getMessage() : "";
+                throw new RegistrationException("Invalid function specification: " + spec + " - " + msg, e);
             }
         }
     }

=== modified file 'src/main/java/com/akiban/server/service/restdml/EndpointMetadata.java'
--- src/main/java/com/akiban/server/service/restdml/EndpointMetadata.java	2013-04-16 00:56:31 +0000
+++ src/main/java/com/akiban/server/service/restdml/EndpointMetadata.java	2013-04-23 01:21:27 +0000
@@ -267,6 +267,7 @@
                         + specification);
             }
         }
+        em.validate();
         return em;
     }
 
@@ -314,7 +315,6 @@
             }
         }
         return pm;
-
     }
 
     /**
@@ -506,6 +506,8 @@
                 String s;
                 if (content instanceof byte[]) {
                     s = new String((byte[]) content, UTF8);
+                } else if (content == null) {
+                    s = "";
                 } else {
                     s = (String) content;
                 }
@@ -660,7 +662,7 @@
                     index++;
                     break;
                 }
-                if (!Character.isLetterOrDigit(c) || (first && !Character.isLetter(c))) {
+                if (!Character.isJavaIdentifierPart(c) || (first && !Character.isJavaIdentifierStart(c))) {
                     throw new IllegalArgumentException("Invalid character in name: " + source);
                 }
                 result.append(c);
@@ -751,6 +753,22 @@
         }
     }
 
+    private void validate() {
+        notNull(schemaName, "schema name");
+        notNull(routineName, "routine name");
+        notNull(method, "'method='");
+        notNull(name, "'path='");
+        notNull(function, "'function='");
+        notNull(inParams, "'in='");
+        notNull(outParam, "'out='");
+    }
+
+    private void notNull(final Object v, final String name) {
+        if (v == null) {
+            throw new IllegalArgumentException(name + " not specified in " + toString());
+        }
+    }
+
     @Override
     public String toString() {
         StringBuilder sb = new StringBuilder();
@@ -759,11 +777,15 @@
             append(sb, pattern.toString());
         }
         append(sb, " ", FUNCTION, "=", function, " ", IN, "=(");
-        for (int index = 0; index < inParams.length; index++) {
-            if (index > 0) {
-                sb.append(", ");
+        if (inParams == null) {
+            sb.append("null");
+        } else {
+            for (int index = 0; index < inParams.length; index++) {
+                if (index > 0) {
+                    sb.append(", ");
+                }
+                sb.append(inParams[index]);
             }
-            sb.append(inParams[index]);
         }
         append(sb, ") ", OUT, "=");
         if (outParam == null) {

=== added file 'src/test/java/com/akiban/rest/RestServiceScriptsIT.java'
--- src/test/java/com/akiban/rest/RestServiceScriptsIT.java	1970-01-01 00:00:00 +0000
+++ src/test/java/com/akiban/rest/RestServiceScriptsIT.java	2013-04-23 01:21:27 +0000
@@ -0,0 +1,460 @@
+/**
+ * Copyright (C) 2009-2013 Akiban Technologies, Inc.
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package com.akiban.rest;
+
+import static com.akiban.util.JsonUtils.readTree;
+import static org.junit.Assert.fail;
+
+import java.io.ByteArrayInputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.UnsupportedEncodingException;
+import java.net.MalformedURLException;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.net.URLEncoder;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+
+import junit.framework.ComparisonFailure;
+
+import org.eclipse.jetty.client.ContentExchange;
+import org.eclipse.jetty.client.HttpClient;
+import org.eclipse.jetty.client.HttpExchange;
+import org.junit.After;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.akiban.http.HttpConductor;
+import com.akiban.junit.NamedParameterizedRunner;
+import com.akiban.junit.Parameterization;
+import com.akiban.server.service.is.BasicInfoSchemaTablesService;
+import com.akiban.server.service.is.BasicInfoSchemaTablesServiceImpl;
+import com.akiban.server.service.servicemanager.GuicedServiceManager;
+import com.akiban.server.test.it.ITBase;
+import com.akiban.sql.RegexFilenameFilter;
+import com.akiban.util.Strings;
+import com.fasterxml.jackson.core.JsonParseException;
+import com.fasterxml.jackson.databind.JsonNode;
+
+/**
+ * Scripted tests for REST end-points. Code was largely copied from
+ * RestServiceFilesIT. Difference is that this version finds files with the
+ * suffix ".script" and executes the command stream located in them. Commands are:
+ * 
+ * <pre>
+ * GET address
+ * DELETE address
+ * QUERY query
+ * EXPLAIN query
+ * POST address content
+ * PUT address content
+ * PATCH address content
+ * EQUALS expected
+ * CONTAINS expected
+ * JSONEQ expected
+ * HEADERS expected
+ * EMPTY
+ * NOTEMPTY
+ * </pre>
+ * 
+ * where address is a path relative the resource end-point, content is a string
+ * value that is converted to bytes and sent with POST, PUT and PATCH
+ * operations, and expected is a value used in comparison with the most recently
+ * returned content. The values of the query, content and expected fields may be
+ * specified in-line, or as a reference to another file as in @filename. For
+ * in-line values, the character sequences "\n", "\t" and "\r" are converted to
+ * the corresponding new-line, tab and return characters. This transformation is
+ * not done if the value is supplied as a file reference. An empty string can be
+ * specified as simply @, e.g.:
+ * 
+ * <pre>
+ * POST    /builder/implode/test.customers @
+ * </pre>
+ * 
+ * @author peter
+ */
+@RunWith(NamedParameterizedRunner.class)
+public class RestServiceScriptsIT extends ITBase {
+    private static final Logger LOG = LoggerFactory.getLogger(RestServiceScriptsIT.class.getName());
+
+    private static final File RESOURCE_DIR = new File("src/test/resources/"
+            + RestServiceScriptsIT.class.getPackage().getName().replace('.', '/'));
+
+    public static final String SCHEMA_NAME = "test";
+
+    private static class CaseParams {
+        public final String subDir;
+        public final String caseName;
+        public final String script;
+
+        private CaseParams(String subDir, String caseName, String script) {
+            this.subDir = subDir;
+            this.caseName = caseName;
+            this.script = script;
+        }
+    }
+
+    static class Result {
+        HttpExchange conn;
+        String output = "<not executed>";
+    }
+
+    protected final CaseParams caseParams;
+    protected final HttpClient httpClient;
+    private final List<String> errors = new ArrayList<>();
+    private final Result result = new Result();
+    private int lineNumber = 0;
+
+    public RestServiceScriptsIT(CaseParams caseParams) throws Exception {
+        this.caseParams = caseParams;
+        this.httpClient = new HttpClient();
+        httpClient.setConnectorType(HttpClient.CONNECTOR_SELECT_CHANNEL);
+        httpClient.setMaxConnectionsPerAddress(10);
+        httpClient.start();
+    }
+
+    @Override
+    protected GuicedServiceManager.BindingsConfigurationProvider serviceBindingsProvider() {
+        return super.serviceBindingsProvider().bindAndRequire(RestService.class, RestServiceImpl.class)
+                .bindAndRequire(BasicInfoSchemaTablesService.class, BasicInfoSchemaTablesServiceImpl.class);
+    }
+
+    @Override
+    protected Map<String, String> startupConfigProperties() {
+        return uniqueStartupConfigProperties(RestServiceScriptsIT.class);
+    }
+
+    public static File[] gatherRequestFiles(File dir) {
+        File[] result = dir.listFiles(new RegexFilenameFilter(".*\\.(script)"));
+        Arrays.sort(result, new Comparator<File>() {
+            public int compare(File f1, File f2) {
+                return f1.getName().compareTo(f2.getName());
+            }
+        });
+        return result;
+    }
+
+    @NamedParameterizedRunner.TestParameters
+    public static Collection<Parameterization> gatherCases() throws Exception {
+        Collection<Parameterization> result = new ArrayList<>();
+        for (String subDirName : RESOURCE_DIR.list()) {
+            File subDir = new File(RESOURCE_DIR, subDirName);
+            if (!subDir.isDirectory()) {
+                LOG.warn("Skipping unexpected file: {}", subDir);
+                continue;
+            }
+            for (File requestFile : gatherRequestFiles(subDir)) {
+                String inputName = requestFile.getName();
+                int dotIndex = inputName.lastIndexOf('.');
+                String caseName = inputName.substring(0, dotIndex);
+                String script = Strings.dumpFileToString(requestFile);
+
+                result.add(Parameterization.create(subDirName + File.separator + caseName, new CaseParams(subDirName,
+                        caseName, script)));
+            }
+        }
+        return result;
+    }
+
+    private URL getRestURL(String request) throws MalformedURLException {
+        int port = serviceManager().getServiceByClass(HttpConductor.class).getPort();
+        String context = serviceManager().getServiceByClass(RestService.class).getContextPath();
+        return new URL("http", "localhost", port, context + request);
+    }
+
+    private void loadDatabase(String subDirName) throws Exception {
+        File subDir = new File(RESOURCE_DIR, subDirName);
+        File schemaFile = new File(subDir, "schema.ddl");
+        if (schemaFile.exists()) {
+            loadSchemaFile(SCHEMA_NAME, schemaFile);
+        }
+        for (File data : subDir.listFiles(new RegexFilenameFilter(".*\\.dat"))) {
+            loadDataFile(SCHEMA_NAME, data);
+        }
+    }
+
+    private static void postContents(HttpExchange httpConn, byte[] request) throws IOException {
+        httpConn.setRequestContentType("application/json");
+        httpConn.setRequestHeader("Accept", "application/json");
+        httpConn.setRequestContentSource(new ByteArrayInputStream(request));
+    }
+
+    @After
+    public void finish() throws Exception {
+        httpClient.stop();
+    }
+
+    private void error(String message) {
+        error(message, result.output);
+    }
+
+    private void error(String message, String s) {
+        String error = String.format("%s in %s:%d <%s>", message, caseParams.caseName, lineNumber, s);
+        errors.add(error);
+    }
+
+    @Test
+    public void testRequest() throws Exception {
+        loadDatabase(caseParams.subDir);
+
+        // Execute lines of script
+
+        result.conn = null;
+        result.output = "<not executed>";
+        lineNumber = 0;
+
+        try {
+            for (String line : caseParams.script.split("\n")) {
+                lineNumber++;
+                line = line.trim();
+
+                while (line.contains("  ")) {
+                    line = line.replace("  ", " ");
+                }
+                if (line.startsWith("#") || line.isEmpty()) {
+                    continue;
+                }
+                String[] pieces = line.split(" ");
+                String command = pieces[0].toUpperCase();
+
+                switch (command) {
+                case "DEBUG":
+                    System.out.println("DEBUG executed on line " + lineNumber);
+                    break;
+                case "GET":
+                case "DELETE": {
+                    result.conn = null;
+                    if (pieces.length < 2) {
+                        error("Missing argument");
+                        continue;
+                    }
+                    executeRestCall(command, pieces[1], null);
+                    break;
+                }
+                case "QUERY":
+                    result.conn = null;
+                    if (pieces.length < 2) {
+                        error("Missing argument");
+                        continue;
+                    }
+                    executeRestCall("GET", "/sql/query?q=" + trimAndURLEncode(value(line, 1)), null);
+                    break;
+                case "EXPLAIN":
+                    result.conn = null;
+                    if (pieces.length < 2) {
+                        error("Missing argument");
+                        continue;
+                    }
+                    executeRestCall("GET", "/sql/explain?q=" + trimAndURLEncode(value(line, 1)), null);
+                    break;
+                case "POST":
+                case "PUT":
+                case "PATCH": {
+                    result.conn = null;
+                    pieces = line.split(" ", 3);
+                    if (pieces.length < 3) {
+                        error("Missing argument");
+                        continue;
+                    }
+                    String contents  = value(line, 2);
+                    executeRestCall(command, pieces[1], contents);
+                    break;
+                }
+                case "EQUALS":
+                    if (pieces.length < 2) {
+                        error("Missing argument");
+                        continue;
+                    }
+                    compareStrings("Incorrect response", value(line, 1), result.output);
+                    break;
+                case "CONTAINS":
+                    if (pieces.length < 2) {
+                        error("Missing argument");
+                        continue;
+                    }
+                    if (!result.output.contains(value(line, 1))) {
+                        error("Incorrect response");
+                    }
+                    break;
+                case "JSONEQ":
+                    if (pieces.length < 2) {
+                        error("Missing argument");
+                        continue;
+                    }
+                    compareAsJSON("Unexpected response", value(line, 1), result.output);
+                    break;
+                case "HEADERS":
+                    if (pieces.length < 2) {
+                        error("Missing argument");
+                        continue;
+                    }
+                    compareHeaders(result.conn, value(line, 1));
+                    break;
+                case "NOTEMPTY":
+                    if (result.output.isEmpty() || result.conn == null) {
+                        error("Expected non-empty response");
+                        continue;
+                    }
+                    break;
+                case "EMPTY":
+                    if (!result.output.isEmpty()) {
+                        error("Expected empty response");
+                    }
+                    break;
+                default:
+                    result.conn = null;
+                    error("Unknown script command '" + command + "'");
+                }
+            }
+        } finally {
+            result.conn = null;
+        }
+        if (!errors.isEmpty()) {
+            String failMessage = "Failed with " + errors.size() + " errors:";
+            for (String s : errors) {
+                failMessage += "\n  " + s;
+            }
+            fail(failMessage);
+        }
+    }
+
+    private void executeRestCall(final String command, final String address, final String contents) throws Exception {
+        String[] pieces = address.split("\\|");
+        try {
+            result.conn = openConnection(pieces[0], command);
+            if (contents != null) {
+                postContents(result.conn, contents.getBytes());
+            }
+            // After postContents to override default
+            if (pieces.length > 1) {
+                result.conn.setRequestContentType(pieces[1]);
+            }
+            httpClient.send(result.conn);
+            result.conn.waitForDone();
+            result.output = getOutput(result.conn);
+        } catch (Exception e) {
+            result.output = e.toString();
+        } finally {
+            if (result.conn != null) {
+                fullyDisconnect(result.conn);
+            }
+        }
+    }
+
+    private HttpExchange openConnection(String address, String requestMethod) throws IOException, URISyntaxException {
+        URL url = getRestURL(address);
+        HttpExchange exchange = new ContentExchange(true);
+        exchange.setURI(url.toURI());
+        exchange.setMethod(requestMethod);
+        return exchange;
+    }
+
+    private String getOutput(HttpExchange httpConn) throws IOException {
+        return ((ContentExchange) httpConn).getResponseContent();
+    }
+
+    private String value(String line, int index) throws IOException {
+        String s = line.split(" ", index + 1)[index];
+        if (s.startsWith("@")) {
+            if (s.length() == 1) {
+                s = "";
+            } else {
+                s = Strings.dumpFileToString(new File(new File(RESOURCE_DIR, caseParams.subDir), s.substring(1)));
+            }
+        } else {
+            s = s.replace("\\n", "\n").replace("\\n", "\t");
+        }
+        return s;
+    }
+
+    private static String trimAndURLEncode(String s) throws UnsupportedEncodingException {
+        return URLEncoder.encode(s.trim().replaceAll("\\s+", " "), "UTF-8");
+    }
+
+    private String diff(String a, String b) {
+        return new ComparisonFailure("", a, b).getMessage();
+    }
+
+    private void compareStrings(String assertMsg, String expected, String actual) {
+        if (!expected.equals(actual)) {
+            error(assertMsg, diff(expected, actual));
+        }
+    }
+
+    private void compareAsJSON(String assertMsg, String expected, String actual) throws IOException {
+        JsonNode expectedNode = null;
+        JsonNode actualNode = null;
+        String expectedTrimmed = (expected != null) ? expected.trim() : "";
+        String actualTrimmed = (actual != null) ? actual.trim() : "";
+        try {
+            if (!expectedTrimmed.isEmpty()) {
+                expectedNode = readTree(expected);
+            }
+            if (!actualTrimmed.isEmpty()) {
+                actualNode = readTree(actual);
+            }
+        } catch (JsonParseException e) {
+            // Note: This case handles the jsonp tests. Somewhat fragile, but
+            // not horrible yet.
+        }
+        // Try manual equals and then assert strings for pretty print
+        if (expectedNode != null && actualNode != null) {
+            if (!expectedNode.equals(actualNode)) {
+                error(assertMsg, diff(expectedNode.toString(), actualNode.toString()));
+            }
+        } else {
+            compareStrings(assertMsg, expected, actual);
+        }
+    }
+
+    private void compareHeaders(HttpExchange httpConn, String checkHeaders) throws Exception {
+        ContentExchange exch = (ContentExchange) httpConn;
+
+        String[] headerList = checkHeaders.split(Strings.NL);
+        for (String header : headerList) {
+            String[] nameValue = header.split(":", 2);
+
+            if (nameValue[0].equals("responseCode")) {
+                if (Integer.parseInt(nameValue[1].trim()) != exch.getResponseStatus()) {
+                    error("Incorrect Response Status",
+                            String.format("%d expected %s", exch.getResponseStatus(), nameValue[1]));
+                }
+            } else {
+                if (!nameValue[1].trim().equals(exch.getResponseFields().getStringField(nameValue[0]))) {
+                    error("Incorrect Response Header", String.format("%s expected %s", exch.getResponseFields()
+                            .getStringField(nameValue[0]), nameValue[1].trim()));
+                }
+            }
+        }
+    }
+
+    private void fullyDisconnect(HttpExchange httpConn) throws InterruptedException {
+        // If there is a failure, leaving junk in any of the streams can cause
+        // cascading issues.
+        // Get rid of anything left and disconnect.
+        httpConn.waitForDone();
+        httpConn.reset();
+    }
+}

=== added file 'src/test/resources/com/akiban/rest/direct/bad-direct-create.body'
--- src/test/resources/com/akiban/rest/direct/bad-direct-create.body	1970-01-01 00:00:00 +0000
+++ src/test/resources/com/akiban/rest/direct/bad-direct-create.body	2013-04-23 01:21:27 +0000
@@ -0,0 +1,26 @@
+function _register(registrar) {
+    registrar.register(
+        "method=GET path=cnames function=customerNames out=String");
+        
+    registrar.register(
+        "method=GET path=oids/(\\d*) function=orderNumbers in=(pp:1 int required)"); 
+};
+
+function customerNames(s) {
+    var result = s;
+    var extent = Packages.com.akiban.direct.Direct.context.extent;
+    for (customer in Iterator(extent.customers)) {
+      result += "," + customer.name;
+    }
+    return result;
+}
+
+function orderNumbers(cid) {
+    var result = s;
+    var extent = Packages.com.akiban.direct.Direct.context.extent;
+    var customer = extent.getCustomer(cid); 
+    for (order in customer.orders) {
+       result += "," + order.oid;
+    }    
+    return result;
+}

=== modified file 'src/test/resources/com/akiban/rest/direct/direct-create.body'
--- src/test/resources/com/akiban/rest/direct/direct-create.body	2013-04-08 17:42:31 +0000
+++ src/test/resources/com/akiban/rest/direct/direct-create.body	2013-04-23 01:21:27 +0000
@@ -1,12 +1,12 @@
 function _register(registrar) {
     registrar.register(
-        "method=GET path=cnames function=customerNames in=(QP:prefix String required) out=String");
+        "method=GET path=cnames function=customer_Names in=(QP:prefix String required) out=String");
         
     registrar.register(
-        "method=GET path=oids/(\\d*) function=orderNumbers in=(pp:1 int required) out=String"); 
+        "method=GET path=oids/(\\d*) function=order_Numbers in=(pp:1 int required) out=String"); 
 };
 
-function customerNames(s) {
+function customer_Names(s) {
     var result = s;
     var extent = Packages.com.akiban.direct.Direct.context.extent;
     for (customer in Iterator(extent.customers)) {
@@ -15,7 +15,7 @@
     return result;
 }
 
-function orderNumbers(cid) {
+function order_Numbers(cid) {
     var result = s;
     var extent = Packages.com.akiban.direct.Direct.context.extent;
     var customer = extent.getCustomer(cid); 

=== added file 'src/test/resources/com/akiban/rest/direct/missing-direct-create.body'
--- src/test/resources/com/akiban/rest/direct/missing-direct-create.body	1970-01-01 00:00:00 +0000
+++ src/test/resources/com/akiban/rest/direct/missing-direct-create.body	2013-04-23 01:21:27 +0000
@@ -0,0 +1,9 @@
+
+function customerNames(s) {
+    var result = s;
+    var extent = Packages.com.akiban.direct.Direct.context.extent;
+    for (customer in Iterator(extent.customers)) {
+      result += "," + customer.name;
+    }
+    return result;
+}

=== added file 'src/test/resources/com/akiban/rest/direct/test-basic-direct-functions.script'
--- src/test/resources/com/akiban/rest/direct/test-basic-direct-functions.script	1970-01-01 00:00:00 +0000
+++ src/test/resources/com/akiban/rest/direct/test-basic-direct-functions.script	2013-04-23 01:21:27 +0000
@@ -0,0 +1,30 @@
+# Install a library with two endpoints
+#
+PUT     /direct/library?module=test.proc1&language=Javascript @direct-create.body
+EQUALS  {"functions":2}\n
+
+# Delete the library
+#
+DELETE  /direct/library?module=test.proc1&language=Javascript
+EQUALS  {"functions":0}\n
+
+# Reinstall it
+#
+PUT     /direct/library?module=test.proc1&language=Javascript @direct-create.body
+
+# Verify that it runs and consumes the supplied parameter value correctly
+#
+GET     /direct/call/test.cnames?prefix=SomeParameterValue
+EQUALS  SomeParameterValue,John Smith,Willy Jones,Jane Smith,Jonathan Smyth\n
+GET     /direct/call/test.cnames?prefix=SomeOtherParameterValue
+EQUALS  SomeOtherParameterValue,John Smith,Willy Jones,Jane Smith,Jonathan Smyth\n
+
+# Delete the library#
+#
+DELETE  /direct/library?module=test.proc1&language=Javascript
+
+# Attempt to invoke end point should return a 404
+#
+GET     /direct/call/test.cnames?prefix=SomeOtherParameterValue
+CONTAINS Not Found
+CONTAINS 404

=== added file 'src/test/resources/com/akiban/rest/direct/test-direct-registration-errors.script'
--- src/test/resources/com/akiban/rest/direct/test-direct-registration-errors.script	1970-01-01 00:00:00 +0000
+++ src/test/resources/com/akiban/rest/direct/test-direct-registration-errors.script	2013-04-23 01:21:27 +0000
@@ -0,0 +1,33 @@
+# Normal registration
+#
+PUT     /direct/library?module=test.proc1&language=Javascript @direct-create.body
+EQUALS  {"functions":2}\n
+
+# Attempt to register an invalid specification
+#
+PUT     /direct/library?module=test.proc2&language=Javascript @bad-direct-create.body
+CONTAINS Invalid function specification
+
+# Register a script with no _register function
+#
+PUT     /direct/library?module=test.proc2&language=Javascript @missing-direct-create.body
+EQUALS  {"functions":2}\n
+
+# Verify a valid script is still available
+#
+GET     /direct/call/test.cnames?prefix=SomeParameterValue
+EQUALS  SomeParameterValue,John Smith,Willy Jones,Jane Smith,Jonathan Smyth\n
+
+# Try adding broken scripts in a different order
+#
+DELETE  /direct/library?module=test.proc1
+DELETE  /direct/library?module=test.proc2
+PUT     /direct/library?module=test.proc1&language=Javascript @missing-direct-create.body
+PUT     /direct/library?module=test.proc2&language=Javascript @bad-direct-create.body
+PUT     /direct/library?module=test.proc3&language=Javascript @direct-create.body
+
+# Verify a valid script is still available
+#
+GET     /direct/call/test.cnames?prefix=SomeParameterValue
+EQUALS  SomeParameterValue,John Smith,Willy Jones,Jane Smith,Jonathan Smyth\n
+

=== added file 'src/test/resources/com/akiban/rest/direct/test-invalid-endpoint-errors.script'
--- src/test/resources/com/akiban/rest/direct/test-invalid-endpoint-errors.script	1970-01-01 00:00:00 +0000
+++ src/test/resources/com/akiban/rest/direct/test-invalid-endpoint-errors.script	2013-04-23 01:21:27 +0000
@@ -0,0 +1,46 @@
+# Normal registration
+#
+PUT     /direct/library?module=test.proc1&language=Javascript @direct-create.body
+EQUALS  {"functions":2}\n
+
+# Try mismatched methods - expect 404 errors
+#
+PUT     /direct/call/test.cnames?prefix=SomeParameterValue @
+CONTAINS 404
+CONTAINS Not Found
+POST     /direct/call/test.cnames?prefix=SomeParameterValue @
+CONTAINS 404
+CONTAINS Not Found
+DELETE   /direct/call/test.cnames?prefix=SomeParameterValue @
+CONTAINS 404
+CONTAINS Not Found
+
+PUT     /direct/library?module=test.proc1&language=Javascript function _register(registrar) {registrar.register("method=GET path=x function=x in=(JSON:prefix String required) out=String");};
+GET     /direct/call/test.x|text/plain
+CONTAINS 404
+CONTAINS Not Found
+
+# Mismatched method GET / POST
+#
+PUT     /direct/library?module=test.proc1&language=Javascript function _register(registrar) {registrar.register("method=GET path=x function=x in=(JSON:prefix String required) out=String");};
+POST    /direct/call/test.x|text/plain {"prefix": "SomeParamaterValue"}
+CONTAINS 404
+CONTAINS Not Found
+
+# Mismatched method POST / PUT
+#
+PUT     /direct/library?module=test.proc1&language=Javascript function _register(registrar) {registrar.register("method=POST path=x function=x in=(JSON:prefix String required) out=String");};
+PUT     /direct/call/test.x|application/json {"prefix": "SomeParamaterValue"}
+CONTAINS 404
+CONTAINS Not Found
+
+# Mismatched request type application/json / text/plain
+#
+PUT     /direct/library?module=test.proc1&language=Javascript function _register(registrar) {registrar.register("method=POST path=x function=x in=(JSON:prefix String required) out=String");};
+POST    /direct/call/test.x|text/plain {"prefix": "SomeParamaterValue"}
+CONTAINS 404
+CONTAINS Not Found
+
+
+
+