Merge lp:~leonardr/lazr.restful/make-test-class-public into lp:lazr.restful

[Leonard Richardson]

[Leonard Richardson (leonardr)]

lazr.restful's unit test file test_webservice.py used to have a class called WebServiceTestCase that would set up a dummy web service and run some unit tests against it. Over time, the amount of setup work necessary increased, to the point where WSTC was basically setting up a fully functional web service.

When integrating a new lazr.restful version into Launchpad, I noticed that Launchpad also had unit tests that ran against a web service, and that I could save myself a lot of trouble by writing unit tests that subclassed WebServiceTestCase. I did this, but I pledged to come back and move WSTC (and its dependent classes) into lazr.restful.testing.webservice, so that Launchpad wouldn't be importing classes from a lazr.restful unit test module.

This branch does basically that, and nothing else. The one fix I made in passing was to change SimpleWebServiceConfiguration.createRequest to use the utility function tag_request_with_version_name(), replacing code that was basically a copy of tag_request_with_version_name. I also had to change the WADL unit tests, because I fleshed out the doctest strings of IGenericEntry and IGenericConfiguration.

[Edwin Grubbs (edwin-grubbs)]

Hi Leonard,

The changes look good, but I got these errors when running the tests under python2.6. There were even more problems under python2.5.

-Edwin

Running lazr.restful.example.base.tests.test_integration.WSGILayer tests:
  Set up lazr.restful.example.base.tests.test_integration.FunctionalLayer in 0.328 seconds.
  Set up lazr.restful.example.base.tests.test_integration.WSGILayer in 0.000 seconds.

Failure in test /home/egrubbs/canonical/launchpadlib/refactor-tag-request-with-version/src/lazr/restful/example/base/tests/field.txt
Traceback (most recent call last):
  File "/usr/lib/python2.5/unittest.py", line 260, in run
    testMethod()
  File "/usr/lib/python2.5/doctest.py", line 2128, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for field.txt
  File "/home/egrubbs/canonical/launchpadlib/refactor-tag-request-with-version/src/lazr/restful/example/base/tests/field.txt", line 0

----------------------------------------------------------------------
File "/home/egrubbs/canonical/launchpadlib/refactor-tag-request-with-version/src/lazr/restful/example/base/tests/field.txt", line 55, in field.txt
Failed example:
    print new_value
Differences (ndiff with -expected +actual):
    - "http://.../cookbooks/The%20Joy%20of%20Cooking"
    ? ^^^^
    + "http:\/\/cookbooks.dev\/devel\/cookbooks\/The%20Joy%20of%20Cooking"
    ? + ^ +++++++++++++++++++++++
----------------------------------------------------------------------
File "/home/egrubbs/canonical/launchpadlib/refactor-tag-request-with-version/src/lazr/restful/example/base/tests/field.txt", line 58, in field.txt
Failed example:
    print webservice(link_field_url, 'PATCH', new_value)
Differences (ndiff with -expected +actual):
      HTTP/1.1 209 Content Returned
    - ...
    + Status: 209 Content Returned
    + Content-Length: 68
      Content-Type: application/json
    - ...
    + X-Powered-By: Zope (www.zope.org), Python (www.python.org)
      <BLANKLINE>
    - "http://cookbooks.dev/.../cookbooks/The%20Joy%20of%20Cooking"
    ? ^^^
    + "http:\/\/cookbooks.dev\/devel\/cookbooks\/The%20Joy%20of%20Cooking"
    ? + + + ^^^^^^ +

  Ran 9 tests with 1 failures and 0 errors in 2.391 seconds.
Running lazr.restful.example.multiversion.tests.test_integration.WSGILayer tests:
  Tear down lazr.restful.example.base.tests.test_integration.WSGILayer in 0.000 seconds.
  Tear down lazr.restful.example.base.tests.test_integration.FunctionalLayer in 0.000 seconds.
  Set up lazr.restful.example.multiversion.tests.test_integration.FunctionalLayer in 0.078 seconds.
  Set up lazr.restful.example.multiversion.tests.test_integration.WSGILayer in 0.000 seconds.
  Ran 3 tests with 0 failures and 0 errors in 0.406 seconds.
Running lazr.restful.example.wsgi.tests.test_integration.WSGILayer tests:
  Tear down lazr.restful.example.multiversion.tests.test_integration.WSGILayer in 0.001 seconds.
  Tear down lazr.restful.example.multiversion.tests.test_integration.FunctionalLayer in 0.000 seconds.
  Set up lazr.restful.example.wsgi.tests.test_integration.FunctionalLayer in 0.069 se...

review: Needs Fixing

[Leonard Richardson (leonardr)]

I think you're running the tests on an old version of simplejson. Can you run some code to see what version of simplejson you have? Here's what I get:

$ bin/py
>>> import simplejson
>>> simplejson.__version__
'2.0.9'

Also try making this change to setup.py, rerun bin/buildout, and see if the problem goes away.

=== modified file 'setup.py'
--- setup.py 2009-10-07 18:54:39 +0000
+++ setup.py 2010-03-04 14:03:26 +0000
@@ -64,7 +64,7 @@
         'lazr.uri',
         'martian==0.11',
         'setuptools',
- 'simplejson',
+ 'simplejson>=2.0.9',
         'van.testing',
         'wsgiref',
         'zope.app.pagetemplate',

[Edwin Grubbs (edwin-grubbs)]

Hi Leonard,

Updating simplejson fixed it.

merge-approved

-Edwin

review: Approve (code)

=== modified file 'src/lazr/restful/testing/webservice.py'
--- src/lazr/restful/testing/webservice.py	2010-03-03 13:05:13 +0000
+++ src/lazr/restful/testing/webservice.py	2010-03-03 17:07:18 +0000
@@ -9,7 +9,11 @@
     'ExampleWebServicePublication',
     'FakeRequest',
     'FakeResponse',
+    'IGenericEntry',
+    'IGenericCollection',
     'pprint_entry',
+    'WebServiceTestCase',
+    'WebServiceTestConfiguration'
     'WebServiceTestPublication',
     'WebServiceTestRequest',
     'TestPublication',
@@ -19,25 +23,39 @@
 import os
 import traceback
 import simplejson
+import sys
+from types import ModuleType
 import urllib
+import unittest
 from urlparse import urljoin
 import wsgi_intercept
 
-from zope.component import adapts, getUtility, queryMultiAdapter
-from zope.interface import implements
+from zope.component import (
+    adapts, getGlobalSiteManager, getUtility, queryMultiAdapter)
+from zope.configuration import xmlconfig
+from zope.interface import alsoProvides, implements, Interface
 from zope.publisher.browser import BrowserRequest
 from zope.publisher.interfaces.http import IHTTPApplicationRequest
 from zope.publisher.paste import Application
 from zope.proxy import ProxyBase
+from zope.schema import TextLine
 from zope.security.checker import ProxyFactory
+from zope.testing.cleanup import CleanUp
 from zope.traversing.browser.interfaces import IAbsoluteURL
 
+from lazr.restful.declarations import (
+    collection_default_content, exported,
+    export_as_webservice_collection, export_as_webservice_entry,
+    export_read_operation, operation_parameters)
 from lazr.restful.interfaces import (
     IServiceRootResource, IWebServiceClientRequest,
     IWebServiceConfiguration, IWebServiceLayer, IWebServiceVersion)
 from lazr.restful.publisher import (
     WebServicePublicationMixin, WebServiceRequestTraversal)
-from lazr.restful.simple import Publication, Request
+from lazr.restful.simple import (
+    BaseWebServiceConfiguration, Publication, Request,
+    RootResourceAbsoluteURL, ServiceRootResource)
+from lazr.restful.utils import tag_request_with_version_name
 
 from lazr.uri import URI
 
@@ -385,6 +403,121 @@
             raise ValueError(self.body)
 
 
+class WebServiceTestConfiguration(BaseWebServiceConfiguration):
+    """A simple configuration for tests that need a running web service."""
+    implements(IWebServiceConfiguration)
+    show_tracebacks = False
+    active_versions = ['1.0', '2.0']
+    hostname = "webservice_test"
+    last_version_with_mutator_named_operations = None
+
+    def createRequest(self, body_instream, environ):
+        request = Request(body_instream, environ)
+        request.setPublication(
+            WebServiceTestPublication(getUtility(IServiceRootResource)))
+        tag_request_with_version_name(request, '2.0')
+        return request
+
+
+class IWebServiceTestRequest10(IWebServiceClientRequest):
+    """A marker interface for requests to the '1.0' web service."""
+
+
+class IWebServiceTestRequest20(IWebServiceClientRequest):
+    """A marker interface for requests to the '2.0' web service."""
+
+
+class WebServiceTestCase(CleanUp, unittest.TestCase):
+    """A test case for web service operations."""
+
+    testmodule_objects = []
+
+    def setUp(self):
+        """Set the component registry with the given model."""
+        super(WebServiceTestCase, self).setUp()
+
+        # Register a simple configuration object.
+        webservice_configuration = WebServiceTestConfiguration()
+        sm = getGlobalSiteManager()
+        sm.registerUtility(webservice_configuration)
+
+        # Register IWebServiceVersions for the
+        # '1.0' and '2.0' web service versions.
+        alsoProvides(IWebServiceTestRequest10, IWebServiceVersion)
+        sm.registerUtility(
+            IWebServiceTestRequest10, IWebServiceVersion, name='1.0')
+        alsoProvides(IWebServiceTestRequest20, IWebServiceVersion)
+        sm.registerUtility(
+            IWebServiceTestRequest20, IWebServiceVersion, name='2.0')
+
+        # Register a service root resource
+        service_root = ServiceRootResource()
+        sm.registerUtility(service_root, IServiceRootResource)
+
+        # Register an IAbsoluteURL adapter for the service root resource.
+        sm.registerAdapter(
+            RootResourceAbsoluteURL,
+            [IServiceRootResource, IWebServiceClientRequest], IAbsoluteURL)
+
+        # Build a test module that exposes the given resource interfaces.
+        testmodule = ModuleType('testmodule')
+        for interface in self.testmodule_objects:
+            setattr(testmodule, interface.__name__, interface)
+        sys.modules['lazr.restful.testmodule'] = testmodule
+
+        # Register the test module in the ZCML configuration: adapter
+        # classes will be built automatically.
+        xmlconfig.string("""
+        <configure
+           xmlns="http://namespaces.zope.org/zope"
+           xmlns:webservice="http://namespaces.canonical.com/webservice">
+         <include package="zope.component" file="meta.zcml" />
+         <include package="zope.security" file="meta.zcml"/>
+         <include package="lazr.restful" file="meta.zcml" />
+         <include package="lazr.restful" file="configure.zcml" />
+
+        <adapter for="*"
+            factory="zope.traversing.adapters.DefaultTraversable"
+            provides="zope.traversing.interfaces.ITraversable" />
+
+        <webservice:register module="lazr.restful.testmodule" />
+        </configure>
+        """)
+
+
+class IGenericEntry(Interface):
+    """A simple, reusable entry interface for use in tests.
+
+    The entry publishes one field and one named operation.
+    """
+    export_as_webservice_entry()
+
+    # pylint: disable-msg=E0213
+    a_field = exported(
+        TextLine(
+            title=u'A "field"',
+            description=u'The only field that can be <> 0 in the entry.'))
+
+    @operation_parameters(
+        message=TextLine(title=u'Message to say'))
+    @export_read_operation()
+    def greet(message):
+        """Print an appropriate greeting based on the message.
+
+        :param message: This will be included in the greeting.
+        """
+
+
+class IGenericCollection(Interface):
+    """A simple collection containing `IGenericEntry`, for use in tests."""
+    export_as_webservice_collection(IGenericEntry)
+
+    # pylint: disable-msg=E0211
+    @collection_default_content()
+    def getAll():
+        """Returns all the entries."""
+
+
 class DummyRootResource:
     """A root resource that does nothing."""
     implements(IServiceRootResource)
@@ -403,3 +536,5 @@
     def __str__(self):
         return "http://dummy"
     __call__ = __str__
+
+

=== modified file 'src/lazr/restful/tests/test_webservice.py'
--- src/lazr/restful/tests/test_webservice.py	2010-03-03 13:05:13 +0000
+++ src/lazr/restful/tests/test_webservice.py	2010-03-03 17:07:18 +0000
@@ -6,18 +6,14 @@
 
 from cStringIO import StringIO
 from operator import attrgetter
-import sys
-from types import ModuleType
 import unittest
 
 from zope.component import getGlobalSiteManager, getUtility
-from zope.configuration import xmlconfig
-from zope.interface import alsoProvides, implements, Interface
+from zope.interface import implements, Interface
 from zope.publisher.browser import TestRequest
 from zope.schema import Date, Datetime, TextLine
 from zope.security.management import (
     endInteraction, newInteraction, queryInteraction)
-from zope.testing.cleanup import CleanUp
 from zope.traversing.browser.interfaces import IAbsoluteURL
 
 from lazr.restful.fields import Reference
@@ -25,15 +21,11 @@
     ICollection, IEntry, IEntryResource, IResourceGETOperation,
     IServiceRootResource, IWebServiceConfiguration,
     IWebServiceClientRequest, IWebServiceVersion)
-from lazr.restful import (
-    EntryResource, ServiceRootResource, ResourceGETOperation)
-from lazr.restful.simple import BaseWebServiceConfiguration, Request
-from lazr.restful.declarations import (
-    collection_default_content, exported, export_as_webservice_collection,
-    export_as_webservice_entry, export_read_operation, operation_parameters)
-from lazr.restful.simple import RootResourceAbsoluteURL
+from lazr.restful import EntryResource, ResourceGETOperation
+from lazr.restful.declarations import exported, export_as_webservice_entry
 from lazr.restful.testing.webservice import (
-    create_web_service_request, WebServiceTestPublication)
+    create_web_service_request, IGenericCollection, IGenericEntry,
+    WebServiceTestCase, WebServiceTestPublication)
 from lazr.restful.testing.tales import test_tales
 from lazr.restful.utils import (
     get_current_browser_request, get_current_web_service_request,
@@ -69,125 +61,10 @@
         IResourceGETOperation, name=name)
 
 
-class IGenericEntry(Interface):
-    """A simple, reusable entry interface.
-
-    This is the description of the entry.
-    """
-    export_as_webservice_entry()
-
-    # pylint: disable-msg=E0213
-    a_field = exported(
-        TextLine(
-            title=u'A "field"',
-            description=u'The only field that can be <> 0 in the entry.'))
-
-    @operation_parameters(
-        message=TextLine(title=u'Message to say'))
-    @export_read_operation()
-    def greet(message):
-        """Print an appropriate greeting based on the message.
-
-        :param message: This will be included in the greeting.
-        """
-
-
-class IGenericCollection(Interface):
-    """A simple collection containing `IGenericEntry`."""
-    export_as_webservice_collection(IGenericEntry)
-
-    # pylint: disable-msg=E0211
-    @collection_default_content()
-    def getAll():
-        """Returns all the entries."""
-
-
-class IWebServiceRequest10(IWebServiceClientRequest):
-    """A marker interface for requests to the '1.0' web service."""
-
-
-class IWebServiceRequest20(IWebServiceClientRequest):
-    """A marker interface for requests to the '2.0' web service."""
-
-
-class SimpleWebServiceConfiguration(BaseWebServiceConfiguration):
-     implements(IWebServiceConfiguration)
-     show_tracebacks = False
-     active_versions = ['1.0', '2.0']
-     hostname = "webservice_test"
-     last_version_with_mutator_named_operations = None
-
-     def createRequest(self, body_instream, environ):
-         request = Request(body_instream, environ)
-         request.version = '2.0'
-         alsoProvides(request, IWebServiceRequest20)
-         request.setPublication(WebServiceTestPublication(
-             getUtility(IServiceRootResource)))
-         request.annotations[request.VERSION_ANNOTATION] = '2.0'
-         return request
-
-
-class WebServiceTestCase(CleanUp, unittest.TestCase):
-    """A test case for web service operations."""
-
-    testmodule_objects = []
-
-    def setUp(self):
-        """Set the component registry with the given model."""
-        super(WebServiceTestCase, self).setUp()
-
-        # Register a simple configuration object.
-        webservice_configuration = SimpleWebServiceConfiguration()
-        sm = getGlobalSiteManager()
-        sm.registerUtility(webservice_configuration)
-
-        # Register IWebServiceVersions for the
-        # '1.0' and '2.0' web service versions.
-        alsoProvides(IWebServiceRequest10, IWebServiceVersion)
-        sm.registerUtility(
-            IWebServiceRequest10, IWebServiceVersion, name='1.0')
-        alsoProvides(IWebServiceRequest20, IWebServiceVersion)
-        sm.registerUtility(
-            IWebServiceRequest20, IWebServiceVersion, name='2.0')
-
-        # Register a service root resource
-        service_root = ServiceRootResource()
-        sm.registerUtility(service_root, IServiceRootResource)
-
-        # Register an IAbsoluteURL adapter for the service root resource.
-        sm.registerAdapter(
-            RootResourceAbsoluteURL,
-            [IServiceRootResource, IWebServiceClientRequest], IAbsoluteURL)
-
-        # Build a test module that exposes the given resource interfaces.
-        testmodule = ModuleType('testmodule')
-        for interface in self.testmodule_objects:
-            setattr(testmodule, interface.__name__, interface)
-        sys.modules['lazr.restful.testmodule'] = testmodule
-
-        # Register the test module in the ZCML configuration: adapter
-        # classes will be built automatically.
-        xmlconfig.string("""
-        <configure
-           xmlns="http://namespaces.zope.org/zope"
-           xmlns:webservice="http://namespaces.canonical.com/webservice">
-         <include package="zope.component" file="meta.zcml" />
-         <include package="zope.security" file="meta.zcml"/>
-         <include package="lazr.restful" file="meta.zcml" />
-         <include package="lazr.restful" file="configure.zcml" />
-
-        <adapter for="*"
-            factory="zope.traversing.adapters.DefaultTraversable"
-            provides="zope.traversing.interfaces.ITraversable" />
-
-        <webservice:register module="lazr.restful.testmodule" />
-        </configure>
-        """)
-
-
 class IHas_getitem(Interface):
     pass
 
+
 class Has_getitem:
     implements(IHas_getitem)
     def __getitem__(self, item):
@@ -320,8 +197,8 @@
             'entry/wadl_entry:doc', entry=entry).splitlines()
         self.assertEquals([
             '<wadl:doc xmlns="http://www.w3.org/1999/xhtml">',
-            '<p>A simple, reusable entry interface.</p>',
-            '<p>This is the description of the entry.</p>',
+            '<p>A simple, reusable entry interface for use in tests.</p>',
+            '<p>The entry publishes one field and one named operation.</p>',
             '',
             '</wadl:doc>'], doclines)
 
@@ -339,7 +216,7 @@
             ).splitlines()
         self.assertEquals([
             '<wadl:doc xmlns="http://www.w3.org/1999/xhtml">',
-            'A simple collection containing IGenericEntry.',
+            'A simple collection containing IGenericEntry, for use in tests.',
             '</wadl:doc>'], doclines)
 
     def test_field_wadl_doc (self):