Merge lp:~leonardr/lazr.restful/bump-version into lp:lazr.restful

[Leonard Richardson]

[Leonard Richardson (leonardr)]

This branch adds multi-version tests for the WADL generation. When I tried to give multi-version features to lazr.restfulclient, it didn't work because the WADL was malformed. The two main problems were:

1. Unless you got the WADL for the latest version, lazr.restful saw multiple registrations for an entry adapter and crashed. This is because lazr.restful was looking for registrations that matched the marker interface (let's say) IWebServiceVersion1_0. But IWebServiceVersion2_0 and IWebServiceVersion3_0 subclass IWebServiceVersion1_0, so the entry adpater registrations for those versions were picked up as well.

2. The stub function that signals the removal of a named operation as of a particular version was being treated as a (malformed) named operation in its own right.

I fixed these problems in ways that should make sense when you look at the code.

This branch also fixes a test failure and prepares a new release of lazr.restful. (I thought that would be the only purpose of this branch, until lazr.restfulclient started failing on me.)

[lp:~leonardr/lazr.restful/bump-version]

115. By Leonard Richardson on 2010-02-11

Added missing file.

[Michael Nelson (michael.nelson)]

Thanks for all the changes Leonard. IRC log below for the record (and sorry Bjorn if you read this - I normally put the quoted diff, but I had to rush to finish this one on time!)

17:02 < noodles775> leonardr: on line 45 of the MP diff, is this really the earliest version, or the earliest *active* version? (which
                    line 47 seems to imply?)
17:03 < leonardr> noodles775: inactive versions don't exist in the lazr.restful installation, only in peoples' memories, so the
                  concepts are equivalent
17:03 < leonardr> i can change the wording

17:04 < noodles775> leonardr: in which case, why can len(config.active_versions) == 0, but we still have an earliest_version? or is
                    that what you'll re-word?
17:04 < leonardr> noodles775: it's kind of confusing but i don't want to change the interface becuase that'll cause backward
                  incompatibility problems
17:04 < leonardr> there are two fields in the interface
17:04 < leonardr> active_versions and latest_version_uri_prefix
17:05 < leonardr> active_versions is a list, the other is a stirng
17:05 < leonardr> lazr.restful serves web services for all of those strings
17:05 -!- danilos [~danilo@canonical/launchpad/danilos] has quit [Ping timeout: 265 seconds]
17:05 < leonardr> so if active_versions is =['beta', '1.0'] and latest_version_uri_prefix is 'devel' there are three versions being
                  served
17:05 < leonardr> if active_versions is [] and latest_version_uri_prefix is 'beta' then there is one version
17:05 < leonardr> though i don't recommend doing that
17:06 < noodles775> So if active_versions is =['beta', '1.0'] and latest_version_uri_prefix is 'devel', why are we evaluating
                    earliest_version = 'beta', is that right?
17:06 < noodles775> leonardr: ^^
17:07 < leonardr> noodles775: i'm having trouble parsing that sentence, but i can tell you that in that case earliest_version is 'beta'
17:07 < leonardr> latest_version_uri_prefix can only be the earliest version if active_versions is empty
17:07 < noodles775> OK, as the code implies, I'm just confused by the names.
17:08 < leonardr> noodles775: i'm kind of leaning towards treating the last item in active_versions as latest_version_uri_prefix

17:08 < leonardr> but that would be a separate job
17:09 -!- salgado-lunch is now known as salgado
17:09 < noodles775> Right, that would make sense. (do you mind adding an XXX?)
17:09 < leonardr> sure
17:11 < noodles775> leonardr: just a style question, is the comment on the empty line 69 intentional?
17:11 < leonardr> yeah, it's like a para break. i don't know if we allow that though
17:12 < noodles775> Yeah, I think a blank line (without the comment) serves just as well, but if the style guide doesn't say either
                    way, whatever you prefer :)
17:12 < leonardr> style guide doesn't mention it
17:13 < leonardr> if i see a blank line i think the first comment was talking about the blank line and the second comment is different
17:13 < noodles775> leonardr: s/marker_interface/version_marker on line 72
17:13 < noodles775> sure
17:19 -!- danilos [~danilo@canonical/launchpad/danilos] has joined #launchp...

review: Approve (code)

=== modified file 'src/lazr/restful/NEWS.txt'
--- src/lazr/restful/NEWS.txt	2010-01-12 17:41:25 +0000
+++ src/lazr/restful/NEWS.txt	2010-02-11 15:31:14 +0000
@@ -2,19 +2,20 @@
 NEWS for lazr.restful
 =====================
 
-Development
-===========
+0.9.18 (2010-02-11)
+===================
 
 Special note: this version contains backwards-incompatible
 changes. You *must* change your configuration object to get your code
-to work in this version! See "active_versions" below.
+to work in this version! See "active_versions" and
+"latest_version_uri_prefix" below.
 
 Added a versioning system for web services. Clients can now request
-the "trunk" of a web service as well as one published version. Apart
-from the URIs served, the two web services are exactly the same. There
-is no way to serve two different versions of a web service without
-defining both versions from scratch. (There will eventually be
-annotations to make this easy.)
+any number of distinct versions as well as a floating "trunk" which is
+always the most recent version. By using version-aware annotations,
+developers can publish the same data model differently over time. See
+the example web service in example/multiversion/ to see how the
+annotations work.
 
 This release introduces a new field to IWebServiceConfiguration:
 latest_version_uri_prefix. If you are rolling your own

=== modified file 'src/lazr/restful/_resource.py'
--- src/lazr/restful/_resource.py	2010-02-03 21:22:19 +0000
+++ src/lazr/restful/_resource.py	2010-02-11 15:31:14 +0000
@@ -1598,6 +1598,14 @@
         collection_classes = []
         singular_names = {}
         plural_names = {}
+
+        # Determine the name of the earliest version. We'll be using this later.
+        config = getUtility(IWebServiceConfiguration)
+        if len(config.active_versions) > 0:
+            earliest_version = config.active_versions[0]
+        else:
+            earliest_version = config.latest_version_uri_prefix
+
         for registration in sorted(site_manager.registeredAdapters()):
             provided = registration.provided
             if IInterface.providedBy(provided):
@@ -1610,10 +1618,26 @@
                     # of the classes with schemas, which we do describe.
 
                     # Make sure we have a registration relevant to
-                    # this version. A given entry may be have one
+                    # this version. A given entry may have one
                     # registration for every web service version.
-                    schema, version = registration.required
-                    if not version.providedBy(self.request):
+                    schema, version_marker = registration.required
+
+                    if (version_marker is IWebServiceClientRequest
+                        and self.request.version != earliest_version):
+                        # We are generating WADL for some version
+                        # other than the earliest version, and this is
+                        # a registration for the earliest version. We
+                        # can ignore it.
+                        #
+                        # We need this special test because the normal
+                        # test (below) is useless when
+                        # marker_interface is
+                        # IWebServiceClientRequest. Since all request
+                        # objects provide IWebServiceClientRequest
+                        # directly, it will always show up in
+                        # providedBy(self.request).
+                        continue
+                    if not version_marker in providedBy(self.request):
                         continue
 
                     # Make sure that no other entry class is using this

=== modified file 'src/lazr/restful/example/multiversion/tests/introduction.txt'
--- src/lazr/restful/example/multiversion/tests/introduction.txt	2010-02-08 18:59:13 +0000
+++ src/lazr/restful/example/multiversion/tests/introduction.txt	2010-02-11 15:31:14 +0000
@@ -232,7 +232,8 @@
 If an entry field is not published in a certain version, the
 corresponding field resource does not exist for that version.
 
-    >>> webservice.get('/pairs/foo/deleted', api_version='beta')
+    >>> print webservice.get('/pairs/foo/deleted', api_version='beta').body
+    Object: ...
     Traceback (most recent call last):
     ...
     NotFound: ... name: u'deleted'

=== added file 'src/lazr/restful/example/multiversion/tests/wadl.txt'
--- src/lazr/restful/example/multiversion/tests/wadl.txt	1970-01-01 00:00:00 +0000
+++ src/lazr/restful/example/multiversion/tests/wadl.txt	2010-02-11 15:31:14 +0000
@@ -0,0 +1,67 @@
+Multi-version WADL documents
+****************************
+
+A given version of the web service generates a WADL document which
+describes that version only. Let's go through the WADL documents for
+the different versions and see how they differ.
+
+    >>> from lazr.restful.testing.webservice import WebServiceCaller
+    >>> webservice = WebServiceCaller(domain='multiversion.dev')
+
+We'll start with a helper function that retrieves the WADL description
+for a given version of the key-value web service, and decomposes the
+top-level tags in the WADL document into a dictionary for easy access
+later. This works because all versions of the web service publish a
+single top-level collection and a single entry type, so the document's
+top-level structure is always the same.
+
+    >>> from lxml import etree
+    >>> from lxml.etree import _Comment
+    >>> def wadl_contents_for_version(version):
+    ...     """Parse the key-value service's WADL into a dictionary."""
+    ...     wadl = webservice.get(
+    ...         '/', media_type='application/vnd.sun.wadl+xml',
+    ...         api_version=version).body
+    ...     tree = etree.fromstring(wadl)
+    ...
+    ...     keys = ("base service_root service_root_json pair_collection "
+    ...             "pair_entry pair_full_json pair_diff_jaon pair_page "
+    ...             "pair_page_json hosted_file hosted_file_representation"
+    ...             ).split()
+    ...
+    ...     tags = [child for child in tree if not isinstance(child, _Comment)]
+    ...     contents = {}
+    ...     for i in range(0, len(keys)):
+    ...         contents[keys[i]] = tags[i]
+    ...     return contents
+
+Let's take a look at the differences. in 'beta', the 'by_value' method
+is not present at all.
+
+    >>> contents = wadl_contents_for_version('beta')
+    >>> print contents['base'].attrib['base']
+    http://multiversion.dev/beta/
+
+    >>> pair_collection = contents['pair_collection']
+    >>> sorted([method.attrib['id'] for method in pair_collection])
+    ['key_value_pairs-get']
+
+In '2.0', the by_value method is called 'byValue'.
+
+    >>> contents = wadl_contents_for_version('2.0')
+    >>> print contents['base'].attrib['base']
+    http://multiversion.dev/2.0/
+
+    >>> pair_collection = contents['pair_collection']
+    >>> sorted([method.attrib['id'] for method in pair_collection])
+    ['key_value_pairs-byValue', 'key_value_pairs-get']
+
+In '3.0', the method changes its name to 'byValue'.
+
+    >>> contents = wadl_contents_for_version('3.0')
+    >>> print contents['base'].attrib['base']
+    http://multiversion.dev/3.0/
+
+    >>> pair_collection = contents['pair_collection']
+    >>> sorted([method.attrib['id'] for method in pair_collection])
+    ['key_value_pairs-by_value', 'key_value_pairs-get']

=== modified file 'src/lazr/restful/tales.py'
--- src/lazr/restful/tales.py	2010-01-11 18:27:43 +0000
+++ src/lazr/restful/tales.py	2010-02-11 15:31:14 +0000
@@ -291,7 +291,13 @@
         for interface in (IResourceGETOperation, IResourcePOSTOperation):
             operations.extend(getGlobalSiteManager().adapters.lookupAll(
                     (model_class, request_interface), interface))
-        return [{'name' : name, 'op' : op} for name, op in operations]
+
+        # An operation that was present in an earlier version but was
+        # removed in the current version will show up in this list as
+        # an stub function that returns None. Since we don't want that
+        # operation to show up in this version, we'll filter it out.
+        return [{'name' : name, 'op' : op} for name, op in operations
+                if IResourceOperation.implementedBy(op)]
 
 
 class WadlEntryInterfaceAdapterAPI(WadlResourceAdapterAPI):

=== modified file 'src/lazr/restful/version.txt'
--- src/lazr/restful/version.txt	2009-11-10 14:30:52 +0000
+++ src/lazr/restful/version.txt	2010-02-11 15:31:14 +0000
@@ -1,1 +1,1 @@
-0.9.17
+0.9.18