Merge lp:~leonardr/lazr.restful/bug-619180 into lp:lazr.restful

[Leonard Richardson]

This branch fixes bug 619180. Field-specific validation errors that contained Unicode characters were treated properly if the validation was done as part of a PUT or PATCH request, but not if it was done as part of a named operation call. I fixed the place where the problem was and two other places as well (though I don't think the problem can really happen in those other two places).

I refactored a test helper method to avoid duplicating code.

[Данило Шеган (danilo)]

> leonardr, hi, is .encode(utf-8) safe to do for URLs?
<lifeless> if it is a url, then it must be a subset of ascii
 by definition
<leonardr> danilos: what specific case are you thinking of?
<danilos> ok, let me rephrase that... leonardr, is .encode('utf-8') safe to do before passing to webservice.get() [i.e. does it do escaping as appropriate, and is it documented that our webservice works with UTF-8]
 leonardr, line 48
<leonardr> danilos: well, the test works... that's a good question, though
<danilos> leonardr, heh, ok, fair enough
 leonardr, how do we answer that question?
<leonardr> danilos: i'll take a look at the incoming request
<jam> jtv: are you still around? I got stuck shoveling snow for a couple of hours this morning, but I'm around now
<danilos> leonardr, cool, thanks... even if it might be a problem already, it doesn't have much to do with this branch I suppose, so r=me
 leonardr, please do file a bug if you find it is, though
<leonardr> danilos: actually, i know it works in reality, because ursula's test request went through properly (and once this fix is applied, it works everywhere)
<leonardr> so i think httplib2 percent-encodes the utf-8, and that's how it works
<danilos> leonardr, right, makes sense, thanks

review: Approve

=== modified file 'src/lazr/restful/NEWS.txt'
--- src/lazr/restful/NEWS.txt	2011-01-26 16:50:13 +0000
+++ src/lazr/restful/NEWS.txt	2011-02-02 18:05:04 +0000
@@ -13,6 +13,9 @@
 You can suppress the website link for a particular entry class by
 passing publish_web_link=False into export_as_webservice_entry().
 
+Validation errors for named operations will be properly sent to the
+client even if they contain Unicode characters. (Launchpad bug 619180.)
+
 0.15.4 (2010-01-26)
 ===================
 

=== modified file 'src/lazr/restful/_operation.py'
--- src/lazr/restful/_operation.py	2010-08-13 14:29:52 +0000
+++ src/lazr/restful/_operation.py	2011-02-02 18:05:04 +0000
@@ -191,14 +191,14 @@
                     value = marshaller.marshall_from_request(
                         self.request.form.get(name))
                 except ValueError, e:
-                    errors.append("%s: %s" % (name, e))
+                    errors.append(u"%s: %s" % (name, e))
                     continue
             try:
                 field.validate(value)
             except RequiredMissing:
-                errors.append("%s: Required input is missing." % name)
+                errors.append(u"%s: Required input is missing." % name)
             except ValidationError, e:
-                errors.append("%s: %s" % (name, e))
+                errors.append(u"%s: %s" % (name, e))
             else:
                 validated_values[name] = value
         return (validated_values, errors)

=== modified file 'src/lazr/restful/example/base/tests/collection.txt'
--- src/lazr/restful/example/base/tests/collection.txt	2011-02-02 13:24:08 +0000
+++ src/lazr/restful/example/base/tests/collection.txt	2011-02-02 18:05:04 +0000
@@ -244,6 +244,16 @@
     ...
     search: Required input is missing.
 
+The error message may contain Unicode characters:
+
+    >>> from lazr.restful.testing.helpers import encode_response
+    >>> url = u"/cookbooks?ws.op=find_for_cuisine&cuisine=\N{SNOWMAN}"
+    >>> response = webservice.get(url.encode("utf-8"))
+    >>> print encode_response(response)
+    HTTP/1.1 400 Bad Request
+    ...
+    cuisine: Invalid value "\u2603". Acceptable values are:...
+
 If a named operation takes an argument that's a value for a vocabulary
 (such as Cuisine in the example web service), the client can specify
 the name of the value, just as they would when changing the value with

=== modified file 'src/lazr/restful/example/base/tests/field.txt'
--- src/lazr/restful/example/base/tests/field.txt	2011-01-26 16:14:11 +0000
+++ src/lazr/restful/example/base/tests/field.txt	2011-02-02 18:05:04 +0000
@@ -297,11 +297,7 @@
 ...and the XHTML representation of an ICookbook's description will be the
 result of calling a dummy_renderer object.
 
-  >>> def encode_response(response):
-  ...     """Encode a response that contains weird Unicode characters."""
-  ...     response_unicode = str(response).decode("utf-8")
-  ...     return response_unicode.encode("ascii", "backslashreplace")
-
+  >>> from lazr.restful.testing.helpers import encode_response
   >>> response = webservice.get(field_url, 'application/xhtml+xml')
   >>> print encode_response(response)
   HTTP/1.1 200 Ok

=== modified file 'src/lazr/restful/testing/helpers.py'
--- src/lazr/restful/testing/helpers.py	2010-09-29 16:18:47 +0000
+++ src/lazr/restful/testing/helpers.py	2011-02-02 18:05:04 +0000
@@ -39,6 +39,18 @@
     return new_module
 
 
+def encode_response(response):
+    """Encode a response that may contain weird Unicode characters.
+
+    The resulting string will (eg.) contain the six characters
+    "\u2603" instead of the single Unicode character SNOWMAN.
+
+    :param response: an httplib HTTPResponse object.
+    """
+    response_unicode = str(response).decode("utf-8")
+    return response_unicode.encode("ascii", "backslashreplace")
+
+
 class TestWebServiceConfiguration:
     implements(IWebServiceConfiguration)
     view_permission = "lazr.View"