Merge lp:~cjwatson/lazr.restful/no-json-binary into lp:lazr.restful

[Colin Watson]

Don't attempt to JSON-decode Bytes fields read from a request.

Binary data can't be marshalled via JSON without additional encoding that we don't do, and lazr.restfulclient doesn't JSON-encode binary fields.

[Ioana Lasc (ilasc)]

review: Approve

=== modified file 'NEWS.rst'
--- NEWS.rst	2021-01-04 11:24:14 +0000
+++ NEWS.rst	2021-01-18 23:15:39 +0000
@@ -11,6 +11,10 @@
 allowing robust use of binary arguments on both Python 2 and 3
 (bug 1116954).
 
+Don't attempt to JSON-decode ``Bytes`` fields read from a request, since
+binary data can't be marshalled via JSON without additional encoding that we
+don't do, and ``lazr.restfulclient`` doesn't JSON-encode binary fields.
+
 0.23.0 (2020-09-28)
 ===================
 
 
=== modified file 'src/lazr/restful/docs/webservice-marshallers.rst'
--- src/lazr/restful/docs/webservice-marshallers.rst	2020-09-07 09:30:30 +0000
+++ src/lazr/restful/docs/webservice-marshallers.rst	2021-01-18 23:15:39 +0000
@@ -478,19 +478,16 @@
     None
 
 When coming over the request, the value is also converted into a UTF-8
-encoded string.
+encoded string, with no JSON decoding.
 
     >>> pprint_value(marshaller.marshall_from_request(u"Test"))
     b'Test'
     >>> pprint_value(marshaller.marshall_from_request(u'int\xe9ressant'))
     b'int\xc3\xa9ressant'
-    >>> pprint_value(marshaller.marshall_from_request('1.0'))
+    >>> pprint_value(marshaller.marshall_from_request(b'1.0'))
     b'1.0'
-
-But again, None is returned as is.
-
-    >>> print(marshaller.marshall_from_request('null'))
-    None
+    >>> pprint_value(marshaller.marshall_from_request(b'"not JSON"'))
+    b'"not JSON"'
 
 Since multipart/form-data can be used to upload data, file-like objects
 are read.
 
=== modified file 'src/lazr/restful/marshallers.py'
--- src/lazr/restful/marshallers.py	2020-09-01 13:20:54 +0000
+++ src/lazr/restful/marshallers.py	2021-01-18 23:15:39 +0000
@@ -290,21 +290,27 @@
         return "%s/%s" % (absoluteURL(entry.context, self.request),
                           self.field.__name__)
 
-    def _marshall_from_request(self, value):
-        """See `SimpleFieldMarshaller`.
+    def marshall_from_request(self, value):
+        """See `IFieldMarshaller`.
 
         Reads the data from file-like object, and converts non-strings into
-        one.
+        one.  This overrides `marshall_from_request` rather than
+        implementing the `_marshall_from_request` hook method because we
+        need to suppress the default JSON-decoding behaviour: binary data
+        can't be marshalled via JSON without additional encoding that we
+        don't do, and lazr.restfulclient doesn't JSON-encode binary fields.
         """
+        if value is None:
+            return None
         if safe_hasattr(value, 'seek'):
             value.seek(0)
             value = value.read()
         elif not isinstance(value, (bytes, six.text_type)):
-            value = bytes(value)
+            value = str(value).encode('UTF-8')
         else:
             # Leave string conversion to _marshall_from_json_data.
             pass
-        return super(BytesFieldMarshaller, self)._marshall_from_request(value)
+        return self._marshall_from_json_data(value)
 
     def _marshall_from_json_data(self, value):
         """See `SimpleFieldMarshaller`.