Merge lp:~divmod-dev/divmod.org/nevow-json into lp:divmod.org

[Jean-Paul Calderone]

I added U+2028 and U+2029 to nevow.json.serialize's special cased code points. It already handled \r and \n. I added a test for these four cases specifically, too.

[Tristan Seligmann (mithrandi)]

Looks good! (I would say "Please merge", but I'm about to set the merge proposal to Approved, which I expect means the same thing)

One thing does occur to me: if you have a reference handy documenting the characters that need to be escaped, it might be useful leaving it in a comment near this code for anyone trying to make sense of it later on.

review: Approve

[lp:~divmod-dev/divmod.org/nevow-json]

2677. By Jean-Paul Calderone on 2011-07-10

Explain the two new characters a bit

=== modified file 'Nevow/nevow/json.py'
--- Nevow/nevow/json.py	2008-05-22 18:46:50 +0000
+++ Nevow/nevow/json.py	2011-07-10 13:58:36 +0000
@@ -235,6 +235,7 @@
 
 _translation = dict([(o, u'\\x%02x' % (o,)) for o in range(0x20)])
 
+# Characters which cannot appear as literals in the output
 _translation.update({
     ord(u'\\'): u'\\\\',
     ord(u'"'): ur'\"',
@@ -243,6 +244,10 @@
     ord(u'\n'): ur'\n',
     ord(u'\t'): ur'\t',
     ord(u'\r'): ur'\r',
+    # The next two are sneaky, see
+    # http://timelessrepo.com/json-isnt-a-javascript-subset
+    ord(u'\u2028'): u'\\u2028',
+    ord(u'\u2029'): u'\\u2029',
     })
 
 def stringEncode(s):

=== modified file 'Nevow/nevow/test/test_json.py'
--- Nevow/nevow/test/test_json.py	2010-07-12 19:00:11 +0000
+++ Nevow/nevow/test/test_json.py	2011-07-10 13:58:36 +0000
@@ -107,6 +107,24 @@
             self.assertEquals(unstruct, struct, failMsg)
             self.assert_(isinstance(unstruct, unicode), failMsg)
 
+
+    def test_lineTerminators(self):
+        """
+        When passed a unicode string containing a line terminator,
+        L{json.serialize} emits an escape sequence representing that character
+        (not a UTF-8 sequence directly representing that the line terminator
+        code point).
+
+        Literal line terminators are allowed in JSON, but some parsers do not
+        handle them properly.
+        """
+        # These are the four line terminators currently in Unicode.
+        self.assertEqual('"\\r"', json.serialize(u"\r"))
+        self.assertEqual('"\\n"', json.serialize(u"\n"))
+        self.assertEqual('"\\u2028"', json.serialize(u"\u2028"))
+        self.assertEqual('"\\u2029"', json.serialize(u"\u2029"))
+
+
     def testScientificNotation(self):
         self.assertEquals(json.parse('1e10'), 10**10)
         self.assertEquals(json.parse('1e0'), 1)