Merge lp:~paul-lucas/zorba/feature-json_strip_array into lp:zorba

[Paul J. Lucas]

Added ability to strip the top-level array from a JSON stream.

Added ability to strip the top-level array from a JSON stream.

[Paul J. Lucas (paul-lucas)]

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-json_strip_array-2012-11-29T01-38-00.341Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~paul-lucas/zorba/feature-json_strip_array into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message):
  Validation queue job feature-json_strip_array-2012-11-29T01-38-00.341Z is
  finished. The final status was:

  6 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-json_strip_array-2012-11-29T04-00-01.179Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job feature-json_strip_array-2012-11-29T04-00-01.179Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 1 Approve.

[Matthias Brantner (matthias-brantner)]

- why do we need to pop the entire stack in the destructor of the loader (shouldn't it be always empty)?
- can the huge number of catch clauses in the loader be reduced by only catching a base class without loss of semantics?
- changelog entry is missing (for the new feature and the two fixed bugs)

review: Needs Fixing

[Matthias Brantner (matthias-brantner)]

I added Ghislain as a reviewer. He needs to make changes to the JSONiq spec and, hence, at least needs to agree on the name of the option.

[Paul J. Lucas (paul-lucas)]

> - why do we need to pop the entire stack in the destructor of the loader
> (shouldn't it be always empty)?

It won't be empty if an exception is thrown.

> - can the huge number of catch clauses in the loader be reduced by only
> catching a base class without loss of semantics?

No.

Actually, I was going to open another bug after this was merged (since the bug existed before my changes). The old (and still current) code catches the exception and throws a new exception passing the existing what() message -- that's wrong since the what() message from the json::* exceptions is only in English (which is fine since json::* exceptions should never be seen by the user anyway).

There really need to be diagnostics entries in the XML file for all the error messages.

Also, why are all JSON parser errors squished into the single JNDY0021 error code? Error codes are free. Why not have one error code per parser error (just as I did with my JSON module)?

> - changelog entry is missing (for the new feature and the two fixed bugs)

What bugs?

> I added Ghislain as a reviewer. He needs to make changes to the JSONiq spec and,
> hence, at least needs to agree on the name of the option.

I was going to add him as a reviewer myself when I was "done" (which I'm not quite). I was trying to spare him the useless spam e-mail from launchpad until I was "done." So much for me trying to be nice.

[lp:~paul-lucas/zorba/feature-json_strip_array]

11152. By Paul J. Lucas on 2012-11-29

Added ascii::printable_char().

11153. By Paul J. Lucas on 2012-11-29

Added mention of jsoniq-strip-top-level-array.

11154. By Paul J. Lucas on 2012-11-29

Comment addition.

11155. By Paul J. Lucas on 2012-11-29

Comment fix.

[Matthias Brantner (matthias-brantner)]

>> - why do we need to pop the entire stack in the destructor of the loader
>> (shouldn't it be always empty)?
>
> It won't be empty if an exception is thrown.
Makes sense.

>
>> - can the huge number of catch clauses in the loader be reduced by only
>> catching a base class without loss of semantics?
>
> No.
>
> Actually, I was going to open another bug after this was merged (since the bug existed before my changes). The old (and still current) code catches the exception and throws a new exception passing the existing what() message -- that's wrong since the what() message from the json::* exceptions is only in English (which is fine since json::* exceptions should never be seen by the user anyway).
No, it didn't do that before. In the old json_loader.cpp, only parts of the message
were exposed. For example, the result of e.get_token().

>
> There really need to be diagnostics entries in the XML file for all the error messages.
>
> Also, why are all JSON parser errors squished into the single JNDY0021 error code? Error codes are free. Why not have one error code per parser error (just as I did with my JSON module)?
I don't think they are free from an XQuery point of view. There is no derivation of error codes
in XQuery. You can catch either one or all with one catch clause. Splitting JNDY0021 into
many error codes effectively means that the user has to catch all of them if he wants to
catch a parsing error. Does this make sense?

>
>> - changelog entry is missing (for the new feature and the two fixed bugs)
>
> What bugs?
https://bugs.launchpad.net/zorba/+bug/1081626 and the duplicate one. I think you included
the patch I appended to this bug, right?

>
>> I added Ghislain as a reviewer. He needs to make changes to the JSONiq spec and,
>> hence, at least needs to agree on the name of the option.
>
> I was going to add him as a reviewer myself when I was "done" (which I'm not quite). I was trying to spare him the useless spam e-mail from launchpad until I was "done." So much for me trying to be nice.
Sorry, I didn't know that you were not done, yet.

Matthias

[lp:~paul-lucas/zorba/feature-json_strip_array]

11156. By Paul J. Lucas on 2012-11-29

Better illegal_escape printing of character.

[Paul J. Lucas (paul-lucas)]

> > Actually, I was going to open another bug after this was merged (since the
> bug existed before my changes). The old (and still current) code catches the
> exception and throws a new exception passing the existing what() message --
> that's wrong since the what() message from the json::* exceptions is only in
> English (which is fine since json::* exceptions should never be seen by the
> user anyway).
> No, it didn't do that before.

Yes, it really did (and still does). The <value> for the error was/is simply $1 which means the entire error message came from the json::* exception as-is in English only.

> > What bugs?
> https://bugs.launchpad.net/zorba/+bug/1081626 and the duplicate one. I think
> you included
> the patch I appended to this bug, right?

I actually discovered the glaringly obvious bug independently and fixed it; however, your patch has the order different. Does it matter?

[Paul J. Lucas (paul-lucas)]

BTW: as I mentioned in the other bug, I couldn't reproduce the error (at least it never dumped core) on Mac OS X, so I have no idea whether your/our patch fixed the bug or not.

[lp:~paul-lucas/zorba/feature-json_strip_array]

11157. By Paul J. Lucas on 2012-11-29

Fixed English-only messages.

[Paul J. Lucas (paul-lucas)]

I stand corrected: the English-only issue was of my own making. It's now fixed.

[lp:~paul-lucas/zorba/feature-json_strip_array]

11158. By Paul J. Lucas on 2012-11-29

Updated ChangeLog.

[Matthias Brantner (matthias-brantner)]

I have done some testing and discovered a memory leak. Essentially, the memory isn't released as the stream is processed. Specifically, the memory allocated in json_loader.cpp:46 and json_loader.cpp:49 is never released.

review: Needs Fixing

[lp:~paul-lucas/zorba/feature-json_strip_array]

11159. By Paul J. Lucas on 2012-11-29

Fixed memory leak.

11160. By Paul J. Lucas on 2012-11-29

Comment addition.

[Paul J. Lucas (paul-lucas)]

OK, I fixed the memory leak.

[Matthias Brantner (matthias-brantner)]

review: Approve

[Ghislain Fourny (gislenius)]

Hi Paul,

Thanks for doing this. I like the parsing code.

1. Would it be possible to add tests to cover the new jsoniq-strip-top-level-array flag?

2. I am wondering if the behaviour with both flags set should not be to strip all top-level arrays, not just the first one. For example:

jn:parse-json(
  '[ 1, 2, 3 ][ 4, 5, 6 ]',
  {
    "jsoniq-multiple-top-level-items" : true,
    "jsoniq-strip-top-level-array" : true
  })

returns 1 2 3[4, 5, 6]

but I would expect 1 2 3 4 5 6.

3. I am under the impression that we use different terminologies in Zorba to describe the same behaviour ('skip', 'strip', I also saw 'chop' in a discussion). It would be nice if we could unify, not sure if still possible though.

I hope it helps!

review: Needs Fixing

[Ghislain Fourny (gislenius)]

Hi Paul,

The segmentation fault on the query seems to no longer happen, which indicates that your fixed worked.

[Ghislain Fourny (gislenius)]

s/fixed/fix/

[lp:~paul-lucas/zorba/feature-json_strip_array]

11161. By Paul J. Lucas on 2012-12-03

Merge from trunk.

11162. By Paul J. Lucas on 2012-12-03

Comment clarification.

11163. By Paul J. Lucas on 2012-12-03

Stripping top-level array now works for multiple items.

[Paul J. Lucas (paul-lucas)]

> 1. Would it be possible to add tests to cover the new jsoniq-strip-top-level-
> array flag?

Done.

> 2. I am wondering if the behaviour with both flags set should not be to strip
> all top-level arrays, not just the first one. For example:
>
> jn:parse-json(
> '[ 1, 2, 3 ][ 4, 5, 6 ]',
> {
> "jsoniq-multiple-top-level-items" : true,
> "jsoniq-strip-top-level-array" : true
> })
>
> returns 1 2 3[4, 5, 6]
>
> but I would expect 1 2 3 4 5 6.

Done.

> 3. I am under the impression that we use different terminologies in Zorba to
> describe the same behaviour ('skip', 'strip', I also saw 'chop' in a
> discussion). It would be nice if we could unify, not sure if still possible
> though.

To me, "skip" and "strip" mean slightly different things:

* "skip" means to skip over unrelated items in something like a stream or list.

* "strip" means to remove possibly related items, e.g., the beginning '[' and ending ']' of a JSON array.

* I personally haven't seen "chop," but that means to remove something from the end only.

[Ghislain Fourny (gislenius)]

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-json_strip_array-2012-12-06T14-45-02.357Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job feature-json_strip_array-2012-12-06T14-45-02.357Z is finished. The final status was:

All tests succeeded!

=== modified file 'ChangeLog'
--- ChangeLog	2012-11-20 01:01:49 +0000
+++ ChangeLog	2012-12-03 18:32:22 +0000
@@ -7,6 +7,8 @@
   * (bug #1039284) Implemented jn:json-doc().
   * (bug #867363) Added http-uri-resolution feature to disable HTTP
 	when resolving URIs.
+  * Can now specify jsoniq-strip-top-level-array option to parse-json() to
+    strip the top-level array from JSON streams.
 
 Optimizations:
   * Various optimizations in the implementation of the optimizer rules.
@@ -17,6 +19,8 @@
   * Fixed bug #868640 (Node lifetime is shorter than expected due to let
     variable inlining)
   * Fixed bug in castable expr when the target type is xs:QName.
+  * Fixed bugs #1081626 (Segmentation fault in a side-effecting FLWOR),
+    #1079488 (streaming segfault)
   * Fixed bug #1062093 (bug in var reference iterators with an associated
     position that is negative)
   * Fixed bug #1061222 (bad message for errors in index key type declaration)

=== modified file 'modules/org/jsoniq/www/functions.xq'
--- modules/org/jsoniq/www/functions.xq	2012-10-08 12:09:36 +0000
+++ modules/org/jsoniq/www/functions.xq	2012-12-03 18:32:22 +0000
@@ -170,6 +170,7 @@
  : Allowed options are
  : <ul>
  :   <li>jsoniq-multiple-top-level-items: allow parsing of sequences of JSON Objects and Arrays (boolean; default: true)</li>
+ :   <li>jsoniq-strip-top-level-array: if the top-level JSON item is an array, strip it and return its elements as multiple top-level items (boolean; default: false)</li>
  : </ul>
  :
  : @error jerr:JNDY0021 if the given string is not valid JSON or

=== modified file 'modules/org/jsoniq/www/pregenerated/errors.xq'
--- modules/org/jsoniq/www/pregenerated/errors.xq	2012-10-09 17:37:56 +0000
+++ modules/org/jsoniq/www/pregenerated/errors.xq	2012-12-03 18:32:22 +0000
@@ -141,7 +141,7 @@
 declare variable $jerr:JNUP0019 as xs:QName := fn:QName($jerr:NS, "jerr:JNUP0019");
 
 (:~
- :It is a type error if the prefix is not a string or if the 
+ :It is a type error if the prefix is not a string or if the
  : serialization parameters are not an element.
 :)
 declare variable $jerr:JNTY0023 as xs:QName := fn:QName($jerr:NS, "jerr:JNTY0023");

=== modified file 'modules/w3c/pregenerated/xqt-errors.xq'
--- modules/w3c/pregenerated/xqt-errors.xq	2012-09-19 21:16:15 +0000
+++ modules/w3c/pregenerated/xqt-errors.xq	2012-12-03 18:32:22 +0000
@@ -1264,8 +1264,8 @@
 
 (:~
  :
- : Retrieved resource contains octets that cannot be decoded into Unicode 
- : using the specified encoding, the resulting characters are not 
+ : Retrieved resource contains octets that cannot be decoded into Unicode
+ : using the specified encoding, the resulting characters are not
  : permitted XML characters or requested encoding not supported
  : 
  : @see http://www.w3.org/2005/xqt-errors

=== modified file 'src/compiler/parser/query_loc.h'
--- src/compiler/parser/query_loc.h	2012-09-19 21:16:15 +0000
+++ src/compiler/parser/query_loc.h	2012-12-03 18:32:22 +0000
@@ -17,9 +17,9 @@
 #ifndef ZORBA_QUERY_LOC_H
 #define ZORBA_QUERY_LOC_H
 
+#include <zorba/internal/ztd.h>
 #include "zorbatypes/zstring.h"
 
-
 namespace zorba {
 
 namespace serialization
@@ -27,6 +27,8 @@
   class Archiver;
 }
 
+///////////////////////////////////////////////////////////////////////////////
+
 /**
  * Class to save the query location of zorba elements that correspond to a code
  * snipped in the query.
@@ -36,6 +38,7 @@
  */
 class QueryLoc 
 {
+  typedef zorba::internal::ztd::explicit_bool explicit_bool;
 public:
   static QueryLoc null;
 
@@ -79,21 +82,30 @@
  
   bool equals(const QueryLoc& loc) const;
 
-  bool operator==(const QueryLoc& loc) const
-  {
-    return equals(loc);    
-  }
-  
-  bool operator<(const QueryLoc& loc) const
-  {
-    return theLineBegin < loc.getLineBegin();
+  operator explicit_bool::type() const {
+    return explicit_bool::value_of( !equals( null ) );
   }
 };
 
+inline bool operator==( QueryLoc const &q1, QueryLoc const &q2 ) {
+  return q1.equals( q2 );
+}
+
+inline bool operator!=( QueryLoc const &q1, QueryLoc const &q2 ) {
+  return !(q1 == q2);
+}
+
+inline bool operator<( QueryLoc const &q1, QueryLoc const &q2 ) {
+  return q1.getLineBegin()  <  q2.getLineBegin()
+      || (q1.getLineBegin() == q2.getLineBegin()
+          && q1.getColumnBegin() < q2.getColumnBegin());
+}
 
 std::ostream& operator<< (std::ostream& aOstr, const QueryLoc& aQueryLoc);
 
+///////////////////////////////////////////////////////////////////////////////
+
 } // namespace zorba
 
-#endif
+#endif /* ZORBA_QUERY_LOC_H */
 /* vim:set et sw=2 ts=2: */

=== modified file 'src/diagnostics/diagnostic_en.xml'
--- src/diagnostics/diagnostic_en.xml	2012-12-01 01:04:39 +0000
+++ src/diagnostics/diagnostic_en.xml	2012-12-03 18:32:22 +0000
@@ -1248,23 +1248,23 @@
        An argument to \c fn:deep-equal() contains a function item.
       </comment>
       <value>"$1": argument to fn:deep-equal() contains a function item</value>
-    </diagnostic>  
-    
+    </diagnostic>
+
     <diagnostic code="FOUT1170">
       <comment>
        Identifier cannot be used to retrive a resource containing text
       </comment>
       <value>"$1": error retrieving resource containing text</value>
     </diagnostic>
-    
+
     <diagnostic code="FOUT1190">
       <comment>
-       Retrieved resource contains octets that cannot be decoded into Unicode 
-       using the specified encoding, the resulting characters are not 
+       Retrieved resource contains octets that cannot be decoded into Unicode
+       using the specified encoding, the resulting characters are not
        permitted XML characters or requested encoding not supported
       </comment>
       <value>"$1": can not decode resource retrieved</value>
-    </diagnostic>  
+    </diagnostic>
 
     <!--////////// XQuery Update Facility //////////////////////////////////-->
 
@@ -1287,7 +1287,7 @@
          specified.
       </comment>
       <value>$1</value>
-    
+
       <entry key="CONCAT">
         <value>comma expression with updating and non-updating branches</value>
       </entry>
@@ -1476,7 +1476,7 @@
        in the query are made effective.
       </comment>
       <value>$1${ 2}</value>
-        
+
       <entry key="AttributeName">
         <value>"$3": attribute with the same name already exists</value>
       </entry>
@@ -1945,11 +1945,11 @@
     <diagnostic code="ZAPI0009" name="XQUERY_NOT_COMPILED_IN_DEBUG_MODE">
       <value>XQuery not compiled in debug mode</value>
     </diagnostic>
-     
+
      <diagnostic code="ZAPI0011" name="ELEMENT_NOT_DECLARED">
        <value>"$1": undefined $2</value>
      </diagnostic>
-     
+
     <diagnostic code="ZAPI0014" name="INVALID_ARGUMENT">
       <value>"$1": invalid argument${: 2}</value>
     </diagnostic>
@@ -1997,7 +1997,7 @@
     <diagnostic code="ZAPI0029" name="REFERENCE_ALREADY_PRESENT">
       <value>"$1": reference already present in the store</value>
     </diagnostic>
-   
+
     <diagnostic code="ZAPI0030" name="NO_CURRENT_REFERENCE">
       <value>node has no current reference</value>
     </diagnostic>
@@ -2033,7 +2033,7 @@
     <diagnostic code="ZAPI0080" name="CANNOT_RETRIEVE_NODE_REFERENCE">
       <value>can not retrieve node-reference for a node that is not in a collection</value>
     </diagnostic>
-     
+
     <diagnostic code="ZAPI0090" name="CANNOT_VALIDATE_NON_ROOT">
       <value>non root element cannot be validated in place</value>
     </diagnostic>
@@ -2647,18 +2647,18 @@
 
   <namespace prefix="jerr">
 
-    <!--////////// JSONiq ///////////////////////////////////////////-->
+    <!--////////// JSONiq //////////////////////////////////////////////////-->
 
     <diagnostic code="JNTY0002" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a type error if the right-hand-side expression of a pair constructor does not return exactly one item.</comment>
       <value>pair value returns no, or more than one, item</value>
     </diagnostic>
-    
+
     <diagnostic code="JNDY0003" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a dynamic error if two pairs in an object constructor or in a simple object union have the same name.</comment>
       <value>"$1": pair with the same name already exists in object</value>
     </diagnostic>
-    
+
     <diagnostic code="JNTY0004" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a type error to call fn:data on a sequence containing an array or an object.</comment>
       <value>can not atomize an $1 item: an $1 has probably been passed where an atomic value is expected (e.g., as a key, or to a function expecting an atomic item)</value>
@@ -2670,19 +2670,19 @@
       <value>"$1": duplicate pair to insert</value>
 
     </diagnostic>
-    
+
     <diagnostic code="JNUP0006" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a dynamic error if upd:applyUpdates causes an object to contain two pairs with the same name.</comment>
 
       <value>"$1": pair to insert already exists in object</value>
 
     </diagnostic>
-    
+
     <diagnostic code="JNUP0007" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a type error if, in an updating expression, an array selector cannot be cast to xs:integer or if an object selector cannot be cast to xs:string.</comment>
-      
+
       <value>"$1": wrong type for object/array selector in update expression</value>
-      
+
       <entry key="Object">
         <value>"$2": can not be promoted to type xs:string</value>
       </entry>
@@ -2696,7 +2696,7 @@
       </entry>
 
     </diagnostic>
-    
+
     <diagnostic code="JNUP0008" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a dynamic error if the target of a deleting or replacing expression is not an array or an object.
         It is a dynamic error if the target of a renaming expression is not an object.
@@ -2719,22 +2719,22 @@
       </entry>
 
     </diagnostic>
-    
+
     <diagnostic code="JNUP0009" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a dynamic error if a pending update list contains two replacing update primitives on the same object or array, and with the same selector.</comment>
       <value>"$1": duplicate pair to replace</value>
     </diagnostic>
-    
+
     <diagnostic code="JNUP0010" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a dynamic error if a pending update list contains two renaming update primitives on the same object and with the same selector.</comment>
       <value>"$1": duplicate pair to rename</value>
     </diagnostic>
-    
+
     <diagnostic code="JNTY0011" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a type error if the content sequence in a node constructor or in an XQUF insert or replace update expression contains an object or an array.</comment>
       <value>JSON item cannot appear in content sequence of node constructor or updating expression</value>
     </diagnostic>
-    
+
     <diagnostic code="JNSE0012" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a dynamic error to serialize a sequence of less
       or more than one item with the JSON output method if the
@@ -2770,7 +2770,7 @@
       <comment>It is a dynamic error if it is attempted to create a replace, delete or rename update primitive with a selector that cannot be resolved against the target array or object.</comment>
 
       <value>$1</value>
-      
+
       <entry key="Object">
         <value>"$2": selector cannot be resolved against supplied object</value>
       </entry>
@@ -2779,32 +2779,30 @@
         <value>"$2": selector cannot be resolved against supplied array</value>
       </entry>
     </diagnostic>
-    
+
     <diagnostic code="JNUP0017" if="defined(ZORBA_WITH_JSON)">
       <comment> It is a dynamic error if the value in a replace expression is not exactly a single item.</comment>
       <value>can not replace with less or more than an item</value>
     </diagnostic>
-    
+
     <diagnostic code="JNTY0018" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a dynamic error if there is not exactly one supplied parameter for an object or array selector.</comment>
-      
+
       <value>object or array selection needs exactly one parameter</value>
     </diagnostic>
-    
+
     <diagnostic code="JNUP0019" if="defined(ZORBA_WITH_JSON)">
       <comment>It is a dynamic error if the content expression, in an object insert expression, does not evaluate to a sequence of objects.</comment>
-      
+
       <value>"$1": invalid type (content of insert expression must evaluate to a sequence of objects)</value>
     </diagnostic>
-    
+
     <diagnostic code="JNTY0023" if="defined(ZORBA_WITH_JSON)">
-      <comment>It is a type error if the prefix is not a string or if the 
+      <comment>It is a type error if the prefix is not a string or if the
         serialization parameters are not an element.</comment>
       <value>$1: value of "$2" is not a $3</value>
     </diagnostic>
 
-    <!--////////// JSONIQ ///////////////////////////////////////////-->
-
     <diagnostic code="JNTY0024" if="defined(ZORBA_WITH_JSON)">
       <comment>objects or arrays don't have a string value</comment>
       <value>$1 items do not have string value</value>
@@ -2818,7 +2816,32 @@
     <diagnostic code="JNDY0021" if="defined(ZORBA_WITH_JSON)">
       <comment>parser error raised by jn:parse-json</comment>
       <value>$1</value>
+      <entry key="IllegalCharacter_2">
+        <value>'$2': illegal JSON character</value>
+      </entry>
+      <entry key="IllegalCodepoint_2">
+        <value>"$2": illegal Unicode code-point</value>
+      </entry>
+      <entry key="IllegalEscape_2">
+        <value>'\\$2': illegal JSON character escape</value>
+      </entry>
+      <entry key="IllegalLiteral">
+        <value>illegal JSON literal</value>
+      </entry>
+      <entry key="IllegalNumber">
+        <value>illegal JSON number</value>
+      </entry>
+      <entry key="UnexpectedToken_2">
+        <value>"$2": unexpected JSON token</value>
+      </entry>
+      <entry key="UnterminatedString">
+        <value>unterminated JSON string</value>
+      </entry>
+      <entry key="UnexpectedExtraContent">
+        <value>unexpected extra content at the end of the document (consider using the jsoniq-multiple-top-level-items option)</value>
+      </entry>
     </diagnostic>
+
   </namespace>
 
   <!--////////// Zorba Warnings ////////////////////////////////////////////-->
@@ -3011,7 +3034,7 @@
     <entry key="BadXMLDocument_2o">
       <value>malformed XML document${ at "2"}</value>
     </entry>
-    
+
     <entry key="BadXMLNoOpeningTag">
       <value>closing tag without matching opening tag</value>
     </entry>
@@ -3828,43 +3851,43 @@
     <entry key="JSONiq error">
       <value>JSONIQ error</value>
     </entry>
-    
+
     <entry key="JSONiq type error">
       <value>JSONIQ type error</value>
     </entry>
-    
+
     <entry key="JSONiq static error">
       <value>JSONIQ static error</value>
     </entry>
-    
+
     <entry key="JSONiq dynamic error">
       <value>JSONIQ dynamic error</value>
     </entry>
-    
+
     <entry key="JSONiq serialization error">
       <value>JSONIQ serialization error</value>
     </entry>
-    
+
     <entry key="JSONiq warning">
       <value>JSONIQ warning</value>
     </entry>
-    
+
     <entry key="JSONiq type warning">
       <value>JSONIQ type warning</value>
     </entry>
-    
+
     <entry key="JSONiq static warning">
       <value>JSONIQ static warning</value>
     </entry>
-    
+
     <entry key="JSONiq dynamic warning">
       <value>JSONIQ dynamic warning</value>
     </entry>
-    
+
     <entry key="JSONiq serialization warning">
       <value>JSONIQ serialization warning</value>
     </entry>
-    
+
     <entry key="dynamic error">
       <value>dynamic error</value>
     </entry>
@@ -4012,74 +4035,42 @@
     <entry key="ParseFragmentOptionCombinationNotAllowed">
       <value>only one of the &lt;schema-validate/&gt;, &lt;DTD-validate/&gt; or &lt;parse-external-parsed-entity/&gt; options can be specified</value>
     </entry>
-    
+
     <entry key="ParseFragmentInvalidOptions">
       <value>invalid options passed to the parse-xml:parse() function, the element must be in the schema target namespace</value>
     </entry>
-    
+
     <entry key="ParseFragmentDoctypeNotAllowed">
       <value>a DOCTYPE declaration is not allowed</value>
     </entry>
-    
+
     <entry key="ParseFragmentDoctypeNotAllowedHere">
       <value>a DOCTYPE declaration must appear before any element or text node, and at most once</value>
     </entry>
-    
+
     <entry key="FormatNumberDuplicates">
       <value>: a sub-picture must not contain more than one of the "$3" sign</value>
     </entry>
-    
+
     <entry key="FormatNumberGroupingAdjacentToDecimal">
       <value>: a sub-picture must not contain a grouping-separator-sign adjacent to a decimal-separator-sign</value>
     </entry>
-    
+
     <entry key="FormatNumberIntegerPart">
       <value>: the integer part of a sub-picture must not contain a member of the decimal-digit-family that is followed by an optional-digit-sign</value>
     </entry>
-    
+
     <entry key="FormatNumberFractionalPart">
       <value>: the fractional part of a sub-picture must not contain an optional-digit-sign that is followed by a member of the decimal-digit-family</value>
     </entry>
-    
+
     <entry key="FormatNumberPercentPermille">
       <value>: a sub-picture must not contain more than one percent-sign or per-mille-sign, and it must not contain one of each</value>
     </entry>
-    
+
     <entry key="FormatNumberAtLeastOneOptionalOrDecimal">
       <value>: a sub-picture must contain at least one character that is an optional-digit-sign or a member of the decimal-digit-family</value>
     </entry>
-    
-    <entry key="JSON_ILLEGAL_CHARACTER">
-      <value>'$2': illegal JSON character${ at 3}</value>
-    </entry>
-
-    <entry key="JSON_ILLEGAL_CODEPOINT">
-      <value>"$2": illegal Unicode code-point${ at 3}</value>
-    </entry>
-
-    <entry key="JSON_ILLEGAL_ESCAPE">
-      <value>'\\$2': illegal JSON character escape${ at 3}</value>
-    </entry>
-
-    <entry key="JSON_ILLEGAL_LITERAL">
-      <value>illegal JSON literal${ at 2}</value>
-    </entry>
-
-    <entry key="JSON_ILLEGAL_NUMBER">
-      <value>illegal JSON number${ at 2}</value>
-    </entry>
-
-    <entry key="JSON_UNEXPECTED_TOKEN">
-      <value>"$2": unexpected JSON token${ at 3}</value>
-    </entry>
-
-    <entry key="JSON_UNTERMINATED_STRING">
-      <value>unterminated JSON string${ at 2}</value>
-    </entry>
-
-    <entry key="JSON_UNEXPECTED_EXTRA_CONTENT">
-      <value>unexpected extra content at the end of the document (consider using the jsoniq-multiple-top-level-items option)</value>
-    </entry>
 
     <entry key="FUNCTION">
       <value>function</value>

=== modified file 'src/diagnostics/pregenerated/dict_en.cpp'
--- src/diagnostics/pregenerated/dict_en.cpp	2012-12-01 01:04:39 +0000
+++ src/diagnostics/pregenerated/dict_en.cpp	2012-12-03 18:32:22 +0000
@@ -611,6 +611,14 @@
   { "~HexBinaryMustBeEven", "HexBinary value must contain an even number of characters" },
   { "~IncompleteKeyInIndexBuild", "incomplete key during index build" },
   { "~IncompleteKeyInIndexRefresh", "incomplete key during index refresh" },
+  { "~JNDY0021_IllegalCharacter_2", "'$2': illegal JSON character" },
+  { "~JNDY0021_IllegalCodepoint_2", "\"$2\": illegal Unicode code-point" },
+  { "~JNDY0021_IllegalEscape_2", "'\\$2': illegal JSON character escape" },
+  { "~JNDY0021_IllegalLiteral", "illegal JSON literal" },
+  { "~JNDY0021_IllegalNumber", "illegal JSON number" },
+  { "~JNDY0021_UnexpectedExtraContent", "unexpected extra content at the end of the document (consider using the jsoniq-multiple-top-level-items option)" },
+  { "~JNDY0021_UnexpectedToken_2", "\"$2\": unexpected JSON token" },
+  { "~JNDY0021_UnterminatedString", "unterminated JSON string" },
   { "~JNUP0007_Array", "\"$2\": can not be promoted to type xs:integer" },
   { "~JNUP0007_Object", "\"$2\": can not be promoted to type xs:string" },
   { "~JNUP0007_ObjectArray", "\"$2\": can not be promoted to type xs:anyAtomicType" },
@@ -621,14 +629,6 @@
   { "~JNUP0016_Object", "\"$2\": selector cannot be resolved against supplied object" },
   { "~JSON parser error", "JSON parser error" },
   { "~JSON serialization error", "JSON serialization error" },
-  { "~JSON_ILLEGAL_CHARACTER", "'$2': illegal JSON character${ at 3}" },
-  { "~JSON_ILLEGAL_CODEPOINT", "\"$2\": illegal Unicode code-point${ at 3}" },
-  { "~JSON_ILLEGAL_ESCAPE", "'\\$2': illegal JSON character escape${ at 3}" },
-  { "~JSON_ILLEGAL_LITERAL", "illegal JSON literal${ at 2}" },
-  { "~JSON_ILLEGAL_NUMBER", "illegal JSON number${ at 2}" },
-  { "~JSON_UNEXPECTED_EXTRA_CONTENT", "unexpected extra content at the end of the document (consider using the jsoniq-multiple-top-level-items option)" },
-  { "~JSON_UNEXPECTED_TOKEN", "\"$2\": unexpected JSON token${ at 3}" },
-  { "~JSON_UNTERMINATED_STRING", "unterminated JSON string${ at 2}" },
   { "~JSONiq dynamic error", "JSONIQ dynamic error" },
   { "~JSONiq dynamic warning", "JSONIQ dynamic warning" },
   { "~JSONiq error", "JSONIQ error" },

=== modified file 'src/diagnostics/pregenerated/dict_zed_keys.h'
--- src/diagnostics/pregenerated/dict_zed_keys.h	2012-12-01 01:04:39 +0000
+++ src/diagnostics/pregenerated/dict_zed_keys.h	2012-12-03 18:32:22 +0000
@@ -72,6 +72,14 @@
 #define ZED_JNUP0008_ObjectArray "~JNUP0008_ObjectArray"
 #define ZED_JNUP0016_Object "~JNUP0016_Object"
 #define ZED_JNUP0016_Array "~JNUP0016_Array"
+#define ZED_JNDY0021_IllegalCharacter_2 "~JNDY0021_IllegalCharacter_2"
+#define ZED_JNDY0021_IllegalCodepoint_2 "~JNDY0021_IllegalCodepoint_2"
+#define ZED_JNDY0021_IllegalEscape_2 "~JNDY0021_IllegalEscape_2"
+#define ZED_JNDY0021_IllegalLiteral "~JNDY0021_IllegalLiteral"
+#define ZED_JNDY0021_IllegalNumber "~JNDY0021_IllegalNumber"
+#define ZED_JNDY0021_UnexpectedToken_2 "~JNDY0021_UnexpectedToken_2"
+#define ZED_JNDY0021_UnterminatedString "~JNDY0021_UnterminatedString"
+#define ZED_JNDY0021_UnexpectedExtraContent "~JNDY0021_UnexpectedExtraContent"
 #define ZED_ZWST0005_RETURN_TYPE "~ZWST0005_RETURN_TYPE"
 #define ZED_ZWST0005_PARAM_TYPE "~ZWST0005_PARAM_TYPE"
 #define ZED_ZWST0005_UPDATING "~ZWST0005_UPDATING"
@@ -318,14 +326,6 @@
 #define ZED_FormatNumberFractionalPart "~FormatNumberFractionalPart"
 #define ZED_FormatNumberPercentPermille "~FormatNumberPercentPermille"
 #define ZED_FormatNumberAtLeastOneOptionalOrDecimal "~FormatNumberAtLeastOneOptionalOrDecimal"
-#define ZED_JSON_ILLEGAL_CHARACTER "~JSON_ILLEGAL_CHARACTER"
-#define ZED_JSON_ILLEGAL_CODEPOINT "~JSON_ILLEGAL_CODEPOINT"
-#define ZED_JSON_ILLEGAL_ESCAPE "~JSON_ILLEGAL_ESCAPE"
-#define ZED_JSON_ILLEGAL_LITERAL "~JSON_ILLEGAL_LITERAL"
-#define ZED_JSON_ILLEGAL_NUMBER "~JSON_ILLEGAL_NUMBER"
-#define ZED_JSON_UNEXPECTED_TOKEN "~JSON_UNEXPECTED_TOKEN"
-#define ZED_JSON_UNTERMINATED_STRING "~JSON_UNTERMINATED_STRING"
-#define ZED_JSON_UNEXPECTED_EXTRA_CONTENT "~JSON_UNEXPECTED_EXTRA_CONTENT"
 #define ZED_FUNCTION "~FUNCTION"
 #define ZED_ANNOTATION "~ANNOTATION"
 

=== modified file 'src/runtime/CMakeLists.txt'
--- src/runtime/CMakeLists.txt	2012-10-16 14:30:02 +0000
+++ src/runtime/CMakeLists.txt	2012-12-03 18:32:22 +0000
@@ -175,6 +175,7 @@
 IF (ZORBA_WITH_JSON)
   LIST(APPEND RUNTIME_SRCS
     json/json_constructors.cpp
+    json/json_loader.cpp
   )
   HEADER_GROUP_SUBFOLDER(RUNTIME_SRCS json)
 ENDIF (ZORBA_WITH_JSON)

=== modified file 'src/runtime/json/json_impl.cpp'
--- src/runtime/json/json_impl.cpp	2012-09-19 21:16:15 +0000
+++ src/runtime/json/json_impl.cpp	2012-12-03 18:32:22 +0000
@@ -24,8 +24,8 @@
 #include "store/api/item_factory.h"
 #include "system/globalenv.h"
 
+#include "util/ascii_util.h"
 #include "util/mem_streambuf.h"
-#include "util/string_util.h"
 
 #include "jsonml_array.h"
 #include "snelson.h"
@@ -105,9 +105,7 @@
   catch ( json::illegal_character const &e ) {
     throw XQUERY_EXCEPTION(
       zerr::ZJPE0001_ILLEGAL_CHARACTER,
-      ERROR_PARAMS(zstring("#x") + 
-      BUILD_STRING(std::uppercase << std::hex
-                   << (static_cast<unsigned int>(e.get_char()) & 0xFF)) ),
+      ERROR_PARAMS( ascii::printable_char( e.get_char() ) ),
       ERROR_LOC( e.get_loc() )
     );
   }

=== added file 'src/runtime/json/json_loader.cpp'
--- src/runtime/json/json_loader.cpp	1970-01-01 00:00:00 +0000
+++ src/runtime/json/json_loader.cpp	2012-12-03 18:32:22 +0000
@@ -0,0 +1,277 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "stdafx.h"
+#include <zorba/config.h>
+
+#ifdef ZORBA_WITH_JSON
+
+// Zorba
+#include <store/api/item.h>
+#include <store/api/store.h>
+#include <zorba/store_consts.h>
+
+#include "context/static_context.h"
+#include "diagnostics/xquery_diagnostics.h"
+#include "store/api/item_factory.h"
+#include "system/globalenv.h"
+#include "zorbatypes/zstring.h"
+
+// local
+#include "json_loader.h"
+
+using namespace std;
+
+namespace zorba {
+namespace json {
+
+///////////////////////////////////////////////////////////////////////////////
+
+loader::stack_element::stack_element( type t ) : type_( t ) {
+  switch ( type_ ) {
+    case array_type:
+      array_ = new json_array_type;
+      break;
+    case object_type:
+      object_ = new json_object_type;
+      break;
+    default:
+      key_ = nullptr;
+      break;
+  }
+}
+
+/**
+ * Destroys a %stack_element.  We have this as an ordinary member function
+ * rather than a destructor so that popping the stack and destroying an element
+ * are distinct operations and can be called independently which comes in
+ * handy.
+ */
+void loader::stack_element::destroy() {
+  switch ( type_ ) {
+    case array_type:
+      delete array_;
+      break;
+    case object_type:
+      delete object_;
+      break;
+    case key_type:
+      key_->removeReference();
+      break;
+    default:
+      break;
+  }
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+loader::loader( istream &is, bool allow_multiple, bool strip_top_level_array ) :
+  parser_( is, allow_multiple ),
+  strip_top_level_array_( strip_top_level_array ),
+  stripped_top_level_array_( false )
+{
+}
+
+loader::~loader() {
+  clear_stack();
+}
+
+void loader::add_value( store::Item_t const &value ) {
+  stack_element top( stack_.top() );
+  switch ( top.type_ ) {
+    case stack_element::array_type:
+      top.array_->push_back( value );
+      break;
+    case stack_element::object_type:
+      //
+      // value must be a string that's the name of the object's next key/value
+      // pair.
+      //
+      push( stack_element::key_type ).key_ = value.getp();
+      value->addReference();
+      break;
+    case stack_element::key_type: {
+      //
+      // Otherwise, the top of the stack must be a string which means that the
+      // second-to-top must be an object awaiting a value associated with this
+      // name.
+      //
+      stack_.pop();
+      stack_element &top2 = stack_.top();
+      assert( top2.type_ == stack_element::object_type );
+      top2.object_->keys_.push_back( top.key_ );
+      top2.object_->values_.push_back( value );
+      top.destroy();
+      break;
+    }
+    default:
+      assert( false );
+  }
+}
+
+void loader::clear() {
+  parser_.clear();
+  clear_stack();
+  stripped_top_level_array_ = false;
+}
+
+void loader::clear_stack() {
+  while ( !stack_.empty() ) {
+    stack_.top().destroy();
+    stack_.pop();
+  }
+}
+
+bool loader::next( store::Item_t *result ) {
+  store::Item_t item;
+  zstring s;
+  json::token t;
+
+  try {
+    while ( parser_.next( &t ) ) {
+      switch( t.get_type() ) {
+        case '[':
+          if ( strip_top_level_array_ && !stripped_top_level_array_ )
+            stripped_top_level_array_ = true;
+          else
+            push( stack_element::array_type );
+          continue;
+        case '{':
+          push( stack_element::object_type );
+          continue;
+        case ']':
+          if ( stack_.empty() && strip_top_level_array_ ) {
+            stripped_top_level_array_ = false;
+            continue;
+          }
+        case '}': {
+          stack_element top( stack_.top() );
+          stack_.pop();
+          switch ( top.type_ ) {
+            case stack_element::array_type:
+              GENV_ITEMFACTORY->createJSONArray( item, *top.array_ );
+              break;
+            case stack_element::object_type:
+              GENV_ITEMFACTORY->createJSONObject(
+                item, top.object_->keys_, top.object_->values_
+              );
+              break;
+            default:
+              assert( false );
+          } // switch
+          top.destroy();
+          break;
+        }
+        case ':':
+        case ',':
+          continue;
+        case token::number:
+          s = t.get_value();
+          GENV_ITEMFACTORY->createJSONNumber( item, s );
+          break;
+        case token::string:
+          s = t.get_value();
+          GENV_ITEMFACTORY->createString( item, s );
+          break;
+        case 'F':
+        case 'T':
+          GENV_ITEMFACTORY->createBoolean( item, t.get_type() == 'T' );
+          break;
+        case token::json_null:
+          GENV_ITEMFACTORY->createJSONNull( item );
+          break;
+        default:
+          assert( false );
+      } // switch
+
+      if ( stack_.empty() ) {
+        *result = item;
+        return true;
+      }
+      add_value( item );
+    } // while
+    return false;
+  } // try
+  catch ( json::illegal_character const &e ) {
+    throw XQUERY_EXCEPTION(
+      jerr::JNDY0021,
+      ERROR_PARAMS(
+        ZED( JNDY0021_IllegalCharacter_2 ),
+        ascii::printable_char( e.get_char() )
+      ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::illegal_codepoint const &e ) {
+    throw XQUERY_EXCEPTION(
+      jerr::JNDY0021,
+      ERROR_PARAMS(
+        ZED( JNDY0021_IllegalCodepoint_2 ),
+        e.get_codepoint()
+      ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::illegal_escape const &e ) {
+    throw XQUERY_EXCEPTION(
+      jerr::JNDY0021,
+      ERROR_PARAMS(
+        ZED( JNDY0021_IllegalEscape_2 ),
+        ascii::printable_char( e.get_escape() )
+      ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::illegal_literal const &e ) {
+    throw XQUERY_EXCEPTION(
+      jerr::JNDY0021,
+      ERROR_PARAMS( ZED( JNDY0021_IllegalLiteral ) ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::illegal_number const &e ) {
+    throw XQUERY_EXCEPTION(
+      jerr::JNDY0021,
+      ERROR_PARAMS( ZED( JNDY0021_IllegalNumber ) ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::unexpected_token const &e ) {
+    throw XQUERY_EXCEPTION(
+      jerr::JNDY0021,
+      ERROR_PARAMS(
+        ZED( JNDY0021_UnexpectedToken_2 ),
+        e.get_token()
+      ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::unterminated_string const &e ) {
+    throw XQUERY_EXCEPTION(
+      jerr::JNDY0021,
+      ERROR_PARAMS( ZED( JNDY0021_UnterminatedString ) ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace json
+} // namespace zorba
+
+#endif /* ZORBA_WITH_JSON */
+/* vim:set et sw=2 ts=2: */

=== added file 'src/runtime/json/json_loader.h'
--- src/runtime/json/json_loader.h	1970-01-01 00:00:00 +0000
+++ src/runtime/json/json_loader.h	2012-12-03 18:32:22 +0000
@@ -0,0 +1,139 @@
+/*
+ * Copyright 2006-2011 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef ZORBA_JSON_LOADER_H
+#define ZORBA_JSON_LOADER_H
+
+// standard
+#include <istream>
+#include <new>
+#include <utility>                          /* for pair */
+#include <vector>
+
+// Zorba
+#include "store/api/item.h"
+#include "util/json_parser.h"
+#include "zorbatypes/zstring.h"
+
+namespace zorba {
+namespace json {
+
+///////////////////////////////////////////////////////////////////////////////
+
+class loader {
+public:
+  typedef parser::line_type line_type;
+  typedef parser::column_type column_type;
+
+  /**
+   * Constructs a %loader.
+   *
+   * @param is The istream to read from.
+   * @param allow_multiple If \c true, allow multiple top-level JSON items to
+   * be returned.
+   * @param strip_top_level_array If \c true, strips the top-level array, if
+   * any.
+   */
+  loader( std::istream &is, bool allow_multiple = false,
+          bool strip_top_level_array = false );
+
+  /**
+   * Destroys this %loader.
+   */
+  ~loader();
+
+  /**
+   * Resets this %loader to its initial state.
+   */
+  void clear();
+
+  /**
+   * Gets the next Item, if any.
+   *
+   * @param result A pointer to the Item to receive the Item.
+   * @return Returns \c true only if an Item was gotten.
+   */
+  bool next( store::Item_t *result );
+
+  /**
+   * Sets the file location.
+   *
+   * @param file The source file name.
+   * @param line The source line number.
+   * @param col  The source column number.
+   */
+  void set_loc( char const *file, line_type line, column_type col ) {
+    parser_.set_loc( file, line, col );
+  }
+
+private:
+  typedef std::vector<store::Item_t> json_array_type;
+
+  struct json_object_type {
+    std::vector<store::Item_t> keys_;
+    std::vector<store::Item_t> values_;
+  };
+
+  struct stack_element {
+    enum type {
+      no_type,
+      array_type,
+      object_type,
+      key_type
+    };
+    type const type_;
+    union {
+      json_array_type *array_;
+      json_object_type *object_;
+      store::Item *key_;
+    };
+
+    stack_element( type = no_type );
+    void destroy();
+  };
+
+  typedef std::stack<stack_element> stack_type;
+  stack_type stack_;
+
+  void add_value( store::Item_t const& );
+  void clear_stack();
+
+  stack_element& push( stack_element::type t ) {
+    //
+    // We push in this convoluted manner since we can't use C++11 rvalue
+    // references.
+    //
+    stack_.push( stack_element() );
+    return *new( &stack_.top() ) stack_element( t );
+  }
+
+  parser parser_;
+  bool const strip_top_level_array_;
+  bool stripped_top_level_array_;
+};
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace json
+} // namespace zorba
+
+#endif /* ZORBA_JSON_LOADER_H */
+/*
+ * Local variables:
+ * mode: c++
+ * End:
+ */
+/* vim:set et sw=2 ts=2: */

=== modified file 'src/runtime/json/jsoniq_functions_impl.cpp'
--- src/runtime/json/jsoniq_functions_impl.cpp	2012-11-07 17:48:17 +0000
+++ src/runtime/json/jsoniq_functions_impl.cpp	2012-12-03 18:32:22 +0000
@@ -55,10 +55,13 @@
 #include <store/api/store.h>
 #include <store/api/copymode.h>
 
-#include <util/uri_util.h>
+#include "util/uri_util.h"
+
 #include <zorba/store_consts.h>
 #include <zorbatypes/URI.h>
 
+#include "json_loader.h"
+
 
 namespace zorba {
 
@@ -744,51 +747,59 @@
 {
   PlanIteratorState::init(aState);
   theAllowMultiple = true; // default
-  theInputStream = 0;
+  theInputStream = nullptr;
   theGotOne = false;
+  loader_ = nullptr;
 }
 
 void
 JSONParseIteratorState::reset(PlanState& aState)
 {
   PlanIteratorState::reset(aState);
-  if (theInput == NULL && theInputStream)
-  {
+  if (theInput == NULL) {
     delete theInputStream;
+    theInputStream = nullptr;
   }
+  theGotOne = false;
+  delete loader_;
+  loader_ = nullptr;
 }
 
 JSONParseIteratorState::~JSONParseIteratorState()
 {
-  if (theInput == NULL && theInputStream)
-  {
+  if (theInput == NULL)
     delete theInputStream;
-  }
+  delete loader_;
 }
 
-void
-JSONParseIterator::processOptions(
-    const store::Item_t& aOptions,
-    bool& aAllowMultiple) const
+bool JSONParseIterator::processBooleanOption( store::Item_t const &options,
+                                              char const *option_name,
+                                              bool *option_value ) const
 {
-  store::Item_t lOptionName, lOptionValue;
-
-  zstring s("jsoniq-multiple-top-level-items");
-  GENV_ITEMFACTORY->createString(lOptionName, s);
-  lOptionValue = aOptions->getObjectValue(lOptionName);
-
-  if (lOptionValue != NULL)
-  {
-    store::SchemaTypeCode lType = lOptionValue->getTypeCode();
-    if (!TypeOps::is_subtype(lType, store::XS_BOOLEAN))
-    {
-      const TypeManager* tm = theSctx->get_typemanager();
-      xqtref_t lType = tm->create_value_type(lOptionValue, loc);
-      RAISE_ERROR(jerr::JNTY0020, loc,
-      ERROR_PARAMS(lType->toSchemaString(), s, "xs:boolean"));
+  store::Item_t i_option_name;
+  zstring z_option_name( option_name );
+  GENV_ITEMFACTORY->createString( i_option_name, z_option_name );
+  store::Item_t i_option_value = options->getObjectValue( i_option_name );
+
+  if ( i_option_value ) {
+    store::SchemaTypeCode const option_type = i_option_value->getTypeCode();
+    if ( !TypeOps::is_subtype( option_type, store::XS_BOOLEAN ) ) {
+      TypeManager const *const tm = theSctx->get_typemanager();
+      xqtref_t const option_type = tm->create_value_type( i_option_value, loc );
+      throw XQUERY_EXCEPTION(
+        jerr::JNTY0020,
+        ERROR_PARAMS(
+          option_type->toSchemaString(),
+          z_option_name,
+          "xs:boolean"
+        ),
+        ERROR_LOC( loc )
+      );
     }
-    aAllowMultiple = lOptionValue->getBooleanValue();
+    *option_value = i_option_value->getBooleanValue();
+    return true;
   }
+  return false;
 }
 
 bool
@@ -797,6 +808,7 @@
   PlanState& planState) const
 {
   store::Item_t lInput;
+  bool lStripTopLevelArray = false;
 
   JSONParseIteratorState* state;
   DEFAULT_STACK_INIT(JSONParseIteratorState, state, planState);
@@ -807,7 +819,12 @@
     {
       store::Item_t lOptions;
       consumeNext(lOptions, theChildren[1].getp(), planState);
-      processOptions(lOptions, state->theAllowMultiple);
+      processBooleanOption(
+        lOptions, "jsoniq-multiple-top-level-items", &state->theAllowMultiple
+      );
+      processBooleanOption(
+        lOptions, "jsoniq-strip-top-level-array", &lStripTopLevelArray
+      );
     }
 
     if (lInput->isStreamable())
@@ -818,59 +835,36 @@
     else
     {
       // will be deleted in the state
-      state->theInputStream = new std::stringstream(
-          lInput->getStringValue().c_str());
-    }
-
-    while (true)
-    {
-      try
-      {
-        // streamable string or non-literal string
-        if (state->theInput != NULL || theRelativeLocation == QueryLoc::null)
-        {
-          result = GENV_STORE.parseJSON(*state->theInputStream, 0);
-        }
-        else
-        {
-          // pass the query location of the StringLiteral to the JSON
-          // parser such that it can give better error locations.
-          zorba::internal::diagnostic::location lLoc;
-          lLoc = ERROR_LOC(theRelativeLocation);
-          result = GENV_STORE.parseJSON(*state->theInputStream, &lLoc);
-        }
-      }
-      catch (zorba::XQueryException& e)
-      {
-        // rethrow with JNDY0021
-        XQueryException xq = XQUERY_EXCEPTION(
-            jerr::JNDY0021,
-            ERROR_PARAMS(e.what()),
-            ERROR_LOC(loc));
-
-        // use location of e in case of literal string
-        if (!(theRelativeLocation == QueryLoc::null)) set_source(xq, e);
-        throw xq;
-      }
-
-      if (result != NULL)
-      {
-        if (!state->theAllowMultiple && state->theGotOne)
-        {
-          RAISE_ERROR(jerr::JNDY0021, loc,
-          ERROR_PARAMS(ZED(JSON_UNEXPECTED_EXTRA_CONTENT)));
-        }
-        state->theGotOne = true;
-        STACK_PUSH(true, state);
-        continue;
-      }
-      else
-      {
-        break;
-      }
+      state->theInputStream =
+        new std::stringstream( lInput->getStringValue().c_str() );
+    }
+
+    state->loader_ = new json::loader(
+      *state->theInputStream, true, lStripTopLevelArray
+    );
+
+    if ( state->theInput == NULL && theRelativeLocation ) {
+      // pass the query location of the StringLiteral to the JSON
+      // parser such that it can give better error locations.
+      state->loader_->set_loc(
+        theRelativeLocation.getFilename().c_str(),
+        theRelativeLocation.getLineBegin(),
+        theRelativeLocation.getColumnBegin()
+      );
+    }
+
+    while ( state->loader_->next( &result ) ) {
+      if ( !state->theAllowMultiple && state->theGotOne ) {
+        throw XQUERY_EXCEPTION(
+          jerr::JNDY0021,
+          ERROR_PARAMS( ZED( JNDY0021_UnexpectedExtraContent ) ),
+          ERROR_LOC( loc )
+        );
+      }
+      state->theGotOne = true;
+      STACK_PUSH( true, state );
     }
   }
-
   STACK_END(state);
 }
 
@@ -1606,6 +1600,30 @@
 /*******************************************************************************
 
 ********************************************************************************/
+
+void
+JSONDocIteratorState::init(PlanState& aState)
+{
+  PlanIteratorState::init(aState);
+  theStream = nullptr;
+  theGotOne = false;
+  loader_ = nullptr;
+}
+
+void
+JSONDocIteratorState::reset(PlanState& aState)
+{
+  PlanIteratorState::reset(aState);
+  theGotOne = false;
+  delete loader_;
+  loader_ = nullptr;
+}
+
+JSONDocIteratorState::~JSONDocIteratorState()
+{
+  delete loader_;
+}
+
 bool JSONDocIterator::nextImpl(store::Item_t& result, PlanState& planState) const
 {
   store::Item_t uriItem;
@@ -1646,38 +1664,19 @@
     }
 
     state->theGotOne = false;
+    state->loader_ = new json::loader( *state->theStream, true );
 
-    while (true)
+    while ( state->loader_->next( &result ) )
     {
-      try
-      {
-        result = GENV_STORE.parseJSON(*state->theStream, 0);
-      }
-      catch (zorba::XQueryException& e)
-      {
-        // rethrow with JNDY0021
-        XQueryException xq = XQUERY_EXCEPTION(
-            jerr::JNDY0021,
-            ERROR_PARAMS(e.what()),
-            ERROR_LOC(loc));
-
-        // use location of e in case of literal string
-        throw xq;
-      }
-      if (result != NULL)
-      {
-        if (!state->theGotOne)
-        {
-          state->theGotOne = true;
-          STACK_PUSH(true, state);
-        } else {
-          RAISE_ERROR(
-              jerr::JNDY0021,
-              loc,
-              ERROR_PARAMS(ZED(JSON_UNEXPECTED_EXTRA_CONTENT)));
-        }
+      if (!state->theGotOne)
+      {
+        state->theGotOne = true;
+        STACK_PUSH(true, state);
       } else {
-        break;
+        RAISE_ERROR(
+            jerr::JNDY0021,
+            loc,
+            ERROR_PARAMS(ZED(JNDY0021_UnexpectedExtraContent)));
       }
     }
   }
@@ -1686,6 +1685,6 @@
 }
 
 } /* namespace zorba */
+
+#endif /* ZORBA_WITH_JSON */
 /* vim:set et sw=2 ts=2: */
-
-#endif /* ZORBA_WITH_JSON */

=== modified file 'src/runtime/json/pregenerated/jsoniq_functions.cpp'
--- src/runtime/json/pregenerated/jsoniq_functions.cpp	2012-10-15 13:39:36 +0000
+++ src/runtime/json/pregenerated/jsoniq_functions.cpp	2012-12-03 18:32:22 +0000
@@ -384,16 +384,6 @@
 
 JSONDocIteratorState::JSONDocIteratorState() {}
 
-JSONDocIteratorState::~JSONDocIteratorState() {}
-
-
-void JSONDocIteratorState::init(PlanState& planState) {
-  PlanIteratorState::init(planState);
-}
-
-void JSONDocIteratorState::reset(PlanState& planState) {
-  PlanIteratorState::reset(planState);
-}
 // </JSONDocIterator>
 
 

=== modified file 'src/runtime/json/pregenerated/jsoniq_functions.h'
--- src/runtime/json/pregenerated/jsoniq_functions.h	2012-10-15 13:49:05 +0000
+++ src/runtime/json/pregenerated/jsoniq_functions.h	2012-12-03 18:32:22 +0000
@@ -30,6 +30,7 @@
 #include "runtime/base/noarybase.h"
 #include "runtime/base/narybase.h"
 #include <context/uri_resolver.h>
+#include "runtime/json/json_loader.h"
 
 
 namespace zorba {
@@ -151,6 +152,7 @@
   store::Item_t theInput; //
   std::istream* theInputStream; //
   bool theGotOne; //
+  json::loader* loader_; //
 
   JSONParseIteratorState();
 
@@ -185,7 +187,7 @@
   virtual ~JSONParseIterator();
 
 public:
-  void processOptions(const store::Item_t& aOptions, bool& aAllowMultiple) const;
+  bool processBooleanOption(const store::Item_t& options, char const* option_name, bool* option_value) const;
   void accept(PlanIterVisitor& v) const;
 
   bool nextImpl(store::Item_t& result, PlanState& aPlanState) const;
@@ -468,6 +470,7 @@
   std::auto_ptr<internal::Resource> theResource; //
   std::istream* theStream; //
   bool theGotOne; //
+  json::loader* loader_; //
 
   JSONDocIteratorState();
 

=== modified file 'src/runtime/spec/json/jsoniq_functions.xml'
--- src/runtime/spec/json/jsoniq_functions.xml	2012-10-15 13:49:05 +0000
+++ src/runtime/spec/json/jsoniq_functions.xml	2012-12-03 18:32:22 +0000
@@ -11,6 +11,7 @@
 
 <zorba:header>
   <zorba:include form="Angle-bracket">context/uri_resolver.h</zorba:include>
+  <zorba:include form="Quoted">runtime/json/json_loader.h</zorba:include>
 </zorba:header>
   
 <!--
@@ -180,6 +181,7 @@
     <zorba:member type="store::Item_t" name="theInput" brief=""/>
     <zorba:member type="std::istream*" name="theInputStream" brief=""/>
     <zorba:member type="bool" name="theGotOne"/>
+    <zorba:member type="json::loader*" name="loader_" brief=""/>
   </zorba:state>
 
   <zorba:constructor>
@@ -188,9 +190,10 @@
 
   <zorba:member type="QueryLoc" name="theRelativeLocation" />
 
-  <zorba:method return="void" name="processOptions" const="true">
-    <zorba:param type="const store::Item_t&amp;" name="aOptions"/>
-    <zorba:param type="bool&amp;" name="aAllowMultiple"/>
+  <zorba:method return="bool" name="processBooleanOption" const="true">
+    <zorba:param type="const store::Item_t&amp;" name="options"/>
+    <zorba:param type="char const*" name="option_name"/>
+    <zorba:param type="bool*" name="option_value"/>
   </zorba:method>
 
 </zorba:iterator>
@@ -403,10 +406,11 @@
     
   </zorba:function>
   
-  <zorba:state>
+  <zorba:state generateInit="false" generateReset="false" generateDestructor="false">
     <zorba:member type="std::auto_ptr&lt;internal::Resource&gt;" name="theResource" brief=""/>
     <zorba:member type="std::istream*" name="theStream" brief=""/>
     <zorba:member type="bool" name="theGotOne" brief=""/>
+    <zorba:member type="json::loader*" name="loader_" brief=""/>
   </zorba:state>
   
 </zorba:iterator>

=== modified file 'src/store/api/store.h'
--- src/store/api/store.h	2012-09-19 21:16:15 +0000
+++ src/store/api/store.h	2012-12-03 18:32:22 +0000
@@ -29,14 +29,6 @@
 namespace zorba 
 { 
 
-namespace internal 
-{
-namespace diagnostic 
-{
-  class location;
-}
-}
-
 SYNC_CODE(class Lock;)
 
 class TokenizerProvider;
@@ -368,13 +360,6 @@
   virtual TokenizerProvider const* getTokenizerProvider() const = 0;
 
 #endif /* ZORBA_NO_FULL_TEXT */
-
-#ifdef ZORBA_WITH_JSON
-  virtual Item_t parseJSON(
-      std::istream& stream,
-      internal::diagnostic::location* relative_error_loc
-    ) = 0;
-#endif
 };
 
 } // namespace store

=== modified file 'src/store/naive/CMakeLists.txt'
--- src/store/naive/CMakeLists.txt	2012-09-19 21:16:15 +0000
+++ src/store/naive/CMakeLists.txt	2012-12-03 18:32:22 +0000
@@ -60,7 +60,7 @@
 IF (ZORBA_WITH_JSON)
   LIST(APPEND ZORBA_STORE_IMPL_SRCS
     json_items.cpp
-    json_loader.cpp
   )
 ENDIF (ZORBA_WITH_JSON)
 
+# vim:set et sw=2 ts=2:

=== removed file 'src/store/naive/json_loader.cpp'
--- src/store/naive/json_loader.cpp	2012-10-08 12:09:36 +0000
+++ src/store/naive/json_loader.cpp	1970-01-01 00:00:00 +0000
@@ -1,307 +0,0 @@
-/*
- * Copyright 2006-2011 The FLWOR Foundation.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-#include "json_loader.h"
-#include "util/json_parser.h"
-#include "json_items.h"
-#include "simple_item_factory.h"
-#include "store_defs.h"
-#include "simple_store.h"
-#include "diagnostics/diagnostic.h"
-#include <cassert>
-#include <vector>
-
-namespace zorba
-{
-
-namespace simplestore
-{
-
-namespace json
-{
-
-
-/******************************************************************************
-
-*******************************************************************************/
-JSONLoader::JSONLoader(
-    std::istream& s,
-    internal::diagnostic::location* relative_error_loc
-  )
-  : in(s),
-    theRelativeLoc(relative_error_loc)
-{
-}
-
-
-/******************************************************************************
-
-*******************************************************************************/
-JSONLoader::~JSONLoader()
-{
-}
-
-#define RAISE_JSON_ERROR_NO_PARAM(msg) \
-  if (theRelativeLoc) \
-  { \
-    throw XQUERY_EXCEPTION( \
-        jerr::JNDY0021, \
-        ERROR_PARAMS( \
-          ZED(msg), \
-          "" \
-        ), \
-        ERROR_LOC(e.get_loc()) \
-     ); \
-  } \
-  else \
-  { \
-    throw ZORBA_EXCEPTION( \
-        jerr::JNDY0021, \
-        ERROR_PARAMS( \
-          ZED(msg), \
-          BUILD_STRING(e.get_loc().line(), ", ", e.get_loc().column()) \
-        ) \
-     ); \
-  } 
-
-#define RAISE_JSON_ERROR_WITH_PARAM(msg, param) \
-  if (theRelativeLoc) \
-  { \
-    throw XQUERY_EXCEPTION( \
-        jerr::JNDY0021, \
-        ERROR_PARAMS( \
-          ZED(msg), \
-          param,  \
-          "" \
-        ), \
-        ERROR_LOC(e.get_loc()) \
-     ); \
-  } \
-  else \
-  { \
-    throw ZORBA_EXCEPTION( \
-        jerr::JNDY0021, \
-        ERROR_PARAMS( \
-          ZED(msg), \
-          param, \
-          BUILD_STRING(e.get_loc().line(), ", ", e.get_loc().column()) \
-        ) \
-     ); \
-  } 
-
-/******************************************************************************
-
-*******************************************************************************/
-store::Item_t
-JSONLoader::next( )
-{
-  using namespace zorba::json;
-  using namespace zorba::simplestore;
-  using namespace zorba::simplestore::json;
-
-  try
-  {
-    BasicItemFactory& lFactory = GET_FACTORY();
-
-    JSONItem_t lRootItem;
-
-    // stack of objects, arrays, and object pairs
-    std::vector<store::Item_t> lStack;
-
-    parser lParser(in);
-    if (theRelativeLoc)
-    {
-      lParser.set_loc(theRelativeLoc->file(), theRelativeLoc->line(), theRelativeLoc->column()+1);
-    }
-
-    token lToken;
-
-    while (lParser.next(&lToken))
-    {
-      switch (lToken.get_type())
-      {
-        case token::begin_array:
-          lStack.push_back(new SimpleJSONArray());
-          break;
-
-        case token::begin_object:
-          lStack.push_back(new SimpleJSONObject());
-          break;
-
-        case token::end_array:
-        case token::end_object:
-          {
-            store::Item_t lItem = lStack.back();
-
-            lStack.pop_back();
-
-            if (lStack.empty())
-            {
-              lRootItem = lItem.cast<JSONItem>();
-            }
-            else
-            {
-              addValue(lStack, lItem);
-            }
-
-            break;
-          }
-        case token::name_separator:
-        case token::value_separator:
-            break;
-        case token::string:
-          {
-            store::Item_t lValue;
-            zstring s = lToken.get_value();
-            lFactory.createString(lValue, s);
-
-            addValue(lStack, lValue);
-            break;
-          }
-        case token::number:
-          {
-            store::Item_t lValue;
-            zstring s = lToken.get_value();
-            lFactory.createJSONNumber(lValue, s);
-            // todo check return type
-            addValue(lStack, lValue);
-            break;
-          }
-        case token::json_false:
-          {
-            store::Item_t lValue;
-            lFactory.createBoolean(lValue, false);
-            addValue(lStack, lValue);
-            break;
-          }
-        case token::json_true:
-          {
-            store::Item_t lValue;
-            lFactory.createBoolean(lValue, true);
-            addValue(lStack, lValue);
-            break;
-          }
-        case token::json_null:
-          {
-            store::Item_t lValue;
-            lFactory.createJSONNull(lValue);
-            addValue(lStack, lValue);
-            break;
-          }
-        default:
-          assert(false);
-      }
-    }
-    return lRootItem;
-  }
-  catch (zorba::json::unterminated_string& e)
-  {
-    RAISE_JSON_ERROR_NO_PARAM(JSON_UNTERMINATED_STRING)
-  }
-  catch (zorba::json::unexpected_token& e)
-  {
-    RAISE_JSON_ERROR_WITH_PARAM(JSON_UNEXPECTED_TOKEN, e.get_token())
-  }
-  catch (zorba::json::illegal_number& e)
-  {
-    RAISE_JSON_ERROR_NO_PARAM(JSON_ILLEGAL_NUMBER)
-  }
-  catch (zorba::json::illegal_literal& e)
-  {
-    RAISE_JSON_ERROR_NO_PARAM(JSON_ILLEGAL_LITERAL)
-  }
-  catch (zorba::json::illegal_escape& e)
-  {
-    RAISE_JSON_ERROR_WITH_PARAM(JSON_ILLEGAL_ESCAPE, e.get_escape())
-  }
-  catch (zorba::json::illegal_codepoint& e)
-  {
-    RAISE_JSON_ERROR_WITH_PARAM(JSON_ILLEGAL_CODEPOINT, e.get_codepoint())
-  }
-  catch (zorba::json::illegal_character& e)
-  {
-    RAISE_JSON_ERROR_WITH_PARAM(JSON_ILLEGAL_CHARACTER, e.get_char())
-  }
-  return NULL;
-}
-#undef RAISE_JSON_ERROR_WITH_PARAM
-#undef RAISE_JSON_ERROR_NO_PARAM
-
-void
-JSONLoader::addValue(
-  std::vector<store::Item_t>& aStack,
-  const store::Item_t& aValue)
-{
-  store::Item_t lLast = aStack.back();
-
-  JSONObject* lObject = dynamic_cast<JSONObject*>(lLast.getp());
-
-  if (lObject)
-  {
-    // if the top of the stack is an object, then
-    // the value must be a string which is the name
-    // of the object's next name/value pair
-    aStack.push_back(aValue);
-    return;
-  }
-
-  JSONArray* lArray  = dynamic_cast<JSONArray*>(lLast.getp());
-  if (lArray)
-  {
-    // if the top of the stack is an array, then
-    // the value must be appended to it
-    lArray->push_back(aValue);
-    return;
-  }
-
-  // Otherwise, the top of the stack must be a string, which means
-  // that the second-to-top must be an object awaiting a value associated with
-  // this name.
-  store::Item_t lString = aStack.back();
-  aStack.pop_back();
-  
-  lLast = aStack.back();
-
-  lObject = dynamic_cast<JSONObject*>(lLast.getp());
-
-  assert(lObject);
-  lObject->add(lString, aValue, false);
-}
-
-template<typename T> T*
-JSONLoader::cast(const JSONItem_t& j)
-{
-#ifndef NDEBUG
-  T* t = dynamic_cast<T*>(j.getp());
-  assert(t);
-#else
-  T* t = static_cast<T*>(j.getp());
-#endif
-  return t;
-}
-
-} /* namespace json */
-
-} /* namespace simplestore */
-
-} /* namespace zorba */
-
-/*
- * Local variables:
- * mode: c++
- * End:
- */
-/* vim:set et sw=2 ts=2: */
-

=== removed file 'src/store/naive/json_loader.h'
--- src/store/naive/json_loader.h	2012-09-19 21:16:15 +0000
+++ src/store/naive/json_loader.h	1970-01-01 00:00:00 +0000
@@ -1,80 +0,0 @@
-/*
- * Copyright 2006-2011 The FLWOR Foundation.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-#ifndef ZORBA_SIMPLE_STORE_JSON_LOADER_H
-#define ZORBA_SIMPLE_STORE_JSON_LOADER_H
-
-#include "common/common.h"
-#include "shared_types.h"
-#include "store/api/item.h"
-
-namespace zorba
-{
-
-namespace internal 
-{
-namespace diagnostic 
-{
-  class location;
-}
-}
-
-
-namespace simplestore
-{
-
-namespace json
-{
-
-class JSONLoader
-{
-protected:
-  std::istream                   & in;
-  internal::diagnostic::location * theRelativeLoc;
-
-public:
-  JSONLoader(
-    std::istream& s,
-    internal::diagnostic::location* relative_error_loc);
-
-  ~JSONLoader();
-
-  store::Item_t
-  next( );
-
-private:
-  template<typename T>
-  static T*
-  cast(const JSONItem_t&);
-
-  void
-  addValue(std::vector<store::Item_t>&, const store::Item_t&);
-
-}; /* class JSONLoader */
-
-
-} /* namespace json */
-
-} /* namespace simplestore */
-
-} /* namespace zorba */
-
-#endif /* ZORBA_SIMPLE_STORE_JSON_LOADER_H */
-/*
- * Local variables:
- * mode: c++
- * End:
- */
-/* vim:set et sw=2 ts=2: */

=== modified file 'src/store/naive/simple_store.cpp'
--- src/store/naive/simple_store.cpp	2012-10-08 12:09:36 +0000
+++ src/store/naive/simple_store.cpp	2012-12-03 18:32:22 +0000
@@ -35,10 +35,6 @@
 #include <zorba/util/uuid.h>
 #include "zorbautils/string_util.h"
 
-#ifdef ZORBA_WITH_JSON
-#include "json_loader.h"
-#endif
-
 
 namespace zorba
 {
@@ -419,18 +415,5 @@
 }
 
 
-#ifdef ZORBA_WITH_JSON
-/*******************************************************************************
-
-********************************************************************************/
-store::Item_t SimpleStore::parseJSON(
-    std::istream& stream,
-    internal::diagnostic::location* relative_error_loc)
-{
-  json::JSONLoader lLoader(stream, relative_error_loc);
-  return lLoader.next();
-}
-#endif /* ZORBA_WITH_JSON */
-
 } // namespace simplestore
 } // namespace zorba

=== modified file 'src/store/naive/simple_store.h'
--- src/store/naive/simple_store.h	2012-09-19 21:16:15 +0000
+++ src/store/naive/simple_store.h	2012-12-03 18:32:22 +0000
@@ -113,12 +113,6 @@
   bool assignReference(const store::Item* node, const zstring& reference);
 
   bool getNodeByReference(store::Item_t& result, const zstring& reference);
-
-#ifdef ZORBA_WITH_JSON
-  store::Item_t parseJSON(
-      std::istream& stream,
-      internal::diagnostic::location* relative_error_loc);
-#endif
 };
 
 } // namespace store

=== modified file 'src/util/ascii_util.cpp'
--- src/util/ascii_util.cpp	2012-09-19 21:16:15 +0000
+++ src/util/ascii_util.cpp	2012-12-03 18:32:22 +0000
@@ -15,10 +15,15 @@
  */
 #include "stdafx.h"
 
+// standard
 #include <cstring>
+#include <iomanip>
 
+// local
 #include "ascii_util.h"
 
+using namespace std;
+
 namespace zorba {
 namespace ascii {
 
@@ -32,6 +37,17 @@
   return true;
 }
 
+ostream& printable_char( ostream &o, char c ) {
+  if ( ascii::is_print( c ) )
+    o << c;
+  else {
+    ios::fmtflags const old_flags = o.flags();
+    o << "#x" << uppercase << hex << (static_cast<unsigned>( c ) & 0xFF);
+    o.flags( old_flags );
+  }
+  return o;
+}
+
 size_type remove_chars( char *s, size_type s_len, char const *chars ) {
   char *end = s + s_len;
   char *c;

=== modified file 'src/util/ascii_util.h'
--- src/util/ascii_util.h	2012-09-19 21:16:15 +0000
+++ src/util/ascii_util.h	2012-12-03 18:32:22 +0000
@@ -17,11 +17,14 @@
 #ifndef ZORBA_STRING_UTIL_ASCII_UTIL_H
 #define ZORBA_STRING_UTIL_ASCII_UTIL_H
 
+// standard
 #include <algorithm>
 #include <cctype>
 #include <cstddef>
 #include <cstring>
 
+// local
+#include "omanip.h"
 #include "stl_util.h"
 
 namespace zorba {
@@ -967,6 +970,20 @@
 ////////// Miscellaneous //////////////////////////////////////////////////////
 
 /**
+ * Prints the given character in a printable way: if \c is_print(c) is \c true,
+ * prints \a c as-is; otherwise prints \c #x followed by the hexadecimal value
+ * of the character.
+ *
+ * @param o The ostream to print to.
+ * @param c The \c char to print.
+ * @return Returns \a o.
+ */
+std::ostream& printable_char( std::ostream &o, char c );
+
+// An ostream manipulator version of the above.
+DEF_OMANIP1( printable_char, char )
+
+/**
  * Reverses the characters in a string.
  *
  * @tparam InputStringType The input string type.

=== modified file 'src/util/json_parser.cpp'
--- src/util/json_parser.cpp	2012-09-19 21:16:15 +0000
+++ src/util/json_parser.cpp	2012-12-03 18:32:22 +0000
@@ -15,6 +15,7 @@
  */
 
 #include "stdafx.h"
+
 #include "diagnostics/assert.h"
 
 #include "ascii_util.h"
@@ -23,7 +24,6 @@
 #include "utf8_util.h"
 
 #define DEBUG_JSON_PARSER 0
-
 #if DEBUG_JSON_PARSER
 # include "indent.h"
 #endif /* DEBUG_JSON_PARSER */
@@ -79,7 +79,10 @@
 }
 
 illegal_character::illegal_character( location const &loc, char c ) :
-  exception( loc, BUILD_STRING( '\'', c, "': illegal character" ) ),
+  exception(
+    loc,
+    BUILD_STRING( '\'', ascii::printable_char( c ), "': illegal character" )
+  ),
   c_( c )
 {
 }
@@ -100,7 +103,12 @@
 }
 
 illegal_escape::illegal_escape( location const &loc, char c ) :
-  exception( loc, BUILD_STRING( "\"\\", c, "\": illegal character escape" ) ),
+  exception(
+    loc,
+    BUILD_STRING(
+      "\"\\", ascii::printable_char( c ), "\": illegal character escape"
+    )
+  ),
   esc_( c )
 {
 }
@@ -148,9 +156,7 @@
 
 ///////////////////////////////////////////////////////////////////////////////
 
-token::token() :
-  type_( none )
-{
+token::token() : type_( none ) {
 }
 
 ostream& operator<<( ostream &o, token::type tt ) {
@@ -556,7 +562,20 @@
 
 ///////////////////////////////////////////////////////////////////////////////
 
-parser::parser( istream &in ) : lexer_( in ) {
+parser::parser( std::istream &in, bool allow_multiple ) :
+  allow_multiple_( allow_multiple ),
+  lexer_( in )
+{
+  init();
+}
+
+void parser::clear() {
+  peeked_token_.clear();
+  ztd::clear_stack( state_stack_ );
+  init();
+}
+
+void parser::init() {
 #if DEBUG_JSON_PARSER
   get_indent( cout ) = 0;
 #endif /* DEBUG_JSON_PARSER */
@@ -601,7 +620,7 @@
     switch ( state_ ) {
 
       // <JSON> ::= <Array> | <Object>
-      case J0:  PUSH_STATE( J1 );
+      case J0:  PUSH_STATE( allow_multiple_ ? J0 : J1 );
                 switch ( PEEK_TOKEN() ) {
                   case token::begin_array : GOTO_STATE( A0 );
                   case token::begin_object: GOTO_STATE( O0 );

=== modified file 'src/util/json_parser.h'
--- src/util/json_parser.h	2012-09-19 21:16:15 +0000
+++ src/util/json_parser.h	2012-12-03 18:32:22 +0000
@@ -501,8 +501,15 @@
    * Constructs a %parser on the given istream.
    *
    * @param in The istream to read from.
-   */
-  parser( std::istream &in );
+   * @param allow_multiple If \c true, allow multiple top-level JSON items.
+   */
+  parser( std::istream &in, bool allow_multiple = false );
+
+  /**
+   * Resets this %parser to its almost-initial state.  (It does not, however,
+   * reset the file location.)
+   */
+  void clear();
 
   /**
    * Gets the next token, if any.
@@ -546,6 +553,7 @@
 
   friend std::ostream& operator<<( std::ostream&, state );
 
+  void init();
   bool get_token( token* );
   bool get_token_debug( int, token* );
   bool matches_token( token::type, token* );
@@ -555,6 +563,7 @@
   void require_token( token::type, token* );
   void require_token_debug( int, token::type, token* );
 
+  bool allow_multiple_;
   lexer lexer_;
   token peeked_token_;
   std::stack<state> state_stack_;

=== modified file 'src/util/stl_util.h'
--- src/util/stl_util.h	2012-09-19 21:16:15 +0000
+++ src/util/stl_util.h	2012-12-03 18:32:22 +0000
@@ -155,6 +155,18 @@
 ///////////////////////////////////////////////////////////////////////////////
 
 /**
+ * Clears the given stack.
+ *
+ * @tparam T The stack's \c value_type.
+ * @param s The stack to clear.
+ */
+template<typename T> inline
+void clear_stack( std::stack<T> &s ) {
+  while ( !s.empty() )
+    s.pop();
+}
+
+/**
  * A less-verbose way to determine whether the given map or set contains a
  * particular element.
  */

=== renamed file 'test/rbkt/ExpQueryResults/zorba/json/json-snelson-numbers-and-decimals.xml.res' => 'test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-numbers.xml.res'
--- test/rbkt/ExpQueryResults/zorba/json/json-snelson-numbers-and-decimals.xml.res	2012-01-23 23:22:52 +0000
+++ test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-numbers.xml.res	2012-12-03 18:32:22 +0000
@@ -1,7 +1,5 @@
 <json xmlns="http://john.snelson.org.uk/parsing-json-into-xquery" type="object">
-  <pair name="special-numbers" type="object">
-    <pair name="decimal" type="number">2.89</pair>
-    <pair name="e" type="number">2E+6</pair>
-    <pair name="negative" type="number">-1.89</pair>
-  </pair>
+  <pair name="decimal" type="number">2.89</pair>
+  <pair name="e" type="number">2E+6</pair>
+  <pair name="negative" type="number">-1.89</pair>
 </json>

=== renamed file 'test/rbkt/ExpQueryResults/zorba/json/json-snelson-null-handling.xml.res' => 'test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-object-05.xml.res'
--- test/rbkt/ExpQueryResults/zorba/json/json-snelson-null-handling.xml.res	2012-01-23 23:22:52 +0000
+++ test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-object-05.xml.res	2012-12-03 18:32:22 +0000
@@ -1,3 +1,3 @@
 <json xmlns="http://john.snelson.org.uk/parsing-json-into-xquery" type="object">
-  <pair name="web-app" type="null"/>
+  <pair name="a" type="null"/>
 </json>

=== renamed file 'test/rbkt/ExpQueryResults/zorba/json/json-snelson-uncommon-chars.xml.res' => 'test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-uncommon-chars.xml.res'
=== renamed file 'test/rbkt/ExpQueryResults/zorba/json/json-snelson-utf-8.xml.res' => 'test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-utf8-01.xml.res'
=== added file 'test/rbkt/ExpQueryResults/zorba/jsoniq/parse_json-09.xml.res'
--- test/rbkt/ExpQueryResults/zorba/jsoniq/parse_json-09.xml.res	1970-01-01 00:00:00 +0000
+++ test/rbkt/ExpQueryResults/zorba/jsoniq/parse_json-09.xml.res	2012-12-03 18:32:22 +0000
@@ -0,0 +1,1 @@
+1 2

=== added file 'test/rbkt/ExpQueryResults/zorba/jsoniq/parse_json-10.xml.res'
--- test/rbkt/ExpQueryResults/zorba/jsoniq/parse_json-10.xml.res	1970-01-01 00:00:00 +0000
+++ test/rbkt/ExpQueryResults/zorba/jsoniq/parse_json-10.xml.res	2012-12-03 18:32:22 +0000
@@ -0,0 +1,1 @@
+1 2 3 4

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-parse-01.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-parse-01.xq	2011-12-23 06:21:58 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-parse-01.xq	2012-12-03 18:32:22 +0000
@@ -6,4 +6,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:parse( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-parse-02.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-parse-02.xq	2011-12-23 06:21:58 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-parse-02.xq	2012-12-03 18:32:22 +0000
@@ -11,4 +11,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:parse( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-parse-03.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-parse-03.xq	2011-12-23 06:21:58 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-parse-03.xq	2012-12-03 18:32:22 +0000
@@ -15,4 +15,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:parse( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-parse-04.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-parse-04.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-parse-04.xq	2012-12-03 18:32:22 +0000
@@ -1,27 +1,30 @@
 import module namespace json = "http://www.zorba-xquery.com/modules/converters/json";
 
-let $json := '["ul",
-  ["li",
-  { "style" : "color:red" },
-  "First Item"
-  ],
-  ["li",
-  {
-  "title" : "Some hover text.",
-  "style" : "color:green"
-  },
-  "Second Item"
-  ],
-  ["li",
-    ["span",
-    { "class" : "code-example-third" },
-    "Third"
-    ],
-  " Item"
-  ]
-]'
+let $json := '
+  [ "ul",
+    [ "li",
+      { "style" : "color:red" },
+      "First Item"
+    ],
+    [ "li",
+      {
+        "title" : "Some hover text.",
+        "style" : "color:green"
+      },
+      "Second Item"
+    ],
+    [ "li",
+      [ "span",
+        { "class" : "code-example-third" },
+        "Third"
+      ],
+      " Item"
+    ]
+  ]'
 let $options :=
   <options xmlns="http://www.zorba-xquery.com/modules/converters/json-options">
     <json-format value="JsonML-array"/>
   </options>
 return json:parse( $json, $options )
+
+(: vim:se et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-parse-05.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-parse-05.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-parse-05.xq	2012-12-03 18:32:22 +0000
@@ -64,3 +64,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:parse( $json, $options )
+
+(: vim:se et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-parse-06.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-parse-06.xq	2012-01-24 01:28:36 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-parse-06.xq	2012-12-03 18:32:22 +0000
@@ -7,4 +7,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:parse( <a/>, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-parse-wikipedia.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-parse-wikipedia.xq	2011-12-23 06:21:58 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-parse-wikipedia.xq	2012-12-03 18:32:22 +0000
@@ -20,4 +20,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:parse( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-01.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-01.xq	2011-12-23 06:21:58 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-01.xq	2012-12-03 18:32:22 +0000
@@ -7,4 +7,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:serialize( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-02.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-02.xq	2011-12-23 06:21:58 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-02.xq	2012-12-03 18:32:22 +0000
@@ -10,4 +10,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:serialize( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-03.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-03.xq	2011-12-23 06:21:58 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-03.xq	2012-12-03 18:32:22 +0000
@@ -7,4 +7,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:serialize( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-indent-wikipedia.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-indent-wikipedia.xq	2011-12-28 05:41:00 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-indent-wikipedia.xq	2012-12-03 18:32:22 +0000
@@ -17,4 +17,5 @@
     <whitespace value="indent"/>
   </options>
 return json:serialize( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-none-wikipedia.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-none-wikipedia.xq	2011-12-28 05:41:00 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-none-wikipedia.xq	2012-12-03 18:32:22 +0000
@@ -16,4 +16,5 @@
     <json-format value="JsonML-array"/>
   </options>
 return json:serialize( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-some-wikipedia.xq'
--- test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-some-wikipedia.xq	2011-12-28 05:41:00 +0000
+++ test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-some-wikipedia.xq	2012-12-03 18:32:22 +0000
@@ -17,4 +17,5 @@
     <whitespace value="some"/>
   </options>
 return json:serialize( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-array-01.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-array-01.xq	2012-09-19 21:16:15 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-array-01.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,5 @@
 
 let $json := '[ true ]'
 return json:parse( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-array-02.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-array-02.xq	2012-09-19 21:16:15 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-array-02.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,5 @@
 
 let $json := '[ false ]'
 return json:parse( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-array-03.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-array-03.xq	2012-09-19 21:16:15 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-array-03.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,5 @@
 
 let $json := '[ null ]'
 return json:parse( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-array-04.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-array-04.xq	2012-09-19 21:16:15 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-array-04.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,5 @@
 
 let $json := '[ 1 ]'
 return json:parse( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-array-05.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-array-05.xq	2012-09-19 21:16:15 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-array-05.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,5 @@
 
 let $json := '[ 1, 2 ]'
 return json:parse( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-array-07.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-array-07.xq	2012-01-26 01:47:19 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-array-07.xq	2012-12-03 18:32:22 +0000
@@ -3,3 +3,5 @@
 import module namespace json = "http://www.zorba-xquery.com/modules/converters/json";
 
 json:parse( '[ [ 11, 12 ], [ 21, 22 ] ]' )
+
+(: vim:se et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-empty.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-empty.xq	2012-01-24 01:28:36 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-empty.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,6 @@
 
 import module namespace json = "http://www.zorba-xquery.com/modules/converters/json";
 
-json:parse(<a/>)
+json:parse( <a/> )
+
+(: vim:se et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-example.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-example.xq	2012-09-19 21:16:15 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-example.xq	2012-12-03 18:32:22 +0000
@@ -17,4 +17,5 @@
   }
   '
 return json:parse( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== renamed file 'test/rbkt/Queries/zorba/json/json-snelson-numbers-and-decimals.xq' => 'test/rbkt/Queries/zorba/json/json-snelson-parse-numbers.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-numbers-and-decimals.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-numbers.xq	2012-12-03 18:32:22 +0000
@@ -1,10 +1,11 @@
-(: json:parse testing numbers and decimals :)
+(: json:parse testing numbers :)
 
 import module namespace json = "http://www.zorba-xquery.com/modules/converters/json";
 
-json:parse('{ "special-numbers": {
-         "decimal": 2.89,
-         "e": 2E+6,
-         "negative": -1.89
-      }
-    }')
+json:parse( '{
+  "decimal": 2.89,
+  "e": 2E+6,
+  "negative": -1.89
+}' )
+
+(: vim:se et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-object-01.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-object-01.xq	2012-09-19 21:16:15 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-object-01.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,5 @@
 
 let $json := '{ "a" : 1 }'
 return json:parse( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-object-02.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-object-02.xq	2012-09-19 21:16:15 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-object-02.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,5 @@
 
 let $json := '{ "a" : 1, "b" : 2 }'
 return json:parse( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== renamed file 'test/rbkt/Queries/zorba/json/json-snelson-null-handling.xq' => 'test/rbkt/Queries/zorba/json/json-snelson-parse-object-05.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-null-handling.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-object-05.xq	2012-12-03 18:32:22 +0000
@@ -1,5 +1,7 @@
-(: json:parse testing null handling :)
+(: json:parse testing null as a key value :)
 
 import module namespace json = "http://www.zorba-xquery.com/modules/converters/json";
 
-json:parse('{ "web-app": null }')
+json:parse( '{ "a" : null }' )
+
+(: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-parse-serialize.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-parse-serialize.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-serialize.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,6 @@
   <pair name="html" type="string">&lt;b&gt;bold&lt;/b&gt;</pair>
 </json>;
 
-json:parse(json:serialize($json-element))
+json:parse( json:serialize( $json-element ) )
+
+(: vim:se et sw=2 ts=2: :)

=== renamed file 'test/rbkt/Queries/zorba/json/json-snelson-uncommon-chars.xq' => 'test/rbkt/Queries/zorba/json/json-snelson-parse-uncommon-chars.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-uncommon-chars.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-uncommon-chars.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,6 @@
 
 import module namespace json = "http://www.zorba-xquery.com/modules/converters/json";
 
-json:parse('{ "web-app": "!_\"-\\?*.$+" }')
+json:parse( '{ "web-app" : "!_\"-\\?*.$+" }' )
+
+(: vim:se et sw=2 ts=2: :)

=== renamed file 'test/rbkt/Queries/zorba/json/json-snelson-utf-8.xq' => 'test/rbkt/Queries/zorba/json/json-snelson-parse-utf8-01.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-utf-8.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-parse-utf8-01.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,4 @@
 
 import module namespace json = "http://www.zorba-xquery.com/modules/converters/json";
 
-json:parse('{"mdash": "–"}')
+json:parse( '{ "mdash": "–" }' )

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-01.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-01.xq	2012-09-19 21:16:15 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-01.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <item type="boolean">true</item>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-02.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-02.xq	2011-12-22 04:01:56 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-02.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <item type="boolean">false</item>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-03.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-03.xq	2011-12-22 04:01:56 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-03.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <item type="null"/>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-04.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-04.xq	2011-12-22 04:01:56 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-04.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <item type="number">1</item>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-05.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-05.xq	2011-12-22 04:01:56 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-05.xq	2012-12-03 18:32:22 +0000
@@ -6,4 +6,5 @@
     <item type="number">2</item>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-06.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-06.xq	2011-12-22 04:56:34 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-06.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <foo type="boolean">true</foo>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-07.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-07.xq	2011-12-22 04:56:34 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-07.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <item type="boolean">foo</item>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-08.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-08.xq	2011-12-22 04:56:34 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-08.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <item>false</item>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-09.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-09.xq	2011-12-22 04:56:34 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-09.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <item type="bool">false</item>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-10.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-10.xq	2011-12-22 04:56:34 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-10.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     hello
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-11.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-11.xq	2011-12-22 04:56:34 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-11.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <item type="boolean"><foo/></item>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-array-12.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-array-12.xq	2012-01-26 04:43:56 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-array-12.xq	2012-12-03 18:32:22 +0000
@@ -8,4 +8,5 @@
     <item type="boolean">true</item>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-comment-node.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-comment-node.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-comment-node.xq	2012-12-03 18:32:22 +0000
@@ -2,4 +2,5 @@
 
 let $json := <!--comment--> 
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-indent-example.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-indent-example.xq	2011-12-28 05:41:00 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-indent-example.xq	2012-12-03 18:32:22 +0000
@@ -21,4 +21,5 @@
     <whitespace value="indent"/>
   </options>
 return json:serialize( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-invalid-value-for-attribute.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-invalid-value-for-attribute.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-invalid-value-for-attribute.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
   <pair name="a" type="nothing">a</pair>
 </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-none-example.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-none-example.xq	2011-12-28 05:41:00 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-none-example.xq	2012-12-03 18:32:22 +0000
@@ -16,4 +16,5 @@
     </pair>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-object-01.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-object-01.xq	2011-12-22 04:01:56 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-object-01.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
     <pair name="a" type="number">1</pair>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-object-02.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-object-02.xq	2011-12-22 04:01:56 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-object-02.xq	2012-12-03 18:32:22 +0000
@@ -6,4 +6,5 @@
     <pair name="b" type="number">2</pair>
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-object-03.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-object-03.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-object-03.xq	2012-12-03 18:32:22 +0000
@@ -4,9 +4,10 @@
 
 let $json :=
   <json type="object">
-  <!--comment-->
+    <!--comment-->
     <pair name="a" type="number">1</pair>
-  <!--comment-->
+    <!--comment-->
   </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-parse.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-parse.xq	2012-01-26 04:43:56 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-parse.xq	2012-12-03 18:32:22 +0000
@@ -3,3 +3,5 @@
 declare variable $json-value := '{ "key" : "value" }';
 
 json:serialize( json:parse( $json-value ) )
+
+(: vim:se et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-some-example.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-some-example.xq	2011-12-28 05:41:00 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-some-example.xq	2012-12-03 18:32:22 +0000
@@ -21,4 +21,5 @@
     <whitespace value="some"/>
   </options>
 return json:serialize( $json, $options )
+
 (: vim:set et sw=2 ts=2: :)

=== modified file 'test/rbkt/Queries/zorba/json/json-snelson-serialize-type-value-missing.xq'
--- test/rbkt/Queries/zorba/json/json-snelson-serialize-type-value-missing.xq	2012-01-23 23:22:52 +0000
+++ test/rbkt/Queries/zorba/json/json-snelson-serialize-type-value-missing.xq	2012-12-03 18:32:22 +0000
@@ -5,4 +5,5 @@
   <pair name="a">a</pair>
 </json>
 return json:serialize( $json )
+
 (: vim:set et sw=2 ts=2: :)

=== added file 'test/rbkt/Queries/zorba/jsoniq/parse_json-09.xq'
--- test/rbkt/Queries/zorba/jsoniq/parse_json-09.xq	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/jsoniq/parse_json-09.xq	2012-12-03 18:32:22 +0000
@@ -0,0 +1,5 @@
+let $json := '[ 1, 2 ]'
+let $options := { "jsoniq-strip-top-level-array" : true }
+return jn:parse-json( $json, $options )
+
+(: vim:set et sw=2 ts=2: :)

=== added file 'test/rbkt/Queries/zorba/jsoniq/parse_json-10.xq'
--- test/rbkt/Queries/zorba/jsoniq/parse_json-10.xq	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/jsoniq/parse_json-10.xq	2012-12-03 18:32:22 +0000
@@ -0,0 +1,8 @@
+let $json := '[ 1, 2 ][ 3, 4 ]'
+let $options := {
+  "jsoniq-multiple-top-level-items" : true,
+  "jsoniq-strip-top-level-array" : true
+}
+return jn:parse-json( $json, $options )
+
+(: vim:set et sw=2 ts=2: :)