Merge lp:~zorba-coders/zorba/feature-json_parser into lp:zorba

[Paul J. Lucas]

New JSON parser and module.

New JSON parser and module.
Fixed the missing quote in the documentation.
Added checking of stream state for manipulators.
All bugs fixed.
Yet another bug fixed.
I think I've finally got it.

[Sorin Marian Nasoi (sorin.marian.nasoi)]

I have resubmitted the merge proposal after committing the fix for the JSON tests.

[William Candillon (wcandillon)]

The documentation is missing some examples.
The example section at http://docs.basex.org/wiki/JSON_Module is a good place to get inspired.

Why A chars have a backslash in error code descriptions:
ZJSE0001 if \a $xml is not a document or element node.
ZJSE0002 if \a $xml contains an element that is missing a required attribute.
ZJSE0003 if \a $xml contains an attribute having an illegal value.
ZJSE0004 if \a $xml contains an illegal element.
ZJSE0005 if \a $xml contains an illegal child element for a JSON type.
ZJSE0006 if \a $xml contains an illegal child element.
ZJSE0007 if \a $xml contains an illegal text node.
ZJSE0008 if \a $xml contains an illegal value for a JSON type.

review: Needs Fixing

[Paul J. Lucas (paul-lucas)]

The \a is the doxygen way to say the following thing is an argument and should be italicized.

[William Candillon (wcandillon)]

Doesn't seem to work for xqdoc

[Paul J. Lucas (paul-lucas)]

review: Approve

[William Candillon (wcandillon)]

The module works great.
There is room for improvement regarding its documentation.
"There are many ways to represent JSON data in XML" -> There are two ways to represent JSON data in XML:
- bullet 1 (John Snelson's format)
- bullet 2 (JSON)
Then add two examples, you can take some inspiration at http://docs.basex.org/wiki/JSON_Module

For parse#1, add a small example.
For parse#2, add two small examples and some description of what is excepted for the option element (which namespace, example of schema instance instance)
For serialize#1, please elaborate on what the following means: "Serializes an XDM into JSON using one of the representations described above." and add two small examples.
For serialize#2, same comment than for serialize#1 + add some explanation of what is excepted for the option element.

review: Needs Fixing

[Paul J. Lucas (paul-lucas)]

No, the statement "There are many ways to represent JSON data in XML" is correct as it is. There *are* *many* ways: Zorba only implements *two* of those *many* ways.

Adding *two* examples is too much. API documentation isn't supposed to be the primary documentation.

[Matthias Brantner (matthias-brantner)]

I think William is right. We should have a couple of examples which help the user to get started with this module. Those could either be inline in the text or links generated using the @example xqdoc tag.

The serialize functions should be annotated %ann:streamable because they return a streamable string (see modules/com/zorba-xquery/www/modules/fetch.xq).

Some error codes lack a prefix (e.g. ZJPE0001). If I understood correctly, it should be zerr:ZJPE0001. Otherwise, the user doesn't know how to catch the error.

Under which circumstances does json:parse return the empty sequence? What does json:serialize return if the input is the empty sequence?

review: Needs Fixing

[Sorin Marian Nasoi (sorin.marian.nasoi)]

1) Should be possible to call json:serialize(json:parse(VALID_JSON))
where VALID_JSON is any valid JSON string
see failing test test/rbkt/zorba/json/json-snelson-serialize-parse
added bug lp:920717

2) array and object closed prematurely in json:parse
see failing tests test/rbkt/zorba/json/json-snelson-parse-array-06 and test/rbkt/zorba/json/json-snelson-parse-array-07
added bug lp:920719

3) When trying to parsing a valid JSON string with invalid JSON option parameter the error raised is:
http://www.w3.org/2005/xqt-errors:XPST0003
Please add a more useful error message: see failing test test/rbkt/zorba/json/json-invalid-option-parameter
added bug lp:920720

4) Parsing an empty value has wrong behavior for both JSON mappings (Snelson and JSON-ML).
Please see failing tests:
- test/rbkt/zorba/json/json-snelson-empty-value
- test/rbkt/zorba/json/json-jsonml-empty-value
added bug lp:920721

5) Comment nodes should be ignored by json:serialize
Please see failing tests:
- test/rbkt/zorba/json/json-snelson-serialize-object-03
- test/rbkt/zorba/json/json-snelson-serialize-array-12
added bug lp:920722

6) XQDoc issues:
- there is no @project tag for the new json.xq module: as a result, in the XQDoc documentation is generated in the www.zorba-xquery.com/modules/converters instead of data processing/data converters
Please add a @project data processing/data converters in the module description in order to fix this.

- first parse function does not state
zerr:ZJPE0006 as a possible error condition if the passes JSON string is invalid
See added test json-snelson-invalid-json.xq
Added bug lp:920724

review: Needs Fixing

[Sorin Marian Nasoi (sorin.marian.nasoi)]

r10618 contains all the tests mentioned above.

The tests are passing because they are marked as EXPECTED_FAILURES.

[Paul J. Lucas (paul-lucas)]

Why did you mark them as EXPECTED_FAILURE if they're not expected to fail? Presumably, you *want* them fixed, right?

[Chris Hillery (ceejatec)]

EXPECTED_FAILURE does *not* mean "negative test". It means "this is broken, we know it's broken, and we're working on it". That's why you have to supply a bug number to the macro. It probably should be renamed "KNOWN_BUG".

[Paul J. Lucas (paul-lucas)]

It means "known bug" for things that are in the trunk. If it's on a branch (as this is), then either (1) it will be fixed before it's merged into the trunk (at which point the EXPECTED_FAILURE will have to be removed since it will no longer be failing -- which begs my question of, "Why put it in in the first place?") or (2) the test will be deemed invalid and removed.

[Chris Hillery (ceejatec)]

Hmm... valid points. I think there's some value in using EXPECTED_FAILURE() anyway, since it documents the relationship to new bugs that are filed.

It does introduce the possibility of unintentionally merging a new bug onto the trunk, but the diff will clearly show a new EXPECTED_FAILURE() marker and I would hope that any reviewer would raise a serious question about that.

I guess I would say that it's probably unnecessary to add EXPECTED_FAILURE()s (and associated bugs) for small review comments. However, for anything which might reasonably take more than a few hours to fix, IMHO it's probably good practice just to help ensure that no issues get forgotten. In general I would leave that decision up to the team working on the branch in question, though.

[Sorin Marian Nasoi (sorin.marian.nasoi)]

> It means "known bug" for things that are in the trunk. If it's on a branch
> (as this is), then either (1) it will be fixed before it's merged into the
> trunk (at which point the EXPECTED_FAILURE will have to be removed since it
> will no longer be failing -- which begs my question of, "Why put it in in the
> first place?") or (2) the test will be deemed invalid and removed.

Paul, IMO:
- adding some bugs on Launchpad
- adding tests that fail and marking them as KNOWN_ISSUE (a.k.a. EXPECTED_FAILURES)
was far better than adding a comment in your merge proposal.

Keep in mind that I have spent my time in order to give you a hand in pointing out the issues I found.

Adding separate bugs allows us to:
- discussed the raised issues separately
- make commits to fix them separately into the branch

And to prove my point: you already wrote 2 comments *without* even looking over *any* of the issues I have raised.

Anyway, I already committed a fix in the branch for bug lp:920724.

[Paul J. Lucas (paul-lucas)]

Chris Hillery wrote: "I think there's some value in using EXPECTED_FAILURE() anyway, since it documents the relationship to new bugs that are filed."

The relationship to new bugs *from* _______?

[Chris Hillery (ceejatec)]

Documents the relationship of the newly-added failing tests to the bugs tracking those issues.

[Paul J. Lucas (paul-lucas)]

IMO, you didn't *need* to add any comment to the merge proposal other than "Bugs filed" -- I can read the bugs myself.

Sorin wrote: "Keep in mind that I have spent my time in order to give you a hand in pointing out the issues I found."

It's all of our job to review each others' code under the "New Order" so please don't act like it was a personal favor.

I never questioned your adding separate bugs. For the record, I *like* separate bugs. I *only* questioned why you added EXPECTED_FAILURE for those bugs that, as I've pointed out, is unnecessary and not only creates more work for you (because you put them in) but more work for me (because I have to take them out).

Sorin wrote: "And to prove my point: you already wrote 2 comments *without* even looking over *any* of the issues I have raised."

*My* point has *nothing* to do with any bugs in particular. Chris understands my point. Do you?

[Paul J. Lucas (paul-lucas)]

@Chris: Hmmm... again, because this is a branch and no Zorba user will ever see this stuff, it's at best only marginally useful.

[Sorin Marian Nasoi (sorin.marian.nasoi)]

Paul, let me ask you something: are you sure you are not missing the point?

IMHO the point is to fix the issues that were raised ASAP.

I have spent my time in order to give as many details as possible in order to fix them ASAP.
Also I have committed a fix in the branch for one of the opened bugs.

Please look over the bugs and let's try to fix them in order to commit the new improved JSON 2.0 module to the trunk.

If you think that the decision I took while reviewing your merge deserves further discussion let's put it on the next weekly conference agenda.

[Sorin Marian Nasoi (sorin.marian.nasoi)]

> IMO, you didn't *need* to add any comment to the merge proposal other than
> "Bugs filed" -- I can read the bugs myself.
FYI: Matthias asked me (in a separate email discussion) to add a small review of the issues I fount in the merge proposal: please take this issue with him.

> Sorin wrote: "Keep in mind that I have spent my time in order to give you a
> hand in pointing out the issues I found."
>
> It's all of our job to review each others' code under the "New Order" so
> please don't act like it was a personal favor.
OK, next time I will review your code I will simply add a one sentence in the merge proposal and keep adding comments because you clearly think this is better.

> I never questioned your adding separate bugs. For the record, I *like*
> separate bugs. I *only* questioned why you added EXPECTED_FAILURE for those
> bugs that, as I've pointed out, is unnecessary and not only creates more work
> for you (because you put them in) but more work for me (because I have to take
> them out).
I will never do this again.

> Sorin wrote: "And to prove my point: you already wrote 2 comments *without*
> even looking over *any* of the issues I have raised."
>
> *My* point has *nothing* to do with any bugs in particular. Chris understands
> my point. Do you?
No, I do not. IMHO you are missing the point, that is: let's try to fix the *bugs* and discuss the procedures later.
Do you understand my point?

[Paul J. Lucas (paul-lucas)]

No, the point is *not* to fix this issues ASAP. The feature is not a high-priority feature, so there's no reason to do this ASAP. My *only* point was why you added EXPECTED_FAILURE lines to the CMakeLists.txt file -- that's it.

But it's moot now since I've removed all the new EXPECTED_FAILURES that you've added. They're of no use to me. Now when I run the test suite and it reports 100% tests passed, I can be confident that I've fixed all the bugs and not have an errant EXPECTED_FAILURE in there.

In the future, please don't add EXPECTED_FAILURE lines to code on a *branch* -- thanks.

[Sorin Marian Nasoi (sorin.marian.nasoi)]

> No, the point is *not* to fix this issues ASAP. The feature is not a high-
> priority feature, so there's no reason to do this ASAP. My *only* point was
> why you added EXPECTED_FAILURE lines to the CMakeLists.txt file -- that's it.
>
> But it's moot now since I've removed all the new EXPECTED_FAILURES that you've
> added. They're of no use to me. Now when I run the test suite and it reports
> 100% tests passed, I can be confident that I've fixed all the bugs and not
> have an errant EXPECTED_FAILURE in there.
>
> In the future, please don't add EXPECTED_FAILURE lines to code on a *branch*
> -- thanks.
My 2 cents: no matter what the brief description of the "ctest -R SOME_TEST" shows, one should *always* check out the details inside the ctest logs.

http://en.wikipedia.org/wiki/The_Devil_is_in_the_details

[Paul J. Lucas (paul-lucas)]

review: Approve

[William Candillon (wcandillon)]

I believe that bug #920717 isn't fixed.

The intro has two examples of XML.
For each of the XML, you should add the associated JSON.

Would it be possible to get all the bug reports that Sorin created associated to this merge request?

review: Needs Fixing

[Paul J. Lucas (paul-lucas)]

review: Approve

[Paul J. Lucas (paul-lucas)]

It was an unrelated bug -- fixed.

[William Candillon (wcandillon)]

This merge request is pending on bug #920717

Also there are some documentation issues that have not been addressed yet.
How do you we do this?
Do you make another shot at it first or we should have call with Matthias to discuss how to review it?

review: Needs Fixing

[William Candillon (wcandillon)]

Do you take another shot at it first or should we have a call with Matthias to discuss how to improve the module documentation?*

[Paul J. Lucas (paul-lucas)]

If you want the documentation fixed, you need to specify exactly what needs fixing -- I'm not a mind-reader.

[William Candillon (wcandillon)]

"Zorba implements that proposed by John Snelson." -> is that correct english? I'm not sure.
For John Snelson representation example, please add the corresponding JSON and XQuery function call above.
For the JSONML example, please add the corresponding JSON and XQuery function call above.

I think that this fix will dramatically improve the time needed for someone to get started with the module.

In parse#1, add a small example.
"Returns: said XDM instance." is that correct?

In parse#2, add a small example.
Add a description of how to build element(json-options:options)

In serialize#1, add a small example.

In serialize#2, add a small example.
Add a description of how to build element(json-options:options)

[William Candillon (wcandillon)]

I have reopened bug #920717.
This merge is still pending on the resolution of this bug.

[Paul J. Lucas (paul-lucas)]

> "Zorba implements that proposed by John Snelson." -> is that correct english?
> I'm not sure.

Yes.

> In parse#1, add a small example.
> "Returns: said XDM instance." is that correct?

Yes.

> In parse#2, add a small example.
> Add a description of how to build element(json-options:options)

I would except XQDoc is broken and renders it very badly.

> In serialize#1, add a small example.
> In serialize#2, add a small example.

It's just the reverse of what's above!

[William Candillon (wcandillon)]

If the documentation is correct in the source, I will fix everything else.

[Paul J. Lucas (paul-lucas)]

review: Approve

[William Candillon (wcandillon)]

Still pending on bug #930573

review: Needs Fixing

[William Candillon (wcandillon)]

I approve the fix for bug #930573 and opened #932186 which depends on this merge proposal.

[William Candillon (wcandillon)]

The intro is great!

There are examples missing for each function description.

Also I would like to understand the best practice for documenting XML parameters.
Currently it's pretty confusing.
Matthias what's your take on it?

[Paul J. Lucas (paul-lucas)]

As far as I can tell, I've already given examples. Since you're so picky and only you seem to know what you want, why don't you just add the documentation you want yourself?

[William Candillon (wcandillon)]

If you give/write a good overview of the possible options for parse/serialize I can definitely do that. Right know I have no clue of what's possible or not which is gonna the case of our users too.

[Paul J. Lucas (paul-lucas)]

Why can't you just look at the json-options schema? There are only two options.

[Sorin Marian Nasoi (sorin.marian.nasoi)]

The reported bugs were fixed, the documentation also looks good.

As far as William's last remark, IMO building the XQDoc documentation and looking at the 'json-options' schema solves the problem.

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-json_parser-2012-02-15T17-27-26.321Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/feature-json_parser into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job feature-json_parser-2012-02-15T17-27-26.321Z is
  finished. The final status was:

  41 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[lp:~zorba-coders/zorba/feature-json_parser]

10650. By Paul J. Lucas on 2012-02-16

Removed data-converters tag.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/feature-json_parser-2012-02-16T00-25-35.097Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job feature-json_parser-2012-02-16T00-25-35.097Z is finished. The final status was:

All tests succeeded!

=== modified file 'ChangeLog'
--- ChangeLog	2012-02-15 10:25:02 +0000
+++ ChangeLog	2012-02-16 00:23:20 +0000
@@ -41,6 +41,7 @@
   * Fixed bug #918211 (xqueryx fulltext w3c conformance generation)
   * Fixed bug #918157 (Add XQFTTS to validation queue)
   * Fixed bug with unversioned modules with C++ external functions
+  * Fixed bug #878508 (JSON Module not escaping escape characters)
 
 version 2.1
 

=== modified file 'include/zorba/diagnostic.h'
--- include/zorba/diagnostic.h	2011-07-01 16:07:54 +0000
+++ include/zorba/diagnostic.h	2012-02-16 00:23:20 +0000
@@ -239,7 +239,10 @@
   ZORBA_DEBUGGER,                       // Zorba Debugger
   ZORBA_OS,                             // Operating System
   ZORBA_SERIALIZATION,
-  ZORBA_STORE
+  ZORBA_STORE,
+
+  JSON_PARSER,
+  JSON_SERIALIZATION
 };
 
 /**

=== modified file 'include/zorba/internal/ztd.h'
--- include/zorba/internal/ztd.h	2011-08-23 13:32:16 +0000
+++ include/zorba/internal/ztd.h	2012-02-16 00:23:20 +0000
@@ -355,7 +355,8 @@
 template<typename T> inline
 typename std::enable_if<ZORBA_TR1_NS::is_pointer<T>::value,std::string>::type
 to_string( T p ) {
-  return p ? to_string( *p ) : "<null>";
+  typedef typename ZORBA_TR1_NS::remove_pointer<T>::type const* T_const_ptr;
+  return p ? to_string( *static_cast<T_const_ptr>( p ) ) : "<null>";
 }
 
 /**

=== modified file 'include/zorba/pregenerated/diagnostic_list.h'
--- include/zorba/pregenerated/diagnostic_list.h	2011-12-21 14:40:33 +0000
+++ include/zorba/pregenerated/diagnostic_list.h	2012-02-16 00:23:20 +0000
@@ -746,6 +746,40 @@
 
 extern ZORBA_DLL_PUBLIC ZorbaErrorCode XSST0010;
 
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJPE0001_ILLEGAL_CHARACTER;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJPE0002_ILLEGAL_CODEPOINT;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJPE0003_ILLEGAL_ESCAPE;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJPE0004_ILLEGAL_LITERAL;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJPE0005_ILLEGAL_NUMBER;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJPE0006_UNEXPECTED_TOKEN;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJPE0007_UNTERMINATED_STRING;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJPE0008_ILLEGAL_QNAME;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJPE0009_ILLEGAL_EMPTY_STRING;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJSE0001_NOT_DOCUMENT_OR_ELEMENT_NODE;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJSE0002_ELEMENT_MISSING_ATTRIBUTE;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJSE0003_BAD_ATTRIBUTE_VALUE;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJSE0004_BAD_ELEMENT;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJSE0005_BAD_CHILD_ELEMENT;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJSE0006_NO_ELEMENT_CHILD;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJSE0007_NO_TEXT_CHILD;
+
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZJSE0008_BAD_VALUE;
+
 } // namespace zerr
 
 namespace zwarn {

=== modified file 'modules/ExternalModules.conf'
--- modules/ExternalModules.conf	2012-02-15 10:25:02 +0000
+++ modules/ExternalModules.conf	2012-02-16 00:23:20 +0000
@@ -27,7 +27,7 @@
 #       currently only works for bzr, since svn tags are just different URLS)
 
 data-cleaning   bzr  lp:zorba/data-cleaning-module       zorba-2.1
-data-converters bzr  lp:zorba/data-converters-module     zorba-2.1
+data-converters bzr  lp:zorba/data-converters-module     
 data-formatting bzr  lp:zorba/data-formatting-module     zorba-2.1
 email           bzr  lp:zorba/email-module               zorba-2.1
 excel           bzr  lp:zorba/excel-module               zorba-2.1

=== modified file 'modules/com/zorba-xquery/www/modules/CMakeLists.txt'
--- modules/com/zorba-xquery/www/modules/CMakeLists.txt	2012-02-15 10:25:02 +0000
+++ modules/com/zorba-xquery/www/modules/CMakeLists.txt	2012-02-16 00:23:20 +0000
@@ -71,6 +71,10 @@
 # Subdirectories
 DECLARE_ZORBA_MODULE(FILE converters/base64.xq VERSION 2.0
   URI "http://www.zorba-xquery.com/modules/converters/base64")
+DECLARE_ZORBA_MODULE(FILE converters/json.xq VERSION 2.0
+  URI "http://www.zorba-xquery.com/modules/converters/json")
+DECLARE_ZORBA_SCHEMA(FILE converters/json-options.xsd
+  URI "http://www.zorba-xquery.com/modules/converters/json-options")
 DECLARE_ZORBA_MODULE(FILE introspection/sctx.xq VERSION 2.0
   URI "http://www.zorba-xquery.com/modules/introspection/sctx")
 DECLARE_ZORBA_MODULE(FILE xqdoc2xhtml/error.xq VERSION 2.0

=== added file 'modules/com/zorba-xquery/www/modules/converters/json-options.xsd'
--- modules/com/zorba-xquery/www/modules/converters/json-options.xsd	1970-01-01 00:00:00 +0000
+++ modules/com/zorba-xquery/www/modules/converters/json-options.xsd	2012-02-16 00:23:20 +0000
@@ -0,0 +1,60 @@
+<!--
+ ! Copyright 2006-2008 The FLWOR Foundation.
+ !
+ ! Licensed under the Apache License, Version 2.0 (the "License");
+ ! you may not use this file except in compliance with the License.
+ ! You may obtain a copy of the License at
+ !
+ ! http://www.apache.org/licenses/LICENSE-2.0
+ !
+ ! Unless required by applicable law or agreed to in writing, software
+ ! distributed under the License is distributed on an "AS IS" BASIS,
+ ! WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ! See the License for the specific language governing permissions and
+ ! limitations under the License.
+-->
+
+<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
+  xmlns:json="http://www.zorba-xquery.com/modules/converters/json-options"
+  targetNamespace="http://www.zorba-xquery.com/modules/converters/json-options"
+  elementFormDefault="qualified"
+  attributeFormDefault="unqualified">
+
+  <xs:element name="options">
+    <xs:complexType>
+      <xs:all>
+
+        <xs:element name="json-format" minOccurs="1" maxOccurs="1">
+          <xs:complexType>
+            <xs:attribute name="value" use="required">
+              <xs:simpleType>
+                <xs:restriction base="xs:string">
+                  <xs:enumeration value="Snelson"/>
+                  <xs:enumeration value="JsonML-array"/>
+                  <xs:enumeration value="JsonML-object"/>
+                </xs:restriction>
+              </xs:simpleType>
+            </xs:attribute>
+          </xs:complexType>
+        </xs:element>
+
+        <xs:element name="whitespace" minOccurs="0" maxOccurs="1">
+          <xs:complexType>
+            <xs:attribute name="value" use="required">
+              <xs:simpleType>
+                <xs:restriction base="xs:string">
+                  <xs:enumeration value="none"/>
+                  <xs:enumeration value="some"/>
+                  <xs:enumeration value="indent"/>
+                </xs:restriction>
+              </xs:simpleType>
+            </xs:attribute>
+          </xs:complexType>
+        </xs:element>
+
+      </xs:all>
+    </xs:complexType>
+  </xs:element>
+</xs:schema>
+
+<!-- vim:set et sw=2 ts=2: -->

=== added file 'modules/com/zorba-xquery/www/modules/converters/json.xq'
--- modules/com/zorba-xquery/www/modules/converters/json.xq	1970-01-01 00:00:00 +0000
+++ modules/com/zorba-xquery/www/modules/converters/json.xq	2012-02-16 00:23:20 +0000
@@ -0,0 +1,260 @@
+(:
+ : Copyright 2006-2009 The FLWOR Foundation.
+ :
+ : Licensed under the Apache License, Version 2.0 (the "License");
+ : you may not use this file except in compliance with the License.
+ : You may obtain a copy of the License at
+ :
+ : http://www.apache.org/licenses/LICENSE-2.0
+ :
+ : Unless required by applicable law or agreed to in writing, software
+ : distributed under the License is distributed on an "AS IS" BASIS,
+ : WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ : See the License for the specific language governing permissions and
+ : limitations under the License.
+ :)
+
+xquery version "3.0";
+
+ (:~
+  : Using this module, you can parse JSON data into XML, manipulate it like any
+  : other XML data using XQuery, and serialize the result back as JSON.
+  :
+  : There are many ways to represent JSON data in XML, some loss-less ("round
+  : tripable") and some lossy ("one way").  Loss-less representations preserve
+  : the JSON data types <i>boolean</i>, <i>number</i>, and <i>null</i>; lossy
+  : representations convert all data to strings.
+  : <p/>
+  : For a loss-less representation, Zorba implements that proposed by
+  : <a href="http://john.snelson.org.uk/parsing-json-into-xquery">John Snelson</a>.
+  : For example:
+  : <pre>
+  :   {
+  :     "firstName" : "John",
+  :     "lastName" : "Smith",
+  :     "address" : {
+  :       "streetAddress" : "21 2nd Street",
+  :       "city" : "New York",
+  :       "state" : "NY",
+  :       "postalCode" : 10021
+  :     },
+  :     "phoneNumbers" : [ "212 732-1234", "646 123-4567" ]
+  :   }
+  : </pre>
+  : would be represented as:
+  : <pre>
+  :   &lt;json type="object"&gt;
+  :     &lt;pair name="firstName" type="string"&gt;John&lt;/pair&gt;
+  :     &lt;pair name="lastName" type="string"&gt;Smith&lt;/pair&gt;
+  :     &lt;pair name="address" type="object"&gt;
+  :       &lt;pair name="streetAddress" type="string"&gt;21 2nd Street&lt;/pair&gt;
+  :       &lt;pair name="city" type="string"&gt;New York&lt;/pair&gt;
+  :       &lt;pair name="state" type="string"&gt;NY&lt;/pair&gt;
+  :       &lt;pair name="postalCode" type="number"&gt;10021&lt;/pair&gt;
+  :     &lt;/pair&gt;
+  :     &lt;pair name="phoneNumbers" type="array"&gt;
+  :       &lt;item type="string"&gt;212 732-1234&lt;/item&gt;
+  :       &lt;item type="string"&gt;646 123-4567&lt;/item&gt;
+  :     &lt;/pair&gt;
+  :   &lt;/json&gt;
+  : </pre>
+  : For a lossy representation, Zorba implements
+  : <a href="http://jsonml.org/">JsonML</a> (the array form).
+  : For example:
+  : <pre>
+  :   [ "person",
+  :     { "created" : "2006-11-11T19:23",
+  :       "modified" : "2006-12-31T23:59" },
+  :     [ "firstName", "Robert" ],
+  :     [ "lastName", "Smith" ],
+  :     [ "address",
+  :       { "type" : "home" },
+  :       [ "street", "12345 Sixth Ave" ],
+  :       [ "city", "Anytown" ],
+  :       [ "state", "CA" ],
+  :       [ "postalCode", "98765-4321" ]
+  :     ]
+  :   ]
+  : </pre>
+  : would be represented as:
+  : <pre>
+  :   &lt;person created="2006-11-11T19:23" modified="2006-12-31T23:59"&gt;
+  :     &lt;firstName&gt;Robert&lt;/firstName&gt;
+  :     &lt;lastName&gt;Smith&lt;/lastName&gt;
+  :     &lt;address type="home"&gt;
+  :       &lt;street&gt;12345 Sixth Ave&lt;/street&gt;
+  :       &lt;city&gt;Anytown&lt;/city&gt;
+  :       &lt;state&gt;CA&lt;/state&gt;
+  :       &lt;postalCode&gt;98765-4321&lt;/postalCode&gt;
+  :     &lt;/address&gt;
+  :   &lt;/person&gt;
+  : </pre>
+  :
+  : @author Paul J. Lucas
+  : @project data processing/data converters
+  :)
+module namespace json = "http://www.zorba-xquery.com/modules/converters/json";
+
+import module namespace schema = "http://www.zorba-xquery.com/modules/schema";
+
+import schema namespace json-options =
+  "http://www.zorba-xquery.com/modules/converters/json-options";
+
+declare namespace ann = "http://www.zorba-xquery.com/annotations";
+declare namespace err = "http://www.w3.org/2005/xqt-errors";
+declare namespace zerr = "http://www.zorba-xquery.com/errors";
+
+declare namespace ver = "http://www.zorba-xquery.com/options/versioning";
+declare option ver:module-version "2.0";
+
+(:~
+ : Parses JSON data from a string and returns an XDM instance using one of the
+ : representations described above.
+ :
+ : @param $json The JSON data to parse.
+ : @param $options The parsing options, for example:
+ : <pre>
+ : &lt;options xmlns="http://www.zorba-xquery.com/modules/converters/json-options"&gt;
+ :   &lt;json-format value="JsonML-array"/&gt;
+ : &lt;/options&gt;
+ : </pre>
+ : @return said XDM instance.
+ : @error err:XQDY0027 if $options can not be validated against the
+ : json-options schema.
+ : @error zerr:ZJPE0001 if $json contains an illegal JSON character.
+ : @error zerr:ZJPE0002 if $json contains an illegal Unicode code-point.
+ : @error zerr:ZJPE0003 if $json contains an illegal JSON character escape.
+ : @error zerr:ZJPE0004 if $json contains an illegal JSON literal.
+ : @error zerr:ZJPE0005 if $json contains an illegal JSON number.
+ : @error zerr:ZJPE0006 if $json is not a valid JSON string.
+ : @error zerr:ZJPE0007 if $json contains an unterminated string.
+ : @error zerr:ZJPE0008 if $json contains an illegal QName.
+ : @example test/rbkt/Queries/zorba/json/json-jsonml_array-parse-01.xq
+ :)
+declare function json:parse(
+  $json as xs:string?,
+  $options as element(json-options:options)
+) as element(*,xs:untyped)*
+{
+  let $validated-options := if ( schema:is-validated( $options ) ) then
+                              $options
+                            else
+                              validate { $options }
+  return json:parse-internal( $json, $validated-options )
+};
+
+(:~
+ : Parses JSON data from a string and returns an XDM instance using the Snelson
+ : representation described above.
+ :
+ : @param $json The JSON data to parse.
+ : @return said XDM instance.
+ : @error zerr:ZJPE0001 if $json contains an illegal JSON character.
+ : @error zerr:ZJPE0002 if $json contains an illegal Unicode code-point.
+ : @error zerr:ZJPE0003 if $json contains an illegal JSON character escape.
+ : @error zerr:ZJPE0004 if $json contains an illegal JSON literal.
+ : @error zerr:ZJPE0005 if $json contains an illegal JSON number.
+ : @error zerr:ZJPE0006 if $json is not a valid JSON string.
+ : @error zerr:ZJPE0007 if $json contains an unterminated string.
+ : @error zerr:ZJPE0008 if $json contains an illegal QName.
+ : @example test/rbkt/Queries/zorba/json/json-snelson-parse-array-01.xq
+ :)
+declare function json:parse(
+  $json as xs:string?
+) as element(*,xs:untyped)*
+{
+  json:parse-internal(
+    $json,
+    validate {
+      <options
+          xmlns="http://www.zorba-xquery.com/modules/converters/json-options">
+        <json-format value="Snelson"/>
+      </options>
+    }
+  )
+};
+
+(:~
+ : Serializes an XDM into JSON using one of the representations described
+ : above.
+ :
+ : @param $xml The XDM to serialize.
+ : @param $options The serializing options, for example:
+ : <pre>
+ : &lt;options xmlns="http://www.zorba-xquery.com/modules/converters/json-options"&gt;
+ :   &lt;json-format value="JsonML-array"/&gt;
+ :   &lt;whitespace value="indent"/&gt;
+ : &lt;/options&gt;
+ : </pre>
+ : @return a JSON string.
+ : @error err:XQDY0027 if $options can not be validated against the
+ : json-options schema.
+ : @error zerr:ZJSE0001 if $xml is not a document or element node.
+ : @error zerr:ZJSE0002 if $xml contains an element that is missing a required
+ : attribute.
+ : @error zerr:ZJSE0003 if $xml contains an attribute having an illegal value.
+ : @error zerr:ZJSE0004 if $xml contains an illegal element.
+ : @error zerr:ZJSE0005 if $xml contains an illegal child element for a JSON
+ : type.
+ : @error zerr:ZJSE0006 if $xml contains an illegal child element.
+ : @error zerr:ZJSE0007 if $xml contains an illegal text node.
+ : @error zerr:ZJSE0008 if $xml contains an illegal value for a JSON type.
+ : @example test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-01.xq
+ :)
+declare function json:serialize(
+  $xml as item()*,
+  $options as element(json-options:options)
+) as xs:string
+{
+  let $validated-options := if ( schema:is-validated( $options ) ) then
+                              $options
+                            else
+                              validate { $options }
+  return json:serialize-internal( $xml, $validated-options )
+};
+
+(:~
+ : Serializes an XDM into JSON using one of the representations described
+ : above.
+ :
+ : @param $xml The XDM to serialize.
+ : @return a JSON string.
+ : @error zerr:ZJSE0001 if $xml is not a document or element node.
+ : @error zerr:ZJSE0002 if $xml contains an element that is missing a required
+ : attribute.
+ : @error zerr:ZJSE0003 if $xml contains an attribute having an illegal value.
+ : @error zerr:ZJSE0004 if $xml contains an illegal element.
+ : @error zerr:ZJSE0005 if $xml contains an illegal child element for a JSON
+ : type.
+ : @error zerr:ZJSE0006 if $xml contains an illegal child element.
+ : @error zerr:ZJSE0007 if $xml contains an illegal text node.
+ : @error zerr:ZJSE0008 if $xml contains an illegal value for a JSON type.
+ : @example test/rbkt/Queries/zorba/json/json-snelson-serialize-array-01.xq
+ :)
+declare function json:serialize(
+  $xml as item()*
+) as xs:string
+{
+  json:serialize-internal($xml,
+    validate {
+      <options
+          xmlns="http://www.zorba-xquery.com/modules/converters/json-options">
+        <json-format value="Snelson"/>
+      </options>
+   }
+ )
+};
+
+(:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::)
+
+declare %private function json:parse-internal(
+  $json as xs:string?,
+  $options as item()?
+) as element()* external;
+
+declare %ann:streamable %private function json:serialize-internal(
+  $xml as item()*,
+  $options as item()?
+) as xs:string external;
+
+(: vim:set et sw=2 ts=2: :)

=== modified file 'modules/com/zorba-xquery/www/modules/pregenerated/errors.xq'
--- modules/com/zorba-xquery/www/modules/pregenerated/errors.xq	2011-12-21 14:40:33 +0000
+++ modules/com/zorba-xquery/www/modules/pregenerated/errors.xq	2012-02-16 00:23:20 +0000
@@ -784,4 +784,72 @@
 
 (:~
 :)
-declare variable $zerr:XSST0010 as xs:QName := fn:QName($zerr:NS, "zerr:XSST0010");
\ No newline at end of file
+declare variable $zerr:XSST0010 as xs:QName := fn:QName($zerr:NS, "zerr:XSST0010");
+
+(:~
+:)
+declare variable $zerr:ZJPE0001 as xs:QName := fn:QName($zerr:NS, "zerr:ZJPE0001");
+
+(:~
+:)
+declare variable $zerr:ZJPE0002 as xs:QName := fn:QName($zerr:NS, "zerr:ZJPE0002");
+
+(:~
+:)
+declare variable $zerr:ZJPE0003 as xs:QName := fn:QName($zerr:NS, "zerr:ZJPE0003");
+
+(:~
+:)
+declare variable $zerr:ZJPE0004 as xs:QName := fn:QName($zerr:NS, "zerr:ZJPE0004");
+
+(:~
+:)
+declare variable $zerr:ZJPE0005 as xs:QName := fn:QName($zerr:NS, "zerr:ZJPE0005");
+
+(:~
+:)
+declare variable $zerr:ZJPE0006 as xs:QName := fn:QName($zerr:NS, "zerr:ZJPE0006");
+
+(:~
+:)
+declare variable $zerr:ZJPE0007 as xs:QName := fn:QName($zerr:NS, "zerr:ZJPE0007");
+
+(:~
+:)
+declare variable $zerr:ZJPE0008 as xs:QName := fn:QName($zerr:NS, "zerr:ZJPE0008");
+
+(:~
+:)
+declare variable $zerr:ZJPE0009 as xs:QName := fn:QName($zerr:NS, "zerr:ZJPE0009");
+
+(:~
+:)
+declare variable $zerr:ZJSE0001 as xs:QName := fn:QName($zerr:NS, "zerr:ZJSE0001");
+
+(:~
+:)
+declare variable $zerr:ZJSE0002 as xs:QName := fn:QName($zerr:NS, "zerr:ZJSE0002");
+
+(:~
+:)
+declare variable $zerr:ZJSE0003 as xs:QName := fn:QName($zerr:NS, "zerr:ZJSE0003");
+
+(:~
+:)
+declare variable $zerr:ZJSE0004 as xs:QName := fn:QName($zerr:NS, "zerr:ZJSE0004");
+
+(:~
+:)
+declare variable $zerr:ZJSE0005 as xs:QName := fn:QName($zerr:NS, "zerr:ZJSE0005");
+
+(:~
+:)
+declare variable $zerr:ZJSE0006 as xs:QName := fn:QName($zerr:NS, "zerr:ZJSE0006");
+
+(:~
+:)
+declare variable $zerr:ZJSE0007 as xs:QName := fn:QName($zerr:NS, "zerr:ZJSE0007");
+
+(:~
+:)
+declare variable $zerr:ZJSE0008 as xs:QName := fn:QName($zerr:NS, "zerr:ZJSE0008");
\ No newline at end of file

=== modified file 'src/context/static_context.cpp'
--- src/context/static_context.cpp	2012-02-15 10:25:02 +0000
+++ src/context/static_context.cpp	2012-02-16 00:23:20 +0000
@@ -280,6 +280,11 @@
 static_context::ZORBA_BASE64_FN_NS = 
 "http://www.zorba-xquery.com/modules/converters/base64";
 
+
+const char*
+static_context::ZORBA_JSON_FN_NS =
+"http://www.zorba-xquery.com/modules/converters/json";
+
 const char*
 static_context::ZORBA_NODEREF_FN_NS = 
 "http://www.zorba-xquery.com/modules/node-reference";
@@ -436,6 +441,7 @@
             ns == ZORBA_REFLECTION_FN_NS ||
             ns == ZORBA_SCRIPTING_FN_NS ||
             ns == ZORBA_STRING_FN_NS ||
+            ns == ZORBA_JSON_FN_NS ||
             ns == ZORBA_FETCH_FN_NS ||
             ns == ZORBA_NODE_FN_NS ||
             ns == ZORBA_XML_FN_NS);
@@ -482,6 +488,7 @@
   {
     return (ns == ZORBA_MATH_FN_NS ||
             ns == ZORBA_INTROSP_SCTX_FN_NS ||
+            ns == ZORBA_JSON_FN_NS ||
             ns == ZORBA_RANDOM_FN_NS);
   }
 

=== modified file 'src/context/static_context.h'
--- src/context/static_context.h	2012-02-02 09:56:52 +0000
+++ src/context/static_context.h	2012-02-16 00:23:20 +0000
@@ -446,6 +446,7 @@
   // Namespaces of external modules declaring zorba builtin functions
   static const char* ZORBA_MATH_FN_NS;
   static const char* ZORBA_BASE64_FN_NS;
+  static const char* ZORBA_JSON_FN_NS;
   static const char* ZORBA_NODEREF_FN_NS;
   static const char* ZORBA_NODEPOS_FN_NS;
   static const char* ZORBA_STORE_DYNAMIC_COLLECTIONS_DDL_FN_NS;

=== modified file 'src/diagnostics/diagnostic.cpp'
--- src/diagnostics/diagnostic.cpp	2011-07-11 21:01:20 +0000
+++ src/diagnostics/diagnostic.cpp	2012-02-16 00:23:20 +0000
@@ -134,6 +134,9 @@
     case ZORBA_STORE         : o << "Zorba store"           ; break;
     case ZORBA_XQP           : o << "Zorba"                 ; break;
 
+    case JSON_PARSER         : o << "JSON parser"           ; break;
+    case JSON_SERIALIZATION  : o << "JSON serialization"    ; break;
+
     default                  : /* suppresses warning */       break;
   }
   return o;

=== modified file 'src/diagnostics/diagnostic_en.xml'
--- src/diagnostics/diagnostic_en.xml	2012-02-15 10:25:02 +0000
+++ src/diagnostics/diagnostic_en.xml	2012-02-16 00:23:20 +0000
@@ -2312,6 +2312,63 @@
       <value>"continue loop" statement not inside while statement</value>
     </diagnostic>
 
+   <!--////////// JSON Parse Errors ////////////////////////////////////////-->
+
+    <diagnostic code="ZJPE0001" name="ILLEGAL_CHARACTER">
+      <value>'$1': illegal JSON character</value>
+    </diagnostic>
+    <diagnostic code="ZJPE0002" name="ILLEGAL_CODEPOINT">
+      <value>"$1": illegal Unicode code-point</value>
+    </diagnostic>
+    <diagnostic code="ZJPE0003" name="ILLEGAL_ESCAPE">
+      <value>'\\$1': illegal JSON character escape</value>
+    </diagnostic>
+    <diagnostic code="ZJPE0004" name="ILLEGAL_LITERAL">
+      <value>illegal JSON literal</value>
+    </diagnostic>
+    <diagnostic code="ZJPE0005" name="ILLEGAL_NUMBER">
+      <value>illegal JSON number</value>
+    </diagnostic>
+    <diagnostic code="ZJPE0006" name="UNEXPECTED_TOKEN">
+      <value>"$1": unexpected JSON token</value>
+    </diagnostic>
+    <diagnostic code="ZJPE0007" name="UNTERMINATED_STRING">
+      <value>unterminated JSON string</value>
+    </diagnostic>
+    <diagnostic code="ZJPE0008" name="ILLEGAL_QNAME">
+      <value>"$1": illegal QName</value>
+    </diagnostic>
+    <diagnostic code="ZJPE0009" name="ILLEGAL_EMPTY_STRING">
+      <value>illegal empty string</value>
+    </diagnostic>
+
+   <!--////////// JSON Serialization Errors ////////////////////////////////-->
+
+    <diagnostic code="ZJSE0001" name="NOT_DOCUMENT_OR_ELEMENT_NODE">
+      <value>JSON serialization requires document or element node</value>
+    </diagnostic>
+    <diagnostic code="ZJSE0002" name="ELEMENT_MISSING_ATTRIBUTE">
+      <value>"$1" element missing required "$2" attribute</value>
+    </diagnostic>
+    <diagnostic code="ZJSE0003" name="BAD_ATTRIBUTE_VALUE">
+      <value>"$1": illegal value for attribute "$2"</value>
+    </diagnostic>
+    <diagnostic code="ZJSE0004" name="BAD_ELEMENT">
+      <value>"$1": illegal element${; must be "2"}${ or "3"}</value>
+    </diagnostic>
+    <diagnostic code="ZJSE0005" name="BAD_CHILD_ELEMENT">
+      <value>"$1": illegal child element of "$2" type; must be "$3"</value>
+    </diagnostic>
+    <diagnostic code="ZJSE0006" name="NO_ELEMENT_CHILD">
+      <value>JSON type "$1" can not have a child element node</value>
+    </diagnostic>
+    <diagnostic code="ZJSE0007" name="NO_TEXT_CHILD">
+      <value>JSON type "$1" can not have a child text node</value>
+    </diagnostic>
+    <diagnostic code="ZJSE0008" name="BAD_VALUE">
+      <value>"$1": illegal value for JSON type "$2"</value>
+    </diagnostic>
+
   </namespace>
 
   <!--////////// Zorba Warnings ////////////////////////////////////////////-->
@@ -3269,6 +3326,14 @@
       <value>Zorba warning</value>
     </entry>
 
+    <entry key="JSON parser error">
+      <value>JSON parser error</value>
+    </entry>
+
+    <entry key="JSON serialization error">
+      <value>JSON serialization error</value>
+    </entry>
+
     <entry key="dynamic error">
       <value>dynamic error</value>
     </entry>

=== modified file 'src/diagnostics/pregenerated/diagnostic_list.cpp'
--- src/diagnostics/pregenerated/diagnostic_list.cpp	2011-12-21 14:40:33 +0000
+++ src/diagnostics/pregenerated/diagnostic_list.cpp	2012-02-16 00:23:20 +0000
@@ -1097,6 +1097,57 @@
 ZorbaErrorCode XSST0010( "XSST0010" );
 
 
+ZorbaErrorCode ZJPE0001_ILLEGAL_CHARACTER( "ZJPE0001" );
+
+
+ZorbaErrorCode ZJPE0002_ILLEGAL_CODEPOINT( "ZJPE0002" );
+
+
+ZorbaErrorCode ZJPE0003_ILLEGAL_ESCAPE( "ZJPE0003" );
+
+
+ZorbaErrorCode ZJPE0004_ILLEGAL_LITERAL( "ZJPE0004" );
+
+
+ZorbaErrorCode ZJPE0005_ILLEGAL_NUMBER( "ZJPE0005" );
+
+
+ZorbaErrorCode ZJPE0006_UNEXPECTED_TOKEN( "ZJPE0006" );
+
+
+ZorbaErrorCode ZJPE0007_UNTERMINATED_STRING( "ZJPE0007" );
+
+
+ZorbaErrorCode ZJPE0008_ILLEGAL_QNAME( "ZJPE0008" );
+
+
+ZorbaErrorCode ZJPE0009_ILLEGAL_EMPTY_STRING( "ZJPE0009" );
+
+
+ZorbaErrorCode ZJSE0001_NOT_DOCUMENT_OR_ELEMENT_NODE( "ZJSE0001" );
+
+
+ZorbaErrorCode ZJSE0002_ELEMENT_MISSING_ATTRIBUTE( "ZJSE0002" );
+
+
+ZorbaErrorCode ZJSE0003_BAD_ATTRIBUTE_VALUE( "ZJSE0003" );
+
+
+ZorbaErrorCode ZJSE0004_BAD_ELEMENT( "ZJSE0004" );
+
+
+ZorbaErrorCode ZJSE0005_BAD_CHILD_ELEMENT( "ZJSE0005" );
+
+
+ZorbaErrorCode ZJSE0006_NO_ELEMENT_CHILD( "ZJSE0006" );
+
+
+ZorbaErrorCode ZJSE0007_NO_TEXT_CHILD( "ZJSE0007" );
+
+
+ZorbaErrorCode ZJSE0008_BAD_VALUE( "ZJSE0008" );
+
+
 } // namespace zerr
 
 namespace zwarn {

=== modified file 'src/diagnostics/pregenerated/dict_en.cpp'
--- src/diagnostics/pregenerated/dict_en.cpp	2012-02-15 10:25:02 +0000
+++ src/diagnostics/pregenerated/dict_en.cpp	2012-02-16 00:23:20 +0000
@@ -332,6 +332,23 @@
 #if defined(ZORBA_WITH_DEBUGGER)
   { "ZGDB0001", "" },
 #endif
+  { "ZJPE0001", "'$1': illegal JSON character" },
+  { "ZJPE0002", "\"$1\": illegal Unicode code-point" },
+  { "ZJPE0003", "'\\$1': illegal JSON character escape" },
+  { "ZJPE0004", "illegal JSON literal" },
+  { "ZJPE0005", "illegal JSON number" },
+  { "ZJPE0006", "\"$1\": unexpected JSON token" },
+  { "ZJPE0007", "unterminated JSON string" },
+  { "ZJPE0008", "\"$1\": illegal QName" },
+  { "ZJPE0009", "illegal empty string" },
+  { "ZJSE0001", "JSON serialization requires document or element node" },
+  { "ZJSE0002", "\"$1\" element missing required \"$2\" attribute" },
+  { "ZJSE0003", "\"$1\": illegal value for attribute \"$2\"" },
+  { "ZJSE0004", "\"$1\": illegal element${; must be \"2\"}${ or \"3\"}" },
+  { "ZJSE0005", "\"$1\": illegal child element of \"$2\" type; must be \"$3\"" },
+  { "ZJSE0006", "JSON type \"$1\" can not have a child element node" },
+  { "ZJSE0007", "JSON type \"$1\" can not have a child text node" },
+  { "ZJSE0008", "\"$1\": illegal value for JSON type \"$2\"" },
   { "ZOSE0001", "\"$1\": file not found" },
   { "ZOSE0002", "\"$1\": not plain file" },
   { "ZOSE0003", "stream read failure" },
@@ -494,6 +511,8 @@
   { "~HexBinaryMustBeEven", "HexBinary value must contain an even number of characters" },
   { "~IncompleteKeyInIndexBuild", "incomplete key during index build" },
   { "~IncompleteKeyInIndexRefresh", "incomplete key during index refresh" },
+  { "~JSON parser error", "JSON parser error" },
+  { "~JSON serialization error", "JSON serialization error" },
   { "~LibModVersionMismatch_3", "XQuery library version can not be imported by a $3 version module" },
   { "~ModuleDeclNotInMain", "module declaration must not be in main module" },
   { "~ModuleNotFound", "module not found" },

=== modified file 'src/diagnostics/qname.cpp'
--- src/diagnostics/qname.cpp	2011-07-01 16:07:54 +0000
+++ src/diagnostics/qname.cpp	2012-02-16 00:23:20 +0000
@@ -79,9 +79,15 @@
     case 'C': return ZORBA_SERIALIZATION;
     case 'D': return ZORBA_DDF;
     case 'G': return ZORBA_DEBUGGER;
+    case 'J': switch ( name[2] ) {
+                case 'P': return JSON_PARSER;
+                case 'S': return JSON_SERIALIZATION;
+                default : ZORBA_ASSERT( false );
+              }
     case 'O': return ZORBA_OS;
     case 'S': return ZORBA_STORE;
     case 'X': return ZORBA_XQP;
+
     default : ZORBA_ASSERT( false );
   }
 }

=== modified file 'src/functions/library.cpp'
--- src/functions/library.cpp	2011-10-14 07:35:51 +0000
+++ src/functions/library.cpp	2012-02-16 00:23:20 +0000
@@ -57,6 +57,7 @@
 #include "functions/func_sequences.h"
 #include "functions/func_sequences_impl.h"
 #include "functions/func_strings.h"
+#include "functions/func_json.h"
 #include "functions/func_var_decl.h"
 #include "functions/func_xqdoc.h"
 #include "functions/func_documents.h"
@@ -107,6 +108,7 @@
   populate_context_fnput(sctx);
   populate_context_index_ddl(sctx);
   populate_context_ic_ddl(sctx);
+  populate_context_json(sctx);
   populate_context_maths(sctx);
   populate_context_nodes(sctx);
   populate_context_node_position(sctx);

=== added file 'src/functions/pregenerated/func_json.cpp'
--- src/functions/pregenerated/func_json.cpp	1970-01-01 00:00:00 +0000
+++ src/functions/pregenerated/func_json.cpp	2012-02-16 00:23:20 +0000
@@ -0,0 +1,87 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+ 
+// ******************************************
+// *                                        *
+// * THIS IS A GENERATED FILE. DO NOT EDIT! *
+// * SEE .xml FILE WITH SAME NAME           *
+// *                                        *
+// ******************************************
+
+
+#include "stdafx.h"
+#include "runtime/json/json.h"
+#include "functions/func_json.h"
+
+
+namespace zorba{
+
+
+
+PlanIter_t fn_zorba_json_parse_internal::codegen(
+  CompilerCB*,
+  static_context* sctx,
+  const QueryLoc& loc,
+  std::vector<PlanIter_t>& argv,
+  AnnotationHolder& ann) const
+{
+  return new JSONParseInternal(sctx, loc, argv);
+}
+
+PlanIter_t fn_zorba_json_serialize_internal::codegen(
+  CompilerCB*,
+  static_context* sctx,
+  const QueryLoc& loc,
+  std::vector<PlanIter_t>& argv,
+  AnnotationHolder& ann) const
+{
+  return new JSONSerializeInternal(sctx, loc, argv);
+}
+
+void populate_context_json(static_context* sctx)
+{
+  {
+    
+
+    DECL_WITH_KIND(sctx, fn_zorba_json_parse_internal,
+        (createQName("http://www.zorba-xquery.com/modules/converters/json","","parse-internal"), 
+        GENV_TYPESYSTEM.STRING_TYPE_ONE, 
+        GENV_TYPESYSTEM.ITEM_TYPE_QUESTION, 
+        GENV_TYPESYSTEM.ELEMENT_TYPE_STAR),
+        FunctionConsts::FN_ZORBA_JSON_PARSE_INTERNAL_2);
+
+  }
+
+
+  {
+    
+
+    DECL_WITH_KIND(sctx, fn_zorba_json_serialize_internal,
+        (createQName("http://www.zorba-xquery.com/modules/converters/json","","serialize-internal"), 
+        GENV_TYPESYSTEM.ITEM_TYPE_STAR, 
+        GENV_TYPESYSTEM.ITEM_TYPE_QUESTION, 
+        GENV_TYPESYSTEM.STRING_TYPE_ONE),
+        FunctionConsts::FN_ZORBA_JSON_SERIALIZE_INTERNAL_2);
+
+  }
+
+}
+
+
+}
+
+
+

=== added file 'src/functions/pregenerated/func_json.h'
--- src/functions/pregenerated/func_json.h	1970-01-01 00:00:00 +0000
+++ src/functions/pregenerated/func_json.h	2012-02-16 00:23:20 +0000
@@ -0,0 +1,79 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+ 
+// ******************************************
+// *                                        *
+// * THIS IS A GENERATED FILE. DO NOT EDIT! *
+// * SEE .xml FILE WITH SAME NAME           *
+// *                                        *
+// ******************************************
+
+
+#ifndef ZORBA_FUNCTIONS_JSON_H
+#define ZORBA_FUNCTIONS_JSON_H
+
+
+#include "common/shared_types.h"
+#include "functions/function_impl.h"
+
+
+namespace zorba {
+
+
+void populate_context_json(static_context* sctx);
+
+
+
+
+//fn-zorba-json:parse-internal
+class fn_zorba_json_parse_internal : public function
+{
+public:
+  fn_zorba_json_parse_internal(const signature& sig, FunctionConsts::FunctionKind kind)
+    : 
+    function(sig, kind)
+  {
+
+  }
+
+  CODEGEN_DECL();
+};
+
+
+//fn-zorba-json:serialize-internal
+class fn_zorba_json_serialize_internal : public function
+{
+public:
+  fn_zorba_json_serialize_internal(const signature& sig, FunctionConsts::FunctionKind kind)
+    : 
+    function(sig, kind)
+  {
+
+  }
+
+  CODEGEN_DECL();
+};
+
+
+} //namespace zorba
+
+
+#endif
+/*
+ * Local variables:
+ * mode: c++
+ * End:
+ */ 

=== modified file 'src/functions/pregenerated/function_enum.h'
--- src/functions/pregenerated/function_enum.h	2012-01-11 17:30:25 +0000
+++ src/functions/pregenerated/function_enum.h	2012-02-16 00:23:20 +0000
@@ -166,6 +166,8 @@
   FN_ZORBA_INTROSPECT_SCTX_IN_SCOPE_ATTRIBUTE_GROUPS_0,
   FN_ZORBA_INTROSPECT_SCTX_OPTION_1,
   FN_ZORBA_INTROSPECT_SCTX_FUNCTION_ANNOTATIONS_2,
+  FN_ZORBA_JSON_PARSE_INTERNAL_2,
+  FN_ZORBA_JSON_SERIALIZE_INTERNAL_2,
   MATH_SQRT_1,
   MATH_EXP_1,
   MATH_EXP10_1,

=== modified file 'src/runtime/CMakeLists.txt'
--- src/runtime/CMakeLists.txt	2011-06-01 13:16:28 +0000
+++ src/runtime/CMakeLists.txt	2012-02-16 00:23:20 +0000
@@ -115,6 +115,9 @@
   durations_dates_times/DurationsDatesTimesImpl.cpp
   indexing/doc_indexer.cpp
   indexing/index_ddl.cpp
+  json/common.cpp
+  json/jsonml_array.cpp
+  json/snelson.cpp
   numerics/NumericsImpl.cpp
   numerics/format_integer_impl.cpp
   sequences/SequencesImpl.cpp

=== modified file 'src/runtime/full_text/ft_match.cpp'
--- src/runtime/full_text/ft_match.cpp	2011-06-14 17:26:33 +0000
+++ src/runtime/full_text/ft_match.cpp	2012-02-16 00:23:20 +0000
@@ -31,7 +31,7 @@
   return o << "0x" << hex << reinterpret_cast<unsigned long>( obj ) << dec;
 }
 
-DEF_OMANIP1( print_addr, void const*, obj )
+DEF_OMANIP1( print_addr, void const* )
 
 ostream& operator<<( ostream &o, ft_string_match const &sm ) {
   return  o << "{SM: "
@@ -52,8 +52,7 @@
   return o;
 }
 
-DEF_OMANIP2( print_string_matches, char const*, label,
-             ft_string_matches const&, sms )
+DEF_OMANIP2( print_string_matches, char const*, ft_string_matches const& )
 
 ostream& operator<<( ostream &o, ft_match const &m ) {
   return  o << indent << "ft_match @ " << print_addr( &m ) << '\n'

=== added directory 'src/runtime/json'
=== added file 'src/runtime/json/common.cpp'
--- src/runtime/json/common.cpp	1970-01-01 00:00:00 +0000
+++ src/runtime/json/common.cpp	2012-02-16 00:23:20 +0000
@@ -0,0 +1,62 @@
+/*
+ * Copyright 2006-2011 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "stdafx.h"
+
+#include "store/api/iterator.h"
+
+#include "common.h"
+
+using namespace std;
+
+namespace zorba {
+
+///////////////////////////////////////////////////////////////////////////////
+
+bool get_attribute_value( store::Item_t const &element, char const *att_name,
+                          zstring *att_value ) {
+  store::Iterator_t i( element->getAttributes() );
+  bool found = false;
+  i->open();
+  store::Item_t att_item;
+  while ( i->next( att_item ) ) {
+    if ( att_item->getNodeName()->getStringValue() == att_name ) {
+      att_item->getStringValue2( *att_value );
+      found = true;
+      break;
+    }
+  }
+  i->close();
+  return found;
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+#if ZORBA_DEBUG_JSON
+
+ostream& operator<<( ostream &o, parse_state s ) {
+  static char const *const string_of[] = {
+    "in_array",
+    "in_object"
+  };
+  return o << string_of[ s ];
+}
+
+#endif /* ZORBA_DEBUG_JSON */
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace zorba
+/* vim:set et sw=2 ts=2: */

=== added file 'src/runtime/json/common.h'
--- src/runtime/json/common.h	1970-01-01 00:00:00 +0000
+++ src/runtime/json/common.h	2012-02-16 00:23:20 +0000
@@ -0,0 +1,120 @@
+/*
+ * Copyright 2006-2011 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef ZORBA_RUNTIME_JSON_COMMON_H
+#define ZORBA_RUNTIME_JSON_COMMON_H
+
+#include <iostream>
+#include <stack>
+
+#include "store/api/item.h"
+#include "store/api/item_factory.h"
+#include "util/indent.h"
+#include "util/omanip.h"
+#include "zorbatypes/zstring.h"
+
+#define ZORBA_DEBUG_JSON 0
+
+namespace zorba {
+
+///////////////////////////////////////////////////////////////////////////////
+
+typedef std::stack<store::Item*> item_stack_type;
+
+enum parse_state {
+  in_array,
+  in_object
+};
+
+typedef std::stack<int> state_stack_type;
+
+namespace whitespace {
+  enum type {
+    none,
+    some,
+    indent
+  };
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+bool get_attribute_value( store::Item_t const &element, char const *att_name,
+                          zstring *att_value );
+
+typedef std::ostream& (*std_omanip_type)(std::ostream&);
+
+inline std::ostream& if_do( std::ostream &o, bool expr, std_omanip_type fn ) {
+  if ( expr )
+    o << fn;
+  return o;
+}
+DEF_OMANIP2( if_do, bool, std_omanip_type )
+
+#define if_indent(WS,FN) if_do( (WS) == whitespace::indent, FN )
+
+inline std::ostream& if_emit( std::ostream &o, bool expr, char c ) {
+  if ( expr )
+    o << c;
+  return o;
+}
+DEF_OMANIP2( if_emit, bool, char )
+
+///////////////////////////////////////////////////////////////////////////////
+
+#define IN_STATE(S) ztd::top_stack_equals( state_stack, (S) )
+
+#if ZORBA_DEBUG_JSON
+
+std::ostream& operator<<( std::ostream &o, parse_state s );
+
+# define PUSH_ITEM(I)                                                     \
+    do {                                                                  \
+      cout << __LINE__ << ":PUSH_ITEM( " << (I)->show() << " )" << endl;  \
+      item_stack.push( (I).getp() );                                      \
+    } while (0)
+
+# define POP_ITEM()                               \
+    do {                                          \
+      cout << __LINE__ << ":POP_ITEM()" << endl;  \
+      cur_item = ztd::pop_stack( item_stack );    \
+    } while (0)
+
+# define PUSH_STATE(S) \
+    do {                                                          \
+      cout << __LINE__ << ":PUSH_STATE( " << (S) << " )" << endl; \
+      state_stack.push( S );                                      \
+    } while (0)
+
+# define POP_STATE()                              \
+    do {                                          \
+      cout << __LINE__ << ":POP_STATE()" << endl; \
+      state_stack.pop();                          \
+    } while (0)                                   \
+
+#else
+
+# define PUSH_ITEM(I)   item_stack.push( (I).getp() )
+# define POP_ITEM()     cur_item = ztd::pop_stack( item_stack )
+# define PUSH_STATE(S)  state_stack.push( S )
+# define POP_STATE()    state_stack.pop()
+
+#endif /* ZORBA_DEBUG_JSON */
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace zorba
+#endif /* ZORBA_RUNTIME_JSON_COMMON_H */
+/* vim:set et sw=2 ts=2: */

=== added file 'src/runtime/json/json_impl.cpp'
--- src/runtime/json/json_impl.cpp	1970-01-01 00:00:00 +0000
+++ src/runtime/json/json_impl.cpp	2012-02-16 00:23:20 +0000
@@ -0,0 +1,217 @@
+/*
+ * Copyright 2006-2011 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "stdafx.h"
+
+#include <map>
+#include <sstream>
+
+#include <zorba/diagnostic_list.h>
+
+#include "runtime/json/json.h"
+#include "store/api/item_factory.h"
+#include "system/globalenv.h"
+#include "util/mem_streambuf.h"
+
+#include "jsonml_array.h"
+#include "snelson.h"
+
+using namespace std;
+
+namespace zorba {
+
+///////////////////////////////////////////////////////////////////////////////
+
+typedef map<zstring,zstring> options_type;
+
+static void get_options( store::Item_t const &options_element,
+                         options_type *options ) {
+  ZORBA_ASSERT( options_element->getNodeKind() ==
+    store::StoreConsts::elementNode );
+  store::Iterator_t i = options_element->getChildren();
+  i->open();
+  store::Item_t option_item;
+  while ( i->next( option_item ) ) {
+    if ( option_item->getNodeKind() == store::StoreConsts::elementNode ) {
+      zstring const name( option_item->getNodeName()->getStringValue() );
+      zstring value;
+      get_attribute_value( option_item, "value", &value );
+      (*options)[ name ] = value;
+    }
+  }
+  i->close();
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+bool JSONParseInternal::nextImpl( store::Item_t& result,
+                                  PlanState &planState ) const {
+  store::Item_t cur_item;
+  options_type options;
+  istringstream iss;
+  mem_streambuf buf;
+
+  PlanIteratorState *state;
+  DEFAULT_STACK_INIT( PlanIteratorState, state, planState );
+
+  ZORBA_ASSERT( theChildren.size() == 2 );
+  consumeNext( cur_item, theChildren[1], planState );
+  get_options( cur_item, &options );
+
+  consumeNext( cur_item, theChildren[0], planState );
+  result = nullptr;
+
+  istream *is;
+  if ( cur_item->isStreamable() ) {
+    is = &cur_item->getStream();
+  } else {
+    zstring s;
+    cur_item->getStringValue2( s );
+    // Doing it this way uses the string data in-place with no copy.
+    buf.set( s.data(), s.size() );
+    iss.ios::rdbuf( &buf );
+    is = &iss;
+  }
+
+  try {
+    json::parser p( *is );
+    p.set_loc(
+      loc.getFilename().c_str(), loc.getLineBegin(), loc.getColumnBegin()
+    );
+
+    options_type::mapped_type const &format = options[ "json-format" ];
+    ZORBA_ASSERT( !format.empty() );
+    if ( format == "Snelson" )
+      snelson::parse( p, &result );
+    else if ( format == "JsonML-array" )
+      jsonml_array::parse( p, &result );
+    else
+      ZORBA_ASSERT( false );
+  }
+  catch ( json::illegal_character const &e ) {
+    throw XQUERY_EXCEPTION(
+      zerr::ZJPE0001_ILLEGAL_CHARACTER,
+      ERROR_PARAMS( e.get_char() ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::illegal_codepoint const &e ) {
+    throw XQUERY_EXCEPTION(
+      zerr::ZJPE0002_ILLEGAL_CODEPOINT,
+      ERROR_PARAMS( e.get_codepoint() ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::illegal_escape const &e ) {
+    throw XQUERY_EXCEPTION(
+      zerr::ZJPE0003_ILLEGAL_ESCAPE,
+      ERROR_PARAMS( e.get_escape() ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::illegal_literal const &e ) {
+    throw XQUERY_EXCEPTION(
+      zerr::ZJPE0004_ILLEGAL_LITERAL,
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::illegal_number const &e ) {
+    throw XQUERY_EXCEPTION(
+      zerr::ZJPE0005_ILLEGAL_NUMBER,
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::unexpected_token const &e ) {
+    throw XQUERY_EXCEPTION(
+      zerr::ZJPE0006_UNEXPECTED_TOKEN,
+      ERROR_PARAMS( e.get_token() ),
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+  catch ( json::unterminated_string const &e ) {
+    throw XQUERY_EXCEPTION(
+      zerr::ZJPE0007_UNTERMINATED_STRING,
+      ERROR_LOC( e.get_loc() )
+    );
+  }
+
+  STACK_PUSH( !!result, state );
+  STACK_END( state );
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+bool JSONSerializeInternal::nextImpl( store::Item_t& result,
+                                      PlanState &planState ) const {
+  store::Item_t cur_item;
+  options_type options;
+
+  PlanIteratorState *state;
+  DEFAULT_STACK_INIT( PlanIteratorState, state, planState );
+
+  ZORBA_ASSERT( theChildren.size() == 2 );
+  consumeNext( cur_item, theChildren[1], planState );
+  get_options( cur_item, &options );
+
+  consumeNext( cur_item, theChildren[0], planState );
+  try {
+    options_type::mapped_type const &format_opt = options[ "json-format" ];
+    ZORBA_ASSERT( !format_opt.empty() );
+
+    whitespace::type ws;
+    options_type::mapped_type const &whitespace_opt = options[ "whitespace" ];
+    if ( whitespace_opt.empty() || whitespace_opt == "none" )
+      ws = whitespace::none;
+    else if ( whitespace_opt == "some" )
+      ws = whitespace::some;
+    else if ( whitespace_opt == "indent" )
+      ws = whitespace::indent;
+    else
+      ZORBA_ASSERT( false );
+
+    ostringstream oss;
+    switch ( cur_item->getNodeKind() ) {
+      case store::StoreConsts::documentNode:
+      case store::StoreConsts::elementNode:
+        if ( format_opt == "Snelson" )
+          snelson::serialize( oss, cur_item, ws );
+        else if ( format_opt == "JsonML-array" )
+          jsonml_array::serialize( oss, cur_item, ws );
+        else
+          ZORBA_ASSERT( false );
+        break;
+      default:
+        throw XQUERY_EXCEPTION(
+          zerr::ZJSE0001_NOT_DOCUMENT_OR_ELEMENT_NODE,
+          ERROR_LOC( loc )
+        );
+    }
+    // This string copying is inefficient, but I can't see another way.
+    zstring temp( oss.str() );
+    GENV_ITEMFACTORY->createString( result, temp );
+  }
+  catch ( ZorbaException &e ) {
+    set_source( e, loc );
+    throw;
+  }
+
+  STACK_PUSH( !!result, state );
+  STACK_END( state );
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace zorba
+/* vim:set et sw=2 ts=2: */

=== added file 'src/runtime/json/jsonml_array.cpp'
--- src/runtime/json/jsonml_array.cpp	1970-01-01 00:00:00 +0000
+++ src/runtime/json/jsonml_array.cpp	2012-02-16 00:23:20 +0000
@@ -0,0 +1,285 @@
+/*
+ * Copyright 2006-2011 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "stdafx.h"
+
+#include <sstream>
+
+#include <zorba/diagnostic_list.h>
+
+#include "runtime/json/json.h"
+#include "store/api/item_factory.h"
+#include "system/globalenv.h"
+#include "types/root_typemanager.h"
+#include "util/ascii_util.h"
+#include "util/cxx_util.h"
+#include "util/json_parser.h"
+#include "util/mem_streambuf.h"
+#include "util/omanip.h"
+#include "util/oseparator.h"
+#include "util/stl_util.h"
+
+#include "jsonml_array.h"
+
+using namespace std;
+
+namespace zorba {
+
+///////////////////////////////////////////////////////////////////////////////
+
+static void split_name( zstring const &name, zstring *prefix, zstring *local ) {
+  zstring::size_type const colon = name.find( ':' );
+  if ( colon != zstring::npos ) {
+    *prefix = name.substr( 0, colon );
+    *local = name.substr( colon + 1 );
+    if ( prefix->empty() || local->empty() )
+      throw XQUERY_EXCEPTION(
+        zerr::ZJPE0008_ILLEGAL_QNAME,
+        ERROR_PARAMS( name )
+      );
+  } else {
+    prefix->clear();
+    *local = name;
+  }
+}
+
+namespace expect {
+  enum type {
+    none,
+    element_name,
+    attribute_name,
+    attribute_value
+  };
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+namespace jsonml_array {
+
+void parse( json::parser &p, store::Item_t *result ) {
+  ZORBA_ASSERT( result );
+
+  state_stack_type state_stack;
+
+  store::Item_t cur_item, junk_item, value_item;
+  store::Item_t att_name, element_name, type_name;
+
+  zstring base_uri;
+  bool got_something = false;
+  item_stack_type item_stack;
+  expect::type expect_what = expect::none;
+  store::NsBindings ns_bindings;
+  zstring value;
+
+  json::token token;
+  while ( p.next( &token ) ) {
+    got_something = true;
+    switch ( token.get_type() ) {
+
+      case '[':
+        PUSH_STATE( in_array );
+        expect_what = expect::element_name;
+        break;
+
+      case '{':
+        PUSH_STATE( in_object );
+        expect_what = expect::attribute_name;
+        break;
+
+      case ']':
+        POP_ITEM();
+        // no break;
+      case '}':
+        POP_STATE();
+        expect_what = expect::none;
+        break;
+
+      case ',':
+        expect_what = IN_STATE( in_object ) ?
+          expect::attribute_name : expect::none;
+        break;
+
+      case ':':
+        expect_what = expect::attribute_value;
+        break;
+
+      case json::token::number:
+      case 'F':
+      case 'T':
+      case json::token::json_null:
+      case json::token::string: {
+        value = token.get_value();
+        zstring prefix, local;
+        switch ( expect_what ) {
+          case expect::element_name:
+            split_name( value, &prefix, &local );
+            GENV_ITEMFACTORY->createQName( element_name, "", prefix, local );
+            type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+            GENV_ITEMFACTORY->createElementNode(
+              cur_item,
+              item_stack.empty() ? nullptr : item_stack.top(),
+              element_name, type_name, false, false, ns_bindings, base_uri
+            );
+            PUSH_ITEM( cur_item );
+            if ( !*result )
+              *result = cur_item;
+            break;
+          case expect::attribute_name:
+            split_name( value, &prefix, &local );
+            GENV_ITEMFACTORY->createQName( att_name, "", prefix, local );
+            break;
+          case expect::attribute_value:
+            type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+            GENV_ITEMFACTORY->createString( value_item, value );
+            GENV_ITEMFACTORY->createAttributeNode(
+              junk_item, cur_item, att_name, type_name, value_item
+            );
+            break;
+          case expect::none:
+            GENV_ITEMFACTORY->createTextNode( junk_item, cur_item, value );
+            break;
+        }
+        break;
+      }
+
+      case json::token::none:
+        break;
+
+      default:
+        assert( false );
+    } // switch
+  } // while
+  if ( !got_something )
+    throw XQUERY_EXCEPTION( zerr::ZJPE0009_ILLEGAL_EMPTY_STRING );
+}
+
+} // namespace jsonml_array
+
+///////////////////////////////////////////////////////////////////////////////
+
+static ostream& serialize_attributes( ostream &o, store::Item_t const &element,
+                                      oseparator &sep, whitespace::type ws ) {
+  bool emitted_attributes = false;
+  oseparator att_sep;
+  switch ( ws ) {
+    case whitespace::none  : att_sep.sep( ","   ); break;
+    case whitespace::some  : att_sep.sep( ", "  ); break;
+    case whitespace::indent: att_sep.sep( ",\n" ); break;
+  }
+
+  store::Iterator_t i( element->getAttributes() );
+  i->open();
+  store::Item_t att_item;
+  while ( i->next( att_item ) ) {
+    zstring const att_name( att_item->getNodeName()->getStringValue() );
+    if ( att_name == "xmlns" )
+      continue;
+    if ( !emitted_attributes ) {
+      o << sep
+        << if_emit( ws == whitespace::indent, '\n' )
+        << if_indent( ws, indent ) << '{'
+        << if_indent( ws, inc_indent );
+      emitted_attributes = true;
+    }
+    bool const was_printing = att_sep.printing();
+    o << att_sep;
+    if ( was_printing )
+      o << if_indent( ws, indent );
+    else
+      o << if_emit( ws, ' ' );
+    
+    o << '"' << att_name << '"'
+      << if_emit( ws, ' ' ) << ':' << if_emit( ws, ' ' )
+      << '"' << att_item->getStringValue() << '"';
+  }
+  i->close();
+  if ( emitted_attributes )
+    o << if_emit( ws, ' ' ) << '}' << if_indent( ws, dec_indent );
+  return o;
+}
+DEF_OMANIP3( serialize_attributes, store::Item_t const&, oseparator&,
+             whitespace::type )
+
+static ostream& serialize_children( ostream&, store::Item_t const &parent,
+                                    oseparator&, whitespace::type );
+DEF_OMANIP3( serialize_children, store::Item_t const&, oseparator&,
+             whitespace::type )
+
+static ostream& serialize_element( ostream &o, store::Item_t const &element,
+                                   oseparator &sep, whitespace::type ws ) {
+  if ( sep.printing() )
+    o << if_emit( ws == whitespace::indent, '\n' );
+  sep.printing( true );
+  o << if_indent( ws, indent ) << '[' << if_emit( ws, ' ' )
+    << '"' << element->getNodeName()->getStringValue() << '"'
+    << if_indent( ws, inc_indent )
+    << serialize_attributes( element, sep, ws )
+    << serialize_children( element, sep, ws )
+    << if_emit( ws, ' ' ) << ']'
+    << if_indent( ws, dec_indent );
+  return o;
+}
+DEF_OMANIP3( serialize_element, store::Item_t const&, oseparator&,
+             whitespace::type )
+
+static ostream& serialize_children( ostream &o, store::Item_t const &parent,
+                                    oseparator &sep, whitespace::type ws ) {
+  store::Iterator_t i( parent->getChildren() );
+  i->open();
+  store::Item_t child;
+  while ( i->next( child ) ) {
+    switch ( child->getNodeKind() ) {
+      case store::StoreConsts::elementNode:
+        o << sep << serialize_element( child, sep, ws );
+        break;
+      case store::StoreConsts::textNode:
+        o << sep << '"' << child->getStringValue() << '"';
+        break;
+      default:
+        break;
+    }
+  }
+  i->close();
+  return o;
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+namespace jsonml_array {
+
+void serialize( ostream &o, store::Item_t const &item, whitespace::type ws ) {
+  oseparator sep;
+  if ( ws )
+    sep.sep( ", " );
+  else
+    sep.sep( "," );
+  switch ( item->getNodeKind() ) {
+    case store::StoreConsts::documentNode:
+      o << serialize_children( item, sep, ws );
+      break;
+    case store::StoreConsts::elementNode:
+      o << serialize_element( item, sep, ws );
+      break;
+    default:
+      throw XQUERY_EXCEPTION( zerr::ZJSE0001_NOT_DOCUMENT_OR_ELEMENT_NODE );
+  }
+}
+
+} // namespace jsonml_array
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace zorba
+/* vim:set et sw=2 ts=2: */

=== added file 'src/runtime/json/jsonml_array.h'
--- src/runtime/json/jsonml_array.h	1970-01-01 00:00:00 +0000
+++ src/runtime/json/jsonml_array.h	2012-02-16 00:23:20 +0000
@@ -0,0 +1,41 @@
+/*
+ * Copyright 2006-2011 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "stdafx.h"
+
+#ifndef ZORBA_RUNTIME_JSON_JSONML_ARRAY_H
+#define ZORBA_RUNTIME_JSON_JSONML_ARRAY_H
+
+#include <iostream>
+
+#include "store/api/item.h"
+#include "util/json_parser.h"
+
+#include "common.h"
+
+namespace zorba {
+namespace jsonml_array {
+
+///////////////////////////////////////////////////////////////////////////////
+
+void parse( json::parser &p, store::Item_t *result );
+void serialize( std::ostream&, store::Item_t const &item, whitespace::type );
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace jsonml_array
+} // namespace zorba
+#endif /* ZORBA_RUNTIME_JSON_JSONML_ARRAY_H */
+/* vim:set et sw=2 ts=2: */

=== added directory 'src/runtime/json/pregenerated'
=== added file 'src/runtime/json/pregenerated/json.cpp'
--- src/runtime/json/pregenerated/json.cpp	1970-01-01 00:00:00 +0000
+++ src/runtime/json/pregenerated/json.cpp	2012-02-16 00:23:20 +0000
@@ -0,0 +1,94 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+ 
+// ******************************************
+// *                                        *
+// * THIS IS A GENERATED FILE. DO NOT EDIT! *
+// * SEE .xml FILE WITH SAME NAME           *
+// *                                        *
+// ******************************************
+
+#include "stdafx.h"
+#include "zorbatypes/rchandle.h"
+#include "zorbatypes/zstring.h"
+#include "runtime/visitors/planiter_visitor.h"
+#include "runtime/json/json.h"
+#include "system/globalenv.h"
+
+
+
+namespace zorba {
+
+// <JSONParseInternal>
+const char* JSONParseInternal::class_name_str = "JSONParseInternal";
+JSONParseInternal::class_factory<JSONParseInternal>
+JSONParseInternal::g_class_factory;
+
+const serialization::ClassVersion 
+JSONParseInternal::class_versions[] ={{ 1, 0x000905, false}};
+
+const int JSONParseInternal::class_versions_count =
+sizeof(JSONParseInternal::class_versions)/sizeof(struct serialization::ClassVersion);
+
+void JSONParseInternal::accept(PlanIterVisitor& v) const {
+  v.beginVisit(*this);
+
+  std::vector<PlanIter_t>::const_iterator lIter = theChildren.begin();
+  std::vector<PlanIter_t>::const_iterator lEnd = theChildren.end();
+  for ( ; lIter != lEnd; ++lIter ){
+    (*lIter)->accept(v);
+  }
+
+  v.endVisit(*this);
+}
+
+JSONParseInternal::~JSONParseInternal() {}
+
+// </JSONParseInternal>
+
+
+// <JSONSerializeInternal>
+const char* JSONSerializeInternal::class_name_str = "JSONSerializeInternal";
+JSONSerializeInternal::class_factory<JSONSerializeInternal>
+JSONSerializeInternal::g_class_factory;
+
+const serialization::ClassVersion 
+JSONSerializeInternal::class_versions[] ={{ 1, 0x000905, false}};
+
+const int JSONSerializeInternal::class_versions_count =
+sizeof(JSONSerializeInternal::class_versions)/sizeof(struct serialization::ClassVersion);
+
+void JSONSerializeInternal::accept(PlanIterVisitor& v) const {
+  v.beginVisit(*this);
+
+  std::vector<PlanIter_t>::const_iterator lIter = theChildren.begin();
+  std::vector<PlanIter_t>::const_iterator lEnd = theChildren.end();
+  for ( ; lIter != lEnd; ++lIter ){
+    (*lIter)->accept(v);
+  }
+
+  v.endVisit(*this);
+}
+
+JSONSerializeInternal::~JSONSerializeInternal() {}
+
+// </JSONSerializeInternal>
+
+
+
+}
+
+

=== added file 'src/runtime/json/pregenerated/json.h'
--- src/runtime/json/pregenerated/json.h	1970-01-01 00:00:00 +0000
+++ src/runtime/json/pregenerated/json.h	2012-02-16 00:23:20 +0000
@@ -0,0 +1,114 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+ 
+// ******************************************
+// *                                        *
+// * THIS IS A GENERATED FILE. DO NOT EDIT! *
+// * SEE .xml FILE WITH SAME NAME           *
+// *                                        *
+// ******************************************
+#ifndef ZORBA_RUNTIME_JSON_JSON_H
+#define ZORBA_RUNTIME_JSON_JSON_H
+
+
+#include "common/shared_types.h"
+
+
+
+#include "runtime/base/narybase.h"
+
+
+namespace zorba {
+
+/**
+ * 
+ *    function for parsing strings into json-xdm
+ *  
+ * Author: Zorba Team
+ */
+class JSONParseInternal : public NaryBaseIterator<JSONParseInternal, PlanIteratorState>
+{ 
+public:
+  SERIALIZABLE_CLASS(JSONParseInternal);
+
+  SERIALIZABLE_CLASS_CONSTRUCTOR2T(JSONParseInternal,
+    NaryBaseIterator<JSONParseInternal, PlanIteratorState>);
+
+  void serialize( ::zorba::serialization::Archiver& ar)
+  {
+    serialize_baseclass(ar,
+    (NaryBaseIterator<JSONParseInternal, PlanIteratorState>*)this);
+  }
+
+  JSONParseInternal(
+    static_context* sctx,
+    const QueryLoc& loc,
+    std::vector<PlanIter_t>& children)
+    : 
+    NaryBaseIterator<JSONParseInternal, PlanIteratorState>(sctx, loc, children)
+  {}
+
+  virtual ~JSONParseInternal();
+
+  void accept(PlanIterVisitor& v) const;
+
+  bool nextImpl(store::Item_t& result, PlanState& aPlanState) const;
+};
+
+
+/**
+ * 
+ *    Function to serialize json/jsonml xdm to string
+ *  
+ * Author: Zorba Team
+ */
+class JSONSerializeInternal : public NaryBaseIterator<JSONSerializeInternal, PlanIteratorState>
+{ 
+public:
+  SERIALIZABLE_CLASS(JSONSerializeInternal);
+
+  SERIALIZABLE_CLASS_CONSTRUCTOR2T(JSONSerializeInternal,
+    NaryBaseIterator<JSONSerializeInternal, PlanIteratorState>);
+
+  void serialize( ::zorba::serialization::Archiver& ar)
+  {
+    serialize_baseclass(ar,
+    (NaryBaseIterator<JSONSerializeInternal, PlanIteratorState>*)this);
+  }
+
+  JSONSerializeInternal(
+    static_context* sctx,
+    const QueryLoc& loc,
+    std::vector<PlanIter_t>& children)
+    : 
+    NaryBaseIterator<JSONSerializeInternal, PlanIteratorState>(sctx, loc, children)
+  {}
+
+  virtual ~JSONSerializeInternal();
+
+  void accept(PlanIterVisitor& v) const;
+
+  bool nextImpl(store::Item_t& result, PlanState& aPlanState) const;
+};
+
+
+}
+#endif
+/*
+ * Local variables:
+ * mode: c++
+ * End:
+ */ 

=== added file 'src/runtime/json/snelson.cpp'
--- src/runtime/json/snelson.cpp	1970-01-01 00:00:00 +0000
+++ src/runtime/json/snelson.cpp	2012-02-16 00:23:20 +0000
@@ -0,0 +1,515 @@
+/*
+ * Copyright 2006-2011 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "stdafx.h"
+
+#include <sstream>
+
+#include <zorba/diagnostic_list.h>
+
+#include "runtime/json/json.h"
+#include "store/api/item_factory.h"
+#include "system/globalenv.h"
+#include "types/root_typemanager.h"
+#include "util/ascii_util.h"
+#include "util/cxx_util.h"
+#include "util/indent.h"
+#include "util/json_parser.h"
+#include "util/mem_streambuf.h"
+#include "util/omanip.h"
+#include "util/oseparator.h"
+#include "util/stl_util.h"
+
+#include "snelson.h"
+
+#define SNELSON_NS "http://john.snelson.org.uk/parsing-json-into-xquery"
+
+using namespace std;
+
+namespace zorba {
+
+///////////////////////////////////////////////////////////////////////////////
+
+static void add_type_attribute( store::Item *parent, char const *value ) {
+  store::Item_t junk_item, att_name, type_name, value_item;
+  GENV_ITEMFACTORY->createQName( att_name, "", "", "type" );
+  type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+  zstring value_string( value );
+  GENV_ITEMFACTORY->createString( value_item, value_string );
+  GENV_ITEMFACTORY->createAttributeNode(
+    junk_item, parent, att_name, type_name, value_item
+  );
+}
+
+#define ADD_TYPE_ATTRIBUTE(T)             \
+  do {                                    \
+    if ( needs_type_attribute ) {         \
+      add_type_attribute( cur_item, T );  \
+      needs_type_attribute = false;       \
+    }                                     \
+  } while (0)
+
+static void add_item_element( item_stack_type &item_stack,
+                              state_stack_type &state_stack,
+                              store::Item_t &cur_item,
+                              char const *type ) {
+  store::Item_t element_name, type_name;
+  zstring base_uri;
+  store::NsBindings ns_bindings;
+  GENV_ITEMFACTORY->createQName( element_name, SNELSON_NS, "", "item" );
+  type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+  GENV_ITEMFACTORY->createElementNode(
+    cur_item, item_stack.top(),
+    element_name, type_name, false, false, ns_bindings, base_uri
+  );
+  add_type_attribute( cur_item.getp(), type );
+  PUSH_ITEM( cur_item );
+}
+
+#define ADD_ITEM_ELEMENT(T)                                 \
+  if ( !IN_STATE( in_array ) ) ; else                       \
+  add_item_element( item_stack, state_stack, cur_item, T )
+
+#define POP_ITEM_ELEMENT()  \
+  if ( !IN_STATE( in_array ) ) ; else POP_ITEM()
+
+static void escape_json_chars( zstring *s ) {
+  ascii::replace_all( *s, "\"", 1, "\\\"", 2 );
+  ascii::replace_all( *s, "\\", 1, "\\\\", 2 );
+  ascii::replace_all( *s, "\b", 1, "\\b", 2 );
+  ascii::replace_all( *s, "\f", 1, "\\f", 2 );
+  ascii::replace_all( *s, "\n", 1, "\\n", 2 );
+  ascii::replace_all( *s, "\r", 1, "\\r", 2 );
+  ascii::replace_all( *s, "\t", 1, "\\t", 2 );
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+namespace snelson {
+
+void parse( json::parser &p, store::Item_t *result ) {
+  ZORBA_ASSERT( result );
+
+  store::Item_t cur_item, junk_item, value_item;
+  store::Item_t att_name, element_name, type_name;
+
+  zstring base_uri;
+  bool got_something = false;
+  item_stack_type item_stack;
+  bool needs_type_attribute = false;
+  bool next_string_is_key = false;
+  store::NsBindings ns_bindings;
+  state_stack_type state_stack;
+  zstring value;
+
+  json::token token;
+  while ( p.next( &token ) ) {
+    got_something = true;
+
+    if ( !*result ) {
+      GENV_ITEMFACTORY->createQName( element_name, SNELSON_NS, "", "json" );
+      type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+      GENV_ITEMFACTORY->createElementNode(
+        cur_item, nullptr,
+        element_name, type_name, false, false, ns_bindings, base_uri
+      );
+      *result = cur_item;
+      needs_type_attribute = true;
+      PUSH_ITEM( cur_item );
+    }
+
+    switch ( token.get_type() ) {
+
+      case '[':
+        if ( IN_STATE( in_object ) )
+          PUSH_ITEM( cur_item );
+        ADD_TYPE_ATTRIBUTE( "array" );
+        ADD_ITEM_ELEMENT( "array" );
+        PUSH_STATE( in_array );
+        break;
+
+      case '{':
+        if ( IN_STATE( in_object ) )
+          PUSH_ITEM( cur_item );
+        ADD_TYPE_ATTRIBUTE( "object" );
+        ADD_ITEM_ELEMENT( "object" );
+        PUSH_STATE( in_object );
+        next_string_is_key = true;
+        break;
+
+      case ']':
+      case '}':
+        POP_STATE();
+        POP_ITEM_ELEMENT();
+        if ( IN_STATE( in_object ) )
+          POP_ITEM();
+        break;
+
+      case ',':
+        next_string_is_key = IN_STATE( in_object );
+        break;
+
+      case json::token::number:
+        ADD_TYPE_ATTRIBUTE( "number" );
+        ADD_ITEM_ELEMENT( "number" );
+        value = token.get_value();
+        GENV_ITEMFACTORY->createTextNode( junk_item, cur_item, value );
+        POP_ITEM_ELEMENT();
+        break;
+
+      case json::token::string:
+        ADD_TYPE_ATTRIBUTE( "string" );
+        value = token.get_value();
+#if 0
+        escape_json_chars( &value );
+#endif
+
+        if ( next_string_is_key ) {
+          // <pair name="..." ...>
+          GENV_ITEMFACTORY->createQName( element_name, SNELSON_NS, "", "pair" );
+          type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+          GENV_ITEMFACTORY->createElementNode(
+            cur_item, item_stack.top(),
+            element_name, type_name, false, false, ns_bindings, base_uri
+          );
+
+          GENV_ITEMFACTORY->createQName( att_name, "", "", "name" );
+          type_name = GENV_TYPESYSTEM.XS_UNTYPED_QNAME;
+          GENV_ITEMFACTORY->createString( value_item, value );
+          GENV_ITEMFACTORY->createAttributeNode(
+            junk_item, cur_item, att_name, type_name, value_item
+          );
+
+          needs_type_attribute = true;
+          next_string_is_key = false;
+        } else {
+          ADD_ITEM_ELEMENT( "string" );
+          GENV_ITEMFACTORY->createTextNode( junk_item, cur_item, value );
+          POP_ITEM_ELEMENT();
+        }
+        break;
+
+      case 'F':
+      case 'T':
+        ADD_TYPE_ATTRIBUTE( "boolean" );
+        ADD_ITEM_ELEMENT( "boolean" );
+        value = token.get_type() == 'F' ? "false" : "true";
+        GENV_ITEMFACTORY->createTextNode( junk_item, cur_item, value );
+        POP_ITEM_ELEMENT();
+        break;
+
+      case json::token::json_null:
+        ADD_TYPE_ATTRIBUTE( "null" );
+        ADD_ITEM_ELEMENT( "null" );
+        POP_ITEM_ELEMENT();
+        break;
+
+      case ':':
+      case json::token::none:
+        break;
+
+      default:
+        assert( false );
+    } // switch
+  } // while
+  if ( !got_something )
+    throw XQUERY_EXCEPTION( zerr::ZJPE0009_ILLEGAL_EMPTY_STRING );
+}
+
+} // namespace snelson
+
+///////////////////////////////////////////////////////////////////////////////
+
+static void assert_json_type( json::type t, zstring const &s ) {
+  // Doing it this way uses the string data in-place with no copy.
+  mem_streambuf::char_type *const p =
+    const_cast<mem_streambuf::char_type*>( s.data() );
+  mem_streambuf buf( p, s.size() );
+  istringstream iss;
+  iss.ios::rdbuf( &buf );
+
+  json::lexer lex( iss );
+  json::token token;
+  try {
+    if ( lex.next( &token ) && json::map_type( token.get_type() ) == t )
+      return;
+  }
+  catch ( json::exception const& ) {
+    // do nothing
+  }
+  throw XQUERY_EXCEPTION(
+    zerr::ZJSE0008_BAD_VALUE,
+    ERROR_PARAMS( s, t )
+  );
+}
+
+static void require_attribute_value( store::Item_t const &element,
+                                     char const *att_name,
+                                     zstring *att_value ) {
+  if ( !get_attribute_value( element, att_name, att_value ) )
+    throw XQUERY_EXCEPTION(
+      zerr::ZJSE0002_ELEMENT_MISSING_ATTRIBUTE,
+      ERROR_PARAMS( element->getNodeName()->getStringValue(), att_name )
+    );
+}
+
+static json::type get_json_type( store::Item_t const &element,
+                                 bool allow_all_types = true ) {
+  zstring att_value;
+  require_attribute_value( element, "type", &att_value );
+  if ( att_value == "array" )
+    return json::array;
+  if ( att_value == "object" )
+    return json::object;
+  if ( allow_all_types ) {
+    if ( att_value == "boolean" )
+      return json::boolean;
+    if ( att_value == "null" )
+      return json::null;
+    if ( att_value == "number" )
+      return json::number;
+    if ( att_value == "string" )
+      return json::string;
+  }
+  throw XQUERY_EXCEPTION(
+    zerr::ZJSE0003_BAD_ATTRIBUTE_VALUE,
+    ERROR_PARAMS( att_value, "type" )
+  );
+}
+
+inline std::ostream& if_space_or_newline( std::ostream &o,
+                                          whitespace::type ws ) {
+  if ( ws == whitespace::some )
+    o << ' ';
+  else
+    o << if_emit( ws == whitespace::indent, '\n' );
+  return o;
+}
+DEF_OMANIP1( if_space_or_newline, whitespace::type )
+
+static ostream& serialize_begin( ostream &o, json::type t,
+                                 whitespace::type ws ) {
+  switch ( t ) {
+    case json::array :
+      o << '[' << if_emit( ws, ' ' );
+      break;
+    case json::object:
+      o << '{' << if_space_or_newline( ws ) << if_indent( ws, inc_indent );
+      break;
+    default:
+      /* suppress warning */;
+  }
+  return o;
+}
+DEF_OMANIP2( serialize_begin, json::type, whitespace::type )
+
+static ostream& serialize_end( ostream &o, json::type t, whitespace::type ws ) {
+  switch ( t ) {
+    case json::array:
+      o << if_emit( ws, ' ' ) << ']';
+      break;
+    case json::object:
+      o << if_space_or_newline( ws ) << if_indent( ws, dec_indent )
+        << if_indent( ws, indent ) << '}';
+      break;
+    default:
+      /* suppress warning */;
+  }
+  return o;
+}
+DEF_OMANIP2( serialize_end, json::type, whitespace::type )
+
+static ostream& serialize_boolean( ostream &o, zstring const &s ) {
+  assert_json_type( json::boolean, s );
+  return o << s;
+}
+DEF_OMANIP1( serialize_boolean, zstring const& )
+
+static ostream& serialize_number( ostream &o, zstring const &s ) {
+  assert_json_type( json::number, s );
+  return o << s;
+}
+DEF_OMANIP1( serialize_number, zstring const& )
+
+static ostream& serialize_string( ostream &o, zstring const &s ) {
+  zstring temp( s );
+  escape_json_chars( &temp );
+  temp.insert( (zstring::size_type)0, 1, '"' );
+  temp.append( 1, '"' );
+  assert_json_type( json::string, temp );
+  return o << temp;
+}
+DEF_OMANIP1( serialize_string, zstring const& )
+
+static ostream& serialize_children( ostream&, store::Item_t const&, json::type,
+                                    whitespace::type );
+DEF_OMANIP3( serialize_children, store::Item_t const&, json::type,
+             whitespace::type )
+
+static ostream& serialize_json_element( ostream &o,
+                                        store::Item_t const &element,
+                                        whitespace::type ws ) {
+  zstring const element_name( element->getNodeName()->getStringValue() );
+  if ( element_name != "json" )
+    throw XQUERY_EXCEPTION(
+      zerr::ZJSE0004_BAD_ELEMENT,
+      ERROR_PARAMS( element_name, "json" )
+    );
+
+  json::type const t = get_json_type( element, false );
+
+  return o
+    << serialize_begin( t, ws )
+    << serialize_children( element, t, ws )
+    << serialize_end( t, ws );
+}
+DEF_OMANIP2( serialize_json_element, store::Item_t const&, whitespace::type )
+
+static ostream& serialize_item_element( ostream &o,
+                                        store::Item_t const &element,
+                                        whitespace::type ws ) {
+  zstring const element_name( element->getNodeName()->getStringValue() );
+  if ( element_name != "item" )
+    throw XQUERY_EXCEPTION(
+      zerr::ZJSE0005_BAD_CHILD_ELEMENT,
+      ERROR_PARAMS( element_name, "array", "item" )
+    );
+
+  json::type const t = get_json_type( element );
+
+  return o
+    << serialize_begin( t, ws )
+    << serialize_children( element, t, ws )
+    << serialize_end( t, ws );
+}
+DEF_OMANIP2( serialize_item_element, store::Item_t const&, whitespace::type )
+
+static ostream& serialize_pair_element( ostream &o,
+                                        store::Item_t const &element,
+                                        whitespace::type ws ) {
+  zstring const element_name( element->getNodeName()->getStringValue() );
+  if ( element_name != "pair" )
+    throw XQUERY_EXCEPTION(
+      zerr::ZJSE0005_BAD_CHILD_ELEMENT,
+      ERROR_PARAMS( element_name, "object", "pair" )
+    );
+
+  zstring name_att_value;
+  require_attribute_value( element, "name", &name_att_value );
+  json::type const t = get_json_type( element );
+
+  return o
+    << if_indent( ws, indent ) << serialize_string( name_att_value )
+    << if_emit( ws, ' ' ) << ':' << if_emit( ws, ' ' )
+    << serialize_begin( t, ws )
+    << serialize_children( element, t, ws )
+    << serialize_end( t, ws );
+}
+DEF_OMANIP2( serialize_pair_element, store::Item_t const&, whitespace::type )
+
+static ostream& serialize_children( ostream &o, store::Item_t const &parent,
+                                    json::type parent_type,
+                                    whitespace::type ws ) {
+  if ( parent_type == json::null )
+    o << "null";
+  else {
+    oseparator sep;
+    if ( ws == whitespace::none )
+      sep.sep( "," );
+    else if ( ws == whitespace::some || parent_type == json::array )
+      sep.sep( ", " );
+    else
+      sep.sep( ",\n" );
+
+    store::Iterator_t i = parent->getChildren();
+    i->open();
+    store::Item_t child;
+    while ( i->next( child ) ) {
+
+      switch ( child->getNodeKind() ) {
+
+        case store::StoreConsts::elementNode:
+          o << sep;
+          switch ( parent_type ) {
+            case json::none:
+              o << serialize_json_element( child, ws );
+              break;
+            case json::array:
+              o << serialize_item_element( child, ws );
+              break;
+            case json::object:
+              o << serialize_pair_element( child, ws );
+              break;
+            default:
+              throw XQUERY_EXCEPTION(
+                zerr::ZJSE0006_NO_ELEMENT_CHILD,
+                ERROR_PARAMS( json::type_string_of[ parent_type ] )
+              );
+          }
+          break;
+
+        case store::StoreConsts::textNode:
+          o << sep;
+          switch ( parent_type ) {
+            case json::boolean:
+              o << serialize_boolean( child->getStringValue() );
+              break;
+            case json::number:
+              o << serialize_number( child->getStringValue() );
+              break;
+            case json::string:
+              o << serialize_string( child->getStringValue() );
+              break;
+            default:
+              throw XQUERY_EXCEPTION(
+                zerr::ZJSE0007_NO_TEXT_CHILD,
+                ERROR_PARAMS( json::type_string_of[ parent_type ] )
+              );
+          }
+          break;
+
+        default:
+          // do nothing
+          break;
+      } // switch
+    } // while
+    i->close();
+  }
+  return o;
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+namespace snelson {
+
+void serialize( ostream &o, store::Item_t const &item, whitespace::type ws ) {
+  switch ( item->getNodeKind() ) {
+    case store::StoreConsts::documentNode:
+      o << serialize_children( item, json::none, ws );
+      break;
+    case store::StoreConsts::elementNode:
+      o << serialize_json_element( item, ws );
+      break;
+    default:
+      throw XQUERY_EXCEPTION( zerr::ZJSE0001_NOT_DOCUMENT_OR_ELEMENT_NODE );
+  }
+}
+
+} // namespace snelson
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace zorba
+/* vim:set et sw=2 ts=2: */

=== added file 'src/runtime/json/snelson.h'
--- src/runtime/json/snelson.h	1970-01-01 00:00:00 +0000
+++ src/runtime/json/snelson.h	2012-02-16 00:23:20 +0000
@@ -0,0 +1,41 @@
+/*
+ * Copyright 2006-2011 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "stdafx.h"
+
+#ifndef ZORBA_RUNTIME_JSON_SNELSON_H
+#define ZORBA_RUNTIME_JSON_SNELSON_H
+
+#include <iostream>
+
+#include "store/api/item.h"
+#include "util/json_parser.h"
+
+#include "common.h"
+
+namespace zorba {
+namespace snelson {
+
+///////////////////////////////////////////////////////////////////////////////
+
+void parse( json::parser &p, store::Item_t *result );
+void serialize( std::ostream&, store::Item_t const &item, whitespace::type );
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace snelson
+} // namespace zorba
+#endif /* ZORBA_RUNTIME_JSON_SNELSON_H */
+/* vim:set et sw=2 ts=2: */

=== added directory 'src/runtime/spec/json'
=== added file 'src/runtime/spec/json/json.xml'
--- src/runtime/spec/json/json.xml	1970-01-01 00:00:00 +0000
+++ src/runtime/spec/json/json.xml	2012-02-16 00:23:20 +0000
@@ -0,0 +1,52 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<!--
+////////////////////////////////////////////////////////////////////////////////
+////////////////////////////////////////////////////////////////////////////////
+-->
+<zorba:iterators
+  xmlns:zorba="http://www.zorba-xquery.com"
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  xsi:schemaLocation="http://www.zorba-xquery.com ../runtime.xsd">
+
+<!--
+/*******************************************************************************
+*******************************************************************************/
+-->
+<zorba:iterator name="JSONParseInternal" arity="nary">
+
+  <zorba:description author="Zorba Team">
+    function for parsing strings into json-xdm
+  </zorba:description>
+
+  <zorba:function isDeterministic="true">
+    <zorba:signature localname="parse-internal" prefix="fn-zorba-json">
+      <zorba:param>xs:string</zorba:param>
+      <zorba:param>item()?</zorba:param>
+      <zorba:output>element()*</zorba:output>
+    </zorba:signature>
+  </zorba:function>
+
+</zorba:iterator>
+
+<!--
+/*******************************************************************************
+*******************************************************************************/
+-->
+<zorba:iterator name="JSONSerializeInternal" arity="nary">
+
+  <zorba:description author="Zorba Team">
+    Function to serialize json/jsonml xdm to string
+  </zorba:description>
+
+  <zorba:function isDeterministic="true">
+    <zorba:signature localname="serialize-internal" prefix="fn-zorba-json">
+      <zorba:param>item()*</zorba:param>
+      <zorba:param>item()?</zorba:param>
+      <zorba:output>xs:string</zorba:output>
+    </zorba:signature>
+  </zorba:function>
+
+</zorba:iterator>
+
+</zorba:iterators>

=== modified file 'src/runtime/spec/mappings.xml'
--- src/runtime/spec/mappings.xml	2011-10-14 07:35:51 +0000
+++ src/runtime/spec/mappings.xml	2012-02-16 00:23:20 +0000
@@ -106,6 +106,10 @@
                      define="ZORBA_STRING_FN_NS" 
                      prefix="fn-zorba-string"/>
 
+    <zorba:namespace uri="http://www.zorba-xquery.com/modules/converters/json"
+                     define="ZORBA_JSON_FN_NS" 
+                     prefix="fn-zorba-json"/>
+
 		<zorba:namespace uri="http://www.zorba-xquery.com/modules/fetch"
                      define="ZORBA_FETCH_FN_NS" 
                      prefix="fn-zorba-fetch"/>

=== modified file 'src/runtime/visitors/pregenerated/planiter_visitor.h'
--- src/runtime/visitors/pregenerated/planiter_visitor.h	2012-01-11 17:30:25 +0000
+++ src/runtime/visitors/pregenerated/planiter_visitor.h	2012-02-16 00:23:20 +0000
@@ -251,6 +251,10 @@
 
     class FunctionAnnotationsIterator;
 
+    class JSONParseInternal;
+
+    class JSONSerializeInternal;
+
     class SqrtIterator;
 
     class ExpIterator;
@@ -929,6 +933,12 @@
     virtual void beginVisit ( const FunctionAnnotationsIterator& ) = 0;
     virtual void endVisit   ( const FunctionAnnotationsIterator& ) = 0;
 
+    virtual void beginVisit ( const JSONParseInternal& ) = 0;
+    virtual void endVisit   ( const JSONParseInternal& ) = 0;
+
+    virtual void beginVisit ( const JSONSerializeInternal& ) = 0;
+    virtual void endVisit   ( const JSONSerializeInternal& ) = 0;
+
     virtual void beginVisit ( const SqrtIterator& ) = 0;
     virtual void endVisit   ( const SqrtIterator& ) = 0;
 

=== modified file 'src/runtime/visitors/pregenerated/printer_visitor.cpp'
--- src/runtime/visitors/pregenerated/printer_visitor.cpp	2012-01-11 17:30:25 +0000
+++ src/runtime/visitors/pregenerated/printer_visitor.cpp	2012-02-16 00:23:20 +0000
@@ -50,6 +50,7 @@
 #include "runtime/function_item/function_item_iter.h"
 #include "runtime/indexing/ic_ddl.h"
 #include "runtime/introspection/sctx.h"
+#include "runtime/json/json.h"
 #include "runtime/maths/maths.h"
 #include "runtime/nodes/node_position.h"
 #include "runtime/nodes/nodes.h"
@@ -1650,6 +1651,34 @@
 // </FunctionAnnotationsIterator>
 
 
+// <JSONParseInternal>
+void PrinterVisitor::beginVisit ( const JSONParseInternal& a) {
+  thePrinter.startBeginVisit("JSONParseInternal", ++theId);
+  printCommons( &a, theId );
+  thePrinter.endBeginVisit( theId );
+}
+
+void PrinterVisitor::endVisit ( const JSONParseInternal& ) {
+  thePrinter.startEndVisit();
+  thePrinter.endEndVisit();
+}
+// </JSONParseInternal>
+
+
+// <JSONSerializeInternal>
+void PrinterVisitor::beginVisit ( const JSONSerializeInternal& a) {
+  thePrinter.startBeginVisit("JSONSerializeInternal", ++theId);
+  printCommons( &a, theId );
+  thePrinter.endBeginVisit( theId );
+}
+
+void PrinterVisitor::endVisit ( const JSONSerializeInternal& ) {
+  thePrinter.startEndVisit();
+  thePrinter.endEndVisit();
+}
+// </JSONSerializeInternal>
+
+
 // <SqrtIterator>
 void PrinterVisitor::beginVisit ( const SqrtIterator& a) {
   thePrinter.startBeginVisit("SqrtIterator", ++theId);

=== modified file 'src/runtime/visitors/pregenerated/printer_visitor.h'
--- src/runtime/visitors/pregenerated/printer_visitor.h	2012-01-11 17:30:25 +0000
+++ src/runtime/visitors/pregenerated/printer_visitor.h	2012-02-16 00:23:20 +0000
@@ -379,6 +379,12 @@
     void beginVisit( const FunctionAnnotationsIterator& );
     void endVisit  ( const FunctionAnnotationsIterator& );
 
+    void beginVisit( const JSONParseInternal& );
+    void endVisit  ( const JSONParseInternal& );
+
+    void beginVisit( const JSONSerializeInternal& );
+    void endVisit  ( const JSONSerializeInternal& );
+
     void beginVisit( const SqrtIterator& );
     void endVisit  ( const SqrtIterator& );
 

=== modified file 'src/unit_tests/CMakeLists.txt'
--- src/unit_tests/CMakeLists.txt	2012-02-02 09:56:52 +0000
+++ src/unit_tests/CMakeLists.txt	2012-02-16 00:23:20 +0000
@@ -19,6 +19,8 @@
   test_uri.cpp
   unique_ptr.cpp
   unit_tests.cpp
+  test_uri.cpp
+  json_parser.cpp
 )
 
 IF (NOT ZORBA_NO_FULL_TEXT)

=== added file 'src/unit_tests/json_parser.cpp'
--- src/unit_tests/json_parser.cpp	1970-01-01 00:00:00 +0000
+++ src/unit_tests/json_parser.cpp	2012-02-16 00:23:20 +0000
@@ -0,0 +1,636 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <sstream>
+
+#include "util/json_parser.h"
+
+using namespace std;
+using namespace zorba;
+using namespace zorba::json;
+
+///////////////////////////////////////////////////////////////////////////////
+
+static int failures;
+
+static bool assert_true( char const *expr, int line, bool result ) {
+  if ( !result ) {
+    cout << "FAILED, line " << line << ": " << expr << endl;
+    ++failures;
+  }
+  return result;
+}
+
+static void print_exception( char const *expr, int line,
+                             std::exception const &e ) {
+  assert_true( expr, line, false );
+  cout << "+ exception: ";
+  if ( json::exception const *j = dynamic_cast<json::exception const*>( &e ) ) {
+    json::location const &loc = j->get_loc();
+    if ( loc.file() && *loc.file() )
+      cout << '"' << loc.file() << "\": ";
+    cout << loc.line();
+    if ( loc.column() )
+      cout << ',' << loc.column();
+    cout << ": ";
+  }
+  cout << e.what() << endl;
+}
+
+#define ASSERT_TRUE( EXPR ) assert_true( #EXPR, __LINE__, !!(EXPR) )
+
+#define ASSERT_EXCEPTION( EXPR, EXCEPTION ) \
+  try { EXPR; assert_true( #EXPR, __LINE__, false ); } \
+  catch ( EXCEPTION const& ) { }
+
+#define ASSERT_NO_EXCEPTION( EXPR ) \
+  try { EXPR; } \
+  catch ( std::exception const &e ) { print_exception( #EXPR, __LINE__, e ); } \
+  catch ( ... ) { assert_true( #EXPR, __LINE__, false ); }
+
+#define ASSERT_TRUE_AND_NO_EXCEPTION( EXPR ) \
+  try { ASSERT_TRUE( EXPR ); } \
+  catch ( std::exception const &e ) { print_exception( #EXPR, __LINE__, e ); } \
+  catch ( ... ) { assert_true( #EXPR, __LINE__, false ); }
+
+///////////////////////////////////////////////////////////////////////////////
+
+static void test_empty_stream() {
+  char const source[] = "";
+  istringstream iss( source );
+  parser p( iss );
+  token t;
+  ASSERT_NO_EXCEPTION( p.next( &t ) );
+}
+
+static void test_illegal_character() {
+  char const source[] = " x ";
+  istringstream iss( source );
+  lexer lex( iss );
+  token t;
+  ASSERT_EXCEPTION( lex.next( &t ), illegal_character );
+}
+
+static void test_illegal_codepoint() {
+  static char const *const sources[] = {
+    " \" \\u  \" ",
+    " \" \\u0  \" ",
+    " \" \\u00  \" ",
+    " \" \\u000  \" ",
+    " \" \\uG  \" ",
+    " \" \\u\" ",
+    0
+  };
+
+  for ( char const *const *s = sources; *s; ++s ) {
+    istringstream iss( *s );
+    lexer lex( iss );
+    token t;
+    ASSERT_EXCEPTION( lex.next( &t ), illegal_codepoint );
+  }
+}
+
+static void test_illegal_escape() {
+  char const source[] = "  \"  \\x  \"  ";
+  istringstream iss( source );
+  lexer lex( iss );
+  token t;
+  ASSERT_EXCEPTION( lex.next( &t ), illegal_escape );
+}
+
+static void test_illegal_literal() {
+  static char const *const sources[] = {
+    " f ",
+    " fa ",
+    " fal ",
+    " fals ",
+    " falsee ",
+    " t ",
+    " tr ",
+    " tru ",
+    " truee ",
+    " n ",
+    " nu ",
+    " nul ",
+    " nulll ",
+    0
+  };
+
+  for ( char const *const *s = sources; *s; ++s ) {
+    istringstream iss( *s );
+    lexer lex( iss );
+    token t;
+    ASSERT_EXCEPTION( lex.next( &t ), illegal_literal );
+  }
+}
+
+static void test_illegal_number() {
+}
+
+static void test_json_org_example() {
+  char const source[] =
+/*  1 */ "{" "\n"
+/*  2 */ "  \"glossary\": {" "\n"
+/*  3 */ "    \"title\": \"example glossary\"," "\n"
+/*  4 */ "    \"GlossDiv\": {" "\n"
+/*  5 */ "      \"title\": \"S\"," "\n"
+/*  6 */ "      \"GlossList\": {" "\n"
+/*  7 */ "        \"GlossEntry\": {" "\n"
+/*  8 */ "          \"ID\": \"SGML\"," "\n"
+/*  9 */ "          \"SortAs\": \"SGML\"," "\n"
+/* 10 */ "          \"GlossTerm\": \"Standard Generalized Markup Language\"," "\n"
+/* 11 */ "          \"Acronym\": \"SGML\"," "\n"
+/* 12 */ "          \"Abbrev\": \"ISO 8879:1986\"," "\n"
+/* 13 */ "          \"GlossDef\": {" "\n"
+/* 14 */ "            \"para\": \"A meta-markup language, used to create markup languages such as DocBook.\"," "\n"
+/* 15 */ "            \"GlossSeeAlso\": [\"GML\", \"XML\"]" "\n"
+/* 16 */ "          }," "\n"
+/* 17 */ "          \"GlossSee\": \"markup\"" "\n"
+/* 18 */ "        }" "\n"
+/* 19 */ "      }" "\n"
+/* 20 */ "    }" "\n"
+/* 21 */ "  }" "\n"
+/* 22 */ "}" "\n" ;
+
+  istringstream iss( source );
+  parser p( iss );
+  token t;
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  1: {
+  ASSERT_TRUE( t == token::begin_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  2: "glossary"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  2: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  2: {
+  ASSERT_TRUE( t == token::begin_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  3: "title"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  3: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  3: "example glossary"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  3: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  4: "GlossDiv"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  4: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  4: {
+  ASSERT_TRUE( t == token::begin_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  5: "title"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  5: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  5: "S"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  5: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  6: "GlossList"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  6: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  6: {
+  ASSERT_TRUE( t == token::begin_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  7: "GlossEntry"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  7: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  7: {
+  ASSERT_TRUE( t == token::begin_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  8: "ID"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  8: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  8: "SGML"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  8: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  9: "SortAs"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  9: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  9: "SGML"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  //  9: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 10: "GlossTerm"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 10: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 10: "Standard ..."
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 10: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 11: "Acronym"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 11: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 11: "SGML"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 11: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 12: "Abbrev"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 12: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 12: "ISO 8879:1986"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 12: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 13: "GlossDef"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 13: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 13: {
+  ASSERT_TRUE( t == token::begin_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 14: "para"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 14: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 14: "A meta-markup ..."
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 14: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 15: "GlossSeeAlso"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 15: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 15: [
+  ASSERT_TRUE( t == token::begin_array );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 15: "GML"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 15: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 15: "XML"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 15: ]
+  ASSERT_TRUE( t == token::end_array );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 16: }
+  ASSERT_TRUE( t == token::end_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 16: ,
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 17: "GlossSee"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 17: :
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 17: "markup"
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 18: }
+  ASSERT_TRUE( t == token::end_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 19: }
+  ASSERT_TRUE( t == token::end_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 20: }
+  ASSERT_TRUE( t == token::end_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 21: }
+  ASSERT_TRUE( t == token::end_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );  // 22: }
+  ASSERT_TRUE( t == token::end_object );
+
+  ASSERT_TRUE( !p.next( &t ) );
+}
+
+static void test_lexer_array() {
+  char const source[] = "[ 1, \"2\", false, true, null ]";
+  istringstream iss( source );
+  lexer lex( iss );
+  token t;
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::begin_array );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::number );
+  ASSERT_TRUE( t.get_value() == "1" );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::string );
+  ASSERT_TRUE( t.get_value() == "2" );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::json_false );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::json_true );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::json_null );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::end_array );
+
+  ASSERT_TRUE( !lex.next( &t ) );
+}
+
+static void test_lexer_object() {
+  char const source[] = "{ \"a\" : 1, \"b\" : \"2\" }";
+  istringstream iss( source );
+  lexer lex( iss );
+  token t;
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::begin_object );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::number );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE( lex.next( &t ) );
+  ASSERT_TRUE( t == token::end_object );
+
+  ASSERT_TRUE( !lex.next( &t ) );
+}
+
+static void test_parser_array() {
+  char const source[] = "[ 1, \"2\", false, true, null ]";
+  istringstream iss( source );
+  parser p( iss );
+  token t;
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::begin_array );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::number );
+  ASSERT_TRUE( t.get_value() == "1" );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::string );
+  ASSERT_TRUE( t.get_value() == "2" );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::json_false );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::json_true );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::json_null );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::end_array );
+
+  ASSERT_TRUE( !p.next( &t ) );
+}
+
+static void test_parser_object() {
+  char const source[] = "{ \"a\" : 1, \"b\" : \"2\" }";
+  istringstream iss( source );
+  parser p( iss );
+  token t;
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::begin_object );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::number );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::value_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::name_separator );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::string );
+
+  ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+  ASSERT_TRUE( t == token::end_object );
+
+  ASSERT_TRUE( !p.next( &t ) );
+}
+
+static void test_unexpected_token() {
+  token t;
+  {
+    char const source[] = "{ 1 }";
+    istringstream iss( source );
+    parser p( iss );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::begin_object );
+    ASSERT_EXCEPTION( p.next( &t ), unexpected_token );
+  }
+  {
+    char const source[] = "{ \"a\" : 1, }";
+    istringstream iss( source );
+    parser p( iss );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::begin_object );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::string );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::name_separator );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::number );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::value_separator );
+
+    ASSERT_EXCEPTION( p.next( &t ), unexpected_token );
+  }
+  {
+    char const source[] = "{ \"t\" : true \"f\" : false }";
+    istringstream iss( source );
+    parser p( iss );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::begin_object );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::string );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::name_separator );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::json_true );
+
+    ASSERT_EXCEPTION( p.next( &t ), unexpected_token );
+  }
+  {
+    char const source[] = "[ 1";
+    istringstream iss( source );
+    parser p( iss );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::begin_array );
+
+    ASSERT_TRUE_AND_NO_EXCEPTION( p.next( &t ) );
+    ASSERT_TRUE( t == token::number );
+
+    ASSERT_EXCEPTION( p.next( &t ), unexpected_token );
+  }
+}
+
+static void test_unterminated_string() {
+  char const source[] = "  \"hello  ";
+  istringstream iss( source );
+  lexer lex( iss );
+  token t;
+
+  ASSERT_EXCEPTION( lex.next( &t ), unterminated_string );
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+namespace zorba {
+namespace UnitTests {
+
+int json_parser( int, char*[] ) {
+
+  // lexer-only tests
+  test_lexer_array();
+  test_lexer_object();
+  test_illegal_character();
+  test_illegal_codepoint();
+  test_illegal_escape();
+  test_illegal_literal();
+  test_illegal_number();
+  test_unterminated_string();
+
+  // parser tests
+  test_empty_stream();
+  test_parser_array();
+  test_parser_object();
+  test_unexpected_token();
+  test_json_org_example();
+
+  cout << failures << " test(s) failed\n";
+  return failures ? 1 : 0;
+}
+
+} // namespace UnitTests
+} // namespace zorba
+/* vim:set et sw=2 ts=2: */

=== modified file 'src/unit_tests/unit_test_list.h'
--- src/unit_tests/unit_test_list.h	2012-02-02 09:56:52 +0000
+++ src/unit_tests/unit_test_list.h	2012-02-16 00:23:20 +0000
@@ -34,6 +34,7 @@
     /**
      * ADD NEW UNIT TESTS HERE
      */
+    int json_parser( int, char*[] );
 
     void initializeTestList();
   };

=== modified file 'src/unit_tests/unit_tests.cpp'
--- src/unit_tests/unit_tests.cpp	2012-02-02 09:56:52 +0000
+++ src/unit_tests/unit_tests.cpp	2012-02-16 00:23:20 +0000
@@ -39,6 +39,7 @@
     void initializeTestList() {
       libunittests["string"] = test_string;
       libunittests["uri"] = runUriTest;
+      libunittests["json_parser"] = json_parser;
       libunittests["unique_ptr"] = test_unique_ptr;
 #ifndef ZORBA_NO_FULL_TEXT
       libunittests["stemmer"] = test_stemmer;

=== modified file 'src/util/CMakeLists.txt'
--- src/util/CMakeLists.txt	2011-07-18 14:25:21 +0000
+++ src/util/CMakeLists.txt	2012-02-16 00:23:20 +0000
@@ -20,6 +20,8 @@
   dir.cpp
   fs_util.cpp
   indent.cpp
+  json_parser.cpp
+  mem_streambuf.cpp
   regex.cpp
   string_util.cpp
   unicode_util.cpp

=== added file 'src/util/json_parser.cpp'
--- src/util/json_parser.cpp	1970-01-01 00:00:00 +0000
+++ src/util/json_parser.cpp	2012-02-16 00:23:20 +0000
@@ -0,0 +1,662 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "diagnostics/assert.h"
+
+#include "ascii_util.h"
+#include "stl_util.h"
+#include "string_util.h"
+#include "utf8_util.h"
+
+#define DEBUG_JSON_PARSER 0
+
+#if DEBUG_JSON_PARSER
+# include "indent.h"
+#endif /* DEBUG_JSON_PARSER */
+
+#include "json_parser.h"
+
+using namespace std;
+
+namespace zorba {
+namespace json {
+
+///////////////////////////////////////////////////////////////////////////////
+
+char const *const type_string_of[] = {
+  "none",
+  "array",
+  "boolean",
+  "null",
+  "number",
+  "object",
+  "string"
+};
+
+type map_type( token::type tt ) {
+  switch ( tt ) {
+    case token::string:
+      return string;
+    case token::number:
+      return number;
+    case token::json_false:
+    case token::json_true:
+      return boolean;
+    case token::json_null:
+      return null;
+    default:
+      return none;
+  }
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+exception::exception( location const &loc, std::string const &message ) :
+  loc_( loc ), message_( message )
+{
+}
+
+exception::~exception() throw() {
+  // out-of-line since it's virtual
+}
+
+char const* exception::what() const throw() {
+  return message_.c_str();
+}
+
+illegal_character::illegal_character( location const &loc, char c ) :
+  exception( loc, BUILD_STRING( '\'', c, "': illegal character" ) ),
+  c_( c )
+{
+}
+
+illegal_character::~illegal_character() throw() {
+  // out-of-line since it's virtual
+}
+
+illegal_codepoint::illegal_codepoint( location const &loc,
+                                      token::value_type const &cp ) :
+  exception( loc, BUILD_STRING( '"', cp, "\": illegal codepoint" ) ),
+  codepoint_( cp )
+{
+}
+
+illegal_codepoint::~illegal_codepoint() throw() {
+  // out-of-line since it's virtual
+}
+
+illegal_escape::illegal_escape( location const &loc, char c ) :
+  exception( loc, BUILD_STRING( "\"\\", c, "\": illegal character escape" ) ),
+  esc_( c )
+{
+}
+
+illegal_escape::~illegal_escape() throw() {
+  // out-of-line since it's virtual
+}
+
+illegal_literal::illegal_literal( location const &loc ) :
+  exception( loc, "illegal literal" )
+{
+}
+
+illegal_literal::~illegal_literal() throw() {
+  // out-of-line since it's virtual
+}
+
+illegal_number::illegal_number( location const &loc ) :
+  exception( loc, "illegal number" )
+{
+}
+
+illegal_number::~illegal_number() throw() {
+  // out-of-line since it's virtual
+}
+
+unexpected_token::unexpected_token( token const &t ) :
+  exception( t.get_loc(), BUILD_STRING( '"', t, "\": unexpected token" ) ),
+  token_( t )
+{
+}
+
+unexpected_token::~unexpected_token() throw() {
+  // out-of-line since it's virtual
+}
+
+unterminated_string::unterminated_string( location const &loc ) :
+  exception( loc, "unterminated string" )
+{
+}
+
+unterminated_string::~unterminated_string() throw() {
+  // out-of-line since it's virtual
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+token::token() :
+  type_( none )
+{
+}
+
+ostream& operator<<( ostream &o, token::type tt ) {
+  switch ( tt ) {
+    case token::string    : o << "string"; break;
+    case token::number    : o << "number"; break;
+    case token::json_false: o << "false" ; break;
+    case token::json_null : o << "null"  ; break;
+    case token::json_true : o << "true"  ; break;
+    case token::none      : o << "<none>"; break;
+    default               : o << static_cast<char>( tt );
+  }
+  return o;
+}
+
+ostream& operator<<( ostream &o, token const &t ) {
+  switch ( t.get_type() ) {
+    case token::string: o << '"' << t.get_value() << '"'; break;
+    case token::number: o <<        t.get_value()       ; break;
+    default           : o <<        t.get_type()        ;
+  }
+  return o;
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+lexer::lexer( istream &in ) :
+  in_( &in ),
+  line_( 1 ),
+  col_( 1 )
+{
+}
+
+bool lexer::get_char( char *c ) {
+  char const temp = in_->get();
+  if ( in_->good() ) {
+    if ( temp == '\n' )
+      ++line_, col_ = 1;
+    else
+      ++col_;
+    if ( c )
+      *c = temp;
+    return true;
+  }
+  return false;
+}
+
+bool lexer::peek_char( char *c ) {
+  *c = in_->peek();
+  return in_->good();
+}
+
+bool lexer::next( token *t ) {
+  while ( true ) {
+    cur_loc_ = cur_loc();
+    char c;
+    if ( !get_char( &c ) )
+      return false;
+    switch ( c ) {
+      case ' ':
+      case '\n':
+      case '\r':
+      case '\t':
+        continue;
+      case '"':
+        t->type_ = token::string;
+        t->loc_ = cur_loc_;
+        parse_string( &t->value_ );
+        return true;
+      case '-':
+      case '0':
+      case '1':
+      case '2':
+      case '3':
+      case '4':
+      case '5':
+      case '6':
+      case '7':
+      case '8':
+      case '9':
+        t->type_ = token::number;
+        t->loc_ = cur_loc_;
+        parse_number( c, &t->value_ );
+        return true;
+      case 'f':
+      case 'n':
+      case 't':
+        t->type_ = parse_literal( c, &t->value_ );
+        t->loc_ = cur_loc_;
+        return true;
+      case '[':
+      case '{':
+      case ']':
+      case '}':
+      case ':':
+      case ',':
+        t->type_ = static_cast<token::type>( c );
+        t->loc_ = cur_loc_;
+        return true;
+      default:
+        throw illegal_character( cur_loc_, c );
+    }
+  } // while
+}
+
+unicode::code_point lexer::parse_codepoint() {
+  static char const hex_digits[] = "0123456789ABCDEF";
+
+  zstring cp_string( "\\u" );           // needed only for error message
+
+  unicode::code_point cp = 0;
+  for ( int i = 1; i <= 4; ++i ) {
+    char c;
+    if ( !get_char( &c ) || !ascii::is_xdigit( c ) )
+      throw illegal_codepoint( cur_loc_, cp_string );
+    cp_string += c;
+    c = ascii::to_upper( c );
+    char const *const p = std::strchr( hex_digits, c );
+    assert( p );
+    cp = (cp << 4) | (p - hex_digits);
+  }
+  return cp;
+}
+
+token::type lexer::parse_literal( char first_c, token::value_type *value ) {
+  static token::value_type const false_value( "false" );
+  static token::value_type const null_value ( "null"  );
+  static token::value_type const true_value ( "true"  );
+
+  token::type tt;
+  switch ( first_c ) {
+    case 'f': *value = false_value; tt = token::json_false; break;
+    case 'n': *value = null_value ; tt = token::json_null ; break;
+    case 't': *value = true_value ; tt = token::json_true ; break;
+    default : assert( false );
+  }
+
+  char c;
+  for ( char const *s = value->c_str(); *++s; ) {
+    if ( !get_char( &c ) || c != *s )
+      throw illegal_literal( cur_loc_ );
+  }
+  if ( peek_char( &c ) && ascii::is_alnum( c ) )
+    throw illegal_literal( cur_loc_ );
+
+  return tt;
+}
+
+void lexer::parse_number( char first_c, token::value_type *value ) {
+  value->clear();
+
+  // <number> ::= [-] <int> [<frac>] [<exp>]
+  char c = first_c;
+  if ( c == '-' ) {
+    *value += c;
+    if ( !get_char( &c ) )
+      throw illegal_number( cur_loc_ );
+  }
+
+  // <int> := '0' | <1-9> <digit>*
+  if ( !ascii::is_digit( c ) )
+    throw illegal_number( cur_loc_ );
+  *value += c;
+  if ( c == '0' ) {
+    if ( !get_char( &c ) )
+      return;
+  } else {
+    while ( true ) {
+      if ( !get_char( &c ) )
+        return;
+      if ( !ascii::is_digit( c ) )
+        break;
+      *value += c;
+    }
+  }
+
+  // <frac> ::= '.' <digit>+
+  if ( c == '.' ) {
+    *value += c;
+    if ( !get_char( &c ) || !ascii::is_digit( c ) )
+      throw illegal_number( cur_loc_ );
+    *value += c;
+    while ( true ) {
+      if ( !get_char( &c ) )
+        return;
+      if ( !ascii::is_digit( c ) )
+        break;
+      *value += c;
+    }
+  }
+
+  // <exp>  ::= <e> [<sign>] <digit>+
+  // <e>    ::= 'e' | 'E'
+  // <sign> ::= '-' | '+'
+  if ( c == 'e' || c == 'E' ) {
+    *value += c;
+    if ( !get_char( &c ) )
+      throw illegal_number( cur_loc_ );
+    if ( c == '+' || c == '-' ) {
+      *value += c;
+      if ( !get_char( &c ) )
+        throw illegal_number( cur_loc_ );
+    }
+    if ( !ascii::is_digit( c ) )
+      throw illegal_number( cur_loc_ );
+    *value += c;
+    while ( true ) {
+      if ( !get_char( &c ) )
+        return;
+      if ( !ascii::is_digit( c ) )
+        break;
+      *value += c;
+    }
+  }
+
+  in_->putback( c );
+}
+
+void lexer::parse_string( token::value_type *value ) {
+  value->clear();
+  bool got_backslash = false;
+  location const start_loc( cur_loc_ );
+
+  while ( true ) {
+    cur_loc_ = cur_loc();
+    char c;
+    if ( !get_char( &c ) )
+      throw unterminated_string( start_loc );
+    if ( got_backslash ) {
+      got_backslash = false;
+      switch ( c ) {
+        case '"':
+        case '/':
+        case '\\':
+          *value += c;
+          break;
+        case 'b':
+          *value += '\b';
+          break;
+        case 'f':
+          *value += '\f';
+          break;
+        case 'n':
+          *value += '\n';
+          break;
+        case 'r':
+          *value += '\r';
+          break;
+        case 't':
+          *value += '\t';
+          break;
+        case 'u':
+          utf8::encode( parse_codepoint(), value );
+          break;
+        default:
+          throw illegal_escape( cur_loc_, c );
+      }
+      continue;
+    }
+
+    switch ( c ) {
+      case '\\':
+        got_backslash = true;
+        break;
+      case '"':
+        return;
+      default:
+        *value += c;
+    }
+  } // while
+}
+
+void lexer::set_loc( char const *file, line_type line, column_type col ) {
+  if ( file )
+    file_ = file;
+  line_ = line;
+  col_ = col;
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+#if DEBUG_JSON_PARSER
+
+ostream& operator<<( ostream &o, parser::state s ) {
+  static char const *const string_of[] = {
+    "A0", "A1", "A2",
+    "E0", "E1",
+    "J0", "J1",
+    "M0", "M1",
+    "O0", "O1", "O2",
+    "P0", "P1",
+    "V0"
+  };
+  return o << string_of[ s ];
+}
+
+static void throw_unexpected_token( int line, token const &t ) {
+  try {
+    throw unexpected_token( t );
+  }
+  catch ( exception const &e ) {
+    cerr << line << ": " << e.what() << endl;
+    throw;
+  }
+}
+
+bool parser::get_token_debug( int line, token *t ) {
+  bool const got_token = get_token( t );
+  cout << line << ": get_token => " << *t << endl;
+  return got_token;
+}
+
+bool parser::matches_token_debug( int line, token::type tt, token *t ) {
+  bool const matched = matches_token( tt, t );
+  cout << line << ": token " << *t << " matches " << tt << " => " << (matched ? 'T' : 'F') << endl;
+  return matched;
+}
+
+token::type parser::peek_token_debug( int line ) {
+  token::type const tt = peek_token();
+  cout << line << ": peek_token => " << peeked_token_ << endl;
+  return tt;
+}
+
+void parser::require_token_debug( int line, token::type tt, token *t ) {
+  if ( !get_token_debug( line, t ) || t->get_type() != tt )
+    throw_unexpected_token( line, *t );
+}
+
+# define GET_TOKEN(T)               get_token_debug( __LINE__, T )
+# define MATCHES_TOKEN(TT,T)        matches_token_debug( __LINE__, TT, T )
+# define PEEK_TOKEN()               peek_token_debug( __LINE__ )
+# define REQUIRE_TOKEN(TT,T)        require_token_debug( __LINE__, TT, T )
+# define THROW_UNEXPECTED_TOKEN(T)  throw_unexpected_token( __LINE__, T )
+
+# define GOTO_STATE(S)  \
+    if (0) ; else {     \
+      state_ = (S);     \
+      cout << __LINE__ << ':' << indent << "GOTO_STATE( " << state_ << " )" << endl;                    \
+      continue;         \
+    }
+
+# define PUSH_STATE(S)      \
+    if (0) ; else {         \
+      state_stack_.push(S); \
+      cout << __LINE__ << ':' << indent << "PUSH_STATE( " << (S) << " )" << endl << inc_indent;             \
+    }
+
+# define POP_STATE()                            \
+    if (0) ; else {                             \
+      state_ = ztd::pop_stack( state_stack_ );  \
+      cout << __LINE__ << ':' << indent << "POP_STATE() => " << state_ << endl << dec_indent;                                   \
+    }
+
+#else
+
+# define GET_TOKEN(T)               get_token( T )
+# define MATCHES_TOKEN(TT,T)        matches_token( TT, T )
+# define PEEK_TOKEN()               peek_token()
+# define REQUIRE_TOKEN(TT,T)        require_token( TT, T )
+# define THROW_UNEXPECTED_TOKEN(T)  throw unexpected_token( T )
+
+# define GOTO_STATE(S)              { state_ = (S); continue; }
+# define PUSH_STATE(S)              state_stack_.push(S)
+# define POP_STATE()                state_ = ztd::pop_stack( state_stack_ )
+
+#endif /* DEBUG_JSON_PARSER */
+
+///////////////////////////////////////////////////////////////////////////////
+
+parser::parser( istream &in ) : lexer_( in ) {
+#if DEBUG_JSON_PARSER
+  get_indent( cout ) = 0;
+#endif /* DEBUG_JSON_PARSER */
+  PUSH_STATE( J0 );
+}
+
+bool parser::get_token( token *t ) {
+  if ( peeked_token_ ) {
+    *t = peeked_token_;
+    peeked_token_.clear();
+    return true;
+  }
+  t->clear();
+  return lexer_.next( t );
+}
+
+bool parser::matches_token( token::type tt, token *t ) {
+  if ( peek_token() == tt )
+    return get_token( t );
+  *t = peeked_token_;
+  return false;
+}
+
+token::type parser::peek_token() {
+  if ( !peeked_token_ )
+    lexer_.next( &peeked_token_ );
+  return peeked_token_.get_type();
+}
+
+#if ! DEBUG_JSON_PARSER
+void parser::require_token( token::type tt, token *t ) {
+  if ( !get_token( t ) || t->get_type() != tt )
+    THROW_UNEXPECTED_TOKEN( *t );
+}
+#endif /* DEBUG_JSON_PARSER */
+
+bool parser::next( token *t ) {
+  if ( state_stack_.empty() )
+    return false;
+  POP_STATE();
+  while ( true ) {
+    switch ( state_ ) {
+
+      // <JSON> ::= <Array> | <Object>
+      case J0:  PUSH_STATE( J1 );
+                switch ( PEEK_TOKEN() ) {
+                  case token::begin_array : GOTO_STATE( A0 );
+                  case token::begin_object: GOTO_STATE( O0 );
+                  case token::none        : break;
+                  default: THROW_UNEXPECTED_TOKEN( peeked_token_ );
+                }
+      case J1:  return false;
+
+      // <Array> ::= '[' <Element>* ']'
+      case A0:  REQUIRE_TOKEN( token::begin_array, t );
+                PUSH_STATE( A1 );
+                return true;
+      case A1:  if ( MATCHES_TOKEN( token::end_array, t ) )
+                  return true;
+                PUSH_STATE( A2 );
+                GOTO_STATE( E0 );
+      case A2:  REQUIRE_TOKEN( token::end_array, t );
+                return true;
+
+      // <Element> ::= <Value> [ ',' <Element> ]
+      case E0:  PUSH_STATE( E1 );
+                GOTO_STATE( V0 );
+      case E1:  if ( MATCHES_TOKEN( token::value_separator, t ) ) {
+                  PUSH_STATE( E0 );
+                  return true;
+                }
+                POP_STATE();
+                continue;
+
+      // <Object> ::= '{' <Member>* '}'
+      case O0:  REQUIRE_TOKEN( token::begin_object, t );
+                PUSH_STATE( O1 );
+                return true;
+      case O1:  if ( MATCHES_TOKEN( token::end_object, t ) )
+                  return true;
+                PUSH_STATE( O2 );
+                GOTO_STATE( M0 );
+      case O2:  REQUIRE_TOKEN( token::end_object, t );
+                return true;
+
+      // <Member> ::= <Pair> [ ',' <Member> ]
+      case M0:  PUSH_STATE( M1 );
+                GOTO_STATE( P0 );
+      case M1:  if ( MATCHES_TOKEN( token::value_separator, t ) ) {
+                  PUSH_STATE( M0 );
+                  return true;
+                }
+                POP_STATE();
+                continue;
+
+      // <Pair> ::= <String> ':' <Value>
+      case P0:  REQUIRE_TOKEN( token::string, t );
+                PUSH_STATE( P1 );
+                return true;
+      case P1:  REQUIRE_TOKEN( token::name_separator, t );
+                PUSH_STATE( V0 );
+                return true;
+
+      // <Value> ::= <Array> | <Object> | <String> | <Number>
+      //           | false | null | true
+      case V0:  switch ( PEEK_TOKEN() ) {
+                  case token::begin_array:
+                    GOTO_STATE( A0 );
+                  case token::begin_object:
+                    GOTO_STATE( O0 );
+                  case token::string:
+                  case token::number:
+                  case token::json_false:
+                  case token::json_null:
+                  case token::json_true:
+                    GET_TOKEN( t );
+                    return true;
+                  default:
+                    THROW_UNEXPECTED_TOKEN( peeked_token_ );
+                }
+    } // switch ( state_ )
+  } // while
+}
+
+token::type parser::peek( token *t ) {
+  if ( token::type const tt = PEEK_TOKEN() ) {
+    if ( t )
+      *t = peeked_token_;
+    return tt;
+  }
+  return token::none;
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace json
+} // namespace zorba
+
+/* vim:set et sw=2 ts=2: */

=== added file 'src/util/json_parser.h'
--- src/util/json_parser.h	1970-01-01 00:00:00 +0000
+++ src/util/json_parser.h	2012-02-16 00:23:20 +0000
@@ -0,0 +1,570 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef ZORBA_JSON_PARSER_H
+#define ZORBA_JSON_PARSER_H
+
+#include <zorba/config.h>
+
+#include <exception>
+#include <iostream>
+#include <stack>
+#include <string>
+
+#include <zorba/internal/diagnostic.h>
+
+#include "zorbatypes/zstring.h"
+
+#include "cxx_util.h"
+#include "unicode_util.h"
+
+namespace zorba {
+namespace json {
+
+///////////////////////////////////////////////////////////////////////////////
+
+typedef internal::diagnostic::location location;
+
+///////////////////////////////////////////////////////////////////////////////
+
+/**
+ * A JSON %type is the type of JSON data.  This isn't used by the lexer or
+ * parser implementation at all, but it's handy.
+ */
+enum type {
+  none,   // meaning "not set" as opposed to "null"
+  array,
+  boolean,
+  null,
+  number,
+  object,
+  string
+};
+extern char const *const type_string_of[];
+
+inline std::ostream& operator<<( std::ostream &o, type t ) {
+  return o << type_string_of[ t ];
+}
+
+/**
+ * A JSON %token.  Tokens have a type, location at which they were found, and
+ * sometimes a value.
+ *
+ * See: "RFC 4627: The application/json Media Type for JavaScript Object
+ * Notation (JSON)."
+ */
+class token {
+  // see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2333.html
+  struct pointer_conversion { int valid; };
+  typedef int pointer_conversion::*explicit_bool;
+public:
+  typedef zstring value_type;
+
+  /**
+   * The types of tokens in JSON.  The first 6 constants have values that
+   * correspond to the actual structural characters used by JSON; the rest were
+   * assigned non-standard, mnemonic values for convenience.
+   */
+  enum type {
+    none,
+    begin_array     = '[',
+    begin_object    = '{',
+    end_array       = ']',
+    end_object      = '}',
+    name_separator  = ':',
+    value_separator = ',',
+    string          = 'S',
+    number          = 'N',
+    json_false      = 'F',
+    json_null       = '0',
+    json_true       = 'T',
+  };
+
+  /**
+   * Default constructor.
+   */
+  token();
+
+  /**
+   * Clears this %token.
+   */
+  void clear() {
+    type_ = none;
+    value_.clear();
+  }
+
+  /**
+   * Gets the location at which this %token was found.
+   *
+   * @return Returns said location.
+   */
+  location const& get_loc() const {
+    return loc_;
+  }
+
+  /**
+   * Gets the type of this %token.
+   *
+   * @return Returns said type.
+   */
+  type get_type() const {
+    return type_;
+  }
+
+  /**
+   * Gets the value of this %token, if any.  Only %token types string, number,
+   * false, null, and true have a value.
+   *
+   * @return Returns said value or the empty string.
+   */
+  value_type const& get_value() const {
+    return value_;
+  }
+
+  /**
+   * Conversion to \c bool.
+   *
+   * @return Returns \c true only if this token's type is not \c none.
+   */
+  operator explicit_bool() const {
+    return type_ ? &pointer_conversion::valid : nullptr;
+  }
+
+private:
+  location loc_;
+  type type_;
+  value_type value_;
+
+  friend class lexer;
+};
+
+/**
+ * Map a token's type to a JSON type.
+ *
+ * @param tt The token::type to map.
+ * @return Returns the corresponding JSON type or \c none if \a tt doesn't map.
+ */
+type map_type( token::type tt );
+
+/**
+ * Emits the given token type to an ostream.
+ *
+ * @param o The ostream to emit to.
+ * @param tt The token type to emit.
+ * @return Returns \a o.
+ */
+std::ostream& operator<<( std::ostream &o, token::type tt );
+
+/**
+ * Emits the given token to an ostream.
+ *
+ * @param o The ostream to emit to.
+ * @param t The token to emit.
+ * @return Returns \a o.
+ */
+std::ostream& operator<<( std::ostream &o, token const &t );
+
+/**
+ * Compares two tokens for equality.
+ *
+ * @param t1 The first token.
+ * @param t2 The second token.
+ * @return Returns \c true only if the two tokens' types and values are equal.
+ */
+inline bool operator==( token const &t1, token const &t2 ) {
+  return t1.get_type() == t2.get_type() && t1.get_value() == t2.get_value();
+}
+
+/**
+ * Compares a token's type to another type for equality.
+ *
+ * @param t The token whose type to compare.
+ * @param tt The type to compare to.
+ * @return Returns \c true only if the token's type equals \a tt.
+ */
+inline bool operator==( token const &t, token::type tt ) {
+  return t.get_type() == tt;
+}
+
+/**
+ * Compares a token's type to another type for equality.
+ *
+ * @param tt The type to compare.
+ * @param t The token whose type to compare to.
+ * @return Returns \c true only if \a tt equals the token's type.
+ */
+inline bool operator==( token::type tt, token const &t ) {
+  return t == tt;
+}
+
+/**
+ * Compares a token's value to a C string for equality.
+ *
+ * @param t The token whose value to compare.
+ * @param value The value to compare to.
+ * @return Returns \c true only if the token's value equals \a value.
+ */
+inline bool operator==( token const &t, char const *value ) {
+  return t.get_value() == value;
+}
+
+/**
+ * Compares a C string to a token's value for equality.
+ *
+ * @param value The value to compare.
+ * @param t The token whose value to compare to.
+ * @return Returns \c true only if \a value equals the token's value.
+ */
+inline bool operator==( char const *value, token const &t ) {
+  return t == value;
+}
+
+/**
+ * Compares two tokens for inequality.
+ *
+ * @param t1 The first token.
+ * @param t2 The second token.
+ * @return Returns \c true if either the two tokens' types or values are not
+ * equal.
+ */
+inline bool operator!=( token const &t1, token const &t2 ) {
+  return !(t1 == t2);
+}
+
+/**
+ * Compares a token's type to another type for inequality.
+ *
+ * @param t The token whose type to compare.
+ * @param tt The type to compare to.
+ * @return Returns \c true only if the token's type is not equal to \a tt.
+ */
+inline bool operator!=( token const &t, token::type tt ) {
+  return !(t == tt);
+}
+
+/**
+ * Compares a token's type to another type for inequality.
+ *
+ * @param tt The type to compare.
+ * @param t The token whose type to compare to.
+ * @return Returns \c true only if \a tt is not equal to the token's type.
+ */
+inline bool operator!=( token::type tt, token const &t ) {
+  return !(tt == t);
+}
+
+/**
+ * Compares a token's value to a C string for inequality.
+ *
+ * @param t The token whose value to compare.
+ * @param value The value to compare to.
+ * @return Returns \c true only if the token's value is not equal to \a value.
+ */
+inline bool operator!=( token const &t, char const *value ) {
+  return !(t == value);
+}
+
+/**
+ * Compares a token's value to a C string for inequality.
+ *
+ * @param value The value to compare.
+ * @param t The token whose value to compare to.
+ * @return Returns \c true only if \a value is not equal to the token's value.
+ */
+inline bool operator!=( char const *value, token const &t ) {
+  return !(value == t);
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+/**
+ * An %exception is the root of the JSON %exception hierarchy.
+ */
+class exception : public std::exception {
+public:
+  ~exception() throw();
+
+  /**
+   * Gets the location in the JSON source whence this exception was thrown.
+   */
+  location const& get_loc() const {
+    return loc_;
+  }
+
+  // inherited
+  char const* what() const throw();
+
+protected:
+  exception( location const &loc, std::string const &message );
+
+private:
+  location loc_;
+  std::string message_;
+};
+
+/**
+ * This exception is thrown when an illegal character is encountered in a JSON
+ * data stream.
+ */
+class illegal_character : public exception {
+public:
+  illegal_character( location const &loc, char c );
+  ~illegal_character() throw();
+
+  /**
+   * Gets the illegal character.
+   *
+   * @return Returns said character.
+   */
+  char get_char() const {
+    return c_;
+  }
+
+private:
+  char c_;
+};
+
+/**
+ * This exception is thrown when an illegal Unicode code-point escape sequence
+ * (\uHHHH) is encountered.
+ */
+class illegal_codepoint : public exception {
+public:
+  illegal_codepoint( location const &loc, token::value_type const &cp );
+  ~illegal_codepoint() throw();
+
+  /**
+   * Gets the illegal code-point.
+   *
+   * @return Returns said code-point.
+   */
+  token::value_type get_codepoint() const {
+    return codepoint_;
+  }
+
+private:
+  token::value_type codepoint_;
+};
+
+/**
+ * This exception is thrown when an illegal character follows a backslash
+ * (escape) within a string literal.  The legal escape characters are:
+ * ["/\bfnrtu].
+ */
+class illegal_escape : public exception {
+public:
+  illegal_escape( location const &loc, char escape );
+  ~illegal_escape() throw();
+
+  /**
+   * Gets the illegal escape character.
+   *
+   * @return Returns said character.
+   */
+  char get_escape() const {
+    return esc_;
+  }
+
+private:
+  char esc_;
+};
+
+/**
+ * This exception is thrown when a literal other than \c false, \c null, or
+ * \c true is encountered.
+ */
+class illegal_literal : public exception {
+public:
+  illegal_literal( location const &loc );
+  ~illegal_literal() throw();
+};
+
+/**
+ * This exception is thrown when an illegal number is encountered.
+ */
+class illegal_number : public exception {
+public:
+  illegal_number( location const &loc );
+  ~illegal_number() throw();
+};
+
+/**
+ * This exception is thrown when an unexpected token is encountered.
+ */
+class unexpected_token : public exception {
+public:
+  unexpected_token( token const &t );
+  ~unexpected_token() throw();
+
+  /**
+   * Gets the unexpected token
+   *
+   * @return Returns said token.
+   */
+  token const& get_token() const {
+    return token_;
+  }
+
+private:
+  token token_;
+};
+
+/**
+ * This exception is thrown when an EOF is encountered before a string's
+ * terminating quote.
+ */
+class unterminated_string : public exception {
+public:
+  unterminated_string( location const &loc );
+  ~unterminated_string() throw();
+};
+
+///////////////////////////////////////////////////////////////////////////////
+
+/**
+ * A %lexer extracts JSON tokens from an istream.
+ */
+class lexer {
+public:
+  typedef location::line_type line_type;
+  typedef location::column_type column_type;
+
+  /**
+   * Constructs a %lexer on the given istream.
+   *
+   * @param in The istream to read from.
+   */
+  lexer( std::istream &in );
+
+  /**
+   * Gets the next token, if any.
+   *
+   * @param result A pointer to the token to get into.
+   * @return Returns \c true only if there was a next token.
+   * @throws exception upon error.
+   */
+  bool next( token *result );
+
+  /**
+   * Sets the file location.
+   *
+   * @param file The source file name.
+   * @param line The source line number.
+   * @param col  The source column number.
+   */
+  void set_loc( char const *file, line_type line, column_type col );
+
+private:
+  location cur_loc() const {
+    return location( file_, line_, col_ );
+  }
+
+  bool get_char( char* = nullptr );
+  bool peek_char( char* );
+  unicode::code_point parse_codepoint();
+  token::type parse_literal( char, token::value_type* );
+  void parse_number( char, token::value_type* );
+  void parse_string( token::value_type* );
+
+  std::istream *in_;
+  std::string file_;
+  line_type line_;
+  column_type col_;
+  location cur_loc_;
+};
+
+///////////////////////////////////////////////////////////////////////////////
+
+/**
+ * A %parser extracts JSON tokens from an istream while checking to ensure the
+ * token sequence is valid.
+ */
+class parser {
+public:
+  typedef lexer::line_type line_type;
+  typedef lexer::column_type column_type;
+
+  /**
+   * Constructs a %parser on the given istream.
+   *
+   * @param in The istream to read from.
+   */
+  parser( std::istream &in );
+
+  /**
+   * Gets the next token, if any.
+   *
+   * @param result A pointer to the token to receive the token.
+   * @return Returns \c true only if there was a next token.
+   * @throws exception upon error.
+   */
+  bool next( token *result );
+
+  /**
+   * Peeks at the next token, if any.
+   *
+   * @param result A pointer to the token to receive the token, if any.
+   * @return Returns the type of the peeked token.
+   * @throws exception upon error.
+   */
+  token::type peek( token *result = nullptr );
+
+  /**
+   * Sets the file location.
+   *
+   * @param file The source file name.
+   * @param line The source line number.
+   * @param col  The source column number.
+   */
+  void set_loc( char const *file, line_type line, column_type col ) {
+    lexer_.set_loc( file, line, col );
+  }
+
+private:
+  enum state {
+    A0, A1, A2, // Array
+    E0, E1,     // Element
+    J0, J1,     // JSON
+    M0, M1,     // Member
+    O0, O1, O2, // Object
+    P0, P1,     // Pair
+    V0          // Value
+  };
+
+  friend std::ostream& operator<<( std::ostream&, state );
+
+  bool get_token( token* );
+  bool get_token_debug( int, token* );
+  bool matches_token( token::type, token* );
+  bool matches_token_debug( int, token::type, token* );
+  token::type peek_token();
+  token::type peek_token_debug( int );
+  void require_token( token::type, token* );
+  void require_token_debug( int, token::type, token* );
+
+  lexer lexer_;
+  token peeked_token_;
+  std::stack<state> state_stack_;
+  state state_;
+};
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace json
+} // namespace zorba
+
+#endif /* ZORBA_JSON_PARSER_H */
+/* vim:set et sw=2 ts=2: */

=== added file 'src/util/mem_streambuf.cpp'
--- src/util/mem_streambuf.cpp	1970-01-01 00:00:00 +0000
+++ src/util/mem_streambuf.cpp	2012-02-16 00:23:20 +0000
@@ -0,0 +1,119 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <cstring>                      /* for memcpy(3) */
+
+#include "diagnostics/assert.h"
+
+#include "cxx_util.h"
+#include "mem_streambuf.h"
+
+using namespace std;
+
+namespace zorba {
+
+///////////////////////////////////////////////////////////////////////////////
+
+mem_streambuf::mem_streambuf() {
+  set( nullptr, nullptr );
+}
+
+mem_streambuf::mem_streambuf( char_type *begin, char_type *end ) {
+  set( begin, end );
+}
+
+mem_streambuf::mem_streambuf( char_type *begin, off_type size ) {
+  set( begin, size );
+}
+
+mem_streambuf::int_type mem_streambuf::overflow( int_type c ) {
+  if ( traits_type::eq_int_type( c, traits_type::eof() ) )
+    return traits_type::not_eof( c );
+  if ( pptr() >= epptr() )
+    return traits_type::eof();
+  *pptr() = traits_type::to_char_type( c );
+  pbump( 1 );
+  return c;
+}
+
+mem_streambuf::int_type mem_streambuf::pbackfail( int_type c ) {
+  if ( !traits_type::eq_int_type( c, traits_type::eof() ) ) {
+    *pptr() = traits_type::to_int_type( c );
+    pbump( -1 );
+  }
+  return traits_type::to_int_type( *pptr() );
+}
+
+mem_streambuf::pos_type mem_streambuf::seekoff( off_type off,
+                                                ios_base::seekdir dir,
+                                                ios_base::openmode ) {
+  switch ( dir ) {
+    case ios_base::beg:
+      our_setg( begin_ + off );
+      break;
+    case ios_base::cur:
+      our_setg( gptr() + off );
+      break;
+    case ios_base::end:
+      our_setg( end_ + off );
+      break;
+    default:
+      ZORBA_ASSERT( false );
+  }
+  return off;
+}
+
+mem_streambuf::pos_type mem_streambuf::seekpos( pos_type pos,
+                                                ios_base::openmode mode ) {
+  return seekoff( pos, ios_base::beg, mode );
+}
+
+void mem_streambuf::set( char_type *begin, char_type *end ) {
+  begin_ = begin;
+  end_ = end;
+  our_setg( begin );
+  our_setp( end );
+}
+
+streamsize mem_streambuf::showmanyc() {
+  return egptr() - gptr();
+}
+
+mem_streambuf::int_type mem_streambuf::underflow() {
+  return gptr() < egptr() ?
+    traits_type::to_int_type( *gptr() ) : traits_type::eof();
+}
+
+streamsize mem_streambuf::xsgetn( char_type *buf, std::streamsize size ) {
+  streamsize const remaining = showmanyc();
+  if ( size > remaining )
+    size = remaining;
+  ::memcpy( buf, gptr(), size );
+  return size;
+}
+
+streamsize mem_streambuf::xsputn( char_type const *buf, streamsize size ) {
+  streamsize const remaining = epptr() - pptr();
+  if ( size > remaining )
+    size = remaining;
+  ::memcpy( pptr(), buf, size );
+  return size;
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace zorba
+/* vim:set et sw=2 ts=2: */

=== added file 'src/util/mem_streambuf.h'
--- src/util/mem_streambuf.h	1970-01-01 00:00:00 +0000
+++ src/util/mem_streambuf.h	2012-02-16 00:23:20 +0000
@@ -0,0 +1,108 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef ZORBA_MMAP_STREAMBUF_H
+#define ZORBA_MMAP_STREAMBUF_H
+
+#include <streambuf>
+
+namespace zorba {
+
+///////////////////////////////////////////////////////////////////////////////
+
+/**
+ * A %mem_streambuf is-a std::streambuf for a fixed-size chunk of memory.
+ */
+class mem_streambuf : public std::streambuf {
+public:
+  typedef std::streambuf::char_type char_type;
+  typedef std::streambuf::int_type int_type;
+  typedef std::streambuf::off_type off_type;
+  typedef std::streambuf::pos_type pos_type;
+  typedef std::streambuf::traits_type traits_type;
+
+  /**
+   * Default constructor.
+   */
+  mem_streambuf();
+
+  /**
+   * Constructs a %mem_streambuf.
+   *
+   * @param begin A pointer to the beginning of the memory chunk.
+   * @param end A pointer to one past the end of the memory chunk.
+   */
+  mem_streambuf( char_type *begin, char_type *end );
+
+  /**
+   * Constructs a %mem_streambuf.
+   *
+   * @param begin A pointer to the beginning of the memory chunk.
+   * @param size The size of the memory chunk.
+   */
+  mem_streambuf( char_type *begin, off_type size );
+
+  /**
+   * Sets the memory chunk.
+   *
+   * @param begin A pointer to the beginning of the memory chunk.
+   * @param end A pointer to one past the end of the memory chunk.
+   */
+  void set( char_type *begin, char_type *end );
+
+  /**
+   * Sets the memory chunk.
+   *
+   * @param begin A pointer to the beginning of the memory chunk.
+   * @param size The size of the memory chunk.
+   */
+  void set( char_type *begin, off_type size );
+
+protected:
+  int_type overflow( int_type c );
+  int_type pbackfail( int_type c );
+  pos_type seekoff( off_type, std::ios_base::seekdir, std::ios_base::openmode );
+  pos_type seekpos( pos_type, std::ios_base::openmode  );
+  std::streamsize showmanyc();
+  int_type underflow();
+  std::streamsize xsgetn( char_type*, std::streamsize );
+  std::streamsize xsputn( char_type const*, std::streamsize );
+
+private:
+  char_type *begin_, *end_;
+
+  void our_setg( char_type *ptr ) {
+    setg( begin_, ptr, end_ );
+  }
+
+  void our_setp( char_type *ptr ) {
+    setp( ptr, end_ );
+  }
+
+  // forbid
+  mem_streambuf( mem_streambuf const& );
+  mem_streambuf& operator=( mem_streambuf const& );
+};
+
+inline void mem_streambuf::set( char_type *begin, off_type size ) {
+  set( begin, begin + size );
+}
+
+///////////////////////////////////////////////////////////////////////////////
+
+} // namespace zorba
+#endif  /* ZORBA_MMAP_STREAMBUF_H */
+/* vim:set et sw=2 ts=2: */

=== modified file 'src/util/omanip.h'
--- src/util/omanip.h	2011-06-14 17:26:33 +0000
+++ src/util/omanip.h	2012-02-16 00:23:20 +0000
@@ -25,127 +25,275 @@
 ///////////////////////////////////////////////////////////////////////////////
 
 /**
- * An omanip1 is a class for assisting in the creation of ostream manipulators
+ * An %omanip1 is a class for assisting in the creation of ostream manipulators
  * by storing a pointer to a function and its argument to be called later via
  * operator<<().
  *
  * See also: "Standard C++ IOStreams and Locales," Angelika Langer and Klaus
  * Kreft, Addison-Wesley, pp. 179-191.
  */
-template<typename Arg1Type> class omanip1 {
+template<typename Arg1Type>
+class omanip1 {
 public:
 
   /**
-   * The signature of functions this omanip1 can handle.
+   * The signature of functions this %omanip1 can handle.
    */
   typedef std::ostream& (*func_type)( std::ostream&, Arg1Type );
 
   /**
-   * Constructs an omanip1.
+   * Constructs an %omanip1.
    *
-   * @param f     The function to call when this omanip1 is inserted into an
-   *              ostream.
-   * @param arg1  The argument to be passed to the function.
+   * @param f   The function to call when this %omanip1 is inserted into an
+   *            ostream.
+   * @param a1  The argument to be passed to the function.
    */
-  omanip1( func_type f, Arg1Type arg1 ) :
-    f_( f ), arg1_( arg1 )
+  omanip1( func_type f, Arg1Type a1 ) :
+    f_( f ), a1_( a1 )
   {
   }
 
   /**
-   * Inserts the given omanip1 into the given ostream.  This has the effect of
-   * calling the function and argument bound to the omanip1 at the time of its
+   * Inserts the given %omanip1 into the given ostream.  This has the effect of
+   * calling the function and argument bound to the %omanip1 at the time of its
    * construction.
    *
    * @param o The ostream to insert into.
-   * @param m The omanip1 to insert.
+   * @param m The %omanip1 to insert.
    */
   friend std::ostream& operator<<( std::ostream &o, omanip1 const &m ) {
-    return (*m.f_)( o, m.arg1_ );
+    if ( o.good() )
+      (*m.f_)( o, m.a1_ );
+    return o;
   }
 
 private:
   func_type const f_;
-  Arg1Type const arg1_;
+  Arg1Type const a1_;
 };
 
 /**
  * Defines an ostream manipulator "thunk" function that calls an existing
  * non-manipulator function having the same name.
  *
- * @param FN_NAME   The name of the existing function.
- * @param ARG1_TYPE The type of the non-ostream argument.
- * @param ARG1_NAME The name of the non-ostream argument.
+ * @param FN_NAME The name of the existing function.
+ * @param ARG1_T  The type of the non-ostream argument.
  */
-#define DEF_OMANIP1(FN_NAME,ARG1_TYPE,ARG1_NAME)      \
-  inline omanip1<ARG1_TYPE>                           \
-  FN_NAME( ARG1_TYPE ARG1_NAME ) {                    \
-    return omanip1<ARG1_TYPE>( FN_NAME, ARG1_NAME );  \
+#define DEF_OMANIP1(FN_NAME,ARG1_T)        \
+  inline omanip1<ARG1_T>                   \
+  FN_NAME( ARG1_T a1 ) {                   \
+    return omanip1<ARG1_T>( FN_NAME, a1 ); \
   }
 
 ///////////////////////////////////////////////////////////////////////////////
 
 /**
- * An omanip2 is a class for assisting in the creation of ostream manipulators
+ * An %omanip2 is a class for assisting in the creation of ostream manipulators
  * by storing a pointer to a function and its arguments to be called later via
  * operator<<().
  *
  * See also: "Standard C++ IOStreams and Locales," Angelika Langer and Klaus
  * Kreft, Addison-Wesley, pp. 179-191.