Zorba

JSON module snelson parsing problem

Bug #930573 reported by William Candillon
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Zorba
Fix Released
High
Paul J. Lucas
Zorba 2.2 "Coeus"

Bug Description

The following query:

import module namespace json = "http://www.zorba-xquery.com/modules/converters/json";
import module namespace file = "http://expath.org/ns/file";

let $tweet := file:read-text("tweet.json")
let $tweet := json:parse($tweet)
let $tweet := json:serialize($tweet)
return
  $tweet

Where tweet.json (in attachement) comes from the twitter stream API, returns:

JSON serialization error [zerr:ZJSE0005]: "pair": illegal child element of "array" type; must be "item"
=================================================
json:serialize#1 <http://www.zorba-xquery.com/modules/converters/json>
/parse.xq at line 6 column 15

Related branches

lp:~zorba-coders/zorba/feature-json_parser
Sorin Marian Nasoi: Approve
William Candillon: Needs Fixing
Paul J. Lucas: Approve
Matthias Brantner: Pending requested
Diff: 6765 lines (+5533/-73)
164 files modified
ChangeLog (+1/-0)
include/zorba/diagnostic.h (+4/-1)
include/zorba/internal/ztd.h (+2/-1)
include/zorba/pregenerated/diagnostic_list.h (+34/-0)
modules/ExternalModules.conf (+1/-1)
modules/com/zorba-xquery/www/modules/CMakeLists.txt (+4/-0)
modules/com/zorba-xquery/www/modules/converters/json-options.xsd (+60/-0)
modules/com/zorba-xquery/www/modules/converters/json.xq (+260/-0)
modules/com/zorba-xquery/www/modules/pregenerated/errors.xq (+69/-1)
src/context/static_context.cpp (+7/-0)
src/context/static_context.h (+1/-0)
src/diagnostics/diagnostic.cpp (+3/-0)
src/diagnostics/diagnostic_en.xml (+65/-0)
src/diagnostics/pregenerated/diagnostic_list.cpp (+51/-0)
src/diagnostics/pregenerated/dict_en.cpp (+19/-0)
src/diagnostics/qname.cpp (+6/-0)
src/functions/library.cpp (+2/-0)
src/functions/pregenerated/func_json.cpp (+87/-0)
src/functions/pregenerated/func_json.h (+79/-0)
src/functions/pregenerated/function_enum.h (+2/-0)
src/runtime/CMakeLists.txt (+3/-0)
src/runtime/full_text/ft_match.cpp (+2/-3)
src/runtime/json/common.cpp (+62/-0)
src/runtime/json/common.h (+120/-0)
src/runtime/json/json_impl.cpp (+217/-0)
src/runtime/json/jsonml_array.cpp (+285/-0)
src/runtime/json/jsonml_array.h (+41/-0)
src/runtime/json/pregenerated/json.cpp (+94/-0)
src/runtime/json/pregenerated/json.h (+114/-0)
src/runtime/json/snelson.cpp (+515/-0)
src/runtime/json/snelson.h (+41/-0)
src/runtime/spec/json/json.xml (+52/-0)
src/runtime/spec/mappings.xml (+4/-0)
src/runtime/visitors/pregenerated/planiter_visitor.h (+10/-0)
src/runtime/visitors/pregenerated/printer_visitor.cpp (+29/-0)
src/runtime/visitors/pregenerated/printer_visitor.h (+6/-0)
src/unit_tests/CMakeLists.txt (+2/-0)
src/unit_tests/json_parser.cpp (+636/-0)
src/unit_tests/unit_test_list.h (+1/-0)
src/unit_tests/unit_tests.cpp (+1/-0)
src/util/CMakeLists.txt (+2/-0)
src/util/json_parser.cpp (+662/-0)
src/util/json_parser.h (+570/-0)
src/util/mem_streambuf.cpp (+119/-0)
src/util/mem_streambuf.h (+108/-0)
src/util/omanip.h (+205/-57)
src/util/oseparator.h (+17/-5)
src/util/stl_util.h (+51/-2)
src/util/string_util.h (+2/-1)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-parse-01.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-parse-02.xml.res (+4/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-parse-03.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-parse-04.xml.res (+7/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-parse-05.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-parse-wikipedia.xml.res (+10/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-serialize-01.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-serialize-02.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-serialize-03.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-serialize-indent-wikipedia.xml.res (+11/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-serialize-none-wikipedia.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-jsonml_array-serialize-some-wikipedia.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-null-handling.xml.res (+3/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-numbers-and-decimals.xml.res (+7/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-array-01.xml.res (+3/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-array-02.xml.res (+3/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-array-03.xml.res (+3/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-array-04.xml.res (+3/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-array-05.xml.res (+4/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-array-06.xml.res (+8/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-array-07.xml.res (+10/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-array-08.xml.res (+6/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-example.xml.res (+14/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-object-01.xml.res (+3/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-object-02.xml.res (+4/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-object-03.xml.res (+8/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-object-04.xml.res (+6/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-parse-serialize.xml.res (+3/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-array-01.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-array-02.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-array-03.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-array-04.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-array-05.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-array-12.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-indent-example.xml.res (+11/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-none-example.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-object-01.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-object-02.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-object-03.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-parse.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-serialize-some-example.xml.res (+1/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-uncommon-chars.xml.res (+3/-0)
test/rbkt/ExpQueryResults/zorba/json/json-snelson-utf-8.xml.res (+3/-0)
test/rbkt/Queries/zorba/json/json-invalid-option-parameter.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-invalid-option-parameter.xq (+10/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-parse-01.xq (+9/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-parse-02.xq (+14/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-parse-03.xq (+18/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-parse-04.xq (+27/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-parse-05.xq (+66/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-parse-06.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-parse-06.xq (+10/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-parse-wikipedia.xq (+23/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-01.xq (+10/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-02.xq (+13/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-03.xq (+10/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-indent-wikipedia.xq (+20/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-none-wikipedia.xq (+19/-0)
test/rbkt/Queries/zorba/json/json-jsonml_array-serialize-some-wikipedia.xq (+20/-0)
test/rbkt/Queries/zorba/json/json-snelson-invalid-json.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-invalid-json.xq (+9/-0)
test/rbkt/Queries/zorba/json/json-snelson-null-handling.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-numbers-and-decimals.xq (+10/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-array-01.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-array-02.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-array-03.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-array-04.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-array-05.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-array-06.xq (+7/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-array-07.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-array-08.xq (+7/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-empty.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-empty.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-example.xq (+20/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-object-01.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-object-02.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-object-03.xq (+7/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-object-04.xq (+12/-0)
test/rbkt/Queries/zorba/json/json-snelson-parse-serialize.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-01.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-02.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-03.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-04.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-05.xq (+9/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-06.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-06.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-07.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-07.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-08.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-08.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-09.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-09.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-10.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-10.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-11.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-11.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-array-12.xq (+11/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-comment-node.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-comment-node.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-indent-example.xq (+24/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-invalid-value-for-attribute.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-invalid-value-for-attribute.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-none-example.xq (+19/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-object-01.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-object-02.xq (+9/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-object-03.xq (+12/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-parse.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-some-example.xq (+24/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-type-value-missing.spec (+1/-0)
test/rbkt/Queries/zorba/json/json-snelson-serialize-type-value-missing.xq (+8/-0)
test/rbkt/Queries/zorba/json/json-snelson-uncommon-chars.xq (+5/-0)
test/rbkt/Queries/zorba/json/json-snelson-utf-8.xq (+5/-0)
test/unit/CMakeLists.txt (+1/-0)
test/update/Queries/zorba/store/sc3.spec (+2/-0)
test/update/Queries/zorba/store/sc3_ex3.xq (+1/-1)
Revision history for this message
William Candillon (wcandillon) wrote :
Changed in zorba:
assignee: nobody → Paul J. Lucas (paul-lucas)
importance: Undecided → High
Revision history for this message
Paul J. Lucas (paul-lucas) wrote :

As I've tried to explain, this bug (and your previous few bugs) have nothing to do with serialization. The reason the serialization reports the problem is because the *parser* is producing incorrect XDM. The serializer then attempts to serialize the XDM and (correctly) reports the error.

Paul J. Lucas (paul-lucas)
summary: - JSON module serialization problem
+ JSON module snelson parsing problem
Changed in zorba:
status: New → In Progress
Paul J. Lucas (paul-lucas)
Changed in zorba:
status: In Progress → Fix Committed
Revision history for this message
William Candillon (wcandillon) wrote :

Please never mark a bug that I reported as fix committed. Always assign it back to me for closing.
As a good illustration of my point, the parse/serialize function is still completely broken.
I will make a more detailled report later.

Changed in zorba:
status: Fix Committed → In Progress
Revision history for this message
Paul J. Lucas (paul-lucas) wrote :

The query works with the tweet.json you attached.

William Candillon (wcandillon)
Changed in zorba:
status: In Progress → Fix Committed
Paul J. Lucas (paul-lucas)
Changed in zorba:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.