Merge lp:~zorba-coders/zorba/dataguide into lp:zorba

[Nicolae Brinza]

Dataguide implementation.

Dataguide implementation.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for the following merge proposals:
https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue

[Zorba Build Bot (zorba-buildbot)]

Validation queue result for https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Stage "TestZorbaUbuntu" failed.
794 tests failed (8369 total tests run).

Check test results at http://jenkins.lambda.nu/job/TestZorbaUbuntu/49/testReport/ to view the results.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for the following merge proposals:
https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue

[Zorba Build Bot (zorba-buildbot)]

Validation queue result for https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Stage "TestZorbaUbuntu" failed.
794 tests failed (8369 total tests run).

Check test results at http://jenkins.lambda.nu/job/TestZorbaUbuntu/50/testReport/ to view the results.

[lp:~zorba-coders/zorba/dataguide]

11482. By Nicolae Brinza on 2013-07-04

Dataguide work in progress.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for the following merge proposals:
https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue

[Zorba Build Bot (zorba-buildbot)]

Validation queue result for https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Stage "TestZorbaUbuntu" failed.
794 tests failed (8373 total tests run).

Check test results at http://jenkins.lambda.nu/job/TestZorbaUbuntu/51/testReport/ to view the results.

[lp:~zorba-coders/zorba/dataguide]

11483. By Nicolae Brinza on 2013-07-04

Merged with Zorba trunk

11484. By Nicolae Brinza on 2013-07-04

Disabled dataguide for testing

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for the following merge proposals:
https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue

[Zorba Build Bot (zorba-buildbot)]

Validation queue result for https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Stage "TestZorbaUbuntu" failed.
26 tests failed (8373 total tests run).

Check test results at http://jenkins.lambda.nu/job/TestZorbaUbuntu/52/testReport/ to view the results.

[lp:~zorba-coders/zorba/dataguide]

11485. By Nicolae Brinza on 2013-07-05

Re-enabled the dataguide

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for the following merge proposals:
https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue

[lp:~zorba-coders/zorba/dataguide]

11486. By Nicolae Brinza on 2013-07-08

Dataguide fixes for the remote queue; cleanup; documentation;

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for the following merge proposals:
https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue

[Zorba Build Bot (zorba-buildbot)]

Voting criteria failed for the following merge proposals:

https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026 :
Votes: {'Pending': 1}

[Zorba Build Bot (zorba-buildbot)]

Validation queue result for https://code.launchpad.net/~zorba-coders/zorba/dataguide/+merge/173026

Stage "CommitZorba" failed.

Check console output at http://jenkins.lambda.nu/job/CommitZorba/9/console to view the results.

[lp:~zorba-coders/zorba/dataguide]

11487. By Nicolae Brinza on 2013-07-08

Dataguide cleanup

[Nicolae Brinza (nbrinza)]

review: Approve

[Matthias Brantner (matthias-brantner)]

- I find the name dataguide misleading because it's a guide on the query and not on the data. Maybe QueryPruneGuide would be more meaningful
- Can the user also use the zann_explores_json annotation?
- Why is the dataguide parameter on the Store's getCollection() function? Shouldn't it be on the function that returns the iterator? The problem is that a Collection object within the simplestore exists only once per collection. What's the semantics if multiple queries access the collection (possibly in parallel)?
- Did you measure the performance impact of the optimizer on some larger queries?

review: Needs Fixing

[Nicolae Brinza (nbrinza)]

> - I find the name dataguide misleading because it's a guide on the query and
> not on the data. Maybe QueryPruneGuide would be more meaningful

The query itself is not pruned, the data is. I think "dataguide" is the established term -- see for example this paper: http://ilpubs.stanford.edu:8090/264/1/1997-50.pdf .

> - Can the user also use the zann_explores_json annotation?

Yes, the users can use it as well. But does it make sense for them to use it? If they have an external function -- it is automatically handled as if it has the annotation. For a UDF it doesn't really make any sense to add it.

> - Why is the dataguide parameter on the Store's getCollection() function?
> Shouldn't it be on the function that returns the iterator? The problem is that
> a Collection object within the simplestore exists only once per collection.
> What's the semantics if multiple queries access the collection (possibly in
> parallel)?

It very much depends on how the collections are handled. Currently for Zorba collections it doesn't make sense to have any dataguides at all, because they're in-memory collections. I have not taken a look at the Sausalito code and have not seen how e.g. the MongoDB "collections" are managed. getCollection() seemed the most logical place where it should be passed, but the dataguide parameter could be easily propagated to any Store class, including the function that returns the iterator.

Currently each and every db:collection() call has its own dataguide, even if they might refer to the same collection. If the collection manager currently "caches" or reuses the collection iterators, then it might make sense to forbid that so that the dataguide for each individual db:collection call could be used.

Or alternatively, an "union" on the dataguides that refer to the same collection could be performed. But I think it is not always possible to determine if that is the case.

I think this could be investigated and decided upon when implementing the Dataguide push-down into MongoDB or when I would take a better look at the Sausalito's collection manager code.

> - Did you measure the performance impact of the optimizer on some larger
> queries?

The expression tree is traversed in its entirety once and only once, visiting each node, so the performance should not be very different from any other dataflow computation, e.g. ignores sorts/order/etc. If there are no "sources", i.e. db:collection() or jn:parse() calls, then the dataguide computation just propagates NULLs, doing no calculations and almost no memory allocations (at most one dataguide_cb allocation per fo_exprs and several others). If there are "sources" in the tree -- there will be some union operations being performed for some of the nodes.

I will check if any of our larger queries have longer compilation times, but because none of them have db:collection() or jn:parse() calls, I do not expect any differences.

It would make sense to have a specially constructed query that would do a stress-test of the dataguide code -- e.g. a db:collection().navigation.navigation. ... .navigation several thousand times or something similar. I will try that out and see if it manages...

[Matthias Brantner (matthias-brantner)]

> > - I find the name dataguide misleading because it's a guide on the query and
> > not on the data. Maybe QueryPruneGuide would be more meaningful
>
> The query itself is not pruned, the data is. I think "dataguide" is the
> established term -- see for example this paper:
> http://ilpubs.stanford.edu:8090/264/1/1997-50.pdf .
"DataGuides serve as dynamic schemas, generated from the database." What we generate is a schema from the query.

> > - Why is the dataguide parameter on the Store's getCollection() function?
> > Shouldn't it be on the function that returns the iterator? The problem is
> that
> > a Collection object within the simplestore exists only once per collection.
> > What's the semantics if multiple queries access the collection (possibly in
> > parallel)?
>
> It very much depends on how the collections are handled. Currently for Zorba
> collections it doesn't make sense to have any dataguides at all, because
> they're in-memory collections. I have not taken a look at the Sausalito code
> and have not seen how e.g. the MongoDB "collections" are managed.
> getCollection() seemed the most logical place where it should be passed, but
> the dataguide parameter could be easily propagated to any Store class,
> including the function that returns the iterator.
>
> Currently each and every db:collection() call has its own dataguide, even if
> they might refer to the same collection. If the collection manager currently
> "caches" or reuses the collection iterators, then it might make sense to
> forbid that so that the dataguide for each individual db:collection call could
> be used.
>
> Or alternatively, an "union" on the dataguides that refer to the same
> collection could be performed. But I think it is not always possible to
> determine if that is the case.
>
> I think this could be investigated and decided upon when implementing the
> Dataguide push-down into MongoDB or when I would take a better look at the
> Sausalito's collection manager code.
I think we will run into a problem. 28msec has only one buffer that is accessed by all db:collection() calls in a query. Hence, the information needs to be the union.

[Nicolae Brinza (nbrinza)]

> "DataGuides serve as dynamic schemas, generated from the database." What we generate is a
> schema from the query.

Still, it is a data schema, not a query schema. The one in the paper would be a Database DataGuide and ours would be Query DataGuide. I would agree to change it to QueryDataguide but I don't think there would be any confusions if it was simply called Dataguide.

> I think we will run into a problem. 28msec has only one buffer that is
> accessed by all db:collection() calls in a query. Hence, the information
> needs to be the union.

If there is no way of removing that limitation then we can overcome this by doing an union on all db:collection() dataguides and this will ensure correctness. But it would be a pity to loose the individually computed dataguides for each separate call. Still, if the name of fields of different collections are mostly disjoint sets, then we won't loose much of the improvement.

Again I suggest leaving this until I start implementing the push-down of projection info into the db:collection() calls. It has no impact on jn:parse() -- these dataguides can still be computed and kept individually for each call even if we do an union on db:collection() calls.

--

[Nicolae Brinza (nbrinza)]

I've done some additional testing, and these are the results:

For the xray query, the largest that we have in the testsuite, compilation time with --compile-only is pretty much the same with and without the dataguide computaiton, at around ~0.08 sec.

With a specially constructed query that looks like this: (see dataguide-29.jq test)

let $col := dml:collection()
let $col2 := ($col.cat1, $col.cat2, ... , $col.cat10)
return $col2.category.category.category ... category (repeated ~2000 times)

the compilation time goes from ~0.7s without the dataguide to ~10s with the dataguide enabled, so it is significant. But this is a worst-case scenario. The resulting dataguide is an object 2000-levels deep.

The compilation can be improved significantly by:
- keeping track of the "leaves" nodes in the dataguide tree
- rewriting a bit the dataguide structure to store the trees incrementally instead of cloning them
- adding a depth cutoff

What do you think?

[lp:~zorba-coders/zorba/dataguide]

11488. By Nicolae Brinza on 2013-07-23

Added a stress-test for the dataguide computation

11489. By Nicolae Brinza on 2013-07-23

Merged with Zorba trunk

11490. By Nicolae Brinza on 2013-07-23

Fixed the build

11491. By Markos Zaharioudakis on 2013-07-30

cosmetic

11492. By Markos Zaharioudakis on 2013-07-30

merge from trunk

[Matthias Brantner (matthias-brantner)]

Superseded by use-dataguide merge proposal.

Unmerged revisions

11492. By Markos Zaharioudakis on 2013-07-30

merge from trunk

11491. By Markos Zaharioudakis on 2013-07-30

cosmetic

11490. By Nicolae Brinza on 2013-07-23

Fixed the build

11489. By Nicolae Brinza on 2013-07-23

Merged with Zorba trunk

11488. By Nicolae Brinza on 2013-07-23

Added a stress-test for the dataguide computation

11487. By Nicolae Brinza on 2013-07-08

Dataguide cleanup

11486. By Nicolae Brinza on 2013-07-08

Dataguide fixes for the remote queue; cleanup; documentation;

11485. By Nicolae Brinza on 2013-07-05

Re-enabled the dataguide

11484. By Nicolae Brinza on 2013-07-04

Disabled dataguide for testing

11483. By Nicolae Brinza on 2013-07-04

Merged with Zorba trunk

=== modified file 'src/annotations/annotations.cpp'
--- src/annotations/annotations.cpp	2013-07-06 06:59:32 +0000
+++ src/annotations/annotations.cpp	2013-07-30 11:18:26 +0000
@@ -133,6 +133,8 @@
   ZANN(read-only-nodes, read_only_nodes);
   ZANN(mutable-nodes, mutable_nodes);
 
+  ZANN(explores-json, explores_json);
+
 #undef ZANN
 
   // create a set of rules to detect conflicts between annotations
@@ -401,6 +403,13 @@
 }
 
 
+void AnnotationList::push_back(
+    AnnotationInternal::AnnotationId id)
+{
+  theAnnotationList.push_back(new AnnotationInternal(AnnotationInternal::lookup(id)));
+}
+
+
 /*******************************************************************************
   Called from translator to detect duplicates and conflicting declarations
 ********************************************************************************/

=== modified file 'src/annotations/annotations.h'
--- src/annotations/annotations.h	2013-07-06 06:59:32 +0000
+++ src/annotations/annotations.h	2013-07-30 11:18:26 +0000
@@ -86,7 +86,8 @@
     zann_ordered,
     zann_unordered,
     zann_read_only_nodes,
-    zann_mutable_nodes,
+    zann_mutable_nodes,    
+    zann_explores_json,
 
     // must be at the end
     zann_end
@@ -188,6 +189,8 @@
   void push_back(
       const store::Item_t& qname,
       const std::vector<const_expr*>& literals);
+  
+  void push_back(AnnotationInternal::AnnotationId id);
 
   void checkConflictingDeclarations(DeclarationKind k, const QueryLoc& loc) const;
 };

=== modified file 'src/common/shared_types.h'
--- src/common/shared_types.h	2013-07-06 06:59:32 +0000
+++ src/common/shared_types.h	2013-07-30 11:18:26 +0000
@@ -106,6 +106,9 @@
 
 typedef rchandle<CompilerCB> CompilerCB_t;
 
+// JSON dataguide
+class dataguide_cb;
+typedef rchandle<dataguide_cb> dataguide_cb_t;  
 
 /* datetime stuff */
 class DateTime;

=== modified file 'src/compiler/codegen/plan_visitor.cpp'
--- src/compiler/codegen/plan_visitor.cpp	2013-07-01 18:59:06 +0000
+++ src/compiler/codegen/plan_visitor.cpp	2013-07-30 11:18:26 +0000
@@ -49,6 +49,8 @@
 #include "compiler/expression/function_item_expr.h"
 #include "compiler/expression/path_expr.h"
 #include "compiler/expression/var_expr.h"
+#include "compiler/expression/json_dataguide.h"
+
 #include "compiler/parser/parse_constants.h"
 
 #include "context/namespace_context.h"
@@ -98,6 +100,9 @@
 #include "runtime/hof/dynamic_fncall_iterator.h"
 #include "runtime/misc/materialize.h"
 
+#include "runtime/collections/collections.h"
+#include "runtime/json/jsoniq_functions.h"
+
 #ifdef ZORBA_WITH_DEBUGGER
 #include "debugger/debugger_commons.h"
 #endif
@@ -2456,7 +2461,7 @@
   if (is_enclosed_expr(&v))
     theConstructorsStack.push(&v);
 
-	return true;
+  return true;
 }
 
 
@@ -2514,6 +2519,16 @@
       user_function* udf = static_cast<user_function*>(func);
       udf->computeResultCaching(theCCB->theXQueryDiagnostics);
     }
+    else if (dataguide_cb::func_uses_dataguide(func->getKind()) && v.get_dataguide() != NULL)
+    {      
+      ZorbaCollectionIterator* collIter;      
+      if ((collIter = dynamic_cast<ZorbaCollectionIterator*>(iter.getp())))
+        collIter->setDataguide(v.get_dataguide()->get_as_json(&v));  
+      
+      JSONParseIterator* parseIter;
+      if ((parseIter = dynamic_cast<JSONParseIterator*>(iter.getp())))
+        parseIter->setDataguide(v.get_dataguide()->get_as_json(&v));
+    }
   }
   else
   {

=== modified file 'src/compiler/expression/CMakeLists.txt'
--- src/compiler/expression/CMakeLists.txt	2013-02-07 17:24:36 +0000
+++ src/compiler/expression/CMakeLists.txt	2013-07-30 11:18:26 +0000
@@ -30,7 +30,8 @@
     function_item_expr.cpp
     pragma.cpp
     mem_manager.cpp
-    expr_manager.cpp)
+    expr_manager.cpp
+    json_dataguide.cpp)
 
 IF (NOT ZORBA_NO_FULL_TEXT)
   LIST(APPEND EXPRESSION_SRCS

=== modified file 'src/compiler/expression/expr_base.cpp'
--- src/compiler/expression/expr_base.cpp	2013-05-29 04:17:01 +0000
+++ src/compiler/expression/expr_base.cpp	2013-07-30 11:18:26 +0000
@@ -26,6 +26,8 @@
 #include "compiler/expression/expr_visitor.h"
 #include "compiler/expression/expr_manager.h"
 
+#include "compiler/expression/json_dataguide.h"
+
 #include "compiler/api/compilercb.h"
 
 #include "functions/function.h"
@@ -1418,6 +1420,29 @@
 }
 
 
+/*******************************************************************************
+
+********************************************************************************/
+dataguide_cb* expr::get_dataguide()
+{ 
+  return theJsonDataguide; 
+}
+
+
+dataguide_cb* expr::get_dataguide_or_new()
+{
+  if (!theJsonDataguide.getp()) 
+    theJsonDataguide = new dataguide_cb();
+  return theJsonDataguide; 
+}
+
+
+dataguide_cb* expr::set_dataguide(dataguide_cb* a_json_dataguide)
+{ 
+  theJsonDataguide = a_json_dataguide; 
+  return theJsonDataguide; 
+}
+
 
 /*******************************************************************************
 
@@ -1440,6 +1465,90 @@
   return cloneExpr->get_return_type();
 }
 
+/*******************************************************************************
+
+********************************************************************************/
+std::string expr::get_expr_kind_string() const
+{
+  const char* result;
+  
+  switch (get_expr_kind())
+  {
+  case const_expr_kind: result = "const_expr"; break;
+
+  case var_expr_kind: result = "var_expr"; break;
+
+  case doc_expr_kind: result = "doc_expr"; break;
+  case elem_expr_kind: result = "elem_expr"; break;
+  case attr_expr_kind: result = "attr_expr"; break;
+  case namespace_expr_kind: result = "namespace_expr"; break;
+  case text_expr_kind: result = "text_expr"; break;
+  case pi_expr_kind: result = "pi_expr"; break;
+
+  case relpath_expr_kind: result = "relpath_expr"; break;
+  case axis_step_expr_kind: result = "axis_step_expr"; break;
+  case match_expr_kind: result = "match_expr"; break;
+
+  case flwor_expr_kind: result = "flwor_expr"; break;
+  case if_expr_kind: result = "if_expr"; break;
+  case trycatch_expr_kind: result = "trycatch_expr"; break;
+
+  case fo_expr_kind: result = "fo_expr"; break;
+  case dynamic_function_invocation_expr_kind: result = "dynamic_function_invocation_expr"; break;
+  case argument_placeholder_expr_kind: result = "argument_placeholder_expr"; break;
+  case function_item_expr_kind: result = "function_item_expr"; break;
+
+  case castable_expr_kind: result = "castable_expr"; break;
+  case cast_expr_kind: result = "cast_expr"; break;
+  case instanceof_expr_kind: result = "instanceof_expr"; break;
+  case treat_expr_kind: result = "treat_expr"; break;
+  case promote_expr_kind: result = "promote_expr"; break;
+  case name_cast_expr_kind: result = "name_cast_expr"; break;
+
+  case validate_expr_kind: result = "validate_expr"; break;
+
+  case extension_expr_kind: result = "extension_expr"; break;
+
+  case order_expr_kind: result = "order_expr"; break;
+
+#ifndef ZORBA_NO_FULL_TEXT
+  case ft_expr_kind: result = "ft_expr"; break;
+#endif /* ZORBA_NO_FULL_TEXT */
+
+  case delete_expr_kind: result = "delete_expr"; break;
+  case insert_expr_kind: result = "insert_expr"; break;
+  case rename_expr_kind: result = "rename_expr"; break;
+  case replace_expr_kind: result = "replace_expr"; break;
+  case transform_expr_kind: result = "transform_expr"; break;
+
+  case block_expr_kind: result = "block_expr"; break;
+  case var_decl_expr_kind: result = "var_decl_expr"; break;
+  case var_set_expr_kind: result = "var_set_expr"; break;
+  case apply_expr_kind: result = "apply_expr"; break;
+  case exit_expr_kind: result = "exit_expr"; break;
+  case exit_catcher_expr_kind: result = "exit_catcher_expr"; break;
+  case flowctl_expr_kind: result = "flowctl_expr"; break;
+  case while_expr_kind: result = "while_expr"; break;
+
+  case eval_expr_kind: result = "eval_expr"; break;
+  case debugger_expr_kind: result = "debugger_expr"; break;
+  case wrapper_expr_kind: result = "wrapper_expr"; break;
+  case function_trace_expr_kind: result = "function_trace_expr"; break;
+
+  case json_direct_object_expr_kind: result = "json_direct_object_expr"; break;
+  case json_object_expr_kind: result = "json_object_expr"; break;
+  case json_array_expr_kind: result = "json_array_expr"; break;
+
+  case unknown_expr_kind: result = "unknown_expr"; break;
+    
+  default:
+    result = "expr kind not in list. Add it to expr_base::get_expr_kind_string()";  
+    break;
+  }
+  
+  return std::string(result);
+}
+
 
 } // namespace zorba
 /* vim:set et sw=2 ts=2: */

=== modified file 'src/compiler/expression/expr_base.h'
--- src/compiler/expression/expr_base.h	2013-05-29 04:17:01 +0000
+++ src/compiler/expression/expr_base.h	2013-07-30 11:18:26 +0000
@@ -45,6 +45,7 @@
 
 class CompilerCB;
 
+
 enum expr_kind_t
 {
   const_expr_kind,
@@ -198,6 +199,8 @@
   uint8_t            theVisitId;
 
   FreeVars           theFreeVars;
+  
+  dataguide_cb_t     theJsonDataguide;
 
 public:
   static bool is_sequential(unsigned short theScriptingKind);
@@ -223,6 +226,8 @@
   user_function* get_udf() const { return theUDF; }
 
   expr_kind_t get_expr_kind() const { return static_cast<expr_kind_t>(theKind); }
+  
+  std::string get_expr_kind_string() const;
 
   const QueryLoc& get_loc() const { return theLoc; }
 
@@ -404,6 +409,13 @@
   void clear_annotations();
 
   xqtref_t get_return_type_with_empty_input(const expr* input) const;
+  
+  dataguide_cb* get_dataguide(); 
+  
+  // If the object's dataguide is NULL, will create a new one and return it
+  dataguide_cb* get_dataguide_or_new();
+  
+  dataguide_cb* set_dataguide(dataguide_cb* a_json_dataguide); 
 
 protected:
   virtual void compute_scripting_kind() = 0;

=== modified file 'src/compiler/expression/expr_clone.cpp'
--- src/compiler/expression/expr_clone.cpp	2013-07-02 21:32:23 +0000
+++ src/compiler/expression/expr_clone.cpp	2013-07-30 11:18:26 +0000
@@ -27,6 +27,7 @@
 #include "compiler/expression/ft_expr.h"
 #include "compiler/expression/ftnode.h"
 #include "compiler/expression/expr_manager.h"
+#include "compiler/expression/json_dataguide.h"
 
 #include "compiler/api/compilercb.h"
 
@@ -337,6 +338,8 @@
       cloneExpr->theArgs.push_back(e->theArgs[i]->clone(udf, subst));
 
     cloneExpr->theScriptingKind  = e->theScriptingKind;
+    
+    cloneExpr->theJsonDataguide = e->theJsonDataguide;
 
     newExpr = cloneExpr;
     break;

=== modified file 'src/compiler/expression/expr_type.cpp'
--- src/compiler/expression/expr_type.cpp	2013-06-15 02:57:08 +0000
+++ src/compiler/expression/expr_type.cpp	2013-07-30 11:18:26 +0000
@@ -643,6 +643,10 @@
       const FunctionXQType* funcType = static_cast<const FunctionXQType*>(fiType.getp());
       newType = funcType->get_return_type();
     }
+    else if (fiType->type_kind() == XQType::STRUCTURED_ITEM_KIND && e->get_args().size() == 1)
+    {
+      newType = rtm.STRUCTURED_ITEM_TYPE_STAR;
+    } 
     else
     {
       newType = rtm.ITEM_TYPE_STAR;

=== added file 'src/compiler/expression/json_dataguide.cpp'
--- src/compiler/expression/json_dataguide.cpp	1970-01-01 00:00:00 +0000
+++ src/compiler/expression/json_dataguide.cpp	2013-07-30 11:18:26 +0000
@@ -0,0 +1,349 @@
+/*
+ * Copyright 2006-2013 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "stdafx.h"
+
+#include "system/globalenv.h"
+#include "store/api/item_factory.h"
+
+#include "compiler/expression/expr_base.h"
+#include "compiler/expression/json_dataguide.h"
+
+
+namespace zorba
+{
+  
+/*******************************************************************************
+  
+********************************************************************************/
+void dataguide_node::add_to_leaves(store::Item* key)
+{
+  if (is_star)
+    return;
+  
+  if (keys.size() == 0) // no children == leaf
+  {
+    keys.push_back(key);
+    values.push_back(dataguide_node());
+    return;
+  }
+  
+  for (csize i = 0; i != keys.size(); ++i)
+    values[i].add_to_leaves(key);
+}
+
+
+void dataguide_node::add_to_leaves(const dataguide_node* other)
+{
+  if (is_star)
+    return;
+  
+  if (keys.size() == 0) // no children == leaf
+  {
+    clone(other);
+    return;
+  }
+  
+  for (csize i = 0; i != keys.size(); ++i)
+    values[i].add_to_leaves(other);
+}
+
+
+void dataguide_node::set_star()
+{
+  is_star = true;
+  keys.clear();
+  values.clear();  
+}
+
+
+void dataguide_node::set_star_on_leaves()
+{
+  if (is_star)
+    return;
+  
+  if (keys.size() == 0)
+  {
+    set_star();
+    return;
+  }
+  
+  for (csize i = 0; i != keys.size(); i++)
+    values[i].set_star_on_leaves();
+}
+
+
+void dataguide_node::do_union(const dataguide_node* other)
+{
+  if (other->is_star)
+  {
+    set_star();
+    return;
+  }
+  
+  for (unsigned int i=0; i<other->keys.size(); i++)
+  {
+    bool found = false;
+    unsigned int j = 0;
+    for ( ; j<keys.size(); j++)
+      if (keys[j]->equals(other->keys[i]))
+      {
+        found = true;
+        break;
+      }
+    
+    if (found)
+    {
+      values[i].do_union(&other->values[j]);      
+    }    
+    else
+    {          
+      keys.push_back(other->keys[i]);
+      values.push_back(dataguide_node());
+      values.back().do_union(&other->values[i]);      
+    }
+  }    
+}
+
+
+void dataguide_node::clone(const dataguide_node* other)
+{ 
+  if (other->is_star)
+  {
+    set_star();
+    return;
+  }
+  
+  for (csize i = 0; i < other->keys.size(); ++i)
+  {
+    keys.push_back(other->keys[i]);
+    values.push_back(dataguide_node());
+    values.back().clone(&other->values[i]);
+  }
+}
+
+
+dataguide_node* dataguide_node::get(store::Item* key)
+{
+  for (csize i = 0; i < keys.size(); ++i)
+  {
+    if (keys[i]->equals(key))
+      return &values[i];
+  }
+
+  return NULL;
+}
+
+
+store::Item_t dataguide_node::get_as_json()
+{ 
+  std::vector<store::Item_t> vals;
+  std::vector<store::Item_t> ks;
+
+  for (csize i = 0; i < values.size(); ++i)
+  {
+    if (values[i].is_star)
+    {
+      store::Item_t star_string;
+      zstring star = zstring("*");
+      GENV_ITEMFACTORY->createString(star_string, star);
+      vals.push_back(star_string);
+    }
+    else
+    {
+      vals.push_back(values[i].get_as_json());
+    }
+
+    ks.push_back(keys[i]);
+  }
+  
+  if (is_star && values.size() == 0)
+  {
+    store::Item_t star_string;
+    zstring star = zstring("*");
+    GENV_ITEMFACTORY->createString(star_string, star);
+    ks.push_back(star_string);
+    
+    star = "";
+    GENV_ITEMFACTORY->createString(star_string, star);
+    vals.push_back(star_string);
+  }
+    
+  store::Item_t result;
+  GENV_ITEMFACTORY->createJSONObject(result, ks, vals);
+  
+  return result;
+}
+
+
+zstring dataguide_node::toString()
+{
+  return get_as_json()->show();  
+}
+
+
+/*******************************************************************************
+
+********************************************************************************/
+dataguide_cb::dataguide_cb()
+{
+}
+
+
+void dataguide_cb::add_to_leaves(store::Item* object_name)
+{
+  // Append the given object to each leaf node
+  map_type::iterator i = theDataguideMap.begin();
+  for ( ; i != theDataguideMap.end(); i++)
+  {
+    i->second.add_to_leaves(object_name);
+  }
+}
+
+
+void dataguide_cb::add_to_leaves(dataguide_node* other)
+{
+  // Append the given object to each leaf node
+  map_type::iterator i = theDataguideMap.begin();
+  for ( ; i != theDataguideMap.end(); i++)
+  {
+    i->second.add_to_leaves(other);
+  }
+}
+
+
+void dataguide_cb::set_star_on_leaves()
+{
+  map_type::iterator i = theDataguideMap.begin();
+  for ( ; i != theDataguideMap.end(); i++)
+  {
+    i->second.set_star_on_leaves();
+  }
+}
+
+
+void dataguide_cb::set_star_on_roots()
+{
+  map_type::iterator i = theDataguideMap.begin();
+  for ( ; i != theDataguideMap.end(); i++)
+  {
+    i->second.set_star();
+  }
+}
+
+
+dataguide_node* dataguide_cb::add_source(expr* e)
+{
+  theDataguideMap[e] = dataguide_node();
+  return &theDataguideMap[e];
+}
+
+
+void dataguide_cb::do_union(const dataguide_cb *other)
+{
+  if (this == other || other == NULL)
+    return;
+  
+  map_type::const_iterator it = other->theDataguideMap.begin();
+  for (; it != other->theDataguideMap.end(); ++it)
+  {
+    theDataguideMap[it->first].do_union(&it->second);    
+  }  
+}
+
+
+void dataguide_cb::do_union(expr* other)
+{
+  if (other == NULL)
+    return;
+  
+  do_union(other->get_dataguide());
+}
+
+
+dataguide_cb_t dataguide_cb::clone()
+{
+  dataguide_cb_t new_dg = new dataguide_cb();
+  
+  map_type::iterator it = theDataguideMap.begin();
+  for (; it != theDataguideMap.end(); ++it)
+  {    
+    dataguide_node* new_node = new_dg->add_source(it->first);
+    new_node->clone(&it->second);
+  }
+  
+  return new_dg;
+}
+
+
+dataguide_node* dataguide_cb::get_dataguide_for_source(expr* e)
+{
+  map_type::iterator it = theDataguideMap.find(e);
+  
+  if (it != theDataguideMap.end())
+    return &it->second;
+  else
+    return NULL;
+}
+
+
+bool dataguide_cb::is_empty(expr* e) 
+{  
+  return theDataguideMap[e].keys.size() == 0 || theDataguideMap[e].is_star;
+}
+
+
+store::Item_t dataguide_cb::get_as_json(expr* e)
+{
+  if (theDataguideMap[e].is_star)
+    return NULL;
+  else
+    return theDataguideMap[e].get_as_json();  
+}
+
+
+zstring dataguide_cb::toString()
+{
+  std::stringstream str;
+  str << this;
+  str << " {";
+  
+  map_type::iterator it = theDataguideMap.begin();
+  for (; it != theDataguideMap.end(); ++it)
+  {
+    str << " " << it->first << ": " << it->second.toString();
+  }  
+  
+  str << "}";
+  return str.str();
+}
+
+bool dataguide_cb::func_uses_dataguide(FunctionConsts::FunctionKind kind)
+{
+  if (kind == FunctionConsts::STATIC_COLLECTIONS_DML_COLLECTION_1 ||
+      kind == FunctionConsts::STATIC_COLLECTIONS_DML_COLLECTION_2 ||
+      kind == FunctionConsts::STATIC_COLLECTIONS_DML_COLLECTION_3 ||
+      kind == FunctionConsts::DYNAMIC_COLLECTIONS_DML_COLLECTION_1 ||
+      kind == FunctionConsts::DYNAMIC_COLLECTIONS_DML_COLLECTION_2 ||
+      kind == FunctionConsts::DYNAMIC_COLLECTIONS_DML_COLLECTION_3 ||
+      kind == FunctionConsts::FN_JSONIQ_PARSE_JSON_1 ||
+      kind == FunctionConsts::FN_JSONIQ_PARSE_JSON_2)
+    return true;
+  else
+    return false;
+}
+
+}
+/* vim:set et sw=2 ts=2: */

=== added file 'src/compiler/expression/json_dataguide.h'
--- src/compiler/expression/json_dataguide.h	1970-01-01 00:00:00 +0000
+++ src/compiler/expression/json_dataguide.h	2013-07-30 11:18:26 +0000
@@ -0,0 +1,153 @@
+/*
+ * Copyright 2006-2013 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+#ifndef ZORBA_COMPILER_JSON_DATAGUIDE
+#define ZORBA_COMPILER_JSON_DATAGUIDE
+
+#include "zorbautils/string_util.h"
+#include "store/api/item.h"
+#include "functions/function_consts.h"
+
+
+namespace zorba
+{
+
+class expr;  
+  
+
+/******************************************************************************
+  The node of a dataguide tree.
+
+  is_star:
+  --------
+  Set to true if all the key/value pairs of an object need to retrieved from
+  the JSON store. This includes all child objects.
+
+  keys:
+  values:
+  -------
+  Key=value pairs of the dataguide. If key X is contained in the keys vector,
+  then the corresponding object must be retrieved from the store. The values
+  is either:
+    -- another list of key/value pairs denoting child objects that must be
+       retreived
+    -- star: all child objects beyond this point must be retrieved.
+    -- empty: no projection information. All child nodes will be retrieved if
+       the list remains empty at the end of the analysis.
+*******************************************************************************/
+class dataguide_node
+{
+public:
+  bool                        is_star;
+  
+  std::vector<store::Item_t>  keys;
+  
+  std::vector<dataguide_node> values;
+  
+  
+public:  
+  dataguide_node() : is_star(false) { }
+  
+  void add_to_leaves(store::Item* key);
+  
+  void add_to_leaves(const dataguide_node* other);
+  
+  void set_star();
+  
+  void set_star_on_leaves();
+  
+  void do_union(const dataguide_node* other);
+  
+  // returns the child dataguide node associated with the given key,
+  // or NULL if there is none
+  dataguide_node* get(store::Item* key);
+    
+  // recursively constructs "this" to be a clone of the other dataguide
+  void clone(const dataguide_node* other);
+  
+  store::Item_t get_as_json();
+
+  zstring toString();
+};
+
+
+/******************************************************************************
+  The JSON dataguide control block
+
+  theDataguideMap:
+  ----------------
+  A mapping of expressions (keys) to dataguide trees that contain the projection
+  information. Each key in the map is a source expression which currently
+  include JSON database collection retrieval functions (db:collection) and
+  JSON parse function.
+  In a UDF, all formal parameters are added to the sources set in order to
+  allow projection information to be "added" to the expressions that invoke
+  the UDF.
+*******************************************************************************/
+class dataguide_cb : public SimpleRCObject
+{
+friend class JsonDataguide;
+
+public:
+  typedef std::map<expr*, dataguide_node> map_type;
+  
+protected:  
+  std::map<expr*, dataguide_node> theDataguideMap;
+  
+  
+public:
+  dataguide_cb();
+    
+  dataguide_node* add_source(expr* e);
+  
+  void add_to_leaves(store::Item* object_name);
+  
+  void add_to_leaves(dataguide_node* other);
+  
+  void set_star_on_leaves();
+  
+  void set_star_on_roots();
+      
+  void do_union(const dataguide_cb* other);
+  
+  void do_union(expr *other);
+  
+  dataguide_cb_t clone();
+  
+  dataguide_node* get_dataguide_for_source(expr* e);
+  
+  bool is_empty(expr *e);                                             
+  
+  store::Item_t get_as_json(expr *e);  
+  
+  zstring toString();  
+  
+public:
+
+  static bool func_uses_dataguide(FunctionConsts::FunctionKind kind);
+};
+
+
+}
+
+#endif  // ZORBA_COMPILER_JSON_DATAGUIDE
+
+/*
+ * Local variables:
+ * mode: c++
+ * End:
+ */
+/* vim:set et sw=2 ts=2: */

=== modified file 'src/compiler/expression/var_expr.cpp'
--- src/compiler/expression/var_expr.cpp	2013-04-24 01:35:58 +0000
+++ src/compiler/expression/var_expr.cpp	2013-07-30 11:18:26 +0000
@@ -23,6 +23,7 @@
 #include "compiler/expression/flwor_expr.h"
 #include "compiler/expression/expr_visitor.h"
 #include "compiler/api/compilercb.h"
+#include "compiler/expression/json_dataguide.h"
 
 #include "types/typeops.h"
 

=== modified file 'src/compiler/rewriter/framework/default_optimizer.cpp'
--- src/compiler/rewriter/framework/default_optimizer.cpp	2013-05-15 10:27:06 +0000
+++ src/compiler/rewriter/framework/default_optimizer.cpp	2013-07-30 11:18:26 +0000
@@ -266,7 +266,16 @@
       driverMarkNodeCopyProps.rewrite(rCtx);
     }
   }
-
+  
+  // Compute Json Dataguide
+  RuleOnceDriver<JsonDataguide> driverJsonDataguide;
+  driverJsonDataguide.rewrite(rCtx);
+  
+  if (Properties::instance()->printDataguide())
+  {
+    driverJsonDataguide.getRule()->printDataguides(rCtx.getRoot());
+  }
+  
   return modified;
 }
 

=== modified file 'src/compiler/rewriter/rules/nodeid_rules.cpp'
--- src/compiler/rewriter/rules/nodeid_rules.cpp	2013-06-15 02:57:08 +0000
+++ src/compiler/rewriter/rules/nodeid_rules.cpp	2013-07-30 11:18:26 +0000
@@ -36,6 +36,8 @@
 #include "functions/func_node_sort_distinct.h"
 #include "functions/udf.h"
 
+#include "compiler/expression/json_dataguide.h"
+
 #include "diagnostics/assert.h"
 
 
@@ -1538,5 +1540,292 @@
 }
 
 
+/*******************************************************************************
+
+********************************************************************************/
+expr* JsonDataguide::apply(
+    RewriterContext& rCtx,
+    expr* node,
+    bool& modified)
+{       
+  process(node, true);
+  
+  if (node->get_dataguide() != NULL)
+  {    
+    std::map<expr*,dataguide_node>::iterator i = node->get_dataguide()->theDataguideMap.begin();
+    for ( ; i != node->get_dataguide()->theDataguideMap.end(); ++i)
+    {
+      i->first->set_dataguide(node->get_dataguide());
+    }
+  }
+    
+  return NULL;
+}
+
+
+void JsonDataguide::printDataguides(expr* root)
+{
+  dataguide_cb* dg = root->get_dataguide();
+  if (dg == NULL)
+    return;
+  
+  std::map<expr*,dataguide_node>::iterator i = dg->theDataguideMap.begin();
+  for ( ; i != dg->theDataguideMap.end(); ++i)
+  {    
+    if (i->first->get_expr_kind() != fo_expr_kind)
+      continue;
+
+    fo_expr* fo = static_cast<fo_expr*>(i->first);
+    store::Item_t json_dg = dg->get_as_json(fo);
+    if (json_dg.getp())
+      std::cout << "Dataguide for function " << fo->get_func()->getName()->getStringValue() << "() at "
+                << fo->get_loc().getLineBegin() << ":" << fo->get_loc().getColumnBegin() << ": "
+                << json_dg->show() << std::endl;
+  }  
+}
+
+
+// For a given expression that is bound to a clause var (for/let/groupby), return the var_expr 
+// or NULL if expr is not bound to a clause var
+var_expr* getClauseVar(flwor_expr* flwor, expr* node, bool& is_groupby)
+{  
+  is_groupby = false;
+  
+  for (unsigned int i=0; i < flwor->num_clauses(); i++)
+  {
+    flwor_clause* c = flwor->get_clause(i);
+    if (c->get_kind() == flwor_clause::for_clause ||
+        c->get_kind() == flwor_clause::let_clause)
+    {
+      forlet_clause* fc = static_cast<forlet_clause*>(c);
+      if (fc->get_expr() == node)      
+        return fc->get_var();
+    }      
+    else if (c->get_kind() == flwor_clause::groupby_clause)
+    {
+      groupby_clause* gc = static_cast<groupby_clause*>(c);
+      flwor_clause::rebind_list_t::iterator it = gc->beginGroupVars();
+      for ( ; it != gc->endGroupVars(); ++it)
+        if (it->first == node)
+        {
+          is_groupby = true;
+          return it->second;
+        }
+      
+      it = gc->beginNonGroupVars();
+      for ( ; it != gc->endNonGroupVars(); ++it)
+        if (it->first == node)
+        {
+          is_groupby = true;
+          return it->second;
+        }
+    }
+  }
+  
+  return NULL;  
+}
+
+
+// Propagates the dataguide from the child to the parent
+void propagate_dg(expr* child, expr* node)
+{
+  if (child->get_dataguide() == NULL)
+    return;
+  
+  if (node->get_dataguide())
+  {
+    node->set_dataguide(node->get_dataguide()->clone());    
+    node->get_dataguide()->do_union(child);
+  }
+  else 
+  {    
+    node->set_dataguide(child->get_dataguide());
+  }
+}
+
+
+void JsonDataguide::iterateChildren(expr* node, bool propagates_to_output)
+{ 
+  flwor_expr* flwor = NULL;
+  bool is_groupby;
+  var_expr* clause_var;
+  
+  // If we're in a UDF root expr, add all parameter variables to the sources set
+  if (node->get_udf() != NULL && node->get_udf()->getBody() == node)
+  {
+    for (unsigned int i=0; i<node->get_udf()->numArgs(); i++)
+    {
+      var_expr* v = node->get_udf()->getArgVar(i);
+      v->get_dataguide_or_new()->add_source(v);
+    }    
+  }
+  
+  if (node->get_expr_kind() == flwor_expr_kind)
+  {
+    flwor = static_cast<flwor_expr*>(node);
+  }
+      
+  ExprIterator iter(node);
+  while (!iter.done())
+  {    
+    expr* child = (**iter);      
+    if (child == NULL)
+      continue;
+    
+    bool child_propagates_to_output = propagates_to_output;
+    
+    if (child->get_expr_kind() == var_decl_expr_kind ||
+        (node->get_expr_kind() == fo_expr_kind && static_cast<fo_expr*>(node)->get_func()->isUdf()) || 
+        (node->get_expr_kind() == if_expr_kind && static_cast<if_expr*>(node)->get_cond_expr() == child))      
+      child_propagates_to_output = false;
+      
+    if (flwor && (clause_var = getClauseVar(flwor, child, is_groupby)))
+    {           
+      process(child, false);
+      clause_var->set_dataguide(child->get_dataguide());      
+      if (is_groupby)
+        propagate_dg(child, node);
+    }    
+    else
+    {       
+      process(child, child_propagates_to_output);
+      propagate_dg(child, node);
+    }
+            
+    iter.next();
+  } // while
+    
+  // std::cerr << "--> " << node << " = " << node->get_expr_kind_string() << " propagates_to_output: " << propagates_to_output << " dataguide: " << (node->get_dataguide() ? node->get_dataguide()->toString() : "") << std::endl;
+}
+
+
+void JsonDataguide::process(expr* node, bool propagates_to_output)
+{  
+  iterateChildren(node, propagates_to_output);
+  
+  switch (node->get_expr_kind())
+  {  
+  case dynamic_function_invocation_expr_kind:
+  {
+    dynamic_function_invocation_expr* fo = static_cast<dynamic_function_invocation_expr*>(node);
+    expr* sourceExpr = fo->get_function();
+    TypeManager* tm = sourceExpr->get_type_manager();
+    xqtref_t sourceType = sourceExpr->get_return_type();
+
+    if ((TypeOps::is_subtype(tm, *sourceType, *GENV_TYPESYSTEM.JSON_ITEM_TYPE_STAR) 
+          || TypeOps::is_subtype(tm, *sourceType, *GENV_TYPESYSTEM.STRUCTURED_ITEM_TYPE_STAR))
+        && fo->get_args().size() == 1
+        && fo->get_args()[0]->get_expr_kind() == const_expr_kind)
+    {
+      dataguide_cb_t dg = fo->get_dataguide() ? fo->get_dataguide()->clone().getp() : new dataguide_cb();      
+      dg->add_to_leaves(static_cast<const_expr*>(fo->get_args()[0])->get_val());         
+      fo->set_dataguide(dg);
+    }
+
+    break;
+  }   
+  case fo_expr_kind :
+  {
+    fo_expr* fo = static_cast<fo_expr*>(node);
+    function* f = fo->get_func();
+    if (fo->get_dataguide() && f->getKind() == FunctionConsts::FN_JSONIQ_VALUE_2)
+    {      
+      if (fo->get_arg(1)->get_expr_kind() == const_expr_kind)
+      {                 
+        fo->set_dataguide(fo->get_dataguide()->clone());
+        fo->get_dataguide()->add_to_leaves(static_cast<const_expr*>(fo->get_arg(1))->get_val());
+      }
+      else
+      {        
+        fo->get_dataguide()->set_star_on_leaves();
+      }      
+    }
+    else if (dataguide_cb::func_uses_dataguide(f->getKind()))
+    {      
+      fo->get_dataguide_or_new()->add_source(fo);
+    }
+    else if (f->isExternal()
+             ||
+             (f->getAnnotationList()
+              &&
+              f->getAnnotationList()->contains(AnnotationInternal::zann_explores_json)))
+    {
+      if (fo->get_dataguide())
+        fo->get_dataguide()->set_star_on_leaves();
+    }
+    else if (f->isUdf())
+    {
+      user_function* udf = static_cast<user_function*>(f);
+
+      // The body of a UDF will be NULL when compiling an eval expression and
+      // plan serialization has been used. Assume the function is recursive in this case.
+      if (udf->getBody() == NULL || udf->isRecursive())
+      {
+        if (fo->get_dataguide())
+          fo->get_dataguide()->set_star_on_leaves();
+      }    
+      else
+      {
+        // Iterate through the effective parameters and prepend to them the function's formal parameter's dataguide
+        for (unsigned int i=0; i<udf->numArgs(); i++)
+        { 
+          dataguide_cb_t new_dg;
+          dataguide_node* var_dg = NULL;
+
+          if (udf->getBody()->get_dataguide())
+              var_dg = udf->getBody()->get_dataguide()->get_dataguide_for_source(udf->getArgVar(i));
+
+          if (var_dg)     
+          {            
+            if (fo->get_arg(i)->get_dataguide())
+              new_dg = fo->get_arg(i)->get_dataguide()->clone();
+            else
+              new_dg = new dataguide_cb();
+            
+            new_dg->add_to_leaves(var_dg);
+          }
+            
+          if (fo->get_dataguide())
+            fo->get_dataguide()->do_union(new_dg);
+          else
+            fo->set_dataguide(new_dg);
+        } // for
+      }
+    }
+    
+    break;
+  }  
+  case var_decl_expr_kind:
+  {
+    var_decl_expr* vd = static_cast<var_decl_expr*>(node);
+    if (vd->get_expr() && vd->get_var_expr())
+      vd->get_var_expr()->set_dataguide(vd->get_expr()->get_dataguide());
+    break;
+  }
+  case eval_expr_kind:
+  {
+    // invalidate all dataguides
+    dataguide_cb* dg = node->get_dataguide_or_new();
+    dg->set_star_on_roots();    
+    break;
+  }
+
+  case wrapper_expr_kind:
+  case var_expr_kind:
+    // skip setting star even if the result propagates to output
+    return;
+
+  default:
+    break;
+  } // switch
+
+  if (propagates_to_output && node->get_dataguide())
+  {     
+    node->get_dataguide()->set_star_on_leaves();
+  }  
+  
+}
+
+
 }
 /* vim:set et sw=2 ts=2: */

=== modified file 'src/compiler/rewriter/rules/rule_base.h'
--- src/compiler/rewriter/rules/rule_base.h	2013-02-07 17:24:36 +0000
+++ src/compiler/rewriter/rules/rule_base.h	2013-07-30 11:18:26 +0000
@@ -55,7 +55,8 @@
     InlineFunctions,
     PartialEval,
     EchoNodes,
-    PlanPrinter
+    PlanPrinter,
+    JsonDataguide
   } RuleKind;
 
 private:

=== modified file 'src/compiler/rewriter/rules/ruleset.h'
--- src/compiler/rewriter/rules/ruleset.h	2013-05-13 10:10:08 +0000
+++ src/compiler/rewriter/rules/ruleset.h	2013-07-30 11:18:26 +0000
@@ -223,6 +223,31 @@
 };
 
 
+/*******************************************************************************
+
+********************************************************************************/
+class JsonDataguide : public RewriteRule
+{
+public:
+  JsonDataguide()
+    :
+    RewriteRule(RewriteRule::JsonDataguide, "JsonDataguide")
+  {
+  }
+
+  expr* apply(RewriterContext& rCtx, expr* node, bool& modified);
+  
+public:
+  // used for automated testing
+  void printDataguides(expr* root); 
+  
+protected:
+  void iterateChildren(expr* node, bool propagates_to_output);
+  
+  void process(expr* node, bool propagates_to_output);  
+};
+
+
 }
 
 #endif /* ZORBA_REWRITE_RULE_H */

=== modified file 'src/functions/function.cpp'
--- src/functions/function.cpp	2013-07-06 06:59:32 +0000
+++ src/functions/function.cpp	2013-07-30 11:18:26 +0000
@@ -135,6 +135,30 @@
 }
 
 
+void function::addAnnotation(AnnotationInternal::AnnotationId id)
+{
+  if (theAnnotationList == NULL)
+    theAnnotationList = new AnnotationList();
+  
+  if (theAnnotationList->contains(id))
+    return;
+  
+  theAnnotationList->push_back(id);
+  
+  if (theAnnotationList->contains(AnnotationInternal::zann_nondeterministic))
+    setDeterministic(false);
+
+  setPrivate(theAnnotationList->contains(AnnotationInternal::fn_private));
+
+  if (isUpdating() &&
+      theAnnotationList->contains(AnnotationInternal::zann_sequential))
+  {
+    throw XQUERY_EXCEPTION(zerr::XSST0001,
+    ERROR_PARAMS(getName()->getStringValue()));
+  }
+}
+
+
 /*******************************************************************************
   This is a virstual method. It is redefined by udf and external-function
   classes. 

=== modified file 'src/functions/function.h'
--- src/functions/function.h	2013-07-06 06:59:32 +0000
+++ src/functions/function.h	2013-07-30 11:18:26 +0000
@@ -51,7 +51,7 @@
 class function : public SimpleRCObject
 {
 protected:
-	signature                    theSignature;
+  signature                    theSignature;
   FunctionConsts::FunctionKind theKind;
   uint32_t                     theFlags;
   AnnotationList             * theAnnotationList;
@@ -163,8 +163,10 @@
   void setAnnotations(AnnotationList* annotations);
 
   const AnnotationList* getAnnotationList() const { return theAnnotationList; }
+  
+  void addAnnotation(AnnotationInternal::AnnotationId id);
 
-	bool validate_args(std::vector<PlanIter_t>& argv) const;
+  bool validate_args(std::vector<PlanIter_t>& argv) const;
 
   bool isUpdating() const { return (getScriptingKind() & UPDATING_EXPR) != 0; }
 

=== modified file 'src/functions/pregenerated/func_accessors.h'
--- src/functions/pregenerated/func_accessors.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_accessors.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -62,7 +62,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -79,7 +78,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -95,7 +94,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -112,7 +110,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -129,7 +126,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -154,7 +150,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -171,7 +166,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -187,7 +182,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -204,7 +198,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }

=== modified file 'src/functions/pregenerated/func_any_uri.h'
--- src/functions/pregenerated/func_any_uri.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_any_uri.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }

=== modified file 'src/functions/pregenerated/func_base64.h'
--- src/functions/pregenerated/func_base64.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_base64.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_booleans.h'
--- src/functions/pregenerated/func_booleans.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_booleans.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -63,7 +62,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -80,7 +78,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }

=== modified file 'src/functions/pregenerated/func_collections.h'
--- src/functions/pregenerated/func_collections.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_collections.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -67,7 +66,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -88,7 +86,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -105,7 +102,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -122,7 +118,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -141,7 +136,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -172,7 +166,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -191,7 +184,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -210,7 +202,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -233,7 +225,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -256,7 +248,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -279,7 +271,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -302,7 +294,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -325,7 +317,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return APPLYING_EXPR; }
@@ -350,7 +342,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return APPLYING_EXPR; }
@@ -375,7 +367,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return APPLYING_EXPR; }
@@ -400,7 +392,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return APPLYING_EXPR; }
@@ -425,7 +417,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return APPLYING_EXPR; }
@@ -450,7 +442,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -475,7 +466,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -496,7 +486,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -517,7 +506,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -540,7 +529,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -559,7 +547,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -576,7 +563,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -593,7 +579,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -610,7 +595,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -627,7 +611,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -642,7 +625,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -657,7 +639,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -672,7 +653,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -687,7 +667,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -702,7 +681,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -717,7 +695,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_context.h'
--- src/functions/pregenerated/func_context.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_context.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -63,7 +62,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -80,7 +78,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -97,7 +94,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -114,7 +110,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -131,7 +126,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -146,7 +140,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -161,7 +154,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_datetime.h'
--- src/functions/pregenerated/func_datetime.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_datetime.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -76,7 +74,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -91,7 +88,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -106,7 +102,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -121,7 +116,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -136,7 +130,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -151,7 +144,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -166,7 +158,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_documents.h'
--- src/functions/pregenerated/func_documents.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_documents.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -67,7 +66,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -86,7 +84,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -103,7 +100,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -120,7 +116,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }

=== modified file 'src/functions/pregenerated/func_durations_dates_times.h'
--- src/functions/pregenerated/func_durations_dates_times.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_durations_dates_times.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -76,7 +74,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -91,7 +88,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -106,7 +102,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -121,7 +116,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -136,7 +130,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -151,7 +144,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -166,7 +158,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -181,7 +172,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -196,7 +186,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -211,7 +200,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -226,7 +214,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -241,7 +228,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -256,7 +242,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -271,7 +256,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -286,7 +270,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -301,7 +284,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -316,7 +298,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -331,7 +312,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -346,7 +326,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_errors_and_diagnostics.h'
--- src/functions/pregenerated/func_errors_and_diagnostics.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_errors_and_diagnostics.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return VACUOUS_EXPR; }
@@ -65,7 +64,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool isMap(csize producer) const { return producer == 0; }

=== modified file 'src/functions/pregenerated/func_fetch.h'
--- src/functions/pregenerated/func_fetch.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_fetch.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -63,7 +62,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -80,7 +78,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }

=== modified file 'src/functions/pregenerated/func_fn_hof_functions.h'
--- src/functions/pregenerated/func_fn_hof_functions.h	2013-06-04 21:47:40 +0000
+++ src/functions/pregenerated/func_fn_hof_functions.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -61,7 +61,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -76,7 +76,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -91,7 +91,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -106,7 +106,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -121,7 +121,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -136,7 +136,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_fnput.h'
--- src/functions/pregenerated/func_fnput.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_fnput.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }

=== modified file 'src/functions/pregenerated/func_ft_module.h'
--- src/functions/pregenerated/func_ft_module.h	2013-06-26 00:10:23 +0000
+++ src/functions/pregenerated/func_ft_module.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -62,7 +61,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -78,7 +76,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -94,7 +91,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -110,7 +106,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -126,7 +121,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -142,7 +136,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -158,7 +151,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -174,7 +166,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -190,7 +181,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -206,7 +196,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -222,7 +211,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -238,7 +226,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -254,7 +241,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -270,7 +256,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_ic_ddl.h'
--- src/functions/pregenerated/func_ic_ddl.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_ic_ddl.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -65,7 +64,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -84,7 +82,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }

=== modified file 'src/functions/pregenerated/func_index_func.h'
--- src/functions/pregenerated/func_index_func.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_index_func.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }

=== modified file 'src/functions/pregenerated/func_item.h'
--- src/functions/pregenerated/func_item.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_item.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_json.h'
--- src/functions/pregenerated/func_json.h	2013-07-11 00:46:02 +0000
+++ src/functions/pregenerated/func_json.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_jsoniq_functions.h'
--- src/functions/pregenerated/func_jsoniq_functions.h	2013-06-08 05:33:57 +0000
+++ src/functions/pregenerated/func_jsoniq_functions.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return false; }
@@ -65,7 +64,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return false; }
@@ -84,7 +83,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -103,7 +101,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -122,7 +119,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return producer == 0; }
@@ -141,7 +137,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return false; }
@@ -162,7 +158,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return false; }
@@ -181,7 +176,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return producer == 0; }
@@ -200,7 +194,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return producer == 0; }
@@ -219,7 +213,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return producer == 0; }
@@ -238,7 +231,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return producer == 0; }
@@ -259,7 +251,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return producer == 0; }
@@ -278,7 +269,7 @@
     : 
     function(sig, kind)
   {
-
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return false; }
@@ -297,7 +288,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool propagatesInputNodes(expr* fo, csize producer) const { return true; }
@@ -316,7 +306,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -331,7 +320,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -346,7 +334,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -367,7 +354,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -388,7 +374,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -407,7 +392,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -428,7 +412,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -447,7 +430,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -468,7 +450,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;

=== modified file 'src/functions/pregenerated/func_maps.h'
--- src/functions/pregenerated/func_maps.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_maps.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return SEQUENTIAL_FUNC_EXPR; }
@@ -65,7 +64,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return SEQUENTIAL_FUNC_EXPR; }
@@ -84,7 +82,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return SEQUENTIAL_FUNC_EXPR; }
@@ -103,7 +100,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -120,7 +116,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return SEQUENTIAL_FUNC_EXPR; }
@@ -141,7 +136,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return SEQUENTIAL_FUNC_EXPR; }
@@ -160,7 +154,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -177,7 +170,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -194,7 +186,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -211,7 +202,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }

=== modified file 'src/functions/pregenerated/func_maths.h'
--- src/functions/pregenerated/func_maths.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_maths.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -76,7 +74,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -91,7 +88,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -106,7 +102,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -121,7 +116,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -136,7 +130,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -151,7 +144,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -166,7 +158,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -181,7 +172,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -196,7 +186,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -211,7 +200,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -226,7 +214,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -241,7 +228,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -256,7 +242,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -271,7 +256,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -286,7 +270,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -301,7 +284,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -316,7 +298,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -331,7 +312,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -346,7 +326,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -361,7 +340,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -376,7 +354,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -391,7 +368,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -406,7 +382,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -421,7 +396,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_node_position.h'
--- src/functions/pregenerated/func_node_position.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_node_position.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -76,7 +74,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -91,7 +88,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -106,7 +102,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -121,7 +116,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -136,7 +130,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -151,7 +144,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -166,7 +158,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -181,7 +172,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -196,7 +186,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -211,7 +200,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -226,7 +214,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -241,7 +228,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -256,7 +242,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -271,7 +256,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -286,7 +270,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -301,7 +284,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -316,7 +298,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -331,7 +312,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -346,7 +326,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -361,7 +340,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -376,7 +354,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -391,7 +368,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_nodes.h'
--- src/functions/pregenerated/func_nodes.h	2013-07-24 08:12:12 +0000
+++ src/functions/pregenerated/func_nodes.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -63,7 +62,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -80,7 +78,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -97,7 +94,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -112,7 +108,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return false; }
@@ -129,7 +125,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -146,7 +142,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -163,7 +159,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -180,7 +176,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -197,7 +192,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -214,7 +208,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -231,7 +224,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -248,7 +240,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -265,7 +256,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -282,7 +272,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -299,7 +288,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -316,7 +304,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -333,7 +320,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -350,7 +336,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -365,7 +351,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const;

=== modified file 'src/functions/pregenerated/func_numerics.h'
--- src/functions/pregenerated/func_numerics.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_numerics.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -76,7 +74,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -93,7 +90,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -107,7 +104,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -122,7 +118,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -137,7 +132,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -152,7 +147,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_other_diagnostics.h'
--- src/functions/pregenerated/func_other_diagnostics.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_other_diagnostics.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -63,7 +62,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }

=== modified file 'src/functions/pregenerated/func_parse_fragment.h'
--- src/functions/pregenerated/func_parse_fragment.h	2013-03-06 07:39:18 +0000
+++ src/functions/pregenerated/func_parse_fragment.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -63,7 +62,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -78,7 +76,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   bool accessesDynCtx() const { return true; }

=== modified file 'src/functions/pregenerated/func_parsing_and_serializing.h'
--- src/functions/pregenerated/func_parsing_and_serializing.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_parsing_and_serializing.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   bool accessesDynCtx() const { return true; }
@@ -65,7 +65,8 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    addAnnotation(AnnotationInternal::zann_explores_json);
   }
 
   bool accessesDynCtx() const { return true; }

=== modified file 'src/functions/pregenerated/func_qnames.h'
--- src/functions/pregenerated/func_qnames.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_qnames.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return producer == 1; }
@@ -63,7 +62,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -78,7 +76,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -93,7 +90,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -108,7 +104,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -123,7 +118,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -138,7 +132,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return producer == 1; }
@@ -155,7 +148,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }

=== modified file 'src/functions/pregenerated/func_random.h'
--- src/functions/pregenerated/func_random.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_random.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -77,7 +75,6 @@
     function(sig, kind)
   {
 setDeterministic(false);
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_reference.h'
--- src/functions/pregenerated/func_reference.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_reference.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const { return true; }
@@ -63,7 +62,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -78,7 +76,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return SEQUENTIAL_FUNC_EXPR; }
@@ -95,7 +92,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_schema.h'
--- src/functions/pregenerated/func_schema.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_schema.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   unsigned short getScriptingKind() const { return UPDATING_EXPR; }
@@ -67,7 +66,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const;
@@ -84,7 +82,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool mustCopyInputNodes(expr* fo, csize producer) const;

=== modified file 'src/functions/pregenerated/func_sctx.h'
--- src/functions/pregenerated/func_sctx.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_sctx.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -76,7 +74,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -91,7 +88,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -106,7 +102,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -121,7 +116,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -136,7 +130,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -151,7 +144,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -166,7 +158,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -181,7 +172,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -196,7 +186,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -211,7 +200,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -226,7 +214,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -241,7 +228,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -256,7 +242,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -271,7 +256,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -286,7 +270,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -301,7 +284,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -316,7 +298,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -331,7 +312,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -346,7 +326,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -361,7 +340,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -376,7 +354,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -391,7 +368,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -406,7 +382,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_sequences.h'
--- src/functions/pregenerated/func_sequences.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_sequences.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -69,7 +68,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -84,7 +82,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize producer) const;
@@ -105,7 +102,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize producer) const;
@@ -126,7 +122,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -147,7 +142,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -170,7 +164,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -197,7 +190,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -222,7 +214,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -247,7 +238,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -270,7 +260,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -293,7 +282,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -314,7 +302,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   xqtref_t getReturnType(const fo_expr* caller) const;
@@ -343,7 +330,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -362,7 +348,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize producer) const;
@@ -381,7 +366,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize producer) const;
@@ -398,7 +382,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool specializable() const { return true; }
@@ -420,7 +403,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize producer) const;
@@ -437,7 +419,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize producer) const;
@@ -454,7 +435,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize producer) const;
@@ -471,7 +451,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize producer) const;
@@ -488,7 +467,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -503,7 +481,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   FunctionConsts::AnnotationValue producesDistinctNodes() const 
@@ -534,7 +511,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   FunctionConsts::AnnotationValue producesDistinctNodes() const 
@@ -565,7 +541,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   FunctionConsts::AnnotationValue producesDistinctNodes() const 
@@ -596,7 +571,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool accessesDynCtx() const { return true; }
@@ -615,7 +589,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -630,7 +603,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -645,7 +618,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -660,7 +633,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -675,7 +648,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -690,7 +663,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_strings.h'
--- src/functions/pregenerated/func_strings.h	2013-07-11 22:08:29 +0000
+++ src/functions/pregenerated/func_strings.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -76,7 +74,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -91,7 +88,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -106,7 +102,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize producer) const;
@@ -125,7 +120,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -139,7 +134,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -154,7 +148,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   bool specializable() const { return true; }
@@ -174,7 +167,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -189,7 +181,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -204,7 +195,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -219,7 +209,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -234,7 +223,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -249,7 +237,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -264,7 +251,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -279,7 +265,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -294,7 +279,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -309,7 +293,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -324,7 +307,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -339,7 +321,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -354,7 +335,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -369,7 +349,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -384,7 +363,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -399,7 +377,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -414,7 +391,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -429,7 +405,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -444,7 +419,7 @@
     : 
     function(sig, kind)
   {
-theXQueryVersion = StaticContextConsts::xquery_version_3_0;
+    theXQueryVersion = StaticContextConsts::xquery_version_3_0;
   }
 
   CODEGEN_DECL();
@@ -459,7 +434,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -474,7 +448,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -504,7 +477,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_uris.h'
--- src/functions/pregenerated/func_uris.h	2013-04-12 21:25:35 +0000
+++ src/functions/pregenerated/func_uris.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -76,7 +74,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/functions/pregenerated/func_xqdoc.h'
--- src/functions/pregenerated/func_xqdoc.h	2013-03-05 23:11:50 +0000
+++ src/functions/pregenerated/func_xqdoc.h	2013-07-30 11:18:26 +0000
@@ -46,7 +46,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();
@@ -61,7 +60,6 @@
     : 
     function(sig, kind)
   {
-
   }
 
   CODEGEN_DECL();

=== modified file 'src/runtime/collections/collections_impl.cpp'
--- src/runtime/collections/collections_impl.cpp	2013-07-25 14:09:47 +0000
+++ src/runtime/collections/collections_impl.cpp	2013-07-30 11:18:26 +0000
@@ -61,8 +61,9 @@
     const static_context* sctx,
     const store::Item_t& name,
     const QueryLoc& loc,
-    bool dynamic,
-    store::Collection_t& coll)
+    bool dynamic,    
+    store::Collection_t& coll,
+    store::Item_t dataguide = NULL)
 {
   const StaticallyKnownCollection* collectionDecl = sctx->lookup_collection(name);
 
@@ -72,7 +73,7 @@
     ERROR_PARAMS(name->getStringValue()));
   }
 
-  coll = GENV_STORE.getCollection(name, dynamic);
+  coll = GENV_STORE.getCollection(name, dynamic, dataguide);
 
   if (coll == NULL)
   {
@@ -287,7 +288,7 @@
 
   consumeNext(name, theChildren[0].getp(), planState);
 
-  (void)getCollection(theSctx, name, loc, theIsDynamic, collection);
+  (void)getCollection(theSctx, name, loc, theIsDynamic, collection, theDataguide);
 
   if (theChildren.size() == 1)
   {

=== modified file 'src/runtime/collections/pregenerated/collections.cpp'
--- src/runtime/collections/pregenerated/collections.cpp	2013-03-05 23:11:50 +0000
+++ src/runtime/collections/pregenerated/collections.cpp	2013-07-30 11:18:26 +0000
@@ -194,6 +194,7 @@
   (NaryBaseIterator<ZorbaCollectionIterator, ZorbaCollectionIteratorState>*)this);
 
     ar & theIsDynamic;
+    ar & theDataguide;
 }
 
 

=== modified file 'src/runtime/collections/pregenerated/collections.h'
--- src/runtime/collections/pregenerated/collections.h	2013-07-15 10:32:25 +0000
+++ src/runtime/collections/pregenerated/collections.h	2013-07-30 11:18:26 +0000
@@ -266,6 +266,7 @@
 { 
 protected:
   bool theIsDynamic; //
+  store::Item_t theDataguide; //
 public:
   SERIALIZABLE_CLASS(ZorbaCollectionIterator);
 
@@ -281,13 +282,16 @@
     bool isDynamic)
     : 
     NaryBaseIterator<ZorbaCollectionIterator, ZorbaCollectionIteratorState>(sctx, loc, children),
-    theIsDynamic(isDynamic)
+    theIsDynamic(isDynamic),
+    theDataguide()
   {}
 
   virtual ~ZorbaCollectionIterator();
 
   bool isDynamic() const { return theIsDynamic; }
 
+  void setDataguide(store::Item_t aValue) { theDataguide= aValue; }
+
 public:
   bool isCountOptimizable() const;
   bool count(store::Item_t& result, PlanState& planState) const;

=== modified file 'src/runtime/json/json_loader.cpp'
--- src/runtime/json/json_loader.cpp	2013-06-19 15:35:24 +0000
+++ src/runtime/json/json_loader.cpp	2013-07-30 11:18:26 +0000
@@ -80,10 +80,11 @@
 
 ///////////////////////////////////////////////////////////////////////////////
 
-loader::loader( istream &is, bool allow_multiple, bool strip_top_level_array ) :
+loader::loader( istream &is, bool allow_multiple, bool strip_top_level_array, const store::Item *aDataguide) :
   parser_( is, allow_multiple ),
   strip_top_level_array_( strip_top_level_array ),
-  stripped_top_level_array_( false )
+  stripped_top_level_array_( false ),
+  dataguide(aDataguide)
 {
 }
 

=== modified file 'src/runtime/json/json_loader.h'
--- src/runtime/json/json_loader.h	2012-12-03 18:29:05 +0000
+++ src/runtime/json/json_loader.h	2013-07-30 11:18:26 +0000
@@ -47,8 +47,10 @@
    * @param strip_top_level_array If \c true, strips the top-level array, if
    * any.
    */
-  loader( std::istream &is, bool allow_multiple = false,
-          bool strip_top_level_array = false );
+  loader( std::istream &is, 
+          bool allow_multiple = false,
+          bool strip_top_level_array = false,
+          const store::Item* aDataguide = NULL);
 
   /**
    * Destroys this %loader.
@@ -123,6 +125,7 @@
   parser parser_;
   bool const strip_top_level_array_;
   bool stripped_top_level_array_;
+  const store::Item* dataguide;
 };
 
 ///////////////////////////////////////////////////////////////////////////////

=== modified file 'src/runtime/json/jsoniq_functions_impl.cpp'
--- src/runtime/json/jsoniq_functions_impl.cpp	2013-06-18 18:55:33 +0000
+++ src/runtime/json/jsoniq_functions_impl.cpp	2013-07-30 11:18:26 +0000
@@ -863,7 +863,7 @@
     }
 
     state->loader_ = new json::loader(
-      *state->theInputStream, true, lStripTopLevelArray
+      *state->theInputStream, true, lStripTopLevelArray, theDataguide
     );
 
     if ( state->theInput == NULL && theRelativeLocation ) {

=== modified file 'src/runtime/json/pregenerated/jsoniq_functions.cpp'
--- src/runtime/json/pregenerated/jsoniq_functions.cpp	2013-06-18 18:55:33 +0000
+++ src/runtime/json/pregenerated/jsoniq_functions.cpp	2013-07-30 11:18:26 +0000
@@ -121,6 +121,7 @@
   (NaryBaseIterator<JSONParseIterator, JSONParseIteratorState>*)this);
 
     ar & theRelativeLocation;
+    ar & theDataguide;
 }
 
 

=== modified file 'src/runtime/json/pregenerated/jsoniq_functions.h'
--- src/runtime/json/pregenerated/jsoniq_functions.h	2013-06-18 18:55:33 +0000
+++ src/runtime/json/pregenerated/jsoniq_functions.h	2013-07-30 11:18:26 +0000
@@ -163,6 +163,7 @@
 { 
 protected:
   QueryLoc theRelativeLocation; //
+  store::Item_t theDataguide; //
 public:
   SERIALIZABLE_CLASS(JSONParseIterator);
 
@@ -178,11 +179,14 @@
     QueryLoc aRelativeLocation)
     : 
     NaryBaseIterator<JSONParseIterator, JSONParseIteratorState>(sctx, loc, children),
-    theRelativeLocation(aRelativeLocation)
+    theRelativeLocation(aRelativeLocation),
+    theDataguide()
   {}
 
   virtual ~JSONParseIterator();
 
+  void setDataguide(store::Item_t aValue) { theDataguide= aValue; }
+
 public:
   bool processBooleanOption(const store::Item_t& options, char const* option_name, bool* option_value) const;
   void accept(PlanIterVisitor& v) const;

=== modified file 'src/runtime/spec/codegen-h.xq'
--- src/runtime/spec/codegen-h.xq	2013-02-07 17:24:36 +0000
+++ src/runtime/spec/codegen-h.xq	2013-07-30 11:18:26 +0000
@@ -89,6 +89,12 @@
                                  then 
                                    "setDeterministic(false);&#xA;"
                                  else ""
+    let $annotations := string-join(
+                          for $a in $function//zorba:annotation
+                          return concat($gen:indent, $gen:indent, "addAnnotation(AnnotationInternal::", $a, ");", $gen:newline),
+                          '')
+    let $funcVersionStr := if ($funcVersion eq "") then ""
+                           else concat($gen:indent, $gen:indent, $funcVersion, $gen:newline)
     return 
       concat($name, '(const signature&amp; sig, FunctionConsts::FunctionKind kind)',
              $gen:newline, $gen:indent,
@@ -96,8 +102,12 @@
              $gen:newline, $gen:indent, $gen:indent,
              local:base-class($function), 
              '(sig, kind)',
-             $gen:newline, $gen:indent,
-             '{&#xA;', $setNoneDeterministic, $funcVersion, '&#xA;', $gen:indent, '}'),
+             $gen:newline, 
+             $gen:indent, '{', $gen:newline,
+               $setNoneDeterministic, 
+               $funcVersionStr,
+               $annotations,
+             $gen:indent, '}'),
   
   $gen:newline,
         

=== modified file 'src/runtime/spec/collections/collections.xml'
--- src/runtime/spec/collections/collections.xml	2013-07-15 10:32:25 +0000
+++ src/runtime/spec/collections/collections.xml	2013-07-30 11:18:26 +0000
@@ -12,7 +12,7 @@
   xsi:schemaLocation="http://www.zorba-xquery.com ../runtime.xsd">
 
 <zorba:source>
-    <zorba:include  form="Quoted">store/api/iterator.h</zorba:include>
+    <zorba:include form="Quoted">store/api/iterator.h</zorba:include>
 </zorba:source>
 
 <zorba:header>
@@ -306,6 +306,7 @@
     </zorba:constructor>
 
     <zorba:member type="bool" name="theIsDynamic" getterName="isDynamic"/>
+    <zorba:member type="store::Item_t" name="theDataguide" setterName="setDataguide"/>
 
     <zorba:method const="true" name="isCountOptimizable" return="bool" />
     
@@ -415,6 +416,7 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
+      <zorba:annotation>zann_explores_json</zorba:annotation>
 
       <zorba:signature localname="insert-nodes" prefix="static-collections-dml">
         <zorba:param>xs:QName</zorba:param>
@@ -471,7 +473,8 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
-
+      <zorba:annotation>zann_explores_json</zorba:annotation>
+      
       <zorba:signature localname="insert-nodes-first" prefix="static-collections-dml">
         <zorba:param>xs:QName</zorba:param>
         <zorba:param>node()*</zorba:param>
@@ -532,6 +535,7 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
+      <zorba:annotation>zann_explores_json</zorba:annotation>
 
       <zorba:signature localname="insert-nodes-last" prefix="static-collections-dml">
         <zorba:param>xs:QName</zorba:param>
@@ -592,6 +596,7 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
+      <zorba:annotation>zann_explores_json</zorba:annotation>
 
       <zorba:signature localname="insert-nodes-before" prefix="static-collections-dml">
         <zorba:param>xs:QName</zorba:param>
@@ -656,7 +661,8 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
-
+      <zorba:annotation>zann_explores_json</zorba:annotation>
+      
       <zorba:signature localname="insert-nodes-after" prefix="static-collections-dml">
         <zorba:param>xs:QName</zorba:param>
         <zorba:param>node()</zorba:param>
@@ -722,6 +728,7 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
+      <zorba:annotation>zann_explores_json</zorba:annotation>
 
       <zorba:signature localname="apply-insert-nodes" prefix="static-collections-dml">
         <zorba:param>xs:QName</zorba:param>
@@ -786,6 +793,7 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
+      <zorba:annotation>zann_explores_json</zorba:annotation>
 
       <zorba:signature localname="apply-insert-nodes-first" 
                        prefix="static-collections-dml">
@@ -856,6 +864,7 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
+      <zorba:annotation>zann_explores_json</zorba:annotation>
 
       <zorba:signature localname="apply-insert-nodes-last" 
                        prefix="static-collections-dml">
@@ -926,6 +935,7 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
+      <zorba:annotation>zann_explores_json</zorba:annotation>
 
       <zorba:signature localname="apply-insert-nodes-before" 
                        prefix="static-collections-dml">
@@ -1000,6 +1010,7 @@
     </zorba:description>
 
     <zorba:function generateCodegen="false">
+      <zorba:annotation>zann_explores_json</zorba:annotation>
 
       <zorba:signature localname="apply-insert-nodes-after" 
                        prefix="static-collections-dml">
@@ -1281,6 +1292,7 @@
     </zorba:description>
     
     <zorba:function generateCodegen="false">
+      <zorba:annotation>zann_explores_json</zorba:annotation>
       
       <zorba:signature localname="edit" prefix="static-collections-dml">
         <zorba:param>structured-item()</zorba:param>

=== modified file 'src/runtime/spec/json/jsoniq_functions.xml'
--- src/runtime/spec/json/jsoniq_functions.xml	2013-06-18 18:55:33 +0000
+++ src/runtime/spec/json/jsoniq_functions.xml	2013-07-30 11:18:26 +0000
@@ -92,6 +92,7 @@
 <zorba:iterator name="JSONEncodeForRoundtripIterator">
 
   <zorba:function isDeterministic="true">
+    <zorba:annotation>zann_explores_json</zorba:annotation>
 
     <zorba:signature localname="encode-for-roundtrip" prefix="fn-jsoniq">
       <zorba:param>item()*</zorba:param>
@@ -188,6 +189,7 @@
   </zorba:constructor>
 
   <zorba:member type="QueryLoc" name="theRelativeLocation" />
+  <zorba:member type="store::Item_t" name="theDataguide" setterName="setDataguide"/>
 
   <zorba:method return="bool" name="processBooleanOption" const="true">
     <zorba:param type="const store::Item_t&amp;" name="options"/>
@@ -273,6 +275,8 @@
 
   <zorba:function isDeterministic="true"
                   generateCodegen="false">
+                  
+    <zorba:annotation>zann_explores_json</zorba:annotation>
 
     <zorba:signature localname="keys" prefix="fn-jsoniq">
       <zorba:param>item()*</zorba:param>
@@ -361,6 +365,10 @@
 <zorba:iterator name="JSONObjectProjectIterator" arity="binary">
 
   <zorba:function isDeterministic="true">
+    <!-- TODO: project() could be used for document projection if the second
+         parameter is all consts. If this is done, this annotation should be
+         removed -->
+    <zorba:annotation>zann_explores_json</zorba:annotation>
 
     <zorba:signature localname="project" prefix="fn-jsoniq">
       <zorba:param>object()</zorba:param>
@@ -475,6 +483,8 @@
 <zorba:iterator name="JSONArraySizeIterator" arity="unary">
 
   <zorba:function isDeterministic="true">
+    <zorba:annotation>zann_explores_json</zorba:annotation>
+    
     <zorba:signature localname="size" prefix="fn-jsoniq">
       <zorba:param>array()</zorba:param>
       <zorba:output>xs:integer</zorba:output>

=== modified file 'src/runtime/spec/parsing_and_serializing/parsing_and_serializing.xml'
--- src/runtime/spec/parsing_and_serializing/parsing_and_serializing.xml	2013-02-07 17:24:36 +0000
+++ src/runtime/spec/parsing_and_serializing/parsing_and_serializing.xml	2013-07-30 11:18:26 +0000
@@ -47,6 +47,7 @@
     <zorba:description author="Zorba Team">fn:serialize</zorba:description>
 
     <zorba:function>
+      <zorba:annotation>zann_explores_json</zorba:annotation>
 
       <zorba:signature localname="serialize" prefix="fn" version="3.0">
         <zorba:param>item()*</zorba:param>

=== modified file 'src/store/api/store.h'
--- src/store/api/store.h	2013-02-07 17:24:36 +0000
+++ src/store/api/store.h	2013-07-30 11:18:26 +0000
@@ -259,10 +259,12 @@
    *
    * @param QName the name of the collection
    * @param bool whether the collection searched is a dynamic or static collection
+   * @param dataguide the dataguie to be used for document projection and query 
+   *        push-down. Pass NULL if no dataguide is available
    * @return handle object of the collection. Returns NULL if the collection
    *         does not exist
    */
-  virtual Collection_t getCollection(const Item* name, bool isDynamic) = 0;
+  virtual Collection_t getCollection(const Item* name, bool isDynamic, const Item* dataguide = NULL) = 0;
 
   /** 
    * Returns an iterator that lists the names of all the available collections.

=== modified file 'src/store/naive/json_items.cpp'
--- src/store/naive/json_items.cpp	2013-06-05 00:37:35 +0000
+++ src/store/naive/json_items.cpp	2013-07-30 11:18:26 +0000
@@ -696,8 +696,19 @@
 zstring SimpleJSONObject::show() const
 {
   std::stringstream str;
-  str << "{ }";
-
+  str << "{";
+  
+  for (unsigned int i=0; i<thePairs.size(); i++)  
+  {
+    str << thePairs[i].first->getStringValue() << ": " ;
+    if (thePairs[i].second->isAtomic())
+      str << "\"" << thePairs[i].second->getStringValue() << "\"";
+    else
+      str << thePairs[i].second->show();        
+    str << (i<thePairs.size()-1 ? ", " : "");
+  }
+  
+  str << "}";  
   return str.str();
 }
 

=== modified file 'src/store/naive/store.cpp'
--- src/store/naive/store.cpp	2013-07-02 21:32:23 +0000
+++ src/store/naive/store.cpp	2013-07-30 11:18:26 +0000
@@ -441,7 +441,8 @@
 ********************************************************************************/
 store::Collection_t Store::getCollection(
     const store::Item* name,
-    bool isDynamic)
+    bool isDynamic, 
+    const store::Item *dataguide)
 {
   if (name == NULL)
     return NULL;

=== modified file 'src/store/naive/store.h'
--- src/store/naive/store.h	2013-06-15 02:57:08 +0000
+++ src/store/naive/store.h	2013-07-30 11:18:26 +0000
@@ -279,7 +279,7 @@
 
   virtual void deleteCollection(const store::Item* name, bool isDynamic);
 
-  virtual store::Collection_t getCollection(const store::Item* name, bool isDynamic);
+  virtual store::Collection_t getCollection(const store::Item* name, bool isDynamic, const store::Item* dataguide = NULL);
 
   virtual store::Iterator_t listCollectionNames(bool dynamic);
 

=== modified file 'src/system/globalenv.cpp'
--- src/system/globalenv.cpp	2013-07-02 21:32:23 +0000
+++ src/system/globalenv.cpp	2013-07-30 11:18:26 +0000
@@ -88,12 +88,12 @@
   m_globalEnv->theRootStaticContext = new root_static_context();
   m_globalEnv->theRootStaticContext->init();
 
+  AnnotationInternal::createBuiltIn();
+  
   m_globalEnv->theFunctionLib = new BuiltinFunctionLibrary();
 
   m_globalEnv->theFunctionLib->populate(m_globalEnv->theRootStaticContext);
 
-  AnnotationInternal::createBuiltIn();
-
 #ifdef ZORBA_XQUERYX
   //libxml2 and libxslt are needed
   xmlInitMemory();

=== modified file 'src/system/zorba_properties.h'
--- src/system/zorba_properties.h	2013-03-22 05:41:14 +0000
+++ src/system/zorba_properties.h	2013-07-30 11:18:26 +0000
@@ -38,12 +38,12 @@
       "--print-normalized", "--print-optimized", "--print-iterator-tree",
       "--print-item-flow", "--print-static-types", "--dump-lib",
       "--stable-iterator-ids", "--no-tree-ids", "--print-intermediate-opt",
-      "--print-locations", "--force-gflwor", "--reorder-globals",
+      "--print-locations", "--print-dataguide", "--force-gflwor", "--reorder-globals",
       "--specialize-num", "--specialize-cmp", "--inline-udf", "--loop-hoisting",
       "--infer-joins", "--no-copy-optim", "--serialize-only-query",
       "--trace-translator", "--trace-codegen", "--trace-fulltext", "--debug",
       "--compile-only", "--lib-module", "--tz", "--external-var", "--serializer-param",
-      "--iter-plan-test", "--dot-plan-file", "--plan", "jsoniq", "--max-udf-call-depth",
+      "--iter-plan-test", "--dot-plan-file", "--plan", "--jsoniq", "--max-udf-call-depth",
       "--CLASSPATH", NULL };
 
     return result;
@@ -72,6 +72,7 @@
   bool theNoTreeIds;
   bool thePrintIntermediateOpt;
   bool thePrintLocations;
+  bool thePrintDataguide;
   bool theForceGflwor;
   bool theReorderGlobals;
   bool theSpecializeNum;
@@ -120,6 +121,7 @@
     theNoTreeIds = false;
     thePrintIntermediateOpt = false;
     thePrintLocations = false;
+    thePrintDataguide = false;
     theForceGflwor = false;
     theReorderGlobals = true;
     theSpecializeNum = true;
@@ -166,6 +168,7 @@
   const bool &noTreeIds () const { return theNoTreeIds; }
   const bool &printIntermediateOpt () const { return thePrintIntermediateOpt; }
   const bool &printLocations () const { return thePrintLocations; }
+  const bool &printDataguide () const { return thePrintDataguide; }
   const bool &forceGflwor () const { return theForceGflwor; }
   const bool &reorderGlobals () const { return theReorderGlobals; }
   const bool &specializeNum () const { return theSpecializeNum; }
@@ -191,6 +194,8 @@
   const bool& jsoniqParser() const { return theJsoniqParser; }
   const uint32_t &maxUdfCallDepth () const { return theMaxUdfCallDepth; }
   const std::string &CLASSPATH () const { return theCLASSPATH; }
+  
+  void setPrintDataguide() { thePrintDataguide = true; }
 
   std::string load_argv (int argc, const char **argv) 
   {
@@ -289,6 +294,9 @@
       else if (strcmp (*argv, "--print-locations") == 0) {
         thePrintLocations = true;
       }
+      else if (strcmp (*argv, "--print-dataguide") == 0) {
+        thePrintDataguide = true;
+      }
       else if (strcmp (*argv, "--force-gflwor") == 0) {
         theForceGflwor = true;
       }
@@ -479,6 +487,7 @@
 "--no-tree-ids\nsuppress ids and locations from compiler tree dumps\n\n"
 "--print-intermediate-opt\nprint intermediate optimizations\n\n"
 "--print-locations\nprint parser locations for compiler expressions\n\n"
+"--print-dataguide\nprint the JSON dataguides\n\n"
 "--force-gflwor\nforce compiler to generate GFLWOR iterators\n\n"
 "--reorder-globals\nreorder global variables (1=enabled (default), 0=off)\n\n"
 "--specialize-num\nspecialize numerics (1=enabled (default), 0=off)\n\n"
@@ -501,6 +510,7 @@
 "--iter-plan-test\nrun as iterator plan test\n\n"
 "--dot-plan-file\ngenerate the dot iterator plan\n\n"
 "--plan\ntest plan serialization, i.e. save the plan, load it back and then execute it\n\n"
+"--jsoniq, -j\nuse the JSONiq grammar to parse the query\n\n"
 "--max-udf-call-depth\nmaximum stack depth of udf function calls\n\n"
 "--CLASSPATH\nJVM classpath to be used by modules using Java implementations\n\n"
 ;

=== modified file 'src/system/zorba_properties.txt'
--- src/system/zorba_properties.txt	2013-03-22 05:41:14 +0000
+++ src/system/zorba_properties.txt	2013-07-30 11:18:26 +0000
@@ -16,6 +16,7 @@
       ("print-iterator-tree,i", "print the iterator tree")
       ("print-item-flow,f", "show items produced by all iterators")
       ("print-static-types", po::value<bool> ()->default_value (true), "print static type inference")
+      ("print-dataguide", "print JSON dataguides")
       ("dump-lib", "dump function library")
       ("stable-iterator-ids", "print the iterator plan with stable ids")
       ("no-tree-ids", "suppress ids and locations from compiler tree dumps")

=== modified file 'test/driver/specification.h'
--- test/driver/specification.h	2013-05-16 08:22:46 +0000
+++ test/driver/specification.h	2013-07-30 11:18:26 +0000
@@ -55,7 +55,8 @@
     theInline(false),
     theComparisonMethod("Fragment"),
     theEnableDtd(false),
-    theEnableUriTestResolver(false)
+    theEnableUriTestResolver(false),
+    thePrintDataguide(false)
 #ifndef ZORBA_NO_FULL_TEXT
     ,
     theStopWordsMapper(zorba::EntityData::STOP_WORDS),
@@ -81,6 +82,7 @@
   bool                     theUseIndent;
   bool                     theEnableDtd;
   bool                     theEnableUriTestResolver;
+  bool                     thePrintDataguide;
 #ifndef ZORBA_NO_FULL_TEXT
   zorba::OneToOneURIMapper theStopWordsMapper;
   zorba::OneToOneURIMapper theThesaurusMapper;
@@ -229,6 +231,10 @@
   bool getEnableUriTestResolver() const {
     return theEnableUriTestResolver;
   }
+  
+  bool getPrintDataguide() const {
+    return thePrintDataguide;
+  }
 
   void tokenize(const std::string& str,
                 std::vector<std::string>& tokens,
@@ -305,7 +311,8 @@
               (str.find("DefaultCollection:")!=std::string::npos) ||
               (str.find("Error:")!=std::string::npos) || 
               (str.find("Date:")!=std::string::npos) || 
-              (str.find("Timezone:")!=std::string::npos));
+              (str.find("Timezone:")!=std::string::npos) || 
+              (str.find("Dataguide:")!=std::string::npos));
     return c;
   }
 
@@ -491,6 +498,12 @@
           if(lIter == tokens.end() ) { return false; }
           setTimezone(lIter->begin(), lIter->end());
         }
+        else if ( *lIter == "Dataguide:" ) 
+        {
+          ++lIter;
+          if(lIter == tokens.end() ) { return false; }
+          thePrintDataguide = true;
+        }
         else
         {
           break;

=== modified file 'test/driver/testdriver.cpp'
--- test/driver/testdriver.cpp	2013-06-15 02:57:08 +0000
+++ test/driver/testdriver.cpp	2013-07-30 11:18:26 +0000
@@ -248,6 +248,10 @@
     // If command-line argument --module-path passed, set up module paths.
     setModulePaths(lModulePath, lContext);
 
+    // If dataguide printing is requested, enable it in the properties
+    if (lSpec.getPrintDataguide())
+      zorba::Properties::instance()->setPrintDataguide();
+            
     // Get the pathnames of the reference-result files found in the .spec
     // file (if any).
     std::vector<std::string> lRefFiles;
@@ -325,8 +329,22 @@
     bool lJSONiqMode = 
     (lQueryFile.rfind(".jq") == lQueryFile.size() - 3);
 
-    if (lJSONiqMode) lContext->setJSONiqVersion(zorba::jsoniq_version_1_0);
-    lQuery->compile(lQueryString.c_str(), lContext, getCompilerHints());
+    if (lJSONiqMode) 
+      lContext->setJSONiqVersion(zorba::jsoniq_version_1_0);
+    
+    // Create and open the results file here so that the dataguide tests can use it to output projection information
+    std::ofstream lResFileStream(lResultFile.c_str());
+    assert (lResFileStream.good());
+    std::streambuf *coutbuf = std::cout.rdbuf(); //save old buf
+    
+    if (lSpec.getPrintDataguide())
+      std::cout.rdbuf(lResFileStream.rdbuf());
+    
+    lQuery->compile(lQueryString.c_str(), lContext, getCompilerHints()); 
+    
+    // reset cout to standard output
+    if (lSpec.getPrintDataguide())
+      std::cout.rdbuf(coutbuf);
 
     errors = -1;
     if ( errHandler.errors() )
@@ -422,8 +440,7 @@
       errors = -1;
       {
         // serialize xml/txt
-        std::ofstream lResFileStream(lResultFile.c_str());
-        assert (lResFileStream.good());
+