Merge lp:~zorba-coders/zorba/xml-in-json-indices into lp:zorba

[Ghislain Fourny]

1. Introducing StructuredItem interface and CollectionTreeInfo class to manage interactions between trees (of JSON and/or XML nodes) and collections.
2. Extending the use of colleciotn-relative tree positions to JSON trees.
3. Fixing index maintenance for XML embedded in JSON.

1. Introducing StructuredItem interface and CollectionTreeInfo class to manage interactions between trees (of JSON and/or XML nodes) and collections.
2. Extending the use of colleciotn-relative tree positions to JSON trees.
3. Fixing index maintenance for XML embedded in JSON.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2012-09-03T16-19-54.674Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/xml-in-json-indices into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job xml-in-json-indices-2012-09-03T16-19-54.674Z is
  finished. The final status was:

  1 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2012-09-04T14-36-58.242Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/xml-in-json-indices into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job xml-in-json-indices-2012-09-04T14-36-58.242Z is
  finished. The final status was:

  No tests were run - build or configure step must have failed.

  Not commiting changes.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2012-09-04T16-08-03.962Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/xml-in-json-indices into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job xml-in-json-indices-2012-09-04T16-08-03.962Z is
  finished. The final status was:

  No tests were run - build or configure step must have failed.

  Not commiting changes.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2012-09-04T16-15-05.617Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2012-09-04T16-15-05.617Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 2 Pending.

[Till Westmann (tillw)]

Seems to work fine :)

There are some minor things that I've fixed myself. Among them is a new test. However, I actually intended this test to do something else that it does now. The intention was to replace the string value of the "name" pair with and XDM value and to verify that the index will indeed be updated. However having a string as the value leads to a type error, as the key expression of the index cannot be evaluated anymore … so I replaced 2 "wrong" pairs with one "right" pair instead, which is close but not quite there …

I also found some other stuff that I would like to keep open for discussion:

The build fails if ZORBA_WITH_JSON is turned off . More generally it seems that we are giving up on using ZORBA_WITH_JSON to turn JSONiq off. Is that intended (agreed on)?

Some comments in the StructuredItem class would help (e.g. the semantics of getStructuredItemRefCount is not immediately obvious).
Should we have zorba::store::Item::isStructured()?

There should be a template function for SimpleJSONObject::setTree and SimpleJSONArray::setTree and for SimpleJSONObject::isInSubTree and SimpleJSONArray::isInSubTree (and possibly others).

It would be nice if there was no additional member needed for XmlTree::theStructuredItemRoot (not sure how to do that though ..).

review: Needs Fixing

[Ghislain Fourny (gislenius)]

Hi Till,

Thanks for your comments. I significantly refactored and simplified the code, added comments, added an isStructuredItem method. I hope it makes sense.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2012-09-05T16-16-00.655Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/xml-in-json-indices into lp:zorba failed. Below is the output from the failed tests.

No current working directory
No current working directory

[Matthias Brantner (matthias-brantner)]

I really like the introduction of the new StructuredItem. It makes a lot of sense and makes the code much cleaner. Regarding the implementation, I have some questions:

- Why is theTreeInfo in XmlTree allocated using new? Is this really required?
- Does every JSONItem have it's own CollectionTreeInfo class? If so, this seems quite some overhead. For example, each JSONItem will have it's own copy of the TreeId and Position.
- The XmlTree seems to contain two pointers to the root node; one from CollectionTreeInfo and one from XmlTree directly. Am I missing something here?

review: Needs Information

[Ghislain Fourny (gislenius)]

Hi Matthias,

Actually, absolutely all items in a tree that is in a collection share a unique CollectionTreeInfo instance, so that the TreeId, position, etc, exist only once physically. JSON items point to it (as before), and node items have their XML tree point to it.

JSON items only have a non-NULL pointer to CollectionTreeInfo if they are in a collection. The CollectionTreeInfo instance is created upon an attachToCollection call and delete upon detachFromCollection.

XML nodes also have a pointer if they are not in a collection, because the tree ID (among others) is also needed at some places (like structural URIs) even if they are not in a collection. In such a case, i.e., the XML tree is not in a collection, the XML tree needs to instantiate its own CollectionTreeInfo (with a NULL collection pointer). It will be deleted upon insertion in a collection and replaced with the collection-tree-wide instance. It will be recreated upon quitting the collection.

Finally, note that the root of an XML tree (mostly used for garbage collection if I am correct) might be different from the overall root of the tree in the collection (which might be a JSON item). An XML tree can "become" in a collection either directly (through attachToCollection), in which case these roots are the same, or indirectly (through setCollectionTreeInfo) if it was inserted in a JSON item that is in a collection.

I hope this helps. Does it make sense?

[Zorba Build Bot (zorba-buildbot)]

Attempt to merge into lp:zorba failed due to conflicts:

text conflict in src/store/naive/node_items.h

[Ghislain Fourny (gislenius)]

Hi Matthias,

Just a tiny correction: If the XML node is the (overall) root of the collection tree, then the existing CollectionTreeInfo instance is used. A new instance thereof is only created if the XML node gets inserted in a tree the root of which is a JSON item -- in which case the root JSON item is the owner of the instance.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2012-09-12T14-39-42.74Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2012-09-12T14-39-42.74Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 1 Needs Fixing, 1 Needs Information.

[Till Westmann (tillw)]

When running foaf-json-xml-index-point-maintenance.xq I run into a memory problem:

==13041== Invalid read of size 8
==13041== at 0x5ED4C32: zorba::simplestore::CollectionTreeInfo::getCollection() const (collection_tree_info.h:54)
==13041== by 0x5EC3C45: zorba::simplestore::XmlTree::setCollectionTreeInfo(zorba::simplestore::CollectionTreeInfo*) (node_items.cpp:272)
==13041== by 0x5EC5AA2: zorba::simplestore::XmlNode::setCollectionTreeInfo(zorba::simplestore::CollectionTreeInfo*) (node_items.cpp:939)
==13041== by 0x5F81BFB: zorba::simplestore::json::SimpleJSONObject::setCollectionTreeInfo(zorba::simplestore::CollectionTreeInfo*) (json_items.cpp:505)
==13041== by 0x5F802D5: zorba::simplestore::json::JSONItem::detachFromCollection() (json_items.cpp:165)
==13041== by 0x5F18251: zorba::simplestore::SimpleCollection::removeNode(zorba::store::Item*, zorba::IntegerImpl<long long>&) (simple_collection.cpp:271)
==13041== by 0x5F08680: zorba::simplestore::UpdDeleteNodesFromCollection::apply() (pul_primitives.cpp:1280)
==13041== by 0x5F4A797: zorba::simplestore::applyList(std::vector<zorba::simplestore::UpdatePrimitive*, std::allocator<zorba::simplestore::UpdatePrimitive*> >&) (simple_pul.cpp:73)
==13041== by 0x5F59674: zorba::simplestore::CollectionPul::applyUpdates() (simple_pul.cpp:3528)
==13041== by 0x5F54319: zorba::simplestore::PULImpl::applyUpdates(bool) (simple_pul.cpp:2471)
==13041== by 0x5BAE9C7: zorba::apply_updates(zorba::CompilerCB*, zorba::dynamic_context*, zorba::static_context*, zorba::store::PUL*, zorba::QueryLoc const&) (apply_updates.cpp:203)
==13041== by 0x5BAE34F: zorba::ApplyIterator::nextImpl(zorba::store::ItemHandle<zorba::store::Item>&, zorba::PlanState&) const (apply_updates.cpp:124)
==13041== Address 0xaec5090 is 0 bytes inside a block of size 32 free'd
==13041== at 0x4C2A4BC: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13041== by 0x5F802B7: zorba::simplestore::json::JSONItem::detachFromCollection() (json_items.cpp:164)
==13041== by 0x5F18251: zorba::simplestore::SimpleCollection::removeNode(zorba::store::Item*, zorba::IntegerImpl<long long>&) (simple_collection.cpp:271)
==13041== by 0x5F08680: zorba::simplestore::UpdDeleteNodesFromCollection::apply() (pul_primitives.cpp:1280)
==13041== by 0x5F4A797: zorba::simplestore::applyList(std::vector<zorba::simplestore::UpdatePrimitive*, std::allocator<zorba::simplestore::UpdatePrimitive*> >&) (simple_pul.cpp:73)
==13041== by 0x5F59674: zorba::simplestore::CollectionPul::applyUpdates() (simple_pul.cpp:3528)
==13041== by 0x5F54319: zorba::simplestore::PULImpl::applyUpdates(bool) (simple_pul.cpp:2471)
==13041== by 0x5BAE9C7: zorba::apply_updates(zorba::CompilerCB*, zorba::dynamic_context*, zorba::static_context*, zorba::store::PUL*, zorba::QueryLoc const&) (apply_updates.cpp:203)
==13041== by 0x5BAE34F: zorba::ApplyIterator::nextImpl(zorba::store::ItemHandle<zorba::store::Item>&, zorba::PlanState&) const (apply_updates.cpp:124)
==13041== by 0x5BB16D0: zorba::Batcher<zorba::ApplyIterator>::produceNext(zorba::store::ItemHandle<zorba::store::Item>&, zorba::PlanState&) const (plan_iterator...

review: Needs Fixing

[Ghislain Fourny (gislenius)]

Hi Till,

Good catch, it was a double free. Only Linux complained about it. Thanks! It should be fixed now, rerunning the tests.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2012-11-09T13-57-20.423Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2012-11-09T13-57-20.423Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 1 Needs Fixing, 1 Needs Information.

[Paul J. Lucas (paul-lucas)]

Why are .h files listed in the CMakefiles? You don't need to do that. The CMake system figures out the .h dependences from the specified .cpp files. Remove the .h dependencies.

[Paul J. Lucas (paul-lucas)]

In detachFromCollection(), you declare a local TreeId. A TreeId is currently defined as a typedef for a ulong. Defining a local variable of a C++ built-in type does no initialization, so the value of lTreeId is garbage.

[Paul J. Lucas (paul-lucas)]

review: Needs Fixing

[Markos Zaharioudakis (markos-za)]

Why is the isStructuredItem() method a virtual one? I don't think it needs to be.

[Markos Zaharioudakis (markos-za)]

In XmlTree, there are now 2 pointers to the root node of the tree: theRootNode and theTreeInfo->theRoot.
Also, theTreeInfo should be an embeded instance of CollectionTreeInfo, instead of being allocated every time an XmlTree is constructed.

There is already an Item::isInSubtreeOf() method at the store API. Currently, this method applies to AnyURI items only. I think it should be extended to apply to structured items as well, and then the StructuredItem::isInSubtree() should be removed. This will make things consistent with the isAncestor, isChild, etc. methods, which apply to both AnyURI items and nodes.

[Markos Zaharioudakis (markos-za)]

review: Needs Fixing

[Markos Zaharioudakis (markos-za)]

Sorry, Ghislain, I did not see Mathias questions and your answers. Now I understand why there 2 root pointers. But I still don't see why a CollectionTreeInfo must be heap-allocated every time an XmlTree is constructed. And I have one more question: an XML tree can have a tree ID even if it is not in a collection. The tree id is used in the construction of structural references, and maybe in other places that I don't remember right now. In your merge proposal I see that the tree id is changed when a tree is inserted in a collection, its tree id changes. Are you sure this is OK? And does it really need to change?

[Ghislain Fourny (gislenius)]

> Why are .h files listed in the CMakefiles? You don't need to do that. The
> CMake system figures out the .h dependences from the specified .cpp files.
> Remove the .h dependencies.

Hi Paul,

It's because it adds them to the XCode project. Not having them makes full-text search very cumbersome (i.e., headers are not considered).

[Ghislain Fourny (gislenius)]

> In detachFromCollection(), you declare a local TreeId. A TreeId is currently
> defined as a typedef for a ulong. Defining a local variable of a C++ built-in
> type does no initialization, so the value of lTreeId is garbage.

This is now fixed. Actually, the variable would not have been used, but I agree it's cleaner that way. Thanks for noticing.

[Ghislain Fourny (gislenius)]

Hi Markos,

Thanks for your thorough analysis.

> Also, theTreeInfo should be an embeded instance of CollectionTreeInfo,
> instead of being allocated every time an XmlTree is constructed.

> But I still don't see why a CollectionTreeInfo must be heap-allocated
> every time an XmlTree is constructed.

The reason is that the CollectionTreeInfo object is not necessarily owned by the XMLTree, but it can also be shared with JSON items higher in the collection tree and possibly other XML trees that are also in the collection tree.

There are basically three cases:
(i) The XMLTree is not in a collection.
(ii) The XMLTree is in a collection and its root is also the overall collection tree root.
(iii) The XMLTree is in a collection, but the overall root is a JSON item.

If it were not for case (iii), it would have worked to embed the CollectionTreeInfo object in the class. But case (iii) is incompatible with this... The code should however only allocate/deallocate this object from the heap when switching between case (iii) and case (i).

Does it make sense?

[Ghislain Fourny (gislenius)]

I will also investigate the other two remarks, which make sense to me (isSubtreeOf, tree ID when not in a collection).

[Zorba Build Bot (zorba-buildbot)]

Attempt to merge into lp:zorba failed due to conflicts:

text conflict in src/store/naive/CMakeLists.txt

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2012-12-12T11-47-09.52Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2012-12-12T11-47-09.52Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 4 Needs Fixing, 1 Needs Information.

[Paul J. Lucas (paul-lucas)]

> It's because it adds them to the XCode project. Not having them makes full-
> text search very cumbersome (i.e., headers are not considered).

Isn't that what our own HEADER_GROUP_SUBFOLDER custom macro is for?

[Ghislain Fourny (gislenius)]

> And I have one more question: an XML tree can have a tree ID even
> if it is not in a collection. The tree id is used in the construction of
> structural references, and maybe in other places that I don't remember right
> now. In your merge proposal I see that the tree id is changed when a tree is
> inserted in a collection, its tree id changes. Are you sure this is OK? And
> does it really need to change?

My understanding of the status quo is that the tree ID is already set using the new collection's tree ID generator (when calling setCollection() with a non-NULL collection):

- if (collection != NULL)
- theId = collection->createTreeId();

And the tree ID is left unchanged when leaving the collection.

setCollection has now been replaced with the more explicit attachToCollection and detachFromCollection of the StructuredItem API, but I think that the behaviour described above should be left intact by this merge proposal in both cases.

Does it make sense?

[Ghislain Fourny (gislenius)]

> In XmlTree, there are now 2 pointers to the root node of the tree: theRootNode
> and theTreeInfo->theRoot.
> Also, theTreeInfo should be an embeded instance of CollectionTreeInfo, instead
> of being allocated every time an XmlTree is constructed.
>
> There is already an Item::isInSubtreeOf() method at the store API. Currently,
> this method applies to AnyURI items only. I think it should be extended to
> apply to structured items as well, and then the StructuredItem::isInSubtree()
> should be removed. This will make things consistent with the isAncestor,
> isChild, etc. methods, which apply to both AnyURI items and nodes.

I added an implementation of Item::isInSubtreeOf() for structured items as you requested. However, the function StructuredItem::isInSubtree() is still needed because (i) it is the other way round and in particular (ii) it is based on a structural recursion on "this" taking advantage of dynamic binding, so that the code depends on the kind of structured item.
Item::isInSubtreeOf() cannot use a structural recursion in the same way, so that it calls StructuredItem::isInSubTree() instead to avoid both (i) code replication and (ii) making structural_item.h aware of its subclasses.

Does it make sense?

[Ghislain Fourny (gislenius)]

> Why is the isStructuredItem() method a virtual one? I don't think it needs to
> be.

Nice catch, I made it non-virtual.

[Ghislain Fourny (gislenius)]

> > It's because it adds them to the XCode project. Not having them makes full-
> > text search very cumbersome (i.e., headers are not considered).
>
> Isn't that what our own HEADER_GROUP_SUBFOLDER custom macro is for?

Thanks for the pointer. I removed the .h files for now and will address this in another merge proposal.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2012-12-14T12-50-24.017Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2012-12-14T12-50-24.017Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 4 Needs Fixing, 1 Needs Information.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2013-01-29T10-15-10.37Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/xml-in-json-indices into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message):
  Validation queue job xml-in-json-indices-2013-01-29T10-15-10.37Z is
  finished. The final status was:

  7 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2013-01-29T13-59-36.602Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2013-01-29T13-59-36.602Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 4 Needs Fixing, 1 Needs Information.

[Zorba Build Bot (zorba-buildbot)]

Attempt to merge into lp:zorba failed due to conflicts:

text conflict in src/store/naive/simple_collection.cpp

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2013-01-30T14-46-35.086Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2013-01-30T14-46-35.086Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 4 Needs Fixing, 1 Needs Information.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2013-02-04T15-47-42.372Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2013-02-04T15-47-42.372Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 4 Needs Fixing, 1 Needs Information.

[Markos Zaharioudakis (markos-za)]

review: Approve

[Markos Zaharioudakis (markos-za)]

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2013-02-07T14-13-01.44Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2013-02-07T14-13-01.44Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 2 Approve, 2 Needs Fixing, 1 Needs Information.

[Matthias Brantner (matthias-brantner)]

review: Approve

[Matthias Brantner (matthias-brantner)]

review: Approve

[Paul J. Lucas (paul-lucas)]

In several places, I see calls to to_xs_unsignedInt() and to to_xs_unsignedLong() without catching the possible std::range_error.

[Till Westmann (tillw)]

review: Approve

[Ghislain Fourny (gislenius)]

Good catch Paul. Do the changes fulfil your expectations provided all tests pass? I am not pushing yet to the patch queue because it might get through without your approval.

[Ghislain Fourny (gislenius)]

I just noticed you are in a Need Fixing state, so I am pushing to the patch queue.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2013-02-18T15-20-10.215Z/log.html

[Paul J. Lucas (paul-lucas)]

> Do the changes fulfil your expectations provided all tests pass?

No because you should catch by const& not by value. And is "assert(false)" really the only/best thing you can do? Is there any legitimate way the position can exceed the size of a long? If the answer is "yes," then you need to do something better.

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2013-02-18T15-20-10.215Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 4 Approve, 1 Needs Fixing.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2013-02-18T16-14-00.593Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2013-02-18T16-14-00.593Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 4 Approve, 1 Needs Fixing.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2013-02-21T14-37-15.369Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2013-02-21T14-37-15.369Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 4 Approve, 1 Needs Fixing.

[lp:~zorba-coders/zorba/xml-in-json-indices]

11065. By Ghislain Fourny on 2013-02-21

Changing exception to const&.

[Ghislain Fourny (gislenius)]

Hi Paul,

I formerly assessed the risk of reaching the limits of the int/long range for a position as highly unlikely, but I do have doubts about whether a negative number might go through. So I changed assert(false) to a proper processor exception. I also changed to const&.

[Paul J. Lucas (paul-lucas)]

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/xml-in-json-indices-2013-02-21T15-43-09.091Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job xml-in-json-indices-2013-02-21T15-43-09.091Z is finished. The final status was:

All tests succeeded!

=== modified file 'src/runtime/collections/collections_impl.cpp'
--- src/runtime/collections/collections_impl.cpp	2013-01-31 11:32:47 +0000
+++ src/runtime/collections/collections_impl.cpp	2013-02-21 15:32:25 +0000
@@ -1816,12 +1816,12 @@
   consumeNext(content, theChildren[1].getp(), planState);
 
   // Target check.
-  if (! target->getCollection())
+  if (!target->getCollection() || !target->isStructuredItem())
   {
     throw XQUERY_EXCEPTION(zerr::ZDDY0017_NODE_IS_ORPHAN, ERROR_LOC(loc));
   }
 
-  if ((target->isNode() || target->isJSONItem()) && !target->isRoot())
+  if (!target->isCollectionRoot())
   {
     throw XQUERY_EXCEPTION(
       zerr::ZDDY0039_NON_ROOT_NODE_EDIT,

=== modified file 'src/runtime/json/json_constructors.cpp'
--- src/runtime/json/json_constructors.cpp	2012-10-08 12:09:36 +0000
+++ src/runtime/json/json_constructors.cpp	2013-02-21 15:32:25 +0000
@@ -304,7 +304,7 @@
       consumeNext(name, theChildren[i], planState);
       consumeNext(value, theChildren[numPairs + i], planState);
 
-      if (theCopyInputs[i] && (value->isNode() || value->isJSONItem()))
+      if (theCopyInputs[i] && (value->isStructuredItem()))
         value = value->copy(NULL, copymode);
 
       names[i].transfer(name);

=== modified file 'src/runtime/json/jsoniq_functions_impl.cpp'
--- src/runtime/json/jsoniq_functions_impl.cpp	2013-02-16 17:16:13 +0000
+++ src/runtime/json/jsoniq_functions_impl.cpp	2013-02-21 15:32:25 +0000
@@ -975,7 +975,7 @@
     {
       value = obj->getObjectValue(key);
 
-      if (value->isNode() || value->isJSONItem())
+      if (value->isStructuredItem())
         value = value->copy(NULL, copymode);
 
       newValues.push_back(value);
@@ -1261,7 +1261,7 @@
                theSctx->preserve_ns(),
                theSctx->inherit_ns());
 
-  if (content->isNode() || content->isJSONItem())
+  if (content->isStructuredItem())
   {
     content = content->copy(NULL, copymode);
   }
@@ -1308,7 +1308,7 @@
 
   while (consumeNext(member, theChildren[2].getp(), planState))
   {
-    if (member->isNode() || member->isJSONItem())
+    if (member->isStructuredItem())
     {
       member = member->copy(NULL, copymode);
     }
@@ -1356,7 +1356,7 @@
 
   while (consumeNext(member, theChildren[1].getp(), planState))
   {
-    if (member->isNode() || member->isJSONItem())
+    if (member->isStructuredItem())
     {
       member = member->copy(NULL, copymode);
     }
@@ -1481,7 +1481,7 @@
   consumeNext(selector, theChildren[1].getp(), planState);
   consumeNext(newValue, theChildren[2].getp(), planState);
 
-  if (theCopyInput && (newValue->isNode() || newValue->isJSONItem()))
+  if (theCopyInput && (newValue->isStructuredItem()))
   {
     copymode.set(true, 
                  theSctx->construction_mode() == StaticContextConsts::cons_preserve,

=== modified file 'src/store/api/item.h'
--- src/store/api/item.h	2013-02-05 04:08:51 +0000
+++ src/store/api/item.h	2013-02-21 15:32:25 +0000
@@ -45,10 +45,16 @@
 
 typedef StoreConsts::NodeKind NodeKind;
 
-/**
- *  Class Item represents an "item" as defined by the XQuery Data Model (XDM)
- *  [http://www.w3.org/TR/xquery-semantics/doc-fs-Item]
- */
+/*******************************************************************************
+   Class Item represents an "item" as defined by the XQuery Data Model (XDM)
+   [http://www.w3.org/TR/xquery-semantics/doc-fs-Item]
+
+  theUnion:
+  ---------
+  It stores either a pointer to an XmlTree or an ItemKind enum value. The 
+  low-order bit is used to distinguish between these 2 cases: a 0 bit indicates
+  an XmlTree pointer, and a 1 bit indicated an ItemKind. 
+********************************************************************************/
 class ZORBA_DLL_PUBLIC Item
 {
 public:
@@ -181,6 +187,15 @@
 
 
   /**
+   *  @return  "true" if the item is a JSON item or a node
+   */
+  bool 
+  isStructuredItem() const
+  {
+    return isNode() || isJSONItem();
+  }
+
+  /**
    * @return a string representation of the item's kind
    */
   zstring printKind() const;
@@ -454,25 +469,21 @@
   virtual const xs_duration&
   getDurationValue() const;
 
-
   /** Accessor for xs:dayTimeDuration
    */
   virtual const xs_dayTimeDuration&
   getDayTimeDurationValue() const;
 
-
   /** Accessor for xs:yearMonthDuration
    */
   virtual const xs_yearMonthDuration&
   getYearMonthDurationValue() const;
 
-
   /** Accessor for xs:hexBinary
    */
   virtual xs_hexBinary
   getHexBinaryValue() const;
 
-
   /**
    * Helper method for numeric atomic items
    * @return true, if containing number is not-a-number (possible for
@@ -481,7 +492,6 @@
   virtual bool
   isNaN() const;
 
-
   /**
    * Helper method for numeric atomic items
    * @return true, if containing numbers represents -INF or +INF
@@ -525,8 +535,25 @@
   virtual bool
   isTextRef() const;
 
-  
-  /* -------------------  Methods for Nodes ------------------------------------- */
+  /* -------------------  Methods for XML and JSON Nodes --------------------- */
+
+  /**
+   * @return true if this node is the root node of a tree that belongs to a
+   * a collection directly.
+   */
+  virtual bool
+  isCollectionRoot() const;
+
+  /**
+   * @return true if this node is in the subtree starting at the given item.
+   *         NOTE: for the purposes of this method, XML trees that are pointed-to
+   *         by a JSON tree are considered part of that JSON tree. As a result,
+   *         an XML node may be in the subtree of a JSON node.
+   */
+  virtual bool
+  isInSubtreeOf(const store::Item_t&) const;
+
+  /* -------------------  Methods for XML Nodes ------------------------------ */
 
   /**
    *  getNodeProperty functions - Accessor of XDM (see XDM specification, Section 5)
@@ -558,10 +585,10 @@
    *  @return True if this is an element node with name N and it has at least one
    *          descendant whose name is also N.
    *
-   * Note: This function is used purely for enabling certain optimizations in the
-   *       query processor. As a result, it is not necessary that a store actually
-   *       provides info about the recursivity of a node; such a store should
-   *       provide a dummy implementation of this method that simply returns true.
+   * Note: This function is used purely for enabling certain optimizations in
+   * the query processor. As a result, it is not necessary that a store actually
+   * provides info about the recursivity of a node; such a store should provide
+   * a dummy implementation of this method that simply returns true.
    */
   virtual bool
   isRecursive() const;
@@ -614,8 +641,8 @@
    */
   virtual void
   getNamespaceBindings(
-        NsBindings& bindings,
-        StoreConsts::NsScoping ns_scoping = StoreConsts::ALL_NAMESPACES) const;
+      NsBindings& bindings,
+      StoreConsts::NsScoping ns_scoping = StoreConsts::ALL_NAMESPACES) const;
 
   /** Accessor for element node
    *  @return  boolean?
@@ -747,12 +774,6 @@
    *
    */
   virtual bool
-  isInSubtreeOf(const store::Item_t&) const;
-
-  /**
-   *
-   */
-  virtual bool
   isDescendant(const store::Item_t&) const;
 
   /**
@@ -844,12 +865,6 @@
   getJSONItemKind() const;
 
   /**
-   * @return true if the JSON item is a root in a collection.
-   */
-  virtual bool
-  isRoot() const;
-
-  /**
    * defined on JSONArray
    * (jdm:size accessor on an array)
    * @return the number of values in the array.

=== modified file 'src/store/naive/CMakeLists.txt'
--- src/store/naive/CMakeLists.txt	2012-11-29 00:47:32 +0000
+++ src/store/naive/CMakeLists.txt	2013-02-21 15:32:25 +0000
@@ -16,45 +16,47 @@
 # Source Files
 #
 SET(ZORBA_STORE_IMPL_SRCS
+    atomic_items.cpp
+    collection.cpp
+    dataguide.cpp
+    inmemorystore.cpp
+    inmemorystorec.cpp
     item.cpp
+    item_iterator.cpp
     item_vector.cpp
+    loader_fast.cpp
+    loader_dtd.cpp
+    node_factory.cpp
     node_items.cpp
-    node_factory.cpp
+    node_iterators.cpp
     node_updates.cpp
-    simple_pul.cpp
+    nsbindings.cpp
+    ordpath.cpp
+    properties.cpp
+    pul_primitive_factory.cpp
     pul_primitives.cpp
-    pul_primitive_factory.cpp
-    atomic_items.cpp
-    item_iterator.cpp
-    node_iterators.cpp
-    simple_item_factory.cpp
-    simple_store.cpp
-    simple_iterator_factory.cpp
+    qname_pool.cpp
     simple_collection.cpp
     simple_collection_set.cpp
     simple_index.cpp
+    simple_index_general.cpp
     simple_index_value.cpp
-    simple_index_general.cpp
+    simple_item_factory.cpp
+    simple_iterator_factory.cpp
+    simple_lazy_temp_seq.cpp
+    simple_pul.cpp
+    simple_store.cpp
     simple_temp_seq.cpp
-    simple_lazy_temp_seq.cpp
-    loader_fast.cpp
-    loader_dtd.cpp
-    qname_pool.cpp
+    store.cpp
     string_pool.cpp
-    ordpath.cpp
-    nsbindings.cpp
-    properties.cpp
-    inmemorystore.cpp
-    inmemorystorec.cpp
-    dataguide.cpp
+    structured_item.cpp
     tree_id_generator.cpp
-    collection.cpp
-    store.cpp
 )
 
 IF (NOT ZORBA_NO_FULL_TEXT)
   LIST(APPEND ZORBA_STORE_IMPL_SRCS
-    ft_token_store.cpp naive_ft_token_iterator.cpp)
+    ft_token_store.cpp
+    naive_ft_token_iterator.cpp)
 ENDIF (NOT ZORBA_NO_FULL_TEXT)
 
 IF (ZORBA_WITH_JSON)

=== modified file 'src/store/naive/collection.cpp'
--- src/store/naive/collection.cpp	2012-09-19 21:16:15 +0000
+++ src/store/naive/collection.cpp	2013-02-21 15:32:25 +0000
@@ -26,11 +26,19 @@
 namespace zorba {
 namespace simplestore {
 
-void Collection::claimOwnership(zorba::simplestore::XmlTree* aTree)
+
+/*******************************************************************************
+
+*******************************************************************************/
+void Collection::getIndexes(std::vector<store::Index*>& indexes)
 {
-  aTree->claimedByCollection(this);
+  getIndexes(getName(), indexes);
 }
 
+
+/*******************************************************************************
+  Static method
+*******************************************************************************/
 void Collection::getIndexes(
     const store::Item* name,
     std::vector<store::Index*>& indexes)
@@ -46,9 +54,9 @@
     const store::IndexSpecification& indexSpec = index->getSpecification();
 
     const std::vector<store::Item_t>& indexSources = indexSpec.theSources;
-    uint64_t numIndexSources = (uint64_t)indexSources.size();
+    csize numIndexSources = indexSources.size();
 
-    for (ulong i = 0; i < numIndexSources; ++i)
+    for (csize i = 0; i < numIndexSources; ++i)
     {
       if (indexSources[i]->equals(name))
       {
@@ -59,6 +67,19 @@
   }
 }
 
+
+/*******************************************************************************
+
+*******************************************************************************/
+void Collection::getActiveICs(std::vector<store::IC*>& ics)
+{
+  getActiveICs(getName(), ics);
+}
+
+
+/*******************************************************************************
+  Static method
+*******************************************************************************/
 void Collection::getActiveICs(
     const store::Item* name,
     std::vector<store::IC*>& ics)
@@ -96,18 +117,6 @@
   activeICNames->close();
 }
 
-/*******************************************************************************
-
-*******************************************************************************/
-void Collection::getActiveICs(std::vector<store::IC*>& ics)
-{
-  getActiveICs(getName(), ics);
-}
-
-void Collection::getIndexes(std::vector<store::Index*>& indexes)
-{
-  getIndexes(getName(), indexes);
-}
 
 } /* namespace simplestore */
 } /* namespace zorba */

=== modified file 'src/store/naive/collection.h'
--- src/store/naive/collection.h	2012-12-17 10:30:41 +0000
+++ src/store/naive/collection.h	2013-02-21 15:32:25 +0000
@@ -26,25 +26,21 @@
 /*******************************************************************************
   Interface for Collection used in the SimpleStore
 
-  theName        : The user-provided qname of the collection.
+  theName: 
+  --------
+  The user-provided qname of the collection.
 ********************************************************************************/
 class Collection : public store::Collection
 {
 protected:
-  store::Item_t                 theName;
+  store::Item_t  theName;
 
 public:
   /***************************** Constructors *********************************/
 
-  Collection(const store::Item_t& aName)
-    : 
-    theName(aName)
-  {
-  }
+  Collection(const store::Item_t& name) : theName(name) { }
 
-  Collection()
-  {
-  }
+  Collection() { }
 
   virtual ~Collection() {}
 
@@ -54,20 +50,21 @@
 
   zorba::xs_integer size() const  = 0;
 
-  zorba::store::Iterator_t getIterator(
-      const xs_integer& aSkip, 
-      const zstring& aStart) = 0;
+  virtual bool isDynamic() const = 0;
+
+  virtual void getAnnotations(std::vector<store::Annotation_t>&) const = 0;
+
+  store::Iterator_t getIterator(const xs_integer& skip, const zstring& start) = 0;
+
+  virtual bool findNode(const store::Item* node, xs_integer& position) const = 0;
 
   virtual zorba::store::Item_t nodeAt(xs_integer position) = 0;
 
-  virtual bool findNode(
-      const store::Item* node,
-      xs_integer& position) const = 0;
-
-  virtual bool isDynamic() const = 0;
-
-  virtual void getAnnotations(
-      std::vector<zorba::store::Annotation_t>&) const = 0;
+  /***************************** ID Management ********************************/
+
+  virtual ulong getId() const = 0;
+
+  virtual TreeId createTreeId() = 0;
 
   /************************* Updates on collection ****************************/
 
@@ -84,39 +81,28 @@
 
   virtual bool removeNode(xs_integer position) = 0;
 
-  virtual zorba::xs_integer removeNodes(
-      xs_integer position,
-      xs_integer num) = 0;
+  virtual zorba::xs_integer removeNodes(xs_integer position, xs_integer num) = 0;
 
   virtual void removeAll() = 0;
   
   virtual void adjustTreePositions() = 0;
 
-  /***************************** ID Management ********************************/
-
-  virtual ulong getId() const = 0;
-
-  virtual TreeId createTreeId() = 0;
-
   /*********************** Indices ********************************************/
+
   static void getIndexes(
       const store::Item* name,
       std::vector<store::Index*>& indexes);
 
   void getIndexes(std::vector<store::Index*>& indexes);
 
-  /**** Returns active integrity constraints referencing this collection ******/
+  /*********************** Integrity Constraints ******************************/
+
   static void getActiveICs(
       const store::Item* name,
       std::vector<store::IC*>& ics);
 
   void getActiveICs(std::vector<store::IC*>& ics);
-
-  /**************************** Claim of ownership ****************************/
-protected:
-  virtual void claimOwnership(simplestore::XmlTree* aTree);
-  
-}; /* class Collection */
+};
 
 
 } /* namespace simplestore */ } /* namespace zorba */

=== added file 'src/store/naive/collection_tree_info.h'
--- src/store/naive/collection_tree_info.h	1970-01-01 00:00:00 +0000
+++ src/store/naive/collection_tree_info.h	2013-02-21 15:32:25 +0000
@@ -0,0 +1,194 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef ZORBA_STORE_COLLECTION_TREE_INFO_H
+#define ZORBA_STORE_COLLECTION_TREE_INFO_H
+
+#include <zorba/config.h>
+
+namespace zorba
+{
+
+namespace simplestore
+{
+
+class Collection;
+class StructuredItem;
+
+/******************************************************************************
+  Contains info about an XML or JSON tree that belongs to a collection, either
+  directly or indirectly. A tree belongs to a collection "directly" if the
+  collection stores a "pointer" to the root of the tree.  A tree belongs to a
+  collection "indirectly" if it is pointed-to by a tree that belongs to the
+  collection directly. Such indirect membership happens only when an XML tree
+  is pointed-to by a JSON tree that belongs to a collection directly. Indirect
+  membership cannot happen for JSON trees.
+
+  A tree may belong to at most one collection at a time.
+
+  A CollectionTreeInfo instance is created when a tree is attached to a collection
+  (directly or indirectly) and is destroyed when the tree is detached from the
+  collection. For XML trees, a pointer to this CollectionTreeInfo is stored in
+  the XmlTree obj representing the tree. For JSON trees, a pointer to the
+  CollectionTreeInfo is stored in each node of the tree. 
+
+  Each tree within a collection must have an id that uniquely identifies the
+  tree within the collection. 
+
+  XML trees need a tree id even if they don't belong to a collection. As a
+  result, XML trees store their id inside their associated XmlTree object.
+  When an XML tree does not belong to a collection, its id uniquely identifies
+  the tree among all the trees that do not belong to any collection. When an
+  XML tree enters a collection, its tree id is updated so that is becomes
+  unique within the target collection. When an XML tree exits a collection,
+  its id is again updated appropriately.
+
+  JSON trees need a tree id only while inside a collection. For this purpose,
+  the CollectionTreeInfoWithTreeId subclass is use to provide the storage for
+  the tree id.
+
+  theCollection:
+  --------------
+  The collection where this tree belongs to.
+
+  theRoot:
+  --------
+  The xml or json node that is directly stored in the collection. For JSON trees
+  this is the root node of the tree. For XML trees, it is the root node of the 
+  tree, if the tree belongs to the collection directly, or it is the root of a
+  JSON tree that belongs to the collection directly and also points to the XML
+  tree.
+
+  thePosition:
+  ------------
+  The position of this tree within its containing collection. It is used as a
+  hint to quickly retrieve the tree of a node within a collection.
+
+  Very simple API (just getters and setters for each of the data members).
+*******************************************************************************/
+
+class CollectionTreeInfo
+{
+protected:
+  Collection*     theCollection;
+  xs_integer      thePosition;
+  StructuredItem* theRoot;
+
+public:
+  CollectionTreeInfo()
+    :
+    theCollection(NULL),
+    thePosition(0),
+    theRoot(NULL)
+  {
+  }
+
+  simplestore::Collection* getCollection() const
+  {
+    return theCollection;
+  }
+
+  void setCollection(simplestore::Collection* aCollection)
+  {
+    theCollection = aCollection;
+  }
+
+  StructuredItem* getRoot() const
+  {
+    return theRoot;
+  }
+
+  void setRoot(StructuredItem* aRoot)
+  {
+    theRoot = aRoot;
+  }
+  
+  const xs_integer& getPosition() const
+  {
+    return thePosition;
+  }
+
+  void setPosition(const xs_integer& aPosition)
+  {
+    thePosition = aPosition;
+  }
+};
+
+
+/******************************************************************************
+
+*******************************************************************************/
+class CollectionTreeInfoWithTreeId : public CollectionTreeInfo
+{
+private:
+  TreeId  theTreeId;
+
+public:
+  CollectionTreeInfoWithTreeId() : theTreeId() {}
+
+  simplestore::Collection* getCollection() const
+  {
+    return theCollection;
+  }
+
+  void setCollection(simplestore::Collection* aCollection)
+  {
+    theCollection = aCollection;
+  }
+
+  const TreeId& getTreeId() const
+  {
+    return theTreeId;
+  }
+
+  void setTreeId(const TreeId& aId)
+  {
+    theTreeId = aId;
+  }
+
+  StructuredItem* getRoot() const
+  {
+    return theRoot;
+  }
+
+  void setRoot(StructuredItem* aRoot)
+  {
+    theRoot = aRoot;
+  }
+  
+  const xs_integer& getPosition() const
+  {
+    return thePosition;
+  }
+
+  void setPosition(const xs_integer& aPosition)
+  {
+    thePosition = aPosition;
+  }
+};
+
+
+} // namespace simplestore
+} // namespace zorba
+#endif /* ZORBA_STORE_COLLECTION_TREE_INFO_H */
+
+/*
+ * Local variables:
+ * mode: c++
+ * End:
+ */
+/* vim:set et sw=2 ts=2: */
+

=== modified file 'src/store/naive/item.cpp'
--- src/store/naive/item.cpp	2013-02-05 04:08:51 +0000
+++ src/store/naive/item.cpp	2013-02-21 15:32:25 +0000
@@ -1348,7 +1348,7 @@
   );
 }
 
-bool Item::isRoot() const
+bool Item::isCollectionRoot() const
 {
   throw ZORBA_EXCEPTION(
     zerr::ZSTR0050_FUNCTION_NOT_IMPLEMENTED_FOR_ITEMTYPE,

=== modified file 'src/store/naive/json_items.cpp'
--- src/store/naive/json_items.cpp	2013-02-17 00:59:54 +0000
+++ src/store/naive/json_items.cpp	2013-02-21 15:32:25 +0000
@@ -37,14 +37,6 @@
 /******************************************************************************
 
 *******************************************************************************/
-JSONTree::~JSONTree()
-{
-  GET_STORE().unregisterReferenceToDeletedNode(theRoot);
-}
-
-/******************************************************************************
-
-*******************************************************************************/
 store::Item* JSONNull::getType() const
 {
   return GET_STORE().JS_NULL_QNAME;
@@ -94,7 +86,7 @@
 *******************************************************************************/
 JSONItem::~JSONItem()
 {
-  delete theTree;
+  delete theCollectionInfo;
 }
 
 
@@ -112,58 +104,39 @@
 *******************************************************************************/
 void JSONItem::destroy()
 {
+  GET_STORE().unregisterReferenceToUnusedNode(this);
   delete this;
 }
 
 
-/******************************************************************************
-
-*******************************************************************************/
-const simplestore::Collection* JSONItem::getCollection() const
-{
-  if (theTree == NULL)
-  {
-    return NULL;
-  }
-  return theTree->getCollection();
-}
-
-
-/******************************************************************************
-  Should only to be called if item is in a collection.
-*******************************************************************************/
-const TreeId& JSONItem::getTreeId() const
-{
-  ZORBA_ASSERT(theTree);
-  return theTree->getTreeId();
-}
-
-
-/******************************************************************************
-  Should only to be called if item is in a collection.
-*******************************************************************************/
-JSONItem* JSONItem::getRoot() const
-{
-  ZORBA_ASSERT(theTree);
-  return theTree->getRoot();
-}
-
-
-/*******************************************************************************
-
-********************************************************************************/
-void JSONItem::attachToCollection(Collection* aCollection, const TreeId& aTreeId)
+/*******************************************************************************
+
+********************************************************************************/
+long JSONItem::getCollectionTreeRefCount() const
+{
+  return getRefCount();
+}
+
+/*******************************************************************************
+
+********************************************************************************/
+void JSONItem::attachToCollection(
+    Collection* aCollection,
+    const TreeId& aTreeId,
+    const xs_integer& aPosition)
 {
   ASSERT_INVARIANT();
-  
+
   assert(aCollection);
 
   // Attach
-  assert(getTree() == NULL);
-  setTree(new JSONTree());
-  getTree()->setRoot(this);
-  getTree()->setCollection(aCollection);
-  getTree()->setTreeId(aTreeId);
+  assert(theCollectionInfo == NULL);
+  CollectionTreeInfoWithTreeId* collectionInfo = new CollectionTreeInfoWithTreeId();
+  collectionInfo->setCollection(aCollection);
+  collectionInfo->setTreeId(aTreeId);
+  collectionInfo->setPosition(aPosition);
+  collectionInfo->setRoot(this);
+  setCollectionTreeInfo(collectionInfo);
   
   ASSERT_INVARIANT();
 }
@@ -175,11 +148,12 @@
 {
   ASSERT_INVARIANT();
   
-  JSONTree* lTree = getTree();
+  
+  CollectionTreeInfo* collectionInfo = theCollectionInfo;
   // Detach
-  assert(lTree);
-  delete lTree;
-  setTree(NULL);
+  assert(collectionInfo);
+  setCollectionTreeInfo(NULL);
+  delete collectionInfo;
 
   ASSERT_INVARIANT();
 }
@@ -190,12 +164,12 @@
 *******************************************************************************/
 void JSONItem::assertInvariant() const
 {
-  if (theTree != NULL)
+  if (theCollectionInfo != NULL)
   {
-    assert(theTree->getCollection() != NULL);
-    assert(theTree->getRoot() != NULL);
-    assert(isThisTreeOfAllDescendants(theTree));
-    assert(theTree->getRoot()->isThisJSONItemInDescendance(this));
+    assert(theCollectionInfo->getCollection() != NULL);
+    assert(theCollectionInfo->getRoot() != NULL);
+    assert(isThisTreeOfAllDescendants(theCollectionInfo));
+    assert(theCollectionInfo->getRoot()->isInSubtree(this));
   }
 }
 #endif
@@ -229,11 +203,11 @@
   {
     store::Item* lName = lIter->first;
     store::Item* lChild = lIter->second;
-    if (getCollection() != NULL && lChild->isJSONItem())
+    if (getCollection() != NULL && lChild->isStructuredItem())
     {
-      assert(dynamic_cast<JSONItem*>(lChild));
-      JSONItem* lJSONItem = static_cast<JSONItem*>(lChild);
-      lJSONItem->setTree(NULL);
+      assert(dynamic_cast<StructuredItem*>(lChild));
+      StructuredItem* lStructuredItem = static_cast<StructuredItem*>(lChild);
+      lStructuredItem->setCollectionTreeInfo(NULL);
     }
     lName->removeReference();
     lChild->removeReference();
@@ -324,11 +298,11 @@
   {
     store::Item* lValue = aValue.getp();
 
-    if (getCollection() != NULL && aValue->isJSONItem())
+    if (getCollection() != NULL && (aValue->isStructuredItem()))
     {
-      assert(dynamic_cast<JSONItem*>(aValue.getp()));
-      JSONItem* lJSONItem = static_cast<JSONItem*>(aValue.getp());
-      lJSONItem->setTree(getTree());
+      assert(dynamic_cast<StructuredItem*>(aValue.getp()));
+      StructuredItem* lStructuredItem = static_cast<StructuredItem*>(aValue.getp());
+      lStructuredItem->setCollectionTreeInfo(theCollectionInfo);
     }
     
     csize lPosition = thePairs.size();
@@ -360,7 +334,7 @@
 
       if (getCollection() != NULL)
       {
-        array->setTree(getTree());
+        array->setCollectionTreeInfo(theCollectionInfo);
       }
 
       lValue->removeReference();
@@ -399,11 +373,11 @@
   lKey = thePairs[lPosition].first;
   lValue = thePairs[lPosition].second;
 
-  if (getCollection() != NULL && lValue->isJSONItem())
+  if (getCollection() != NULL && (lValue->isStructuredItem()))
   {
-    assert(dynamic_cast<JSONItem*>(lValue.getp()));
-    JSONItem* lJSONItem = static_cast<JSONItem*>(lValue.getp());
-    lJSONItem->setTree(NULL);
+    assert(dynamic_cast<StructuredItem*>(lValue.getp()));
+    StructuredItem* lStructuredItem = static_cast<StructuredItem*>(lValue.getp());
+    lStructuredItem->setCollectionTreeInfo(NULL);
   }
 
   lKey->removeReference();
@@ -455,18 +429,20 @@
 
   if (getCollection() != NULL)
   {
-    if (lOldValue->isJSONItem())
+    if (lOldValue->isStructuredItem())
     {
-      assert(dynamic_cast<JSONItem*>(lOldValue.getp()));
-      JSONItem* lJSONItem = static_cast<JSONItem*>(lOldValue.getp());
-      lJSONItem->setTree(NULL);
+      assert(dynamic_cast<StructuredItem*>(lOldValue.getp()));
+      StructuredItem* lStructuredItem =
+          static_cast<StructuredItem*>(lOldValue.getp());
+      lStructuredItem->setCollectionTreeInfo(NULL);
     }
 
-    if (aValue->isJSONItem())
+    if (aValue->isStructuredItem())
     {
-      assert(dynamic_cast<JSONItem*>(aValue.getp()));
-      JSONItem* lJSONItem = static_cast<JSONItem*>(aValue.getp());
-      lJSONItem->setTree(getTree());
+      assert(dynamic_cast<StructuredItem*>(aValue.getp()));
+      StructuredItem* lStructuredItem =
+          static_cast<StructuredItem*>(aValue.getp());
+      lStructuredItem->setCollectionTreeInfo(theCollectionInfo);
     }
   }
 
@@ -518,6 +494,7 @@
   return true;
 }
 
+
 /******************************************************************************
 
 *******************************************************************************/
@@ -527,27 +504,28 @@
   assert(lOther);
   std::swap(theKeys, lOther->theKeys);
   std::swap(thePairs, lOther->thePairs);
-  setTree(getTree());
-  lOther->setTree(lOther->getTree());
+  setCollectionTreeInfo(theCollectionInfo);
+  lOther->setCollectionTreeInfo(lOther->theCollectionInfo);
 }
 
+
 /******************************************************************************
 
 *******************************************************************************/
-void SimpleJSONObject::setTree(JSONTree* aTree)
+void SimpleJSONObject::setCollectionTreeInfo(CollectionTreeInfo* collectionInfo)
 {
-  theTree = aTree;
+  theCollectionInfo = static_cast<CollectionTreeInfoWithTreeId*>(collectionInfo);
 
-  for (Pairs::iterator lIter = thePairs.begin();
-       lIter != thePairs.end();
-       ++lIter)
+  for (Pairs::iterator ite = thePairs.begin();
+       ite != thePairs.end();
+       ++ite)
   {
-    store::Item* lValue = lIter->second;
-    if (lValue->isJSONItem())
+    store::Item* value = ite->second;
+
+    if (value->isStructuredItem())
     {
-      assert(dynamic_cast<JSONItem*>(lValue));
-      JSONItem* lJSONItem = static_cast<JSONItem*>(lValue);
-      lJSONItem->setTree(aTree);
+      StructuredItem* structuredItem = static_cast<StructuredItem*>(value);
+      structuredItem->setCollectionTreeInfo(collectionInfo);
     }
   }
 }
@@ -651,9 +629,10 @@
 /******************************************************************************
 
 *******************************************************************************/
-bool SimpleJSONObject::isThisTreeOfAllDescendants(const JSONTree* aTree) const
+bool SimpleJSONObject::isThisTreeOfAllDescendants(
+    const CollectionTreeInfo* collectionInfo) const
 {
-  if (theTree != aTree)
+  if (theCollectionInfo != collectionInfo)
   {
     return false;
   }
@@ -664,19 +643,21 @@
   {
     store::Item* lValue = lIter->second;
     const JSONItem* lJSONItem = dynamic_cast<const JSONItem*>(lValue);
-    if (lJSONItem != NULL && (!lJSONItem->isThisTreeOfAllDescendants(aTree)))
+    if (lJSONItem != NULL && 
+        !lJSONItem->isThisTreeOfAllDescendants(collectionInfo))
     {
       return false;
     }
   }
   return true;
 }
+#endif // NDEBUG
 
 
 /******************************************************************************
 
 *******************************************************************************/
-bool SimpleJSONObject::isThisJSONItemInDescendance(const store::Item* anItem) const
+bool SimpleJSONObject::isInSubtree(const StructuredItem* anItem) const
 {
   if (this == anItem)
   {
@@ -688,23 +669,18 @@
        ++lIter)
   {
     store::Item* lValue = lIter->second;
-    const SimpleJSONObject* lObject =
-      dynamic_cast<const SimpleJSONObject*>(lValue);
-    const SimpleJSONArray* lArray =
-      dynamic_cast<const SimpleJSONArray*>(lValue);
- 
-    if (lObject != NULL && lObject->isThisJSONItemInDescendance(anItem))
-    {
-      return true;
-    }
-    else if (lArray != NULL && lArray->isThisJSONItemInDescendance(anItem))
-    {
-      return true;
+    if (lValue->isStructuredItem())
+    {
+      const StructuredItem* lStructuredItem =
+        static_cast<const StructuredItem*>(lValue);
+      if (lStructuredItem->isInSubtree(anItem))
+      {
+        return true;
+      }
     }
   }
   return false;
 }
-#endif // NDEBUG
 
 
 /******************************************************************************
@@ -799,11 +775,13 @@
        lIter != theContent.end();
        ++lIter)
   {
-    if (getCollection() != NULL && (*lIter)->isJSONItem())
+    if (getCollection() != NULL &&
+        ((*lIter)->isStructuredItem()))
     {
-      assert(dynamic_cast<JSONItem*>(*lIter));
-      JSONItem* lJSONItem = static_cast<JSONItem*>(*lIter);
-      lJSONItem->setTree(NULL);
+      assert(dynamic_cast<StructuredItem*>(*lIter));
+      StructuredItem* lStructuredItem =
+          static_cast<StructuredItem*>(*lIter);
+      lStructuredItem->setCollectionTreeInfo(NULL);
     }
     (*lIter)->removeReference();
   }
@@ -831,11 +809,12 @@
 {
   ASSERT_INVARIANT();
 
-  if (getCollection() != NULL && aValue->isJSONItem())
+  if (getCollection() != NULL && (aValue->isStructuredItem()))
   {
-    assert(dynamic_cast<JSONItem*>(aValue.getp()));
-    JSONItem* lJSONItem = static_cast<JSONItem*>(aValue.getp());
-    lJSONItem->setTree(getTree());
+    assert(dynamic_cast<StructuredItem*>(aValue.getp()));
+    StructuredItem* lStructuredItem =
+        static_cast<StructuredItem*>(aValue.getp());
+    lStructuredItem->setCollectionTreeInfo(theCollectionInfo);
   }
 
   aValue->addReference();
@@ -878,11 +857,13 @@
 {
   ASSERT_INVARIANT();
 
-  if (getCollection() != NULL && member->isJSONItem())
+  if (getCollection() != NULL &&
+      (member->isStructuredItem()))
   {
-    assert(dynamic_cast<JSONItem*>(member.getp()));
-    JSONItem* lJSONItem = static_cast<JSONItem*>(member.getp());
-    lJSONItem->setTree(getTree());
+    assert(dynamic_cast<StructuredItem*>(member.getp()));
+    StructuredItem* lStructuredItem =
+        static_cast<StructuredItem*>(member.getp());
+    lStructuredItem->setCollectionTreeInfo(theCollectionInfo);
   }
 
   member->addReference();
@@ -939,11 +920,13 @@
   {
     store::Item* lItem = aNewMembers[i].getp();
 
-    if (getCollection() != NULL && lItem->isJSONItem())
+    if (getCollection() != NULL &&
+        (lItem->isStructuredItem()))
     {
-      assert(dynamic_cast<JSONItem*>(lItem));
-      JSONItem* lJSONItem = static_cast<JSONItem*>(lItem);
-      lJSONItem->setTree(getTree());
+      assert(dynamic_cast<StructuredItem*>(lItem));
+      StructuredItem* lStructuredItem =
+          static_cast<StructuredItem*>(lItem);
+      lStructuredItem->setCollectionTreeInfo(theCollectionInfo);
     }
 
     lItem->addReference();
@@ -962,11 +945,13 @@
   ASSERT_INVARIANT();
   store::Item_t lItem = getArrayValue(aPos);
 
-  if (getCollection() != NULL && lItem->isJSONItem())
+  if (getCollection() != NULL &&
+      (lItem->isStructuredItem()))
   {
-    assert(dynamic_cast<JSONItem*>(lItem.getp()));
-    JSONItem* lJSONItem = static_cast<JSONItem*>(lItem.getp());
-    lJSONItem->setTree(NULL);
+    assert(dynamic_cast<StructuredItem*>(lItem.getp()));
+    StructuredItem* lStructuredItem =
+        static_cast<StructuredItem*>(lItem.getp());
+    lStructuredItem->setCollectionTreeInfo(NULL);
   }
 
   lItem->removeReference();
@@ -986,8 +971,8 @@
   SimpleJSONArray* lOther = dynamic_cast<SimpleJSONArray*>(anotherItem);
   assert(lOther);
   std::swap(theContent, lOther->theContent);
-  setTree(getTree());
-  lOther->setTree(lOther->getTree());
+  setCollectionTreeInfo(theCollectionInfo);
+  lOther->setCollectionTreeInfo(lOther->theCollectionInfo);
 }
 
 /******************************************************************************
@@ -1000,26 +985,30 @@
   ASSERT_INVARIANT();
   store::Item_t lItem = getArrayValue(aPos);
 
-  if (getCollection() != NULL && lItem->isJSONItem())
+  if (getCollection() != NULL &&
+      (lItem->isStructuredItem()))
   {
-    assert(dynamic_cast<JSONItem*>(lItem.getp()));
-    JSONItem* lJSONItem = static_cast<JSONItem*>(lItem.getp());
-    lJSONItem->setTree(NULL);
+      assert(dynamic_cast<StructuredItem*>(lItem.getp()));
+      StructuredItem* lStructuredItem =
+          static_cast<StructuredItem*>(lItem.getp());
+      lStructuredItem->setCollectionTreeInfo(NULL);
   }
 
   uint64_t pos = cast(aPos) - 1;
 
-  if (getCollection() != NULL && value->isJSONItem())
+  if (getCollection() != NULL &&
+      (value->isStructuredItem()))
   {
-    assert(dynamic_cast<JSONItem*>(value.getp()));
-    JSONItem* lJSONItem = static_cast<JSONItem*>(value.getp());
-    lJSONItem->setTree(getTree());
+    assert(dynamic_cast<StructuredItem*>(value.getp()));
+    StructuredItem* lStructuredItem =
+        static_cast<StructuredItem*>(value.getp());
+    lStructuredItem->setCollectionTreeInfo(theCollectionInfo);
   }
 
   theContent[pos]->removeReference();
   value->addReference();
   theContent[pos] = value.getp();
-  
+
   ASSERT_INVARIANT();
   return lItem;
 }
@@ -1028,20 +1017,20 @@
 /******************************************************************************
 
 *******************************************************************************/
-void SimpleJSONArray::setTree(JSONTree* aTree)
+void SimpleJSONArray::setCollectionTreeInfo(CollectionTreeInfo* collectionInfo)
 {
-  theTree = aTree;
+  theCollectionInfo = static_cast<CollectionTreeInfoWithTreeId*>(collectionInfo);
 
-  for (Members::const_iterator lIter = theContent.begin();
-       lIter != theContent.end();
-       ++lIter)
+  for (Members::const_iterator ite = theContent.begin();
+       ite != theContent.end();
+       ++ite)
   {
-    store::Item* lValue = *lIter;
-    if (lValue->isJSONItem())
+    store::Item* value = *ite;
+
+    if (value->isStructuredItem())
     {
-      assert(dynamic_cast<JSONItem*>(lValue));
-      JSONItem* lJSONItem = static_cast<JSONItem*>(lValue);
-      lJSONItem->setTree(aTree);
+      StructuredItem* structuredItem = static_cast<StructuredItem*>(value);
+      structuredItem->setCollectionTreeInfo(collectionInfo);
     }
   }
 }
@@ -1195,9 +1184,10 @@
 /******************************************************************************
 
 *******************************************************************************/
-bool SimpleJSONArray::isThisTreeOfAllDescendants(const JSONTree* aTree) const
+bool SimpleJSONArray::isThisTreeOfAllDescendants(
+    const CollectionTreeInfo* collectionInfo) const
 {
-  if (getTree() != aTree)
+  if (theCollectionInfo != collectionInfo)
   {
     return false;
   }
@@ -1208,7 +1198,8 @@
   {
     store::Item* lValue = (*lIter);
     const JSONItem* lJSONItem = dynamic_cast<const JSONItem*>(lValue);
-    if (lJSONItem != NULL && (!lJSONItem->isThisTreeOfAllDescendants(aTree)))
+    if (lJSONItem != NULL &&
+        !lJSONItem->isThisTreeOfAllDescendants(collectionInfo))
     {
       return false;
     }
@@ -1217,37 +1208,39 @@
 }
 
 
+#endif // NDEBUG
+
 /******************************************************************************
 
 *******************************************************************************/
-bool SimpleJSONArray::isThisJSONItemInDescendance(const store::Item* anItem) const
+bool SimpleJSONArray::isInSubtree(const StructuredItem* anItem) const
 {
-  if(this == anItem)
+  if (this == anItem)
   {
     return true;
   }
+
   for (Members::const_iterator lIter = theContent.begin();
        lIter != theContent.end();
        ++lIter)
   {
-    const SimpleJSONObject* lObject =
-        dynamic_cast<const SimpleJSONObject*>(*lIter);
-    const SimpleJSONArray* lArray =
-        dynamic_cast<const SimpleJSONArray*>(*lIter);
-    if (lObject != NULL && lObject->isThisJSONItemInDescendance(anItem))
-    {
-      return true;
-    }
-    else if (lArray != NULL && lArray->isThisJSONItemInDescendance(anItem))
-    {
-      return true;
+    store::Item* lValue = *lIter;
+
+    if (lValue->isStructuredItem())
+    {
+      const StructuredItem* structuredItem = 
+      static_cast<const StructuredItem*>(lValue);
+
+      if (structuredItem->isInSubtree(anItem))
+      {
+        return true;
+      }
     }
   }
+
   return false;
 }
 
-#endif // NDEBUG
-
 
 /******************************************************************************
 

=== modified file 'src/store/naive/json_items.h'
--- src/store/naive/json_items.h	2013-02-17 00:59:54 +0000
+++ src/store/naive/json_items.h	2013-02-21 15:32:25 +0000
@@ -22,11 +22,15 @@
 #include <zorba/config.h>
 #include "util/unordered_map.h"
 
+#include "diagnostics/assert.h"
+
 #include "store/api/item_handle.h"
 #include "store/api/iterator.h"
 
 #include "atomic_items.h"
+#include "collection_tree_info.h"
 #include "simple_collection.h"
+#include "structured_item.h"
 
 
 namespace zorba
@@ -40,6 +44,8 @@
 namespace simplestore
 {
 
+class CollectionTreeInfoGetters;
+
 namespace json
 {
 
@@ -84,66 +90,20 @@
 
 *******************************************************************************/
 
-class JSONTree
-{
-private:
-  simplestore::Collection * theCollection;
-  TreeId                    theId;
-  JSONItem                * theRoot;
-
-public:
-  JSONTree() : theCollection(NULL), theId(), theRoot(NULL)
-  {}
-
-  ~JSONTree();
-
-  simplestore::Collection* getCollection() const
-  {
-    return theCollection;
-  }
-
-  void setCollection(simplestore::Collection* aCollection)
-  {
-    theCollection = aCollection;
-  }
-
-  const TreeId& getTreeId() const
-  {
-    return theId;
-  }
-
-  void setTreeId(const TreeId& aId)
-  {
-    theId = aId;
-  }
-
-  JSONItem* getRoot() const
-  {
-    return theRoot;
-  }
-
-  void setRoot(JSONItem* aRoot)
-  {
-    theRoot = aRoot;
-  }
-};
-
-
-/******************************************************************************
-
-*******************************************************************************/
-
-class JSONItem : public store::Item
-{
+class JSONItem : public StructuredItem
+{
+  // Used to access collection tree information.
+  friend class zorba::simplestore::CollectionTreeInfoGetters;
+
 protected:
   SYNC_CODE(mutable RCLock  theRCLock;)
 
-  JSONTree  * theTree;
+  CollectionTreeInfoWithTreeId * theCollectionInfo;
 
 public:
   SYNC_CODE(RCLock* getRCLock() const { return &theRCLock; })
 
-  JSONItem() : store::Item(JSONIQ), theTree(NULL) {}
+  JSONItem() : StructuredItem(store::Item::JSONIQ), theCollectionInfo(NULL) {}
 
   virtual ~JSONItem();
   
@@ -151,31 +111,28 @@
 
   virtual void destroy();
 
-  const simplestore::Collection* getCollection() const;
-
-  virtual void setTree(JSONTree* aTree) = 0;
-
-  JSONTree* getTree() const
-  {
-    return theTree;
-  }
-  
-  bool isRoot() const
-  {
-    return theTree != NULL && (this == theTree->getRoot());
-  }
-
-  // These two functions are only to be called if in a collection.
-  const TreeId& getTreeId() const;
-
-  JSONItem* getRoot() const;
-  
-  void attachToCollection(Collection* aCollection, const TreeId& aTreeId);
+  // StructuredItem API
+
+  void attachToCollection(
+      Collection* aCollection,
+      const TreeId& aTreeId,
+      const xs_integer& aPosition);
 
   void detachFromCollection();
   
+  CollectionTreeInfo* getCollectionTreeInfo() const { return theCollectionInfo; }
+
+  void setCollectionTreeInfo(CollectionTreeInfo* collectionInfo) = 0;
+  
+  long getCollectionTreeRefCount() const;
+
   // store API
 
+  const store::Collection* getCollection() const
+  {
+    return (theCollectionInfo ? theCollectionInfo->getCollection() : NULL);
+  }
+
   virtual bool equals(
       const store::Item* other,
       long timezone = 0,
@@ -188,9 +145,8 @@
 #ifndef NDEBUG
   virtual void assertInvariant() const;
 
-  virtual bool isThisTreeOfAllDescendants(const JSONTree* aTree) const = 0;
-
-  virtual bool isThisJSONItemInDescendance(const store::Item* aJSONItem) const = 0;
+  virtual bool isThisTreeOfAllDescendants(
+      const CollectionTreeInfo* collectionInfo) const = 0;
 #endif
 };
 
@@ -345,21 +301,19 @@
       const store::Item_t& aName,
       const store::Item_t& aNewName);
 
+  // StructuredItem API
+
+  void setCollectionTreeInfo(CollectionTreeInfo* collectionInfo);
+
+  bool isInSubtree(const StructuredItem* aJSONItem) const;
+
   virtual void swap(store::Item* anotherItem);
 
-  // root management
-  
-protected:
-  void setTree(JSONTree* aTree);
-
   // Invariant handling
-public:
 #ifndef NDEBUG
   void assertInvariant() const;
   
-  bool isThisTreeOfAllDescendants(const JSONTree* aTree) const;
-
-  bool isThisJSONItemInDescendance(const store::Item* aJSONItem) const;
+  bool isThisTreeOfAllDescendants(const CollectionTreeInfo* collectionInfo) const;
 #endif
 };
 
@@ -454,7 +408,7 @@
   {}
 
   virtual ~SimpleJSONArray();
-  
+
   // store API
 
   size_t alloc_size() const;
@@ -506,11 +460,13 @@
   virtual store::Item_t
   replace(const xs_integer& aPos, const store::Item_t& value);
 
+  // StructuredItem API
+  
+  void setCollectionTreeInfo(CollectionTreeInfo* collectionInfo);
+
   virtual void swap(Item* anotherItem);
 
-  // root management
-public:
-  void setTree(JSONTree* aTree);
+  bool isInSubtree(const StructuredItem* aJSONItem) const;
 
 protected:
   void add(uint64_t pos, const std::vector<store::Item_t>& aNewMembers);
@@ -520,9 +476,7 @@
   // Invariant handling
 public:
 #ifndef NDEBUG
-  bool isThisTreeOfAllDescendants(const JSONTree* aTree) const;
-
-  bool isThisJSONItemInDescendance(const store::Item* aJSONItem) const;
+  bool isThisTreeOfAllDescendants(const CollectionTreeInfo* colectionInfo) const;
 #endif
 };
 

=== modified file 'src/store/naive/loader_dtd.cpp'
--- src/store/naive/loader_dtd.cpp	2013-02-13 02:18:07 +0000
+++ src/store/naive/loader_dtd.cpp	2013-02-21 15:32:25 +0000
@@ -37,7 +37,6 @@
 #include "loader.h"
 #include "simple_item_factory.h"
 #include "node_factory.h"
-
 #include "zorbatypes/datetime.h"
 #include "zorbatypes/URI.h"
 
@@ -198,7 +197,7 @@
   if (docUri.empty())
   {
     std::ostringstream uristream;
-    uristream << "zorba://internalDocumentURI-" << theTree->getId();
+    uristream << "zorba://internalDocumentURI-" << theTree->getTreeId();
     theDocUri = uristream.str();
   }
   else
@@ -796,7 +795,7 @@
   if (docUri.empty())
   {
     std::ostringstream uristream;
-    uristream << "zorba://internalDocumentURI-" << theTree->getId();
+    uristream << "zorba://internalDocumentURI-" << theTree->getTreeId();
     theDocUri = uristream.str();
   }
   else

=== modified file 'src/store/naive/loader_fast.cpp'
--- src/store/naive/loader_fast.cpp	2013-02-13 02:10:51 +0000
+++ src/store/naive/loader_fast.cpp	2013-02-21 15:32:25 +0000
@@ -372,7 +372,8 @@
   if (docUri.empty())
   {
     std::ostringstream uristream;
-    uristream << "zorba://internalDocumentURI-" << theTree->getId();
+    uristream << "zorba://internalDocumentURI-" << theTree->getTreeId();
+
     theDocUri = uristream.str();
   }
   else

=== modified file 'src/store/naive/node_items.cpp'
--- src/store/naive/node_items.cpp	2013-02-07 02:43:13 +0000
+++ src/store/naive/node_items.cpp	2013-02-21 15:32:25 +0000
@@ -71,9 +71,7 @@
 XmlTree::XmlTree()
   :
   theRefCount(0),
-  theId(0),
-  thePos(0),
-  theCollection(NULL),
+  theCollectionInfo(NULL),
   theRootNode(NULL),
 #ifdef DATAGUIDE
   theDataGuideRootNode(NULL),
@@ -91,9 +89,8 @@
 XmlTree::XmlTree(XmlNode* root, const TreeId& id)
   :
   theRefCount(0),
-  theId(id),
-  thePos(0),
-  theCollection(NULL),
+  theCollectionInfo(NULL),
+  theTreeId(id),
   theRootNode(root),
 #ifdef DATAGUIDE
   theDataGuideRootNode(NULL),
@@ -111,47 +108,17 @@
 /*******************************************************************************
 
 ********************************************************************************/
-void XmlTree::claimedByCollection(Collection* collection)
-{
-  ZORBA_ASSERT(collection != NULL);
-  theCollection = collection;
-}
-
-
-/*******************************************************************************
-
-********************************************************************************/
-void XmlTree::setCollection(Collection* collection, xs_integer pos)
-{
-  ZORBA_ASSERT(collection == NULL || theCollection == NULL);
-
-  theCollection = collection;
-  thePos = pos;
-
-  if (collection != NULL)
-    theId = collection->createTreeId();
-}
-
-
-/*******************************************************************************
-
-********************************************************************************/
-ulong XmlTree::getCollectionId() const
-{
-  if (theCollection != NULL)
-    return theCollection->getId();
-  else
-    return 0;
-}
-
-
-/*******************************************************************************
-
-********************************************************************************/
 void XmlTree::destroy() throw()
 {
   // std::cout << "Deleting Xml Tree: " << this << std::endl;
 
+  // Only delete if not in a collection, or if it is the overall root.
+  if (theCollectionInfo != NULL &&
+      theCollectionInfo->getRoot() == static_cast<StructuredItem*>(getRoot()))
+  {
+    delete theCollectionInfo;
+  }
+
   if (theRootNode != 0)
   {
     theRootNode->destroy(false);
@@ -285,9 +252,56 @@
   }
 }
 
-
 #endif // #ifndef EMBEDED_TYPE
 
+/*******************************************************************************
+
+********************************************************************************/
+void XmlTree::setCollectionTreeInfo(CollectionTreeInfo* collectionInfo)
+{
+  // If the supplied tree information is not NULL, the existing tree info must
+  // be NULL.
+  assert(!collectionInfo || !theCollectionInfo);
+  // If the supplied tree information is NULL, the existing tree info must
+  // exist, and this node may not be the root.
+  assert(collectionInfo || theCollectionInfo);
+  assert(collectionInfo ||
+         theCollectionInfo->getRoot() != static_cast<StructuredItem*>(getRoot()));
+
+  theCollectionInfo = collectionInfo;
+}
+
+/*******************************************************************************
+
+********************************************************************************/
+void XmlTree::attachToCollection(
+    simplestore::Collection* aCollection,
+    const TreeId& aTreeId,
+    const xs_integer& aPosition)
+{
+  assert(!theCollectionInfo);
+
+  theCollectionInfo = new CollectionTreeInfo();
+  theCollectionInfo->setCollection(aCollection);
+  theCollectionInfo->setPosition(aPosition);
+  theCollectionInfo->setRoot(getRoot());
+  theTreeId = aTreeId;
+}
+  
+/*******************************************************************************
+
+********************************************************************************/
+void XmlTree::detachFromCollection()
+{
+  // The tree info must exist and this XML must be the overall root.
+  assert(theCollectionInfo);
+  assert(theCollectionInfo->getRoot() == static_cast<StructuredItem*>(getRoot()));
+
+  delete theCollectionInfo;
+  theCollectionInfo = NULL;
+}
+
+
 
 /////////////////////////////////////////////////////////////////////////////////
 //                                                                             //
@@ -909,6 +923,55 @@
     GET_STORE().unregisterReferenceToDeletedNode(this);
 }
 
+/*******************************************************************************
+
+********************************************************************************/
+void XmlNode::attachToCollection(
+    simplestore::Collection* aCollection,
+    const TreeId& aTreeId,
+    const xs_integer& aPosition)
+{
+  getTree()->attachToCollection(aCollection, aTreeId, aPosition);
+}
+  
+/*******************************************************************************
+
+********************************************************************************/
+void XmlNode::detachFromCollection()
+{
+  getTree()->detachFromCollection();
+}
+
+/*******************************************************************************
+
+********************************************************************************/
+void XmlNode::setCollectionTreeInfo(CollectionTreeInfo* lTreeInfo)
+{
+  getTree()->setCollectionTreeInfo(lTreeInfo);
+}
+
+/*******************************************************************************
+
+********************************************************************************/
+long XmlNode::getCollectionTreeRefCount() const
+{
+  return getTree()->getRefCount();
+}
+
+/*******************************************************************************
+
+********************************************************************************/
+bool XmlNode::isInSubtree(const StructuredItem* anotherItem) const
+{
+  if (!anotherItem->isNode())
+  {
+    return false;
+  }
+  assert(this == getTree()->getRoot());
+  assert(dynamic_cast<const XmlNode*>(anotherItem));
+  const XmlNode* aNode = static_cast<const XmlNode*>(anotherItem);
+  return getTree() == aNode->getTree();
+}
 
 
 /////////////////////////////////////////////////////////////////////////////////
@@ -1327,6 +1390,7 @@
 ********************************************************************************/
 bool OrdPathNode::isAncestor(const store::Item_t& aOther) const
 {
+  assert(dynamic_cast<XmlNode*>(aOther.getp()));
   assert(!static_cast<XmlNode*>(aOther.getp())->isConnectorNode());
 
   const OrdPathNode* lThisNode = static_cast<const OrdPathNode*>(this);
@@ -1353,6 +1417,7 @@
 ********************************************************************************/
 bool OrdPathNode::isFollowing(const store::Item_t& aOther) const
 { 
+  assert(dynamic_cast<XmlNode*>(aOther.getp()));
   assert(!static_cast<XmlNode*>(aOther.getp())->isConnectorNode());
 
   const OrdPathNode* lThisNode = static_cast<const OrdPathNode*>(this);
@@ -1370,6 +1435,7 @@
 ********************************************************************************/
 bool OrdPathNode::isDescendant(const store::Item_t& aOther) const
 { 
+  assert(dynamic_cast<XmlNode*>(aOther.getp()));
   assert(!static_cast<XmlNode*>(aOther.getp())->isConnectorNode());
 
   const OrdPathNode* lThisNode = static_cast<const OrdPathNode*>(this);
@@ -1387,6 +1453,7 @@
 ********************************************************************************/
 bool OrdPathNode::isPrecedingSibling(const store::Item_t& aOther) const
 {
+  assert(dynamic_cast<XmlNode*>(aOther.getp()));
   assert(!static_cast<XmlNode*>(aOther.getp())->isConnectorNode());
  
   return isPreceding(aOther) && getParent() == aOther->getParent();
@@ -1398,6 +1465,7 @@
 ********************************************************************************/
 bool OrdPathNode::isPreceding(const store::Item_t& aOther) const
 {
+  assert(dynamic_cast<XmlNode*>(aOther.getp()));
   assert(!static_cast<XmlNode*>(aOther.getp())->isConnectorNode());
  
   const OrdPathNode* lThisNode = static_cast<const OrdPathNode*>(this);
@@ -1415,6 +1483,7 @@
 ********************************************************************************/
 bool OrdPathNode::isChild(const store::Item_t& aOther) const
 {
+  assert(dynamic_cast<XmlNode*>(aOther.getp()));
   assert(!static_cast<XmlNode*>(aOther.getp())->isConnectorNode());
  
   return aOther->getParent() == this;
@@ -1426,6 +1495,7 @@
 ********************************************************************************/
 bool OrdPathNode::isParent(const store::Item_t& aOther) const
 {
+  assert(dynamic_cast<XmlNode*>(aOther.getp()));
   assert(!static_cast<XmlNode*>(aOther.getp())->isConnectorNode());
  
   return this->getParent() == aOther;
@@ -1455,6 +1525,7 @@
 ********************************************************************************/
 store::Item_t OrdPathNode::leastCommonAncestor(const store::Item_t& aOther) const
 {
+  assert(dynamic_cast<XmlNode*>(aOther.getp()));
   assert(!static_cast<XmlNode*>(aOther.getp())->isConnectorNode());
 
   const OrdPathNode* lThisNode = static_cast<const OrdPathNode*>(this);
@@ -1889,7 +1960,7 @@
   theDocUri(docUri)
 {
   STORE_TRACE1("{\nConstructing doc node " << this << " tree = "
-              << getTree()->getId() << ":" << getTree()
+              << getTreeId() << ":" << getTree()
               << " doc uri = " << docUri);
 }
 
@@ -2250,7 +2321,7 @@
 
   STORE_TRACE1("Constructed element node " << this << " parent = "
               << std::hex << (parent ? (ulong)parent : 0) << " pos = " << pos
-              << " tree = " << getTree()->getId() << ":" << getTree()
+              << " tree = " << getTreeId() << ":" << getTree()
               << " ordpath = " << theOrdPath.show()
               << " name = " << theName->getStringValue()
               << " type = " << getType()->getStringValue());
@@ -3757,7 +3828,7 @@
 
   STORE_TRACE1("Constructed attribute node " << this << " parent = "
               << std::hex << (parent ? (ulong)parent : 0) << " pos = " << pos
-              << " tree = " << getTree()->getId() << ":" << getTree()
+              << " tree = " << getTreeId() << ":" << getTree()
               << " ordpath = " << theOrdPath.show()
               << " name = " << theName->getStringValue()
               << " value = " << getStringValue());
@@ -4223,7 +4294,7 @@
 #else
   STORE_TRACE1("Constructed text node " << this << " parent = "
               << std::hex << (parent ? (ulong)parent : 0) << " pos = " << pos
-              << " tree = " << getTree()->getId() << ":" << getTree()
+              << " tree = " << getTreeId() << ":" << getTree()
               << " content = " << getText());
 #endif
 }
@@ -4940,7 +5011,7 @@
 
   STORE_TRACE1("Constructed pi node " << this << " parent = "
               << std::hex << (parent ? (ulong)parent : 0) << " pos = " << pos
-              << " tree = " << getTree()->getId() << ":" << getTree()
+              << " tree = " << getTreeId() << ":" << getTree()
               << " ordpath = " << theOrdPath.show() << " target = " << theTarget);
 }
 
@@ -5100,7 +5171,7 @@
 
   STORE_TRACE1("Constructed comment node " << this << " parent = "
               << std::hex << (parent ? (ulong)parent : 0) << " pos = " << pos
-              << " tree = " << getTree()->getId() << ":" << getTree()
+              << " tree = " << getTreeId() << ":" << getTree()
               << " ordpath = " << theOrdPath.show() << " content = "
               << theContent);
 }

=== modified file 'src/store/naive/node_items.h'
--- src/store/naive/node_items.h	2013-02-07 02:43:13 +0000
+++ src/store/naive/node_items.h	2013-02-21 15:32:25 +0000
@@ -44,6 +44,10 @@
 #include "store_defs.h"
 #include "text_node_content.h"
 #include "tree_id.h"
+#include "simple_store.h"
+#include "structured_item.h"
+#include "collection_tree_info.h"
+#include "collection.h"
 
 // Note: whether the EMBEDED_TYPE is defined or not is done in store_defs.h
 #ifndef EMBEDED_TYPE
@@ -66,6 +70,10 @@
 namespace simplestore
 {
 
+namespace json {
+  class JSONItem;
+}
+
 class AttributeNode;
 class CommentNode;
 class DocumentNode;
@@ -95,6 +103,8 @@
 
 class Collection;
 
+class CollectionTreeInfoGetters;
+
 typedef std::vector<NodeTypeInfo> TypeUndoList;
 
 typedef rchandle<NsBindingsContext> NsBindingsContext_t;
@@ -111,38 +121,51 @@
 
 /*******************************************************************************
 
-  theRefCount    : It is the sum of theRefCounts of all the nodes belonging to
-                   this tree. Individual nodes are not destroyed when their
-                   refCount goes to 0. Instead, nodes belonging to a tree are all
-                   destroyed together when theRefCount of the tree goes to 0
-                   (i.e. when there are no refs to any of the nodes in the tree).
-  theRCLock      : Protects theRefCount
-
-  theId          : An internally generated id for the tree. The id uniquely
-                   identifies the tree within its containing collection (see
-                   Collection::createTreeId() method). Trees that do not
-                   belong to any collection, are considered to belong to a
-                   "virtual" collection (with collection id equal to 0), and
-                   their id is created by the SimpleStore::createId() method.
-                   theId it is guaranteed to be stable during the lifetime of
-                   the tree, but may be reused after the tree is destroyed.
-                   theId is used in building node URIs that are returned to
-                   the application. It is not meant to be used (at least not
-                   directly) to sort nodes in doc order.
-  thePos         : The position of this tree within its containing collection.
-                   After inserting/deleting one or more nodes in/from the middle
-
-  theBaseUri     : The base uri property of the tree's root node.
-  theDocUri      : A user provided uri for the tree (may be NULL).
-
-  theCollection  : The collection where this xml tree belongs to, if any. An xml
-                   tree may belong to at most one collection at a time.
-  theRootNode    : The root node of the tree.
-
-  theIsValidated : True if the tree has ever undergone schema validation.
-  theIsRecursive : True if the tree contains at least one pair of element nodes
-                   that have the same tag name and are in an ancestor-descendant
-                   relationship with each other.
+  Class XmlTree contains top-level information about an xml tree. Every node of
+  such a tree has a pointer to the associated XmlTree object.
+
+  theRefCount:
+  ------------
+  It is the sum of theRefCounts of all the nodes belonging to this tree. 
+  Individual nodes are not destroyed when their refCount goes to 0. Instead,
+  nodes belonging to a tree are all destroyed together when theRefCount of
+  the tree goes to 0 (i.e. when there are no refs to any of the nodes in the
+  tree).
+
+  theRCLock:
+  ----------
+  Protects theRefCount
+
+  theTreeId:
+  ----------
+  A collection-relative id for this tree. Uniquely identifies the tree within a
+  collection, or if the tree does not belong to any collection, its id is unique
+  among all the other trees that do not belong to any collection either. NOTE:
+  when a tree becomes member of a colection, it gets a new id. 
+ 
+  theCollectionInfo:
+  ------------------
+  Contains info that is relevant only if the tree belongs to a collection (see
+  class CollectionTreeInfo for more details).
+
+  theRootNode:
+  ------------
+  The root node of this XML tree.
+
+  theIsValidated : 
+  ----------------
+  True if the tree has ever undergone schema validation.
+
+  theIsRecursive :
+  ----------------
+  True if the tree contains at least one pair of element nodes that have the 
+  same tag name and are in an ancestor-descendant relationship with each other.
+
+  theTypesMap:
+  ------------
+
+  theTokens:
+  ----------
 ********************************************************************************/
 class XmlTree
 {
@@ -150,6 +173,9 @@
 
   // make sure that only created by the factory
   friend class NodeFactory;
+  
+  // For setting positions directly.
+  friend class SimpleCollection;
 
 #ifndef EMBEDED_TYPE
   typedef NodePointerHashMap<store::Item_t> NodeTypeMap;
@@ -159,10 +185,9 @@
   mutable long              theRefCount;
   SYNC_CODE(mutable RCLock  theRCLock;)
 
-  TreeId                    theId;
-  xs_integer                thePos;
+  CollectionTreeInfo      * theCollectionInfo;
 
-  Collection              * theCollection;
+  TreeId                    theTreeId;
 
   XmlNode                 * theRootNode;
 
@@ -198,27 +223,41 @@
   long& getRefCount()      { return theRefCount; }
 
   SYNC_CODE(RCLock* getRCLock() const { return &theRCLock; })
-
-  void setId(const TreeId& id) { theId = id; }
-
-  const TreeId& getId() const { return theId; }
-
-  ulong getCollectionId() const;
-
-  const Collection* getCollection() const { return theCollection; }
-
-private:
-friend class zorba::simplestore::Collection;
-  // Allows a collection to claim ownership of a node it already owns, but
-  // which does not have the backpointer yet.
-  void claimedByCollection(Collection* coll);
   
-public:
-  void setCollection(Collection* coll, xs_integer pos);
-
-  void setPosition(xs_integer pos) { thePos = pos; }
-
-  xs_integer getPosition() const { return thePos; }
+  const TreeId& getTreeId() const { return theTreeId; }
+
+  CollectionTreeInfo* getCollectionTreeInfo() const { return theCollectionInfo; }
+
+  void setCollectionTreeInfo(CollectionTreeInfo* collectionInfo);
+
+  void attachToCollection(
+      simplestore::Collection* aCollection,
+      const TreeId& aTreeId,
+      const xs_integer& aPosition);
+
+  void detachFromCollection();
+
+  ulong getCollectionId() const
+  {
+    return (theCollectionInfo ? theCollectionInfo->getCollection()->getId() : 0);
+  }
+
+  const store::Collection* getCollection() const
+  {
+    return (theCollectionInfo ? theCollectionInfo->getCollection() : NULL);
+  }
+
+  const xs_integer& getPosition() const
+  {
+    ZORBA_ASSERT(theCollectionInfo);
+    return theCollectionInfo->getPosition();
+  }
+
+  void setPosition(const xs_integer& pos) const
+  {
+    ZORBA_ASSERT(theCollectionInfo);
+    return theCollectionInfo->setPosition(pos);
+  }
 
   XmlNode* getRoot() const { return theRootNode; }
 
@@ -264,7 +303,7 @@
 
  ******************************************************************************/
 
-class XmlNode : public store::Item
+class XmlNode : public StructuredItem
 {
   friend class XmlTree;
   friend class NodeFactory;
@@ -363,7 +402,9 @@
   {
   }
 
-  XmlNode(store::StoreConsts::NodeKind k) : Item(), theParent(NULL)
+  XmlNode(store::StoreConsts::NodeKind k) :
+      StructuredItem(),
+      theParent(NULL)
   {
     theFlags = (uint32_t)k;
   }
@@ -428,7 +469,7 @@
   const store::Collection* getCollection() const
   {
     assert(!isConnectorNode());
-    return reinterpret_cast<const store::Collection*>(getTree()->getCollection());
+    return getTree()->getCollection();
   }
 
   virtual void getDocumentURI(zstring& uri) const
@@ -441,11 +482,6 @@
     return reinterpret_cast<store::Item*>(theParent);
   }
 
-  bool isRoot() const
-  {
-    return getCollection() != NULL && getParent() == NULL;
-  }
-
   bool equals(
       const store::Item* other,
       long timezone = 0,
@@ -503,17 +539,9 @@
 
   XmlTree* getTree() const { return (XmlTree*)theUnion.treeRCPtr; }
 
-  const TreeId& getTreeId() const { return getTree()->getId(); }
-
   XmlNode* getRoot() const { return getTree()->getRoot(); }
 
-  void setCollection(Collection* coll, xs_integer pos)
-  {
-    assert(!isConnectorNode());
-    getTree()->setCollection(coll, pos);
-  }
-
-  ulong getCollectionId() const 
+  ulong getCollectionId() const
   {
     assert(!isConnectorNode());
     return getTree()->getCollectionId(); 
@@ -567,6 +595,23 @@
       locale::iso639_1::type,
       bool = false ) const;
 #endif /* ZORBA_NO_FULL_TEXT */
+
+  //
+  // StructuredItem Methods
+  //
+
+  virtual void attachToCollection(
+      simplestore::Collection* aCollection,
+      const TreeId& aTreeId,
+      const xs_integer& aPosition);
+
+  virtual void detachFromCollection();
+
+  virtual void setCollectionTreeInfo(CollectionTreeInfo* collectionInfo);
+
+  virtual long getCollectionTreeRefCount() const;
+
+  virtual bool isInSubtree(const StructuredItem* anotherItem) const;
 };
 
 
@@ -1614,8 +1659,8 @@
   {
     if (col1 == 0)
     {
-      TreeId tree1 = this->getTreeId();
-      TreeId tree2 = other->getTreeId();
+      TreeId tree1 = this->getTree()->getTreeId();
+      TreeId tree2 = other->getTree()->getTreeId();
 
       if (tree1 < tree2)
         return -1;
@@ -1664,8 +1709,8 @@
   {
     if (col1 == 0)
     {
-      const TreeId& tree1 = this->getTreeId();
-      const TreeId& tree2 = other->getTreeId();
+      const TreeId& tree1 = this->getTree()->getTreeId();
+      const TreeId& tree2 = other->getTree()->getTreeId();
 
       if (tree1 < tree2)
         return -1;

=== modified file 'src/store/naive/pul_primitives.cpp'
--- src/store/naive/pul_primitives.cpp	2013-02-03 19:01:15 +0000
+++ src/store/naive/pul_primitives.cpp	2013-02-21 15:32:25 +0000
@@ -166,7 +166,8 @@
   
   if (theTarget->isNode())
   {
-    static_cast<zorba::simplestore::XmlNode*>(theTarget.getp())
+    assert(dynamic_cast<XmlNode*>(theTarget.getp()));
+    static_cast<XmlNode*>(theTarget.getp())
         ->unregisterReferencesToDeletedSubtree();
   }
 }
@@ -1037,22 +1038,11 @@
 
   for (uint64_t i = 0; i < size; ++i)
   {
-    long lRefCount = 0;
     store::Item* lItem = collection->nodeAt(xs_integer(i)).getp();
-    if (lItem->isNode())
-    {
-      assert(dynamic_cast<XmlNode*>(lItem));
-      XmlNode* lNode = static_cast<XmlNode*>(lItem);
-      lRefCount = lNode->getTree()->getRefCount();
-#ifdef ZORBA_WITH_JSON
-    }
-    else if (lItem->isJSONItem())
-    {
-      assert(dynamic_cast<json::JSONItem*>(lItem));
-      json::JSONItem* lJSONItem = static_cast<json::JSONItem*>(lItem);
-      lRefCount = lJSONItem->getRefCount();
-#endif
-    }
+    assert(lItem->isStructuredItem());
+    assert(dynamic_cast<StructuredItem*>(lItem));
+    StructuredItem* lNode = static_cast<StructuredItem*>(lItem);
+    long lRefCount = lNode->getCollectionTreeRefCount();
 
     if (lRefCount > 1)
     {

=== modified file 'src/store/naive/simple_collection.cpp'
--- src/store/naive/simple_collection.cpp	2013-01-30 01:13:09 +0000
+++ src/store/naive/simple_collection.cpp	2013-02-21 15:32:25 +0000
@@ -38,13 +38,13 @@
 
 ********************************************************************************/
 SimpleCollection::SimpleCollection(
-    const store::Item_t& aName,
-    const std::vector<store::Annotation_t>& aAnnotations,
+    const store::Item_t& name,
+    const std::vector<store::Annotation_t>& annotations,
     bool isDynamic)
   : 
-  theName(aName),
+  Collection(name),
   theIsDynamic(isDynamic),
-  theAnnotations(aAnnotations)
+  theAnnotations(annotations)
 {
   theId = GET_STORE().createCollectionId();
   theTreeIdGenerator = GET_STORE().getTreeIdGeneratorFactory().createTreeGenerator(0);
@@ -73,69 +73,169 @@
 
 
 /*******************************************************************************
+
+********************************************************************************/
+void SimpleCollection::getAnnotations(
+    std::vector<store::Annotation_t>& annotations) const
+{
+  annotations = theAnnotations;
+}
+
+
+/*******************************************************************************
   Return an iterator over the nodes of this collection.
 
   Note: it is allowed to have several concurrent iterators on the same collection
   but each iterator should be used by a single thread only.
 ********************************************************************************/
-store::Iterator_t SimpleCollection::getIterator(const xs_integer& aSkip,
-                                                const zstring& aStart)
+store::Iterator_t SimpleCollection::getIterator(
+    const xs_integer& skip,
+    const zstring& startRef)
 {
-  store::Item_t lReferencedNode;
-  xs_integer lReferencedPosition = xs_integer::zero();
-  if (aStart.size() != 0
-   && (!GET_STORE().getNodeByReference(lReferencedNode, aStart)
-    || !findNode(lReferencedNode.getp(), lReferencedPosition)))
+  store::Item_t startNode;
+  xs_integer startPos = xs_integer::zero();
+
+  if (startRef.size() != 0 &&
+      (!GET_STORE().getNodeByReference(startNode, startRef) ||
+       !findNode(startNode.getp(), startPos)))
   {
     throw ZORBA_EXCEPTION(zerr::ZSTR0066_REFERENCED_NODE_NOT_IN_COLLECTION,
-    ERROR_PARAMS(aStart, theName->getStringValue()));
-  }
-
-  return new CollectionIter(
-               this, 
-               aSkip + lReferencedPosition);
+    ERROR_PARAMS(startRef, theName->getStringValue()));
+  }
+
+  try
+  {
+    return new CollectionIter(this, skip + startPos);
+  }
+  catch (const std::range_error&)
+  {
+    throw ZORBA_EXCEPTION(
+        zerr::ZXQD0004_INVALID_PARAMETER,
+        ERROR_PARAMS(ZED(ZXQD0004_NOT_WITHIN_RANGE), skip)
+      );
+  }
+}
+
+
+/*******************************************************************************
+  Check if the tree rooted at the given node belongs to this collection. If yes,
+  return true and the position of the tree within the collection. Otherwise, 
+  return false.
+********************************************************************************/
+bool SimpleCollection::findNode(const store::Item* item, xs_integer& position) const
+{
+  if (!(item->isStructuredItem()))
+  {
+    throw ZORBA_EXCEPTION(zerr::ZSTR0013_COLLECTION_ITEM_MUST_BE_STRUCTURED,
+    ERROR_PARAMS(getName()->getStringValue()));
+  }
+
+  const StructuredItem* structuredItem = static_cast<const StructuredItem*>(item);
+  
+  if (structuredItem->isNode())
+  {
+    const XmlNode* node = static_cast<const XmlNode*>(item);
+    if (node->getTree()->getRoot() != node)
+    {
+      throw ZORBA_EXCEPTION(zerr::ZSTR0011_COLLECTION_NON_ROOT_NODE,
+      ERROR_PARAMS(getName()->getStringValue()));
+    }
+  }
+
+  if (theTrees.empty())
+    return false;
+
+  if (item->getCollection() != this)
+    return false;
+
+  position = structuredItem->getPosition();
+
+  csize pos = 0;
+  try
+  {
+    pos = to_xs_unsignedInt(position);
+  } catch (const std::range_error&)
+  {
+    throw ZORBA_EXCEPTION(
+        zerr::ZXQD0004_INVALID_PARAMETER,
+        ERROR_PARAMS(ZED(ZXQD0004_NOT_WITHIN_RANGE), position)
+      );
+  }
+
+  StructuredItem* collectionItem =
+  static_cast<StructuredItem*>(theTrees[pos].getp());
+
+  if (pos < theTrees.size() &&
+      collectionItem->getTreeId() == structuredItem->getTreeId())
+  {
+    return true;
+  }
+
+  csize numTrees = theTrees.size();
+
+  for (csize i = 0; i < numTrees; ++i)
+  {
+    // check if the nodes are the same
+    if (item->equals(theTrees[i]))
+    {
+      ZORBA_ASSERT(theTrees[i]->getCollection() == this);
+      position = i;
+      return true;
+    }
+  }
+
+  return false;
+}
+
+
+/*******************************************************************************
+  Return the node at the given position within the collection, or NULL if the
+  given position is >= than the number of nodes in the collection.
+********************************************************************************/
+store::Item_t SimpleCollection::nodeAt(xs_integer position)
+{
+  try
+  {
+    csize pos = to_xs_unsignedInt(position);
+    if (pos >= theTrees.size())
+    {
+      return NULL;
+    }
+
+    return theTrees[pos];
+  }
+  catch (const std::range_error&)
+  {
+    throw ZORBA_EXCEPTION(
+        zerr::ZXQD0004_INVALID_PARAMETER,
+        ERROR_PARAMS(ZED(ZXQD0004_NOT_WITHIN_RANGE), position)
+      );
+  }
+}
+
+
+/*******************************************************************************
+
+********************************************************************************/
+TreeId SimpleCollection::createTreeId()
+{
+  return theTreeIdGenerator->create();
 }
 
 
 /*******************************************************************************
   Insert the given node to the collection. If the node is in any collection
-  already or if the node has a parent, this method raises an error. Otherwise,
-  the node is inserted into the given position.
+  already or if the node is an xml node with a parent, this method raises an
+  error. Otherwise, the node is inserted into the given position.
 ********************************************************************************/
 void SimpleCollection::addNode(store::Item* item, xs_integer position)
 {
-  XmlNode* node = NULL;
-#ifdef ZORBA_WITH_JSON
-  json::JSONItem* lJSONItem = NULL;
-#endif
+  if (!(item->isStructuredItem()))
+  {
+    throw ZORBA_EXCEPTION(zerr::ZSTR0013_COLLECTION_ITEM_MUST_BE_STRUCTURED,
+    ERROR_PARAMS(getName()->getStringValue()));
+  }
 
-  if (item->isNode())
-  {
-    node = static_cast<XmlNode*>(item);
-    
-    if (node->getParent() != NULL)
-    {
-      throw ZORBA_EXCEPTION(zerr::ZSTR0011_COLLECTION_NON_ROOT_NODE,
-      ERROR_PARAMS(getName()->getStringValue()));
-    }
-  }
-#ifdef ZORBA_WITH_JSON
-  else if (item->isJSONItem())
-  {
-    lJSONItem = static_cast<json::JSONItem*>(item);
-  }
-  else
-  {
-    throw ZORBA_EXCEPTION(zerr::ZSTR0013_COLLECTION_ITEM_MUST_BE_STRUCTURED,
-    ERROR_PARAMS(getName()->getStringValue()));
-  }
-#else
-  else
-  {
-    throw ZORBA_EXCEPTION(zerr::ZSTR0012_COLLECTION_ITEM_MUST_BE_A_NODE,
-    ERROR_PARAMS(getName()->getStringValue()));
-  }
-#endif
   if (item->getCollection() != NULL)
   {
     throw ZORBA_EXCEPTION(zerr::ZSTR0010_COLLECTION_NODE_ALREADY_IN_COLLECTION,
@@ -143,35 +243,54 @@
                  item->getCollection()->getName()->getStringValue()));
   }
 
-  xs_long lPosition = to_xs_long(position);
-  xs_integer pos = xs_integer(0);
-
-  SYNC_CODE(AutoLatch lock(theLatch, Latch::WRITE););
-
-  if (lPosition < 0 || to_xs_unsignedLong(position) >= theXmlTrees.size())
-  {
-    pos = theXmlTrees.size();
-    theXmlTrees.push_back(item);
-  }
-  else
-  {
-    theXmlTrees.insert(theXmlTrees.begin() + lPosition, item);
-  }
-
-#ifdef ZORBA_WITH_JSON
-  if (lJSONItem)
-    lJSONItem->attachToCollection(this, createTreeId());
-  else
-#endif
-    node->setCollection(this, pos);
+  StructuredItem* structuredItem = static_cast<StructuredItem*>(item);
+  
+  if (structuredItem->isNode())
+  {
+    XmlNode* node = static_cast<XmlNode*>(item);
+    if (node->getRoot() != node)
+    {
+      throw ZORBA_EXCEPTION(zerr::ZSTR0011_COLLECTION_NON_ROOT_NODE,
+      ERROR_PARAMS(getName()->getStringValue()));
+    }
+  }
+  
+  try
+  {
+    xs_long pos = to_xs_long(position);
+    SYNC_CODE(AutoLatch lock(theLatch, Latch::WRITE););
+
+    if (pos < 0 || to_xs_unsignedLong(position) >= theTrees.size())
+    {
+      theTrees.push_back(item);
+
+      structuredItem->attachToCollection(this,
+                                         createTreeId(),
+                                         xs_integer(theTrees.size()));
+    }
+    else
+    {
+      theTrees.insert(theTrees.begin() + pos, item);
+
+      structuredItem->attachToCollection(this, createTreeId(), position);
+    }
+  }
+  catch (const std::range_error&)
+  {
+    throw ZORBA_EXCEPTION(
+        zerr::ZXQD0004_INVALID_PARAMETER,
+        ERROR_PARAMS(ZED(ZXQD0004_NOT_WITHIN_RANGE), position)
+      );
+  }
+
 }
 
 
 /*******************************************************************************
   Insert the given nodes to the collection before or after the given target node.
-  If any of the nodes is not a root node or is in any collection already, this
-  method raises an error. The moethod returns the position occupied by the first
-  new node after the insertion is done.
+  If any of the nodes is a non root xml node or is in any collection already,
+  this method raises an error. The moethod returns the position occupied by the
+  first new node after the insertion is done.
 ********************************************************************************/
 xs_integer SimpleCollection::addNodes(
     std::vector<store::Item_t>& items,
@@ -189,87 +308,78 @@
     ERROR_PARAMS(theName->getStringValue()));
   }
 
-  csize targetPos = to_xs_unsignedInt(pos);
+  csize targetPos = 0;
+  try {
+    targetPos = to_xs_unsignedInt(pos);
+  }
+  catch (const std::range_error&)
+  {
+    throw ZORBA_EXCEPTION(
+        zerr::ZXQD0004_INVALID_PARAMETER,
+        ERROR_PARAMS(ZED(ZXQD0004_NOT_WITHIN_RANGE), pos)
+      );
+  }
 
   if (!before)
   {
     ++targetPos;
   }
 
-  csize numNodes = theXmlTrees.size();
+  csize numNodes = theTrees.size();
   csize numNewNodes = items.size();
   
   for (csize i = 0; i < numNewNodes; ++i)
   {
     store::Item* item = items[i].getp();
 
-    XmlNode* node = NULL;
-#ifdef ZORBA_WITH_JSON
-    json::JSONItem* lJSONItem = NULL;
-#endif
-
-    if (item->isNode())
-    {
-      node = static_cast<XmlNode*>(item);
-      
-      if (node->getParent() != NULL)
-      {
-        throw ZORBA_EXCEPTION(zerr::ZSTR0011_COLLECTION_NON_ROOT_NODE,
-        ERROR_PARAMS(getName()->getStringValue()));
-      }
-    }
-#ifdef ZORBA_WITH_JSON
-    else if (item->isJSONItem())
-    {
-      lJSONItem = static_cast<json::JSONItem*>(item);
-    }
-    else
+    if (!(item->isStructuredItem()))
     {
       throw ZORBA_EXCEPTION(zerr::ZSTR0013_COLLECTION_ITEM_MUST_BE_STRUCTURED,
       ERROR_PARAMS(getName()->getStringValue()));
     }
-#else
-    else
-    {
-      throw ZORBA_EXCEPTION(zerr::ZSTR0012_COLLECTION_ITEM_MUST_BE_A_NODE,
-      ERROR_PARAMS(getName()->getStringValue()));
-    }
-#endif
 
     if (item->getCollection() != NULL)
     {
       throw ZORBA_EXCEPTION(zerr::ZSTR0010_COLLECTION_NODE_ALREADY_IN_COLLECTION,
       ERROR_PARAMS(getName()->getStringValue(),
-                   node->getCollection()->getName()->getStringValue()));
+                   item->getCollection()->getName()->getStringValue()));
+    }
+
+    StructuredItem* structuredItem = static_cast<StructuredItem*>(item);
+  
+    if (structuredItem->isNode())
+    {
+      XmlNode* node = static_cast<XmlNode*>(item);
+
+      if (node->getRoot() != node)
+      {
+        throw ZORBA_EXCEPTION(zerr::ZSTR0011_COLLECTION_NON_ROOT_NODE,
+        ERROR_PARAMS(getName()->getStringValue()));
+      }
     }
 
     pos = targetPos + i;
 
-#ifdef ZORBA_WITH_JSON
-    if (lJSONItem)
-      lJSONItem->attachToCollection(this, createTreeId());
-    else
-#endif
-      node->setCollection(this, pos);
+    structuredItem->attachToCollection(this, createTreeId(), pos);
   } // for each new node
 
-  theXmlTrees.resize(numNodes + numNewNodes);
+  theTrees.resize(numNodes + numNewNodes);
 
   if (targetPos < numNodes)
   {
-    memmove(&theXmlTrees[targetPos + numNewNodes], 
-            &theXmlTrees[targetPos],
+    memmove(&theTrees[targetPos + numNewNodes], 
+            &theTrees[targetPos],
             (numNodes-targetPos) * sizeof(store::Item_t));
   }
 
   for (csize i = targetPos; i < targetPos + numNewNodes; ++i)
   {
-    theXmlTrees[i].setNull();
+    theTrees[i].setNull();
   }
 
   for (csize i = 0; i < numNewNodes; ++i)
   {
-    theXmlTrees[targetPos + i].transfer(items[i]);
+    theTrees[targetPos + i].transfer(items[i]);
   }
 
   return xs_integer(targetPos);
@@ -283,32 +393,11 @@
 ********************************************************************************/
 bool SimpleCollection::removeNode(store::Item* item, xs_integer& position)
 {
-  XmlNode* node = NULL;
-#ifdef ZORBA_WITH_JSON
-  json::JSONItem* lJSONItem = NULL;
-#endif
-
-  if (item->isNode())
-  {
-    node = static_cast<XmlNode*>(item);
-  }
-#ifdef ZORBA_WITH_JSON
-  else if (item->isJSONItem())
-  {
-    lJSONItem = static_cast<json::JSONItem*>(item);
-  }
-  else
+  if (!(item->isStructuredItem()))
   {
     throw ZORBA_EXCEPTION(zerr::ZSTR0013_COLLECTION_ITEM_MUST_BE_STRUCTURED,
     ERROR_PARAMS(getName()->getStringValue()));
   }
-#else
-  else
-  {
-    throw ZORBA_EXCEPTION(zerr::ZSTR0012_COLLECTION_ITEM_MUST_BE_A_NODE,
-    ERROR_PARAMS(getName()->getStringValue()));
-  }
-#endif
 
   SYNC_CODE(AutoLatch lock(theLatch, Latch::WRITE);)
 
@@ -318,18 +407,23 @@
   {
     ZORBA_ASSERT(item->getCollection() == this);
 
-    xs_integer const &zero = xs_integer::zero();
-
-#ifdef ZORBA_WITH_JSON
-    if (lJSONItem)
-      lJSONItem->detachFromCollection();
-    else
-#endif
-      node->setCollection(NULL, zero);
-
-    csize pos = to_xs_unsignedInt(position);
-    theXmlTrees.erase(theXmlTrees.begin() + pos);
-    return true;
+    StructuredItem* structuredItem = static_cast<StructuredItem*>(item);
+
+    structuredItem->detachFromCollection();
+    
+    try
+    {
+      csize pos = to_xs_unsignedInt(position);
+      theTrees.erase(theTrees.begin() + pos);
+      return true;
+    }
+    catch (const std::range_error&)
+    {
+      throw ZORBA_EXCEPTION(
+          zerr::ZXQD0004_INVALID_PARAMETER,
+          ERROR_PARAMS(ZED(ZXQD0004_NOT_WITHIN_RANGE), position)
+        );
+    }
   }
   else
   {
@@ -347,38 +441,33 @@
 {
   SYNC_CODE(AutoLatch lock(theLatch, Latch::WRITE);)
 
-  std::size_t pos = to_xs_unsignedInt(position);
+  std::size_t pos = 0;
+  try {
+    pos = to_xs_unsignedInt(position);
+  }
+  catch (const std::range_error&)
+  {
+    throw ZORBA_EXCEPTION(
+        zerr::ZXQD0004_INVALID_PARAMETER,
+        ERROR_PARAMS(ZED(ZXQD0004_NOT_WITHIN_RANGE), position)
+      );
+  }
 
-  if (pos >= theXmlTrees.size()) 
+  if (pos >= theTrees.size())
   {
     return false;
   }
   else
   {
-    store::Item* item = theXmlTrees[pos].getp();
+    store::Item* item = theTrees[pos].getp();
 
     ZORBA_ASSERT(item->getCollection() == this);
-
-    xs_integer const &zero = xs_integer::zero();
-
-    if (item->isNode())
-    {
-      XmlNode* node = static_cast<XmlNode*>(item);
-      node->setCollection(NULL, zero);
-    }
-#ifdef ZORBA_WITH_JSON
-    else if (item->isJSONItem())
-    {
-      json::JSONItem* lJSONItem = static_cast<json::JSONItem*>(item);
-      lJSONItem->detachFromCollection();
-    }
-#endif
-    else
-    {
-      ZORBA_ASSERT(false);
-    }
-
-    theXmlTrees.erase(theXmlTrees.begin() + pos);
+    ZORBA_ASSERT(item->isStructuredItem());
+
+    StructuredItem* structuredItem = static_cast<StructuredItem*>(item);
+    structuredItem->detachFromCollection();
+
+    theTrees.erase(theTrees.begin() + pos);
     return true;
   }
 }
@@ -387,54 +476,61 @@
 /*******************************************************************************
   Remove a given number of trees starting with the tree at the given position.
   If the given number is 0 or the given position is >= than the number of trees
-  in the collection, this mothod is a noop. The method returns the number of 
+  in the collection, this method is a noop. The method returns the number of 
   trees that are actually deleted.
 ********************************************************************************/
 xs_integer SimpleCollection::removeNodes(xs_integer position, xs_integer numNodes)
 {
   SYNC_CODE(AutoLatch lock(theLatch, Latch::WRITE);)
 
-  csize pos = to_xs_unsignedInt(position);
-  csize num = to_xs_unsignedInt(numNodes);
+  csize pos, num;
+  try
+  {
+    pos = to_xs_unsignedInt(position);
+  }
+  catch (const std::range_error&)
+  {
+    throw ZORBA_EXCEPTION(
+        zerr::ZXQD0004_INVALID_PARAMETER,
+        ERROR_PARAMS(ZED(ZXQD0004_NOT_WITHIN_RANGE), position)
+      );
+  }
+  try
+  {
+    num = to_xs_unsignedInt(numNodes);
+  }
+  catch (const std::range_error&)
+  {
+    throw ZORBA_EXCEPTION(
+        zerr::ZXQD0004_INVALID_PARAMETER,
+        ERROR_PARAMS(ZED(ZXQD0004_NOT_WITHIN_RANGE), numNodes)
+      );
+  }
 
-  if (num == 0 || pos >= theXmlTrees.size())
+  if (num == 0 || pos >= theTrees.size())
   {
     return xs_integer::zero();
   }
   else
   {
-    uint64_t last = pos + num;
-    if (last > theXmlTrees.size())
+    csize last = pos + num;
+
+    if (last > theTrees.size())
     {
-      last = theXmlTrees.size();
+      last = theTrees.size();
     }
 
-    xs_integer const &zero = xs_integer::zero();
-
     for (csize i = pos; i < last; ++i)
-    { 
-      store::Item* item = theXmlTrees[pos].getp();
+    {
+      store::Item* item = theTrees[pos].getp();
 
       ZORBA_ASSERT(item->getCollection() == this);
-
-      if (item->isNode())
-      {
-        XmlNode* node = static_cast<XmlNode*>(item);
-        node->setCollection(NULL, zero);
-      }
-#ifdef ZORBA_WITH_JSON
-      else if (item->isJSONItem())
-      {
-        json::JSONItem* lJSONItem = static_cast<json::JSONItem*>(item);
-        lJSONItem->detachFromCollection();
-      }
-#endif
-      else
-      {
-        ZORBA_ASSERT(false);
-      }
-
-      theXmlTrees.erase(theXmlTrees.begin() + pos);
+      ZORBA_ASSERT(item->isStructuredItem());
+
+      StructuredItem* structuredItem = static_cast<StructuredItem*>(item);
+      structuredItem->detachFromCollection();
+
+      theTrees.erase(theTrees.begin() + pos);
     }
 
     return xs_integer(last - pos);
@@ -443,115 +539,26 @@
 
 
 /*******************************************************************************
- * Remove all the nodes from the collection
+  Remove all the nodes from the collection
 ********************************************************************************/
 void SimpleCollection::removeAll()
 {
   SYNC_CODE(AutoLatch lock(theLatch, Latch::WRITE);)
 
-  theXmlTrees.clear();
-}
-
-
-/*******************************************************************************
-  Return the node at the given position within the collection, or NULL if the
-  given position is >= than the number of nodes in the collection.
-********************************************************************************/
-store::Item_t SimpleCollection::nodeAt(xs_integer position)
-{
-  csize pos = to_xs_unsignedInt(position);
-  if (pos >= theXmlTrees.size())
-  {
-    return NULL;
-  }
-
-  return theXmlTrees[pos];
-}
-
-
-/*******************************************************************************
-  Check if the tree rooted at the given node belongs to this collection. If yes,
-  return true and the position of the tree within the collection. Otherwise, 
-  return false.
-********************************************************************************/
-bool SimpleCollection::findNode(const store::Item* item, xs_integer& position) const
-{
-  const XmlNode* node = NULL;
-
-  if (item->isNode())
-  {
-    node = static_cast<const XmlNode*>(item);
-    
-    if (node->getParent() != NULL)
-    {
-      throw ZORBA_EXCEPTION(zerr::ZSTR0011_COLLECTION_NON_ROOT_NODE,
-      ERROR_PARAMS(getName()->getStringValue()));
-    }
-  }
-#ifdef ZORBA_WITH_JSON
-  else if (item->isJSONObject())
-  {
-  }
-  else if (item->isJSONArray())
-  {
-  }
-  else
-  {
-    throw ZORBA_EXCEPTION(zerr::ZSTR0013_COLLECTION_ITEM_MUST_BE_STRUCTURED,
-    ERROR_PARAMS(getName()->getStringValue()));
-  }
-#else
-  else
-  {
-    throw ZORBA_EXCEPTION(zerr::ZSTR0012_COLLECTION_ITEM_MUST_BE_A_NODE,
-    ERROR_PARAMS(getName()->getStringValue()));
-  }
-#endif
-
-  if (theXmlTrees.empty())
-    return false;
-
-  if (item->getCollection() != this)
-    return false;
-
-  if (node)
-  {
-    position = node->getTree()->getPosition();
-
-    csize pos = to_xs_unsignedInt(position);
-
-    if (pos < theXmlTrees.size() &&
-        theXmlTrees[pos]->isNode() &&
-        BASE_NODE(theXmlTrees[pos])->getTreeId() == node->getTreeId())
-    {
-      return true;
-    }
-  }
-
-  csize numTrees = theXmlTrees.size();
+  csize numTrees = theTrees.size();
 
   for (csize i = 0; i < numTrees; ++i)
   {
-    // check if the nodes are the same
-    if (item->equals(theXmlTrees[i]))
-    {
-      ZORBA_ASSERT(theXmlTrees[i]->getCollection() == this);
-      position = i;
-      return true;
-    }
+    store::Item* item = theTrees[i].getp();
+
+    ZORBA_ASSERT(item->getCollection() == this);
+    ZORBA_ASSERT(item->isStructuredItem());
+
+    StructuredItem* structuredItem = static_cast<StructuredItem*>(item);
+    structuredItem->detachFromCollection();
   }
 
-  return false;
-}
-
-
-/*******************************************************************************
-
-********************************************************************************/
-void SimpleCollection::getAnnotations(
-    std::vector<store::Annotation_t>& annotations) const
-{
-  annotations = theAnnotations;
+  theTrees.clear();
 }
 
 
@@ -560,16 +567,11 @@
 ********************************************************************************/
 void SimpleCollection::adjustTreePositions()
 {
-  csize numTrees = theXmlTrees.size();
+  csize numTrees = theTrees.size();
 
   for (csize i = 0; i < numTrees; ++i)
   {
-#ifdef ZORBA_WITH_JSON
-    if (theXmlTrees[i]->isNode())
-      BASE_NODE(theXmlTrees[i])->getTree()->setPosition(xs_integer(i));
-#else
-    BASE_NODE(theXmlTrees[i])->getTree()->setPosition(xs_integer(i));
-#endif
+    static_cast<StructuredItem*>(theTrees[i].getp())->setPosition(xs_integer(i));
   }
 }
 
@@ -577,22 +579,13 @@
 /*******************************************************************************
 
 ********************************************************************************/
-TreeId SimpleCollection::createTreeId()
-{
-  return theTreeIdGenerator->create();
-}
-
-
-/*******************************************************************************
-
-********************************************************************************/
 SimpleCollection::CollectionIter::CollectionIter(
     SimpleCollection* collection,
-    const xs_integer& aSkip)
+    const xs_integer& skip)
   :
   theCollection(collection),
   theHaveLock(false),
-  theSkip(aSkip)
+  theSkip(to_xs_unsignedLong(skip))
 {
 }
 
@@ -607,6 +600,9 @@
 }
 
 
+/*******************************************************************************
+
+********************************************************************************/
 void SimpleCollection::CollectionIter::skip()
 {
   // skip by position
@@ -617,7 +613,7 @@
   }
   else
   {
-    theIterator += to_xs_long(theSkip);
+    theIterator += theSkip;
   }
 }
 
@@ -630,8 +626,8 @@
   SYNC_CODE(theCollection->theLatch.rlock();)
   theHaveLock = true;
 
-  theIterator = theCollection->theXmlTrees.begin();
-  theEnd = theCollection->theXmlTrees.end();
+  theIterator = theCollection->theTrees.begin();
+  theEnd = theCollection->theTrees.end();
   skip();
 }
 
@@ -665,8 +661,8 @@
 ********************************************************************************/
 void SimpleCollection::CollectionIter::reset()
 {
-  theIterator = theCollection->theXmlTrees.begin();
-  theEnd = theCollection->theXmlTrees.end();
+  theIterator = theCollection->theTrees.begin();
+  theEnd = theCollection->theTrees.end();
   skip();
 }
 

=== modified file 'src/store/naive/simple_collection.h'
--- src/store/naive/simple_collection.h	2013-01-30 01:13:09 +0000
+++ src/store/naive/simple_collection.h	2013-02-21 15:32:25 +0000
@@ -33,13 +33,42 @@
 
 
 /*******************************************************************************
-  theId          : An internally generated unique id.
-  theName        : The user provided qname of the collection.
-  theXmlTrees    : The set of root nodes comprising this collection. Implemented
-                   as vector of rchandles to nodes.
-  theTreeCounter : Incremented every time a new tree is added to the collection.
-                   The current value of the counter is then assigned as the id
-                   the new tree.
+  theId:
+  ------
+  An internally generated unique id for the collection.
+
+  theName:
+  --------
+  The user provided qname of the collection (inherited from simplestore::Collection).
+
+  theIsDynamic:
+  -------------
+  Whether the collection is dynamic or not. Static (ie, non-dynamic) collections
+  must be declared in the static context to be usable, whereas dynamic ones do
+  not have any declarations. As far as the store is concerned, this property 
+  is used only for naming purposes: a static and a dynamic collection may have
+  the same name, so the isDynamic property is used to resolve such name conflicts.
+
+  theTrees:
+  ------------
+  The root nodes of the XML and/or JSON trees that comprise this collection.
+
+  theTreeIdGenerator:
+  -------------------
+  A generator of tree ids for the trees that belong to this collection. The
+  generated tree ids are unique within the collection. Every time a tree is
+  added to the collection it is assigned a new tree id that is generated by 
+  this generator.
+
+  theAnnotations:
+  ---------------
+  The properties of the collection. For static collections, the values of these
+  properties are specified by the user in the collection declaration. Dynamic
+  collections use pre-determined default values.
+
+  theLatch:
+  ---------
+  Synchronizes concurrent accesses to the collection.
 ********************************************************************************/
 class SimpleCollection : public Collection
 {
@@ -54,7 +83,7 @@
     checked_vector<store::Item_t>::iterator theIterator;
     checked_vector<store::Item_t>::iterator theEnd;
     bool                                    theHaveLock;
-    xs_integer                              theSkip;
+    csize                                   theSkip;
 
   public:
     CollectionIter(SimpleCollection* collection, const xs_integer& skip);
@@ -65,6 +94,7 @@
     bool next(store::Item_t& result);
     void reset();
     void close();
+
   private:
     void skip();
   };
@@ -72,8 +102,9 @@
 
 protected:
   ulong                                  theId;
-  store::Item_t                          theName;
-  checked_vector<store::Item_t>          theXmlTrees;
+
+  checked_vector<store::Item_t>          theTrees;
+
   bool                                   theIsDynamic;
 
   TreeIdGenerator                      * theTreeIdGenerator;
@@ -82,36 +113,44 @@
 
   SYNC_CODE(Latch                        theLatch;)
 
+protected:
   // default constructor added in order to allow subclasses to instantiate
   // a collection without name
   SimpleCollection();
 
 public:
   SimpleCollection(
-      const store::Item_t& aName,
+      const store::Item_t& name,
       const std::vector<store::Annotation_t>& annotations,
       bool isDynamic = false);
 
   virtual ~SimpleCollection();
   
-  /********************** All these methods implement the **********************
-  ***************** zorba::simplestore::Collection interface ******************/
-
-  ulong getId() const { return theId; }
+  //
+  // Store API methods
+  //
 
   const store::Item* getName() const { return theName.getp(); }
 
-  xs_integer size() const { return xs_integer( theXmlTrees.size() ); }
+  xs_integer size() const { return xs_integer( theTrees.size() ); }
 
   bool isDynamic() const { return theIsDynamic; }
 
   void getAnnotations(std::vector<store::Annotation_t>& annotations) const;
 
+  store::Iterator_t getIterator(const xs_integer& skip, const zstring& start);
+
+  bool findNode(const store::Item* node, xs_integer& position) const;
+
+  store::Item_t nodeAt(xs_integer position);
+
+  //
+  // simplestore methods
+  //
+  ulong getId() const { return theId; }
+
   TreeId createTreeId();
 
-  store::Iterator_t getIterator(const xs_integer& aSkip, 
-                                const zstring& aStart);
-
   void addNode(store::Item* node, xs_integer position = xs_integer(-1));
 
   xs_integer addNodes(
@@ -123,14 +162,10 @@
 
   bool removeNode(xs_integer position);
 
+  xs_integer removeNodes(xs_integer position, xs_integer num);
+
   void removeAll();
 
-  xs_integer removeNodes(xs_integer position, xs_integer num);
-
-  bool findNode(const store::Item* node, xs_integer& position) const;
-
-  store::Item_t nodeAt(xs_integer position);
-
   void adjustTreePositions();
 };
 

=== modified file 'src/store/naive/simple_collection_set.h'
--- src/store/naive/simple_collection_set.h	2013-01-22 18:43:03 +0000
+++ src/store/naive/simple_collection_set.h	2013-02-21 15:32:25 +0000
@@ -45,23 +45,18 @@
   virtual
   ~SimpleCollectionSet();
     
-  virtual void
-  clear();
-    
-  virtual bool
-  insert(const store::Item* aName, store::Collection_t& aCollection);
-    
-  virtual bool
-  get(
-      const zorba::store::Item* aName,
-      store::Collection_t& aCollection,
+  virtual void clear();
+    
+  virtual bool insert(const store::Item* name, store::Collection_t& collection);
+    
+  virtual bool get(
+      const zorba::store::Item* name,
+      store::Collection_t& collection,
       bool isDynamic);
   
-  virtual bool
-  remove(const store::Item* name, bool isDynamic);
+  virtual bool remove(const store::Item* name, bool isDynamic);
     
-  virtual zorba::store::Iterator_t
-  names(bool isDynamic);
+  virtual zorba::store::Iterator_t names(bool isDynamic);
 };
 
 

=== modified file 'src/store/naive/simple_item_factory.cpp'
--- src/store/naive/simple_item_factory.cpp	2013-02-12 04:20:23 +0000
+++ src/store/naive/simple_item_factory.cpp	2013-02-21 15:32:25 +0000
@@ -2159,7 +2159,7 @@
 
     while (source->next(item))
     {
-      if (copymode.theDoCopy && (item->isNode() || item->isJSONItem()))
+      if (copymode.theDoCopy && (item->isStructuredItem()))
         item = item->copy(NULL, copymode);
       
       array->push_back(item);

=== modified file 'src/store/naive/simple_pul.cpp'
--- src/store/naive/simple_pul.cpp	2013-01-17 04:43:05 +0000
+++ src/store/naive/simple_pul.cpp	2013-02-21 15:32:25 +0000
@@ -170,32 +170,20 @@
 
 #ifdef ZORBA_WITH_JSON
   assert(target->isNode()
-      || target->isJSONObject()
-      || target->isJSONArray());
+      || target->isJSONItem());
 #else
   assert(target->isNode());
 #endif
 
-  const store::Collection* lCollection;
-  
-  if (target->isNode())
-  {
-    assert(dynamic_cast<const XmlNode*>(target));
-    const XmlNode* lNode = static_cast<const XmlNode*>(target);
-    lCollection = lNode->getCollection();
-#ifdef ZORBA_WITH_JSON
-  }
-  else if (target->isJSONItem())
-  {
-    assert(dynamic_cast<const json::JSONItem*>(target));
-    const json::JSONItem* lJSONItem = static_cast<const json::JSONItem*>(target);
-    lCollection = lJSONItem->getCollection();
-#endif
-  }
+  assert(dynamic_cast<const StructuredItem*>(target));
+  const StructuredItem* lStructuredItem =
+      static_cast<const StructuredItem*>(target);
+  const store::Collection* lCollection = lStructuredItem->getCollection();
 
   if (lCollection != NULL)
   {
-    collName = static_cast<const QNameItem*>(lCollection->getName())->getNormalized();
+    collName = static_cast<const QNameItem*>(
+        lCollection->getName())->getNormalized();
 
     if (collName == theLastCollection)
       return theLastPul;
@@ -2298,45 +2286,21 @@
     for (; it != end; ++it)
     {
       zorba::store::Item* lItem = (*it).first;
-      if (lItem->isNode())
-      {
-        assert(dynamic_cast<const XmlNode*>(lItem));
-        const XmlNode* lNode = static_cast<const XmlNode*>(lItem);
-        for (csize i = 0; i < numRoots; i++)
-        {
-          if (rootNodes[i]->isNode())
-          {
-            assert(dynamic_cast<const XmlNode*>(rootNodes[i]));
-            XmlNode* lRootNode = static_cast<XmlNode*>(rootNodes[i]);
-            
-            if (lNode->getTree() == lRootNode->getTree())
-            {
-              found = true;
-              break;
-            }
-          }
-        }
-#ifdef ZORBA_WITH_JSON
-      }
-      else if (lItem->isJSONItem())
-      {
-        assert(dynamic_cast<const json::JSONItem*>(lItem));
-        const json::JSONItem* lJSONItem = static_cast<const json::JSONItem*>(lItem);
-        for (csize i = 0; i < numRoots; i++)
-        {
-          if (rootNodes[i]->isJSONItem())
-          {
-            assert(dynamic_cast<const json::JSONItem*>(rootNodes[i]));
-            json::JSONItem* lRootJSONItem = static_cast<json::JSONItem*>(rootNodes[i]);
-            
-            if (lJSONItem->getTree() == lRootJSONItem->getTree())
-            {
-              found = true;
-              break;
-            }
-          }
-        }
-#endif
+      assert(dynamic_cast<const StructuredItem*>(lItem));
+      const StructuredItem* lStructuredItem =
+          static_cast<const StructuredItem*>(lItem);
+
+      for (csize i = 0; i < numRoots; i++)
+      {
+        assert(dynamic_cast<const StructuredItem*>(rootNodes[i]));
+        StructuredItem* lRootStructuredItem =
+            static_cast<StructuredItem*>(rootNodes[i]);
+        
+        if (lRootStructuredItem->isInSubtree(lStructuredItem))
+        {
+          found = true;
+          break;
+        }
       }
         
       if (!found)
@@ -2398,31 +2362,30 @@
     NodeToUpdatesMap::iterator end = pul->theNodeToUpdatesMap.end();
     for (; ite != end; ++ite)
     {
-      store::Item* lItem = (*ite).first;
-#ifdef ZORBA_WITH_JSON
-      ZORBA_ASSERT(lItem->isNode() || lItem->isJSONItem());
+      store::Item* item = (*ite).first;
+      ZORBA_ASSERT(item->isStructuredItem());
 
-      if (lItem->isJSONItem())
-      {
-        json::JSONItem* lJSONItem = dynamic_cast<json::JSONItem*>(lItem);
-        ZORBA_ASSERT(lJSONItem != NULL);
-        pul->theModifiedDocs.insert(const_cast<json::JSONItem*>(lJSONItem->getRoot()));
-        continue;
-      }
-#endif
-      ZORBA_ASSERT(lItem->isNode());
-      XmlNode* node = dynamic_cast<XmlNode*>((*ite).first);
-      ZORBA_ASSERT(node != NULL);
-      pul->theModifiedDocs.insert(node->getRoot());
+      StructuredItem* structuredItem = static_cast<StructuredItem*>(item);
+      pul->theModifiedDocs.insert(structuredItem->getCollectionRoot());
       continue;
     }
 
-    csize numCollUpdates = pul->theInsertIntoCollectionList.size();
-
-    for (csize i = 0; i < numCollUpdates; ++i)
-    {
-      UpdCollection* upd = static_cast<UpdCollection*>
-                           (pul->theInsertIntoCollectionList[i]);
+    csize numCollUpdates = pul->theEditInCollectionList.size();
+
+    for (csize i = 0; i < numCollUpdates; ++i)
+    {
+      UpdEditInCollection* upd = 
+      static_cast<UpdEditInCollection*>(pul->theEditInCollectionList[i]);
+
+      pul->theModifiedDocs.insert(upd->getTarget());
+    }
+
+    numCollUpdates = pul->theInsertIntoCollectionList.size();
+
+    for (csize i = 0; i < numCollUpdates; ++i)
+    {
+      UpdCollection* upd = 
+      static_cast<UpdCollection*>(pul->theInsertIntoCollectionList[i]);
 
       csize numDocs = upd->numNodes();
 
@@ -2430,22 +2393,12 @@
         pul->theInsertedDocs.push_back(upd->getNode(j));
     }
 
-    numCollUpdates = pul->theEditInCollectionList.size();
-
-    for (csize i = 0; i < numCollUpdates; ++i)
-    {
-      UpdEditInCollection* upd = static_cast<UpdEditInCollection*>
-                           (pul->theEditInCollectionList[i]);
-
-      pul->theModifiedDocs.insert(upd->getTarget());
-    }
-
     numCollUpdates = pul->theDeleteFromCollectionList.size();
 
     for (csize i = 0; i < numCollUpdates; ++i)
     {
-      UpdCollection* upd = static_cast<UpdCollection*>
-                           (pul->theDeleteFromCollectionList[i]);
+      UpdCollection* upd =
+      static_cast<UpdCollection*>(pul->theDeleteFromCollectionList[i]);
 
       csize numDocs = upd->numNodes();
 

=== modified file 'src/store/naive/simple_store.cpp'
--- src/store/naive/simple_store.cpp	2013-01-17 04:43:05 +0000
+++ src/store/naive/simple_store.cpp	2013-02-21 15:32:25 +0000
@@ -266,10 +266,10 @@
 
 
 /*******************************************************************************
- Computes the reference of the given node.
+  Computes the reference of the given node.
 
- @param node XDM node
- @return the identifier as an item of type xs:anyURI
+  @param node XDM node
+  @return the identifier as an item of type xs:anyURI
 ********************************************************************************/
 bool SimpleStore::getNodeReference(store::Item_t& result, const store::Item* node)
 {
@@ -308,11 +308,11 @@
 {
   using namespace zorba::simplestore::json;
 
-  bool lHasReference = false;
+  bool hasReference = false;
   if (node->isNode())
   {
     const XmlNode* x = static_cast<const XmlNode*>(node);
-    lHasReference = x->haveReference();
+    hasReference = x->haveReference();
   }
   else
   {
@@ -320,17 +320,21 @@
     JSONItem* j = const_cast<JSONItem*>(static_cast<const JSONItem*>(node));
 
     // only root nodes in a collection can have a reference
-    if (j->getTree() && j->getTree()->getRoot() == j)
+    if (j->isCollectionRoot())
     {
-      ItemRefMap::iterator lIter = theNodeToReferencesMap.find(node);
+      ItemRefMap::iterator iter = theNodeToReferencesMap.find(node);
 
-      lHasReference = (lIter != theNodeToReferencesMap.end());
+      hasReference = (iter != theNodeToReferencesMap.end());
     }
   }
-  return lHasReference;
+
+  return hasReference;
 }
 
 
+/*******************************************************************************
+
+********************************************************************************/
 bool SimpleStore::assignReference(const store::Item* node, const zstring& reference)
 {
   using namespace zorba::simplestore::json;
@@ -352,7 +356,7 @@
     assert(node->isJSONItem());
     const JSONItem* j = static_cast<const JSONItem*>(node);
 
-    if (!j->getTree() || j != j->getTree()->getRoot())
+    if (! j->isCollectionRoot())
       throw ZORBA_EXCEPTION(zerr::ZAPI0080_CANNOT_RETRIEVE_REFERENCE);
   }
 

=== modified file 'src/store/naive/store.cpp'
--- src/store/naive/store.cpp	2012-11-21 03:17:34 +0000
+++ src/store/naive/store.cpp	2013-02-21 15:32:25 +0000
@@ -608,7 +608,7 @@
       bool more = true;
 
 #ifdef ZORBA_WITH_JSON
-      assert(domainNode->isNode() || domainNode->isJSONItem());
+      assert(domainNode->isStructuredItem());
 #else
       assert(domainNode->isNode());
 #endif
@@ -631,7 +631,7 @@
         // If current node has no keys, put it in the "null" entry and continue
         // with the next domain node, if nay.
 #ifdef ZORBA_WITH_JSON
-        if (!more || firstKeyItem->isNode() || firstKeyItem->isJSONItem())
+        if (!more || firstKeyItem->isStructuredItem())
 #else
         if (!more || firstKeyItem->isNode())
 #endif
@@ -648,7 +648,7 @@
         // If current domain node has exactly 1 key, insert it in the index
         // and continue with next domain node, if any.
 #ifdef ZORBA_WITH_JSON
-        if (!more || keyItem->isNode() || keyItem->isJSONItem())
+        if (!more || keyItem->isStructuredItem())
 #else
         if (!more || keyItem->isNode())
 #endif
@@ -673,7 +673,7 @@
         while ((more = sourceIter->next(keyItem)))
         {
 #ifdef ZORBA_WITH_JSON
-          if (keyItem->isNode() || keyItem->isJSONItem())
+          if (keyItem->isStructuredItem())
 #else
           if (keyItem->isNode())
 #endif
@@ -1211,11 +1211,12 @@
 #ifdef TEXT_ORDPATH
   const OrdPathNode* n = static_cast<const OrdPathNode*>(node);
 
-  return theItemFactory->createStructuralAnyURI(result,
-                                                n->getCollectionId(),
-                                                n->getTreeId(),
-                                                n->getNodeKind(),
-                                                n->getOrdPath());
+  return theItemFactory->createStructuralAnyURI(
+      result,
+      n->getCollectionId(),
+      n->getTree()->getTreeId(),
+      n->getNodeKind(),
+      n->getOrdPath());
 #else
   if (node->getNodeKind() == store::StoreConsts::textNode)
   {
@@ -1223,21 +1224,23 @@
     const TextNode* n = static_cast<const TextNode*>(node);
     n->getOrdPath(ordPath);
 
-    return GET_FACTORY().createStructuralAnyURI(result,
-                                                n->getCollectionId(),
-                                                n->getTreeId(),
-                                                store::StoreConsts::textNode,
-                                                ordPath);
+    return GET_FACTORY().createStructuralAnyURI(
+        result,
+        n->getCollectionId(),
+        n->getTree()->getTreeId(),
+        store::StoreConsts::textNode,
+        ordPath);
   }
   else
   {
     const OrdPathNode* n = static_cast<const OrdPathNode*>(node);
 
-    return GET_FACTORY().createStructuralAnyURI(result,
-                                                n->getCollectionId(),
-                                                n->getTreeId(),
-                                                n->getNodeKind(),
-                                                n->getOrdPath());
+    return GET_FACTORY().createStructuralAnyURI(
+        result,
+        n->getCollectionId(),
+        n->getTree()->getTreeId(),
+        n->getNodeKind(),
+        n->getOrdPath());
   }
 #endif
 }

=== modified file 'src/store/naive/store_defs.h'
--- src/store/naive/store_defs.h	2012-09-19 21:16:15 +0000
+++ src/store/naive/store_defs.h	2013-02-21 15:32:25 +0000
@@ -38,22 +38,24 @@
 
 #define GET_PUL_FACTORY() \
   (GET_STORE().getPULFactory())
-  
-#define BASE_NODE(item) (reinterpret_cast<XmlNode*>((item).getp()))
-
-#define INTERNAL_NODE(item) (reinterpret_cast<InternalNode*>((item).getp()))
-
-#define DOC_NODE(item) (reinterpret_cast<DocumentNode*>((item).getp()))
-
-#define ATTR_NODE(item) (reinterpret_cast<AttributeNode*>((item).getp()))
-
-#define ELEM_NODE(item) (reinterpret_cast<ElementNode*>((item).getp()))
-
-#define TEXT_NODE(item) (reinterpret_cast<TextNode*>((item).getp()))
-
-#define PI_NODE(item) (reinterpret_cast<PiNode*>((item).getp()))
-
-#define COMMENT_NODE(item) (reinterpret_cast<CommentNode*>((item).getp()))
+
+#define STRUCT_NODE(item) (static_cast<StructuredItem*>((item).getp()))
+
+#define BASE_NODE(item) (static_cast<XmlNode*>((item).getp()))
+
+#define INTERNAL_NODE(item) (static_cast<InternalNode*>((item).getp()))
+
+#define DOC_NODE(item) (static_cast<DocumentNode*>((item).getp()))
+
+#define ATTR_NODE(item) (static_cast<AttributeNode*>((item).getp()))
+
+#define ELEM_NODE(item) (static_cast<ElementNode*>((item).getp()))
+
+#define TEXT_NODE(item) (static_cast<TextNode*>((item).getp()))
+
+#define PI_NODE(item) (static_cast<PiNode*>((item).getp()))
+
+#define COMMENT_NODE(item) (static_cast<CommentNode*>((item).getp()))
 
 
 #ifndef NDEBUG

=== added file 'src/store/naive/structured_item.cpp'
--- src/store/naive/structured_item.cpp	1970-01-01 00:00:00 +0000
+++ src/store/naive/structured_item.cpp	2013-02-21 15:32:25 +0000
@@ -0,0 +1,144 @@
+/*
+ * Copyright 2006-2013 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "structured_item.h"
+#include "collection_tree_info.h"
+#include "collection.h"
+#include "node_items.h"
+#include "json_items.h"
+
+
+namespace zorba
+{
+namespace simplestore
+{
+
+
+/*******************************************************************************
+  Get the id of the tree this node belongs to
+********************************************************************************/
+const TreeId& StructuredItem::getTreeId() const
+{
+  if (isNode())
+  {
+    const XmlNode* node = static_cast<const XmlNode*>(this);
+    return node->getTree()->getTreeId();
+  }
+  else
+  {
+    assert(isJSONItem());
+
+    const json::JSONItem* node = static_cast<const json::JSONItem*>(this);
+
+    CollectionTreeInfoWithTreeId* info = 
+    static_cast<CollectionTreeInfoWithTreeId*>(node->getCollectionTreeInfo());
+
+    ZORBA_ASSERT(info);
+
+    return info->getTreeId();
+  }
+}
+
+
+/*******************************************************************************
+
+********************************************************************************/
+CollectionTreeInfo* StructuredItem::getCollectionTreeInfo() const
+{
+  if (isNode())
+  {
+    const XmlNode* node = static_cast<const XmlNode*>(this);
+    return node->getTree()->getCollectionTreeInfo();
+  }
+  else
+  {
+    assert(isJSONItem());
+    const json::JSONItem* node = static_cast<const json::JSONItem*>(this);
+    return node->getCollectionTreeInfo();
+  }
+}
+
+
+/*******************************************************************************
+  Let T be the tree containing this node. If T belongs to a collection directly,
+  return the root node of T. If T belongs to a collection indirectly, return the
+  root node of the (JSON) tree that points to T and belongs to the collection
+  directly. If T does not belong to any collection, return NULL.
+********************************************************************************/
+StructuredItem* StructuredItem::getCollectionRoot() const
+{
+  CollectionTreeInfo* info = getCollectionTreeInfo();
+  return info ? info->getRoot() : NULL;
+}
+
+
+/*******************************************************************************
+  Assuming the tree T this node belongs to is in a collection, return the
+  position of T within its containing collection. 
+********************************************************************************/
+const xs_integer& StructuredItem::getPosition() const
+{
+  CollectionTreeInfo* info = getCollectionTreeInfo();
+  ZORBA_ASSERT(info);
+  return info->getPosition();
+}
+
+
+/*******************************************************************************
+  Assuming the tree T this node belongs to is in a collection, store the
+  position of T within its containing collection. The position is stored
+  inside the CollectionTreeInfo object associated with the node.
+********************************************************************************/
+void StructuredItem::setPosition(const xs_integer& pos)
+{
+  CollectionTreeInfo* info = getCollectionTreeInfo();
+  ZORBA_ASSERT(info);
+  info->setPosition(pos);
+}
+
+
+/*******************************************************************************
+  Return true if this node is the root node of a tree that belongs to a
+  a collection directly.
+********************************************************************************/
+bool StructuredItem::isCollectionRoot() const
+{
+  return getCollectionRoot() == this;
+}
+
+
+/*******************************************************************************
+  Return true if this node is in the subtree starting at the given item.
+
+  NOTE: for the purposes of this method, XML trees that are pointed-to by a
+  JSON tree are considered part of that JSON tree. As a result, an XML node
+  may be in the subtree of a JSON node.
+********************************************************************************/
+bool StructuredItem::isInSubtreeOf(const store::Item_t& anItem) const
+{
+  if (!anItem->isStructuredItem())
+  {
+    return false;
+  }
+
+  StructuredItem* structuredItem = static_cast<StructuredItem*>(anItem.getp());
+
+  return structuredItem->isInSubtree(this);
+};
+
+
+} // simplestore
+} // zorba

=== added file 'src/store/naive/structured_item.h'
--- src/store/naive/structured_item.h	1970-01-01 00:00:00 +0000
+++ src/store/naive/structured_item.h	2013-02-21 15:32:25 +0000
@@ -0,0 +1,93 @@
+/*
+ * Copyright 2006-2012 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef ZORBA_SIMPLE_STORE_STRUCTURED_ITEMS
+#define ZORBA_SIMPLE_STORE_STRUCTURED_ITEMS
+
+#include "store/api/item.h"
+#include "tree_id.h"
+
+namespace zorba
+{
+namespace simplestore
+{
+
+class Collection;
+class CollectionTreeInfo;
+
+
+/*******************************************************************************
+  StructuredItem represents either an XML node or a JSON node.
+
+  An important goal of the StructureItem class is to provide methods that
+  manage the interactions between trees of nodes and collections. These methods
+  operate on instances of the CollectionTreeInfo class. See comments in 
+  collection_tree_info.h for more details.
+********************************************************************************/
+class StructuredItem : public store::Item
+{
+public:
+  StructuredItem(ItemKind k) : Item(k) {}
+
+  StructuredItem() : Item() {}
+
+  //--------------------- Item API ---------------------------------------------
+  
+  bool isInSubtreeOf(const store::Item_t& anItem) const;
+  
+  bool isCollectionRoot() const;
+
+  //--------------------- Structured Item API ----------------------------------
+
+  const TreeId& getTreeId() const;
+
+  StructuredItem* getCollectionRoot() const;
+
+  const xs_integer& getPosition() const;
+
+  void setPosition(const xs_integer& pos);
+
+  // Attaches a root node to a collection (populates collection info).
+  virtual void attachToCollection(
+      Collection* collection,
+      const TreeId& treeId,
+      const xs_integer& position) = 0;
+
+  // Detaches a root node from a collection (populates collection info).
+  virtual void detachFromCollection() = 0;
+  
+  // Propagates collection tree info to descendants
+  // (not to be called on a root - use functions above if root).
+  virtual void setCollectionTreeInfo(CollectionTreeInfo* collectionInfo) = 0;
+
+  // Returns total number of outstanding pointers to the tree (for garbage
+  // collection purposes).
+  virtual long getCollectionTreeRefCount() const = 0;
+
+  // Tells if the given item is in the subtree starting at this item.
+  // NOTE: for the purposes of this method, XML trees that are pointed-to
+  // by a JSON tree are considered part of that JSON tree. As a result,
+  // an XML node may be in the subtree of a JSON node.
+  virtual bool isInSubtree(const StructuredItem* anotherItem) const = 0;
+
+protected:
+  CollectionTreeInfo* getCollectionTreeInfo() const;
+};
+
+} // namespace simplestore
+} // namespace zorba
+#endif /* ZORBA_SIMPLE_STORE_NODE_ITEMS */
+

=== modified file 'src/types/typeops.cpp'
--- src/types/typeops.cpp	2013-01-29 06:01:31 +0000
+++ src/types/typeops.cpp	2013-02-21 15:32:25 +0000
@@ -811,7 +811,7 @@
   case XQType::STRUCTURED_ITEM_KIND:
   {
 #ifdef ZORBA_WITH_JSON
-    if (subitem->isJSONItem() || subitem->isNode())
+    if (subitem->isStructuredItem())
 #else
     if (subitem->isNode())
 #endif

=== added file 'test/rbkt/ExpQueryResults/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point-maintenance.xml.res'
--- test/rbkt/ExpQueryResults/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point-maintenance.xml.res	1970-01-01 00:00:00 +0000
+++ test/rbkt/ExpQueryResults/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point-maintenance.xml.res	2013-02-21 15:32:25 +0000
@@ -0,0 +1,1 @@
+{ "name" : "<name><first>Bev</first><last>Picard</last></name>", "age" : 38, "gender" : "female", "friends" : [ "Jean-Luc Picard", "Ensign Crusher" ] }
\ No newline at end of file

=== added file 'test/rbkt/ExpQueryResults/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point.xml.res'
--- test/rbkt/ExpQueryResults/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point.xml.res	1970-01-01 00:00:00 +0000
+++ test/rbkt/ExpQueryResults/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point.xml.res	2013-02-21 15:32:25 +0000
@@ -0,0 +1,1 @@
+{ "name" : "<name><grade>Lieutenant Commander</grade><first>Data</first></name>", "age" : 100, "gender" : "positronic matrix", "friends" : [ "Geordi La Forge" ] }
\ No newline at end of file

=== added file 'test/rbkt/ExpQueryResults/zorba/jsoniq/collection-and-index/foaf-json-xml-update-index-point.xml.res'
--- test/rbkt/ExpQueryResults/zorba/jsoniq/collection-and-index/foaf-json-xml-update-index-point.xml.res	1970-01-01 00:00:00 +0000
+++ test/rbkt/ExpQueryResults/zorba/jsoniq/collection-and-index/foaf-json-xml-update-index-point.xml.res	2013-02-21 15:32:25 +0000
@@ -0,0 +1,1 @@
+{ "age" : 30, "gender" : "male", "name" : "<name><first>William</first><last>Riker</last></name>" }
\ No newline at end of file

=== added file 'test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point-maintenance.spec'
--- test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point-maintenance.spec	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point-maintenance.spec	2013-02-21 15:32:25 +0000
@@ -0,0 +1,1 @@
+Serialization: method=json-xml-hybrid jsoniq-multiple-items=appended jsoniq-extensions=yes

=== added file 'test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point-maintenance.xq'
--- test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point-maintenance.xq	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point-maintenance.xq	2013-02-21 15:32:25 +0000
@@ -0,0 +1,36 @@
+
+import module namespace foaf = "http://www.w3.org/TestModules/foaf" at
+                                   "foaf_module-with-index-and-xml.xqlib";
+
+import module namespace ddl = "http://www.zorba-xquery.com/modules/store/static/collections/ddl";
+import module namespace dml = "http://www.zorba-xquery.com/modules/store/static/collections/dml";
+import module namespace index_ddl = "http://www.zorba-xquery.com/modules/store/static/indexes/ddl";
+import module namespace index_dml = "http://www.zorba-xquery.com/modules/store/static/indexes/dml";
+
+declare namespace err = "http://www.w3.org/2005/xqt-errors";
+
+foaf:create-db();
+
+dml:delete(dml:collection($foaf:network)[. ("name")/first eq "James"]);
+
+dml:insert($foaf:network,
+    {
+      "name" : <name><first>Wesley</first><last>Crusher</last></name>,
+      "age" : 20,
+      "gender" : "male",
+      "friends" : [ "Beverly Crusher" ]
+    }
+);
+
+(replace value of node dml:collection($foaf:network)[.("name")/first eq "Beverly"]("name")/first with "Bev",
+replace value of node dml:collection($foaf:network)[.("name")/first eq "Beverly"]("name")/last with "Picard",
+
+for $person in dml:collection($foaf:network)
+let $friends := $person("friends")
+for $i in 1 to jn:size($friends)
+where $friends($i) eq "Beverly Crusher"
+return
+replace json value of $friends($i) with "Bev Picard")
+;
+
+foaf:probe-point-id($foaf:person, "Bev")

=== added file 'test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point.spec'
--- test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point.spec	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point.spec	2013-02-21 15:32:25 +0000
@@ -0,0 +1,1 @@
+Serialization: method=json-xml-hybrid jsoniq-multiple-items=appended jsoniq-extensions=yes

=== added file 'test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point.xq'
--- test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point.xq	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-index-point.xq	2013-02-21 15:32:25 +0000
@@ -0,0 +1,15 @@
+
+import module namespace foaf = "http://www.w3.org/TestModules/foaf" at
+                                   "foaf_module-with-index-and-xml.xqlib";
+
+import module namespace ddl = "http://www.zorba-xquery.com/modules/store/static/collections/ddl";
+import module namespace dml = "http://www.zorba-xquery.com/modules/store/static/collections/dml";
+import module namespace index_ddl = "http://www.zorba-xquery.com/modules/store/static/indexes/ddl";
+import module namespace index_dml = "http://www.zorba-xquery.com/modules/store/static/indexes/dml";
+
+declare namespace err = "http://www.w3.org/2005/xqt-errors";
+
+foaf:create-db();
+
+for $x in foaf:probe-point-id($foaf:person, "Data")
+return $x

=== added file 'test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-update-index-point.spec'
--- test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-update-index-point.spec	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-update-index-point.spec	2013-02-21 15:32:25 +0000
@@ -0,0 +1,1 @@
+Serialization: method=json-xml-hybrid jsoniq-multiple-items=appended jsoniq-extensions=yes

=== added file 'test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-update-index-point.xq'
--- test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-update-index-point.xq	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf-json-xml-update-index-point.xq	2013-02-21 15:32:25 +0000
@@ -0,0 +1,25 @@
+import module namespace foaf = "http://www.w3.org/TestModules/foaf" at
+                                   "foaf_module-with-index-and-xml.xqlib";
+
+import module namespace dml = "http://www.zorba-xquery.com/modules/store/static/collections/dml";
+
+declare namespace err = "http://www.w3.org/2005/xqt-errors";
+
+foaf:create-db();
+
+dml:insert($foaf:network, {
+    "first-name" : "William",
+    "last-name" : "Riker",
+    "age" : 30,
+    "gender" : "male"
+  });
+
+let $riker := dml:collection($foaf:network)[ .("last-name") eq "Riker" ]
+return {
+  insert json { "name" : <name><first>William</first><last>Riker</last></name> } into $riker;
+  delete json $riker("first-name");
+  delete json $riker("last-name");
+}
+
+for $x in foaf:probe-point-id($foaf:person, "William")
+return $x

=== added file 'test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf_module-with-index-and-xml.xqlib'
--- test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf_module-with-index-and-xml.xqlib	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/jsoniq/collection-and-index/foaf_module-with-index-and-xml.xqlib	2013-02-21 15:32:25 +0000
@@ -0,0 +1,82 @@
+
+module namespace foaf = "http://www.w3.org/TestModules/foaf";
+
+import module namespace ddl = "http://www.zorba-xquery.com/modules/store/static/collections/ddl";
+import module namespace dml = "http://www.zorba-xquery.com/modules/store/static/collections/dml";
+import module namespace index_ddl = "http://www.zorba-xquery.com/modules/store/static/indexes/ddl";
+import module namespace index_dml = "http://www.zorba-xquery.com/modules/store/static/indexes/dml";
+
+declare namespace ann = "http://www.zorba-xquery.com/annotations";
+
+declare variable $foaf:network:= xs:QName("foaf:network");
+declare variable $foaf:person:= xs:QName("foaf:person");
+declare variable $foaf:age:= xs:QName("foaf:age");
+declare variable $foaf:friends:= xs:QName("foaf:friends");
+
+declare collection foaf:network as object()*;
+
+
+(:
+  The person index maps each person name with its information.
+:)
+
+declare %ann:automatic index foaf:person
+on nodes dml:collection(xs:QName("foaf:network"))
+by .("name")/string(first) as xs:string?;
+
+(:
+  Create and populate the collection, and then create the indexes
+:)
+
+declare %ann:sequential function foaf:create-db()
+{
+  ddl:create($foaf:network);
+
+  dml:insert($foaf:network, (
+    {
+      "name" : <name><first>James</first><middle>T.</middle><last>Kirk</last></name>,
+      "age" : 30,
+      "gender" : "male",
+      "friends" : [ "Mister Spock", "Scotty", "Jean-Luc Picard"]
+    },
+
+    {
+      "name" : <name><first>Jean-Luc</first><last>Picard</last></name>,
+      "age" : 40,
+      "gender" : "male",
+      "friends" : [ "James T. Kirk", "Lieutenant Commander Data", "Beverly Crusher" ]
+    },
+
+    {
+      "name" : <name><first>Beverly</first><last>Crusher</last></name>,
+      "age" : 38,
+      "gender" : "female",
+      "friends" : [ "Jean-Luc Picard", "Ensign Crusher" ]
+    },
+
+    {
+      "name" : <name><grade>Lieutenant Commander</grade><first>Data</first></name>,
+      "age" : 100,
+      "gender" : "positronic matrix",
+      "friends" : [ "Geordi La Forge" ]
+    }
+  ));
+
+  index_ddl:create($foaf:person);
+};
+
+
+declare function foaf:probe-point-id($indexName as xs:QName, $id as xs:string)
+{
+  index_dml:probe-index-point-value($indexName, $id)
+};
+
+declare function foaf:probe-range-id($indexName as xs:QName, $lower as xs:integer, $upper as xs:integer)
+{
+  index_dml:probe-index-range-value($indexName, $lower, $upper, true(), true(), true(), true())
+};
+
+declare function foaf:probe-point-general-id($indexName as xs:QName, $names as xs:string*)
+{
+  index_dml:probe-index-point-general($indexName, $names)
+};