Merge lp:~davidagraf/zorba/paging_with_refs into lp:zorba

[David Graf]

feature: skipping nodes in a collection using a node-reference

Resubmit to trigger patchqueue.

[Matthias Brantner (matthias-brantner)]

- Given the documentation, I was not able to figure out why the collection
function takes the $start and $skip parameters. Is one of the ignored if
the other one is given? Does skipping start at the item identified by
$start? If so, would it make sense to separate the two ways of skipping
into separate functions?

- improved
The reference to first node to return
=>
A reference to the first node to return.

- What does "order is implementation dependent" mean for ordered collections?

- Error message can be improved according to Paul's style:

Node reference $1 doesn't reference a node in collection $2
=>
$1: doesn't reference a node in collection ($2)

- typo:
if the passed reference $start doesn't reference
a _not_ from the collection identified by $name.

- the documentation mention zerr:ZAPI0028 if the given URI
is not a valid node reference

review: Needs Fixing

[David Graf (davidagraf)]

> - Given the documentation, I was not able to figure out why the collection
> function takes the $start and $skip parameters. Is one of the ignored if
> the other one is given? Does skipping start at the item identified by
> $start? If so, would it make sense to separate the two ways of skipping
> into separate functions?
No, none is ignored. If you don't wanna skip, you pass 0. If you wanna skip only, you use db:collection($name, $skip). The goal was to not introduce additional function. I think there are already enough dml and ddl functions. If you want, I can make $skip optional.
But as already mentioned in a yammer message, I am thinking about replacing the $skip parameter with an optimisation for fn:subsequence(db:collection(), $start). To get rid of additional stuff.
>
Fixed everything below:
> - improved
> The reference to first node to return
> =>
> A reference to the first node to return.
>
> - What does "order is implementation dependent" mean for ordered collections?
>
> - Error message can be improved according to Paul's style:
>
> Node reference $1 doesn't reference a node in collection $2
> =>
> $1: doesn't reference a node in collection ($2)
>
> - typo:
> if the passed reference $start doesn't reference
> a _not_ from the collection identified by $name.
>
> - the documentation mention zerr:ZAPI0028 if the given URI
> is not a valid node reference

[Till Westmann (tillw)]

There are a few small fixes that I've sent as a patch via email as I could not push them into this branch. Once this patch is applied this branch works for me.

review: Needs Fixing

[Matthias Brantner (matthias-brantner)]

The patch looks great.

As far as I can tell, the documentation should also mention the error zerr:ZAPI0028.

review: Needs Fixing

[David Graf (davidagraf)]

> There are a few small fixes that I've sent as a patch via email as I could not
> push them into this branch. Once this patch is applied this branch works for
> me.

I patched you patch. Unfortunately, I needed to do it by hand. Did you create the patch with bzr diff?

[David Graf (davidagraf)]

> The patch looks great.
>
> As far as I can tell, the documentation should also mention the error
> zerr:ZAPI0028.
done

[Till Westmann (tillw)]

review: Approve

[Matthias Brantner (matthias-brantner)]

review: Approve

[Till Westmann (tillw)]

review: Approve

[Matthias Brantner (matthias-brantner)]

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/paging_with_refs-2012-09-10T19-15-08.776Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~davidagraf/zorba/paging_with_refs into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job paging_with_refs-2012-09-10T19-15-08.776Z is finished.
  The final status was:

  2 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/paging_with_refs-2012-09-12T11-18-03.523Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~davidagraf/zorba/paging_with_refs into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job paging_with_refs-2012-09-12T11-18-03.523Z is finished.
  The final status was:

  1 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/paging_with_refs-2012-09-12T13-08-49.816Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job paging_with_refs-2012-09-12T13-08-49.816Z is finished. The final status was:

All tests succeeded!

=== modified file 'include/zorba/pregenerated/diagnostic_list.h'
--- include/zorba/pregenerated/diagnostic_list.h	2012-09-12 08:06:26 +0000
+++ include/zorba/pregenerated/diagnostic_list.h	2012-09-12 11:09:20 +0000
@@ -750,6 +750,8 @@
 
 extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZSTR0065_STRINGS_IN_POOL;
 
+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZSTR0066_REFERENCED_NODE_NOT_IN_COLLECTION;
+
 extern ZORBA_DLL_PUBLIC ZorbaErrorCode XSST0001;
 
 extern ZORBA_DLL_PUBLIC ZorbaErrorCode XSST0002;
 
=== modified file 'modules/com/zorba-xquery/www/modules/pregenerated/errors.xq'
--- modules/com/zorba-xquery/www/modules/pregenerated/errors.xq	2012-09-10 22:53:04 +0000
+++ modules/com/zorba-xquery/www/modules/pregenerated/errors.xq	2012-09-12 11:09:20 +0000
@@ -781,6 +781,10 @@
 
 (:~
 :)
+declare variable $zerr:ZSTR0066 as xs:QName := fn:QName($zerr:NS, "zerr:ZSTR0066");
+
+(:~
+:)
 declare variable $zerr:XSST0001 as xs:QName := fn:QName($zerr:NS, "zerr:XSST0001");
 
 (:~
 
=== modified file 'modules/com/zorba-xquery/www/modules/store/dynamic/collections/dml.xq'
--- modules/com/zorba-xquery/www/modules/store/dynamic/collections/dml.xq	2012-09-10 22:53:04 +0000
+++ modules/com/zorba-xquery/www/modules/store/dynamic/collections/dml.xq	2012-09-12 11:09:20 +0000
@@ -632,6 +632,35 @@
 
 
 (:~
+ : The collection function returns the sequence of nodes and/or json items
+ : that belong to the collection identified by the given name.
+ : The parameters $start and $skip can be used to skip over some items at
+ : the beginning of the collection. If both are given, both are applied:
+ : first $start to skip to the referenced item and then $skip to skip an
+ : additional number of items.
+ :
+ : @param $name The name of the collection.
+ : @param $start A reference to the first item to return. All items before
+                 are skipped.
+ : @param $skip The number of collection items to skip.
+ :
+ : @return The sequence contained in the given collection.
+ :
+ : @error zerr:ZAPI0028 If the given URI is not a valid node
+ :        position computed by the <tt>np:node-position</tt> function. 
+ : @error zerr:ZDDY0003 If available collections does not provide a mapping
+ :        for the expanded QName $name.
+ : @error zerr:ZSTR0066 If the passed reference $start does not reference
+ :        a node from the collection identified by $name.
+ :
+ :)
+declare function
+dml:collection($name as xs:QName, 
+               $start as xs:anyURI,
+               $skip as xs:integer) as item()* external;
+
+
+(:~
  : The collection-name function returns the name of the collection the given
  : item (node or json item) belongs to.
  :
 
=== modified file 'modules/com/zorba-xquery/www/modules/store/static/collections/dml.xq'
--- modules/com/zorba-xquery/www/modules/store/static/collections/dml.xq	2012-09-10 22:53:04 +0000
+++ modules/com/zorba-xquery/www/modules/store/static/collections/dml.xq	2012-09-12 11:09:20 +0000
@@ -884,6 +884,33 @@
 
 
 (:~
+ : The collection function returns the sequence of nodes and/or json items
+ : that belong to the collection identified by the given name.
+ : The parameters $start and $skip can be used to skip over some items at
+ : the beginning of the collection. If both are given, both are applied:
+ : first $start to skip to the referenced item and then $skip to skip an
+ : additional number of items.
+ :
+ : @param $name The name of the collection.
+ : @param $start The reference to the first item to return. All items before
+                 are skipped.
+ : @param $skip The number of collection items to skip.
+ :
+ : @return The sequence contained in the given collection.
+ :
+ : @error zerr:ZAPI0028 If the given URI is not a valid node
+ :        position computed by the <tt>np:node-position</tt> function. 
+ : @error zerr:ZDDY0001 if the collection identified by $name is not declared.
+ : @error zerr:ZDDY0003 if the collection identified by $name is not available.
+ : @error zerr:ZSTR0066 if the passed reference $start does not reference
+ :        a node from the collection identified by $name.
+ :
+ :)
+declare function 
+cdml:collection($name as xs:QName, 
+                $start as xs:anyURI,
+                $skip as xs:integer) as item()* external;
+(:~
  : The collection-name function returns the name of the collection the given
  : item (node or json item) belongs to.
  :
 
=== modified file 'src/diagnostics/diagnostic_en.xml'
--- src/diagnostics/diagnostic_en.xml	2012-09-12 08:06:26 +0000
+++ src/diagnostics/diagnostic_en.xml	2012-09-12 11:09:20 +0000
@@ -2433,6 +2433,10 @@
       <value>Zorba did not close properly, objects may still in memory.\n$1 referenced URI(s) remain in the string pool.\nFor help avoiding this message please refer to http://www.zorba-xquery.com/html/documentation in section General Architecture -> Memory Leaks.</value>
     </diagnostic>
 
+    <diagnostic code="ZSTR0066" name="REFERENCED_NODE_NOT_IN_COLLECTION">
+      <value>$1: does not reference a node in collection $2.</value>
+    </diagnostic>
+
     <!--////////// XQuery Scripting ////////////////////////////////////////-->
 
     <diagnostic code="XSST0001">
 
=== modified file 'src/diagnostics/pregenerated/diagnostic_list.cpp'
--- src/diagnostics/pregenerated/diagnostic_list.cpp	2012-09-12 08:06:26 +0000
+++ src/diagnostics/pregenerated/diagnostic_list.cpp	2012-09-12 11:09:20 +0000
@@ -1103,6 +1103,9 @@
 ZorbaErrorCode ZSTR0065_STRINGS_IN_POOL( "ZSTR0065" );
 
 
+ZorbaErrorCode ZSTR0066_REFERENCED_NODE_NOT_IN_COLLECTION( "ZSTR0066" );
+
+
 ZorbaErrorCode XSST0001( "XSST0001" );
 
 
 
=== modified file 'src/diagnostics/pregenerated/dict_en.cpp'
--- src/diagnostics/pregenerated/dict_en.cpp	2012-09-12 08:06:26 +0000
+++ src/diagnostics/pregenerated/dict_en.cpp	2012-09-12 11:09:20 +0000
@@ -453,6 +453,7 @@
   { "ZSTR0055", "streamable string has already been consumed" },
   { "ZSTR0060", "out of range: $1" },
   { "ZSTR0065", "Zorba did not close properly, objects may still in memory.\n$1 referenced URI(s) remain in the string pool.\nFor help avoiding this message please refer to http://www.zorba-xquery.com/html/documentation in section General Architecture -> Memory Leaks." },
+  { "ZSTR0066", "$1: does not reference a node in collection $2." },
   { "ZWST0002", "\"$1\": unknown or unsupported annotation" },
   { "ZWST0003", "\"$1\": function declared sequential, but has non-sequential body" },
   { "ZWST0004", "Sequential FLWOR expr may not have the semantics you expect" },
 
=== modified file 'src/functions/func_sequences_impl.cpp'
--- src/functions/func_sequences_impl.cpp	2012-09-10 22:53:04 +0000
+++ src/functions/func_sequences_impl.cpp	2012-09-12 11:09:20 +0000
@@ -543,16 +543,19 @@
     ZorbaCollectionIterator& collection =
     static_cast<ZorbaCollectionIterator&>(*argv[0]);
 
-    return new CountCollectionIterator(
-                 sctx,
-                 loc,
-                 collection.getChildren(),
-                 (
-                   collection.isDynamic()
-                     ? CountCollectionIterator::ZORBADYNAMIC
-                     : CountCollectionIterator::ZORBASTATIC
-                 )
-               );
+    if (collection.isCountOptimizable())
+    {
+      return new CountCollectionIterator(
+                   sctx,
+                   loc,
+                   collection.getChildren(),
+                   (
+                     collection.isDynamic()
+                       ? CountCollectionIterator::ZORBADYNAMIC
+                       : CountCollectionIterator::ZORBASTATIC
+                   )
+                 );
+    }
   }
   else if (typeid(FnCollectionIterator) == counted_type)
   {
 
=== modified file 'src/functions/pregenerated/func_collections.cpp'
--- src/functions/pregenerated/func_collections.cpp	2012-09-10 22:53:04 +0000
+++ src/functions/pregenerated/func_collections.cpp	2012-09-12 11:09:20 +0000
@@ -355,6 +355,20 @@
 
       {
     DECL_WITH_KIND(sctx, static_collections_dml_collection,
+        (createQName("http://www.zorba-xquery.com/modules/store/static/collections/dml","","collection"), 
+        GENV_TYPESYSTEM.QNAME_TYPE_ONE, 
+        GENV_TYPESYSTEM.ANY_URI_TYPE_ONE, 
+        GENV_TYPESYSTEM.INTEGER_TYPE_ONE, 
+        GENV_TYPESYSTEM.STRUCTURED_ITEM_TYPE_STAR),
+        FunctionConsts::STATIC_COLLECTIONS_DML_COLLECTION_3);
+
+  }
+
+
+
+
+      {
+    DECL_WITH_KIND(sctx, static_collections_dml_collection,
         (createQName("http://www.zorba-xquery.com/modules/store/dynamic/collections/dml","","collection"), 
         GENV_TYPESYSTEM.QNAME_TYPE_ONE, 
         GENV_TYPESYSTEM.STRUCTURED_ITEM_TYPE_STAR),
@@ -379,6 +393,20 @@
 
 
       {
+    DECL_WITH_KIND(sctx, static_collections_dml_collection,
+        (createQName("http://www.zorba-xquery.com/modules/store/dynamic/collections/dml","","collection"), 
+        GENV_TYPESYSTEM.QNAME_TYPE_ONE, 
+        GENV_TYPESYSTEM.ANY_URI_TYPE_ONE, 
+        GENV_TYPESYSTEM.INTEGER_TYPE_ONE, 
+        GENV_TYPESYSTEM.STRUCTURED_ITEM_TYPE_STAR),
+        FunctionConsts::DYNAMIC_COLLECTIONS_DML_COLLECTION_3);
+
+  }
+
+
+
+
+      {
     DECL_WITH_KIND(sctx, static_collections_dml_collection_name,
         (createQName("http://www.zorba-xquery.com/modules/store/static/collections/dml","","collection-name"), 
         GENV_TYPESYSTEM.STRUCTURED_ITEM_TYPE_ONE, 
 
=== modified file 'src/functions/pregenerated/function_enum.h'
--- src/functions/pregenerated/function_enum.h	2012-09-11 16:33:21 +0000
+++ src/functions/pregenerated/function_enum.h	2012-09-12 11:09:20 +0000
@@ -55,8 +55,10 @@
   FN_COLLECTION_1,
   STATIC_COLLECTIONS_DML_COLLECTION_1,
   STATIC_COLLECTIONS_DML_COLLECTION_2,
+  STATIC_COLLECTIONS_DML_COLLECTION_3,
   DYNAMIC_COLLECTIONS_DML_COLLECTION_1,
   DYNAMIC_COLLECTIONS_DML_COLLECTION_2,
+  DYNAMIC_COLLECTIONS_DML_COLLECTION_3,
   STATIC_COLLECTIONS_DML_COLLECTION_NAME_1,
   DYNAMIC_COLLECTIONS_DML_COLLECTION_NAME_1,
   STATIC_COLLECTIONS_DML_INDEX_OF_1,
 
=== modified file 'src/runtime/collections/collections_impl.cpp'
--- src/runtime/collections/collections_impl.cpp	2012-09-10 22:53:04 +0000
+++ src/runtime/collections/collections_impl.cpp	2012-09-12 11:09:20 +0000
@@ -367,6 +367,12 @@
   }
 }
 
+bool ZorbaCollectionIterator::isCountOptimizable() const
+{
+  // if ref is passed to the collections function, count cannot be 
+  // optimized anymore.
+  return theChildren.size() <= 2;
+}
 
 bool ZorbaCollectionIterator::nextImpl(
     store::Item_t& result,
@@ -375,6 +381,7 @@
   store::Item_t name;
   store::Collection_t collection;
   xs_integer lSkip;
+  zstring lStart;
 
   ZorbaCollectionIteratorState* state;
   DEFAULT_STACK_INIT(ZorbaCollectionIteratorState, state, planState);
@@ -383,21 +390,42 @@
 
   (void)getCollection(theSctx, name, loc, theIsDynamic, collection);
 
-  if (theChildren.size() > 1)
-  {
-    // skip parameter passed
+  if (theChildren.size() == 1)
+  {
+    state->theIterator = collection->getIterator();
+  }
+  else
+  {
+    bool lRefPassed = theChildren.size() >= 3;
+    
+    // read positional skip parameter
     store::Item_t lSkipItem;
-    consumeNext(lSkipItem, theChildren[1].getp(), planState);
+    consumeNext(lSkipItem, theChildren[(lRefPassed ? 2 : 1)].getp(), planState);
     lSkip = lSkipItem->getIntegerValue(); 
-    // negative is transformed into 0
-    state->theIterator = ( lSkip > xs_integer::zero() 
-                             ? collection->getIterator(lSkip)
-                             : collection->getIterator()
-                         );
-  }
-  else
-  {
-    state->theIterator = collection->getIterator();
+    // negative skip is not allowed
+    if (lSkip < xs_integer::zero())
+    {
+      lSkip = xs_integer::zero();
+    }
+    if (!lRefPassed)
+    {
+      state->theIterator = collection->getIterator(lSkip);
+    }
+    else
+    {
+      store::Item_t lRefItem;
+      consumeNext(lRefItem, theChildren[1].getp(), planState);
+      lStart = lRefItem->getString(); 
+      try
+      {
+        state->theIterator = collection->getIterator(lSkip, lStart);
+      }
+      catch (ZorbaException& e)
+      {
+        set_source(e, loc);
+        throw;
+      }
+    }
   }
 
   ZORBA_ASSERT(state->theIterator != NULL);
 
=== modified file 'src/runtime/collections/pregenerated/collections.h'
--- src/runtime/collections/pregenerated/collections.h	2012-09-10 22:53:04 +0000
+++ src/runtime/collections/pregenerated/collections.h	2012-09-12 11:09:20 +0000
@@ -285,6 +285,8 @@
 
   bool isDynamic() const { return theIsDynamic; }
 
+public:
+  bool isCountOptimizable() const;
   void accept(PlanIterVisitor& v) const;
 
   bool nextImpl(store::Item_t& result, PlanState& aPlanState) const;
 
=== modified file 'src/runtime/spec/collections/collections.xml'
--- src/runtime/spec/collections/collections.xml	2012-09-10 22:53:04 +0000
+++ src/runtime/spec/collections/collections.xml	2012-09-12 11:09:20 +0000
@@ -255,13 +255,29 @@
         <zorba:output>structured-item()*</zorba:output>
       </zorba:signature>
 
-      <zorba:signature localname="collection" prefix="dynamic-collections-dml">
-        <zorba:param>xs:QName</zorba:param>
-        <zorba:output>structured-item()*</zorba:output>
-      </zorba:signature>
-
-      <zorba:signature localname="collection" prefix="dynamic-collections-dml">
-        <zorba:param>xs:QName</zorba:param>
+      <zorba:signature localname="collection"
+                       prefix="static-collections-dml">
+        <zorba:param>xs:QName</zorba:param>
+        <zorba:param>xs:anyURI</zorba:param><!-- start ref -->
+        <zorba:param>xs:integer</zorba:param><!-- nodes to skip -->
+        <zorba:output>structured-item()*</zorba:output>
+      </zorba:signature>
+
+      <zorba:signature localname="collection" prefix="dynamic-collections-dml">
+        <zorba:param>xs:QName</zorba:param>
+        <zorba:output>structured-item()*</zorba:output>
+      </zorba:signature>
+
+      <zorba:signature localname="collection" prefix="dynamic-collections-dml">
+        <zorba:param>xs:QName</zorba:param>
+        <zorba:param>xs:integer</zorba:param><!-- start to skip -->
+        <zorba:output>structured-item()*</zorba:output>
+      </zorba:signature>
+
+      <zorba:signature localname="collection"
+                       prefix="dynamic-collections-dml">
+        <zorba:param>xs:QName</zorba:param>
+        <zorba:param>xs:anyURI</zorba:param><!-- node ref-->
         <zorba:param>xs:integer</zorba:param><!-- nodes to skip -->
         <zorba:output>structured-item()*</zorba:output>
       </zorba:signature>
@@ -281,6 +297,10 @@
 
     <zorba:member type="bool" name="theIsDynamic" getterName="isDynamic"/>
 
+    <zorba:method const="true"
+                  name="isCountOptimizable"
+                  return="bool" />
+    
     <zorba:state generateInit="false" generateReset="false" generateDestructor="false">
       <zorba:member type="store::Iterator_t" name="theIterator"/>
       <zorba:member type="bool" name="theIteratorOpened" defaultValue="false"/>
 
=== modified file 'src/store/api/collection.h'
--- src/store/api/collection.h	2012-09-10 22:53:04 +0000
+++ src/store/api/collection.h	2012-09-12 11:09:20 +0000
@@ -54,11 +54,13 @@
    * It is allowed to have several concurrent iterators on the same Collection,
    * but each iterator should be used by a single thread only.
    *
-   * @param  aSkip The number of collection entries to skip.
+   * @param aSkip The number of collection items to skip.
+   * @param aStart The reference to the first node. All nodes before are skipped.
    * @return Iterator
    */
   virtual Iterator_t getIterator(
-      const xs_integer& aSkip = xs_integer::zero()) = 0;
+      const xs_integer& aSkip = xs_integer::zero(),
+      const zstring& aStart = "") = 0;
 
   /**
    * Get the node at the given position in the collection.
 
=== modified file 'src/store/naive/collection.h'
--- src/store/naive/collection.h	2012-09-10 22:53:04 +0000
+++ src/store/naive/collection.h	2012-09-12 11:09:20 +0000
@@ -54,7 +54,9 @@
 
   zorba::xs_integer size() const  = 0;
 
-  zorba::store::Iterator_t getIterator(const xs_integer& aSkip) = 0;
+  zorba::store::Iterator_t getIterator(
+      const xs_integer& aSkip, 
+      const zstring& aStart) = 0;
 
   virtual zorba::store::Item_t nodeAt(xs_integer position) = 0;
 
 
=== modified file 'src/store/naive/simple_collection.cpp'
--- src/store/naive/simple_collection.cpp	2012-09-10 22:53:04 +0000
+++ src/store/naive/simple_collection.cpp	2012-09-12 11:09:20 +0000
@@ -81,9 +81,22 @@
   Note: it is allowed to have several concurrent iterators on the same collection
   but each iterator should be used by a single thread only.
 ********************************************************************************/
-store::Iterator_t SimpleCollection::getIterator(const xs_integer& aSkip)
+store::Iterator_t SimpleCollection::getIterator(const xs_integer& aSkip,
+                                                const zstring& aStart)
 {
-  return new CollectionIter(this, aSkip);
+  store::Item_t lReferencedNode;
+  xs_integer lReferencedPosition = xs_integer::zero();
+  if (aStart.size() != 0
+   && (!GET_STORE().getNodeByReference(lReferencedNode, aStart)
+    || !findNode(lReferencedNode.getp(), lReferencedPosition)))
+  {
+    throw ZORBA_EXCEPTION(zerr::ZSTR0066_REFERENCED_NODE_NOT_IN_COLLECTION,
+    ERROR_PARAMS(aStart, theName->getStringValue()));
+  }
+
+  return new CollectionIter(
+               this, 
+               aSkip + lReferencedPosition);
 }
 
 
@@ -596,6 +609,21 @@
     theCollection->theLatch.unlock();)
 }
 
+void SimpleCollection::CollectionIter::skip()
+{
+  // skip by position
+  long lToSkip = to_xs_long(theSkip);
+  if (theSkip >= theCollection->size())
+  {
+    // we need to skip more then possible -> jump to the end
+    theIterator = theEnd;
+  }
+  else
+  {
+    theIterator += to_xs_long(theSkip);
+  }
+}
+
 
 /*******************************************************************************
 
@@ -606,8 +634,8 @@
   theHaveLock = true;
 
   theIterator = theCollection->theXmlTrees.begin();
-  theIterator += to_xs_long(theSkip);
   theEnd = theCollection->theXmlTrees.end();
+  skip();
 }
 
 
@@ -641,8 +669,8 @@
 void SimpleCollection::CollectionIter::reset()
 {
   theIterator = theCollection->theXmlTrees.begin();
-  theIterator += to_xs_long(theSkip);
   theEnd = theCollection->theXmlTrees.end();
+  skip();
 }
 
 
 
=== modified file 'src/store/naive/simple_collection.h'
--- src/store/naive/simple_collection.h	2012-09-10 22:53:04 +0000
+++ src/store/naive/simple_collection.h	2012-09-12 11:09:20 +0000
@@ -57,7 +57,8 @@
     xs_integer                              theSkip;
 
   public:
-    CollectionIter(SimpleCollection* collection, const xs_integer& aSkip);
+    CollectionIter(SimpleCollection* collection, 
+                   const xs_integer& aSkip);
 
     ~CollectionIter();
 
@@ -65,6 +66,8 @@
     bool next(store::Item_t& result);
     void reset();
     void close();
+  private:
+    void skip();
   };
 
 
@@ -110,7 +113,8 @@
 
   TreeId createTreeId();
 
-  store::Iterator_t getIterator(const xs_integer& aSkip);
+  store::Iterator_t getIterator(const xs_integer& aSkip, 
+                                const zstring& aStart);
 
   void addNode(store::Item* node, xs_integer position = xs_integer(-1));
 
 
=== modified file 'test/rbkt/ExpQueryResults/zorba/collections/paging_1.xml.res'
--- test/rbkt/ExpQueryResults/zorba/collections/paging_1.xml.res	2012-06-28 18:19:34 +0000
+++ test/rbkt/ExpQueryResults/zorba/collections/paging_1.xml.res	2012-09-12 11:09:20 +0000
@@ -1,1 +1,1 @@
-<d/><e/><a/><b/><c/><d/><e/>
+<d/><e/><delim/><a/><b/><c/><d/><e/><delim/><c/><d/><e/>
 
=== modified file 'test/rbkt/ExpQueryResults/zorba/collections/paging_2.xml.res'
--- test/rbkt/ExpQueryResults/zorba/collections/paging_2.xml.res	2012-06-28 18:19:34 +0000
+++ test/rbkt/ExpQueryResults/zorba/collections/paging_2.xml.res	2012-09-12 11:09:20 +0000
@@ -1,1 +1,1 @@
-2 5 0
+2 5 0 3
 
=== modified file 'test/rbkt/ExpQueryResults/zorba/collections/paging_3.xml.res'
--- test/rbkt/ExpQueryResults/zorba/collections/paging_3.xml.res	2012-06-28 18:19:34 +0000
+++ test/rbkt/ExpQueryResults/zorba/collections/paging_3.xml.res	2012-09-12 11:09:20 +0000
@@ -1,1 +1,1 @@
-<d/><e/><a/><b/><c/><d/><e/>
+<d/><e/><delim/><a/><b/><c/><d/><e/><delim/><c/><d/><e/>
 
=== modified file 'test/rbkt/ExpQueryResults/zorba/collections/paging_4.xml.res'
--- test/rbkt/ExpQueryResults/zorba/collections/paging_4.xml.res	2012-06-28 18:19:34 +0000
+++ test/rbkt/ExpQueryResults/zorba/collections/paging_4.xml.res	2012-09-12 11:09:20 +0000
@@ -1,1 +1,1 @@
-2 5 0
+2 5 0 3
 
=== modified file 'test/rbkt/Queries/zorba/collections/paging_1.xq'
--- test/rbkt/Queries/zorba/collections/paging_1.xq	2012-06-28 18:19:34 +0000
+++ test/rbkt/Queries/zorba/collections/paging_1.xq	2012-09-12 11:09:20 +0000
@@ -1,9 +1,17 @@
 import module namespace ddl = "http://www.zorba-xquery.com/modules/store/static/collections/ddl";
 import module namespace dml = "http://www.zorba-xquery.com/modules/store/static/collections/dml";
 import module namespace ns = "http://example.org/datamodule/" at "collections.xqdata";
+import module namespace ref = "http://www.zorba-xquery.com/modules/node-reference";
 
 declare namespace ann = "http://www.zorba-xquery.com/annotations";
 
+declare function local:order($items)
+{
+  for $item in $items
+  order by fn:local-name($item)
+  return $item
+};
+
 declare %ann:sequential function local:test()
 {
   ddl:create(xs:QName("ns:test2"));
@@ -11,10 +19,11 @@
   dml:insert-nodes(xs:QName("ns:test2"), <b/>);
   dml:insert-nodes(xs:QName("ns:test2"), (<c/>, <d/>, <e/>));
   (
-    dml:collection(xs:QName("ns:test2"), 3),
-    dml:collection(xs:QName("ns:test2"), -1)
+    local:order(dml:collection(xs:QName("ns:test2"), 3)), <delim/>,
+    local:order(dml:collection(xs:QName("ns:test2"), -1)), <delim/>,
+    let $ref := ref:node-reference(dml:collection(xs:QName("ns:test2"))[3])
+    return local:order(dml:collection(xs:QName("ns:test2"), $ref, 0))
   )
 };
 
 local:test()
-
 
=== modified file 'test/rbkt/Queries/zorba/collections/paging_2.xq'
--- test/rbkt/Queries/zorba/collections/paging_2.xq	2012-06-28 18:19:34 +0000
+++ test/rbkt/Queries/zorba/collections/paging_2.xq	2012-09-12 11:09:20 +0000
@@ -1,6 +1,7 @@
 import module namespace ddl = "http://www.zorba-xquery.com/modules/store/static/collections/ddl";
 import module namespace dml = "http://www.zorba-xquery.com/modules/store/static/collections/dml";
 import module namespace ns = "http://example.org/datamodule/" at "collections.xqdata";
+import module namespace ref = "http://www.zorba-xquery.com/modules/node-reference";
 
 declare namespace ann = "http://www.zorba-xquery.com/annotations";
 
@@ -13,9 +14,10 @@
   (
     fn:count(dml:collection(xs:QName("ns:test2"), 3)),
     fn:count(dml:collection(xs:QName("ns:test2"), -1)),
-    fn:count(dml:collection(xs:QName("ns:test2"), 100))
+    fn:count(dml:collection(xs:QName("ns:test2"), 100)),
+    let $ref := ref:node-reference(dml:collection(xs:QName("ns:test2"))[3])
+    return fn:count(dml:collection(xs:QName("ns:test2"), $ref, 0))
   )
 };
 
 local:test()
-
 
=== modified file 'test/rbkt/Queries/zorba/collections/paging_3.xq'
--- test/rbkt/Queries/zorba/collections/paging_3.xq	2012-06-28 18:19:34 +0000
+++ test/rbkt/Queries/zorba/collections/paging_3.xq	2012-09-12 11:09:20 +0000
@@ -1,5 +1,6 @@
 import module namespace ddl = "http://www.zorba-xquery.com/modules/store/dynamic/collections/ddl";
 import module namespace dml = "http://www.zorba-xquery.com/modules/store/dynamic/collections/dml";
+import module namespace ref = "http://www.zorba-xquery.com/modules/node-reference";
 
 declare namespace ann = "http://www.zorba-xquery.com/annotations";
 declare namespace ns = "http://www.zorba-xquery.com/test";
@@ -11,10 +12,11 @@
   dml:insert-nodes-last(xs:QName("ns:test2"), <b/>);
   dml:insert-nodes-last(xs:QName("ns:test2"), (<c/>, <d/>, <e/>));
   (
-    dml:collection(xs:QName("ns:test2"), 3),
-    dml:collection(xs:QName("ns:test2"), -1)
+    dml:collection(xs:QName("ns:test2"), 3), <delim/>,
+    dml:collection(xs:QName("ns:test2"), -1), <delim/>,
+    let $ref := ref:node-reference(dml:collection(xs:QName("ns:test2"))[3])
+    return dml:collection(xs:QName("ns:test2"), $ref, 0)
   )
 };
 
 local:test()
-
 
=== modified file 'test/rbkt/Queries/zorba/collections/paging_4.xq'
--- test/rbkt/Queries/zorba/collections/paging_4.xq	2012-06-28 18:19:34 +0000
+++ test/rbkt/Queries/zorba/collections/paging_4.xq	2012-09-12 11:09:20 +0000
@@ -1,5 +1,6 @@
 import module namespace ddl = "http://www.zorba-xquery.com/modules/store/dynamic/collections/ddl";
 import module namespace dml = "http://www.zorba-xquery.com/modules/store/dynamic/collections/dml";
+import module namespace ref = "http://www.zorba-xquery.com/modules/node-reference";
 
 declare namespace ann = "http://www.zorba-xquery.com/annotations";
 declare namespace ns = "http://www.zorba-xquery.com/test";
@@ -13,9 +14,10 @@
   (
     fn:count(dml:collection(xs:QName("ns:test2"), 3)),
     fn:count(dml:collection(xs:QName("ns:test2"), -1)),
-    fn:count(dml:collection(xs:QName("ns:test2"), 100))
+    fn:count(dml:collection(xs:QName("ns:test2"), 100)),
+    let $ref := ref:node-reference(dml:collection(xs:QName("ns:test2"))[3])
+    return fn:count(dml:collection(xs:QName("ns:test2"), $ref, 0))
   )
 };
 
 local:test()
-
 
=== added file 'test/rbkt/Queries/zorba/collections/paging_5.spec'
--- test/rbkt/Queries/zorba/collections/paging_5.spec	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/collections/paging_5.spec	2012-09-12 11:09:20 +0000
@@ -0,0 +1,1 @@
+Error: http://www.zorba-xquery.com/errors:ZSTR0066
 
=== added file 'test/rbkt/Queries/zorba/collections/paging_5.xq'
--- test/rbkt/Queries/zorba/collections/paging_5.xq	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/collections/paging_5.xq	2012-09-12 11:09:20 +0000
@@ -0,0 +1,20 @@
+import module namespace ddl = "http://www.zorba-xquery.com/modules/store/dynamic/collections/ddl";
+import module namespace dml = "http://www.zorba-xquery.com/modules/store/dynamic/collections/dml";
+import module namespace ref = "http://www.zorba-xquery.com/modules/node-reference";
+
+declare namespace ann = "http://www.zorba-xquery.com/annotations";
+declare namespace ns = "http://www.zorba-xquery.com/test";
+
+declare %ann:sequential function local:test()
+{
+  ddl:create(xs:QName("ns:test2"));
+  dml:insert-nodes-last(xs:QName("ns:test2"), <a/>);
+  dml:insert-nodes-last(xs:QName("ns:test2"), <b/>);
+  dml:insert-nodes-last(xs:QName("ns:test2"), (<c/>, <d/>, <e/>));
+
+  dml:collection(
+    xs:QName("ns:test2"),
+    xs:anyURI("urn:uuid:aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"), 0)
+};
+
+local:test()