Merge lp:~zorba-coders/zorba/gen-flwor-opt into lp:zorba

[Markos Zaharioudakis]

Extended automatic hash-join rule to general flwor and scripting

Extended automatic hash-join rule to general flwor and scripting

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/gen-flwor-opt-2013-02-26T07-24-42.137Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/gen-flwor-opt into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message):
  Validation queue job gen-flwor-opt-2013-02-26T07-24-42.137Z is finished.
  The final status was:

  22 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/gen-flwor-opt-2013-02-26T09-56-30.35Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/gen-flwor-opt into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message):
  Validation queue job gen-flwor-opt-2013-02-26T09-56-30.35Z is finished.
  The final status was:

  16 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/gen-flwor-opt-2013-02-26T16-15-22.294Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/gen-flwor-opt into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message):
  Validation queue job gen-flwor-opt-2013-02-26T16-15-22.294Z is finished.
  The final status was:

  18 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/gen-flwor-opt-2013-02-26T20-53-20.631Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/gen-flwor-opt into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message):
  Validation queue job gen-flwor-opt-2013-02-26T20-53-20.631Z is finished.
  The final status was:

  18 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Attempt to merge into lp:zorba failed due to conflicts:

text conflict in ChangeLog

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/gen-flwor-opt-2013-02-28T11-32-29.632Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/gen-flwor-opt into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message):
  Validation queue job gen-flwor-opt-2013-02-28T11-32-29.632Z is finished.
  The final status was:

  2 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/gen-flwor-opt-2013-02-28T12-50-12.909Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/gen-flwor-opt into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message):
  Validation queue job gen-flwor-opt-2013-02-28T12-50-12.909Z is finished.
  The final status was:

  2 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[Markos Zaharioudakis (markos-za)]

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/gen-flwor-opt-2013-02-28T20-52-18.552Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~zorba-coders/zorba/gen-flwor-opt into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:275 (message):
  Validation queue job gen-flwor-opt-2013-02-28T20-52-18.552Z is finished.
  The final status was:

  5 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[lp:~zorba-coders/zorba/gen-flwor-opt]

10676. By Markos Zaharioudakis on 2013-02-28

bug fix

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/gen-flwor-opt-2013-02-28T21-55-21.647Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job gen-flwor-opt-2013-02-28T21-55-21.647Z is finished. The final status was:

All tests succeeded!

=== modified file 'ChangeLog'
--- ChangeLog	2013-02-28 05:14:35 +0000
+++ ChangeLog	2013-02-28 21:53:23 +0000
@@ -19,11 +19,14 @@
   * Extended optimization rules for subsequence function (pushing $startingLoc
     parameter into collection skip if $sourceSeq parameter is from a
     collection).
+  * Extented index join  rule to general flwor expressions. 
 
 Bug Fixes/Other Changes:
   * Fixed bug #1124273 (xqdoc crash because of annotation literals)
   * Fixed bug #867027 (XQST0059 error messages inconsistent)
   * Fixed bug #1095889 (Improve error message for xml-parsing error).
+  * Fixed bug in index join rule (copy var ids after cloning index domain expr).
+  * Added missing wrapper expressions around some variable references.
   * Fixed bug in the throwing of error XQTY0086 during node construction.
   * Fixed bug #1122396 (Associate origin/destination URI to I/O stream)
   * Fixed bug #1111786 (xml/json parse error location in catch clause).

=== modified file 'src/compiler/codegen/plan_visitor.cpp'
--- src/compiler/codegen/plan_visitor.cpp	2013-02-17 00:59:54 +0000
+++ src/compiler/codegen/plan_visitor.cpp	2013-02-28 21:53:23 +0000
@@ -810,7 +810,7 @@
         ZORBA_ASSERT(clauseVarMap->find_var(&var) < 0);
 
         if (flworExpr == clauseVarMap->theClause->get_flwor_expr() &&
-            ((clauseVarMap->theClause->get_kind() == flwor_clause::order_clause &&
+            ((clauseVarMap->theClause->get_kind() == flwor_clause::orderby_clause &&
               flworExpr->is_general()) ||
              clauseVarMap->theClause->get_kind() == flwor_clause::materialize_clause))
         {
@@ -892,10 +892,12 @@
 {
   CODEGEN_TRACE_IN("");
 
-  bool isGeneral = v.is_general();
-
   csize numClauses = v.num_clauses();
 
+  bool isGeneral = v.compute_is_general();
+
+  v.set_general(isGeneral);
+    
   if (v.is_sequential())
   {
     if (!isGeneral)
@@ -962,6 +964,7 @@
 
     if (isGeneral)
     {
+      static_context* sctx = v.get_sctx();
       std::vector<OrderModifier> modifiers;
       std::vector<expr*> orderingExprs;
 
@@ -971,7 +974,6 @@
       while (i < numClauses)
       {
         const flwor_clause* c = v.get_clause(i);
-
         flwor_clause::ClauseKind k = c->get_kind();
 
         switch (k)
@@ -990,42 +992,32 @@
               ++numForClauses;
           }
 
-          if (domExpr->is_sequential())
+          if (domExpr->is_sequential() &&
+              (k == flwor_clause::for_clause ||
+               k == flwor_clause::window_clause ||
+               numForClauses > 0))
           {
-            if (k == flwor_clause::for_clause ||
-                k == flwor_clause::window_clause ||
-                numForClauses > 0)
-            {
-              if (i > 0 &&
-                  v.get_clause(i-1)->get_kind() != flwor_clause::order_clause &&
-                  v.get_clause(i-1)->get_kind() != flwor_clause::groupby_clause)
-              {
-                orderby_clause* mat = theCCB->theEM->
-                create_orderby_clause(v.get_sctx(),
-                                      c->get_loc(),
-                                      true,
-                                      modifiers,
-                                      orderingExprs);
-
-                v.add_clause(i, mat);
-                ++i;
-                ++numClauses;
-              }
-
-              if (i == numClauses -1 ||
-                  (i < numClauses - 1 &&
-                   v.get_clause(i+1)->get_kind() != flwor_clause::groupby_clause))
-              {
-                orderby_clause* mat = theCCB->theEM->
-                create_orderby_clause(v.get_sctx(),
-                                      c->get_loc(),
-                                      true,
-                                      modifiers,
-                                      orderingExprs);
-
-                v.add_clause(i+1, mat);
-                ++numClauses;
-              }
+            if (i > 0 &&
+                v.get_clause(i-1)->get_kind() != flwor_clause::orderby_clause &&
+                v.get_clause(i-1)->get_kind() != flwor_clause::groupby_clause)
+            {
+              orderby_clause* mat = theCCB->theEM->
+              create_orderby_clause(sctx, c->get_loc(), true, modifiers, orderingExprs);
+
+              v.add_clause(i, mat);
+              ++i;
+              ++numClauses;
+            }
+
+            if (i == numClauses -1 ||
+                (i < numClauses - 1 &&
+                 v.get_clause(i+1)->get_kind() != flwor_clause::groupby_clause))
+            {
+              orderby_clause* mat = theCCB->theEM->
+              create_orderby_clause(sctx, c->get_loc(), true, modifiers, orderingExprs);
+
+              v.add_clause(i+1, mat);
+              ++numClauses;
             }
           }
 
@@ -1033,7 +1025,7 @@
         }
         case flwor_clause::where_clause:
         case flwor_clause::groupby_clause:
-        case flwor_clause::order_clause:
+        case flwor_clause::orderby_clause:
         case flwor_clause::count_clause:
         {
           break;
@@ -1048,11 +1040,11 @@
       const flwor_clause* lastClause = v.get_clause(v.num_clauses()-1);
 
       if (v.get_return_expr()->is_sequential() &&
-          lastClause->get_kind() != flwor_clause::order_clause &&
+          lastClause->get_kind() != flwor_clause::orderby_clause &&
           lastClause->get_kind() != flwor_clause::groupby_clause)
       {
         orderby_clause* mat = theCCB->theEM->
-        create_orderby_clause(v.get_sctx(),
+        create_orderby_clause(sctx,
                               v.get_return_expr()->get_loc(),
                               true,
                               modifiers,
@@ -1123,7 +1115,7 @@
       break;
     }
 
-    case flwor_clause::order_clause:
+    case flwor_clause::orderby_clause:
     {
       const orderby_clause* oc = reinterpret_cast<const orderby_clause*>(c);
 
@@ -1270,7 +1262,7 @@
     break;
   }
 
-  case flwor_clause::order_clause:
+  case flwor_clause::orderby_clause:
   case flwor_clause::materialize_clause:
   {
     break;
@@ -1625,7 +1617,7 @@
   //
   // ORDERBY
   //
-  else if (c.get_kind() == flwor_clause::order_clause)
+  else if (c.get_kind() == flwor_clause::orderby_clause)
   {
     csize numVars = clauseVarMap->theVarRebinds.size();
     csize numForVars = 0;
@@ -1807,7 +1799,7 @@
     //
     // ORDERBY
     //
-    case flwor_clause::order_clause:
+    case flwor_clause::orderby_clause:
     {
       const orderby_clause* obc = static_cast<const orderby_clause*>(&c);
       unsigned numColumns = obc->num_columns();

=== modified file 'src/compiler/expression/expr_base.cpp'
--- src/compiler/expression/expr_base.cpp	2013-01-11 22:58:12 +0000
+++ src/compiler/expression/expr_base.cpp	2013-02-28 21:53:23 +0000
@@ -974,7 +974,7 @@
 
         break;
       }
-      case flwor_clause::order_clause:
+      case flwor_clause::orderby_clause:
       {
         if (found)
           return false;
@@ -1161,6 +1161,22 @@
 
 
 /*******************************************************************************
+
+********************************************************************************/
+expr* expr::skip_wrappers() const
+{
+  const expr* e = this;
+
+  while (e->get_expr_kind() == wrapper_expr_kind)
+  {
+    e = static_cast<const wrapper_expr*>(e)->get_input();
+  }
+
+  return const_cast<expr*>(e);
+}
+
+
+/*******************************************************************************
   If "this" is a const expr that returns a qname, evaluate and return this
   qname. This method is used to extract the qname from the expression that is
   given as an arg to collection and index related functions.

=== modified file 'src/compiler/expression/expr_base.h'
--- src/compiler/expression/expr_base.h	2013-01-11 22:58:12 +0000
+++ src/compiler/expression/expr_base.h	2013-02-28 21:53:23 +0000
@@ -398,6 +398,8 @@
 
   const store::Item* getQName() const;
 
+  expr* skip_wrappers() const;
+
   void clear_annotations();
 
   xqtref_t get_return_type_with_empty_input(const expr* input) const;

=== modified file 'src/compiler/expression/expr_clone.cpp'
--- src/compiler/expression/expr_clone.cpp	2013-01-11 22:58:12 +0000
+++ src/compiler/expression/expr_clone.cpp	2013-02-28 21:53:23 +0000
@@ -602,8 +602,17 @@
 
     checked_vector<expr*> seq2;
     for (csize i = 0; i < e->theArgs.size(); ++i)
+    {
       seq2.push_back(e->theArgs[i]->clone(udf, subst));
 
+      if (e->theArgs[i]->get_expr_kind() == var_decl_expr_kind)
+      {
+        var_decl_expr* varDeclExpr = static_cast<var_decl_expr*>(e->theArgs[i]);
+        var_expr* varExpr = varDeclExpr->get_var_expr();
+        varExpr->set_block_expr(e);
+      }
+    }
+
     newExpr = theCCB->theEM->
     create_block_expr(theSctx, udf, theLoc, true, seq2, NULL);
 

=== modified file 'src/compiler/expression/expr_iter.cpp'
--- src/compiler/expression/expr_iter.cpp	2013-01-11 22:58:12 +0000
+++ src/compiler/expression/expr_iter.cpp	2013-02-28 21:53:23 +0000
@@ -189,7 +189,7 @@
           return;
         }
 
-        case flwor_clause::order_clause:
+        case flwor_clause::orderby_clause:
         {
           orderby_clause* oc = static_cast<orderby_clause *>(c);
 
@@ -384,7 +384,7 @@
         }
       }
 
-      else if (c->get_kind() == flwor_clause::order_clause)
+      else if (c->get_kind() == flwor_clause::orderby_clause)
       {
         oc = static_cast<orderby_clause *>(c);
 

=== modified file 'src/compiler/expression/expr_put.cpp'
--- src/compiler/expression/expr_put.cpp	2013-01-12 22:10:40 +0000
+++ src/compiler/expression/expr_put.cpp	2013-02-28 21:53:23 +0000
@@ -337,8 +337,10 @@
     }
     case flwor_clause::count_clause:
     {
-      os << indent << "COUNT $";
-      put_qname(static_cast<const count_clause *>(&c)->get_var()->get_name(), os);
+      const count_clause* cc = static_cast<const count_clause *>(&c);
+      os << indent << "COUNT " << expr_addr(cc) << " ";
+      put_qname(cc->get_var()->get_name(), os);
+      os << expr_addr(cc->get_var());
       os << endl;
       break;
     }
@@ -358,7 +360,7 @@
       static_cast<const groupby_clause *>(&c)->put(os);
       break;
     }
-    case flwor_clause::order_clause:
+    case flwor_clause::orderby_clause:
     {
       static_cast<const orderby_clause *>(&c)->put(os);
       break;

=== modified file 'src/compiler/expression/flwor_expr.cpp'
--- src/compiler/expression/flwor_expr.cpp	2012-12-21 03:45:19 +0000
+++ src/compiler/expression/flwor_expr.cpp	2013-02-28 21:53:23 +0000
@@ -653,7 +653,7 @@
     const std::vector<OrderModifier>& modifiers,
     const std::vector<expr*>& orderingExprs)
   :
-  flwor_clause(sctx, ccb, loc, flwor_clause::order_clause),
+  flwor_clause(sctx, ccb, loc, flwor_clause::orderby_clause),
   theStableOrder(stable),
   theModifiers(modifiers),
   theOrderingExprs(orderingExprs)
@@ -831,6 +831,33 @@
 /*******************************************************************************
 
 ********************************************************************************/
+void flwor_expr::remove_clause(flwor_clause* c, csize posHint)
+{
+  if (posHint < theClauses.size() && theClauses[posHint] == c)
+    return remove_clause(posHint);
+  
+  csize numClauses = theClauses.size();
+  csize i = 0;
+  for (; i < numClauses; ++i)
+  {
+    if (theClauses[i] != c)
+      continue;
+
+    if (theClauses[i]->theFlworExpr == this)
+      theClauses[i]->theFlworExpr = NULL;
+
+    theClauses.erase(theClauses.begin() + i);
+
+    return;
+  }
+
+  assert(i < numClauses);
+}
+
+
+/*******************************************************************************
+
+********************************************************************************/
 void flwor_expr::add_clause(flwor_clause* c, bool computeScriptingKind)
 {
   theClauses.push_back(c);
@@ -907,24 +934,6 @@
 /*******************************************************************************
   For simple flwor only.
 ********************************************************************************/
-void flwor_expr::remove_where_clause()
-{
-  csize numClauses = num_clauses();
-  for (csize i = 0; i < numClauses; ++i)
-  {
-    if (theClauses[i]->get_kind() == flwor_clause::where_clause)
-    {
-      theClauses[i]->theFlworExpr = NULL;
-      theClauses.erase(theClauses.begin() + i);
-      return;
-    }
-  }
-}
-
-
-/*******************************************************************************
-  For simple flwor only.
-********************************************************************************/
 expr* flwor_expr::get_where() const
 {
   csize numClauses = num_clauses();
@@ -962,7 +971,7 @@
   csize numClauses = num_clauses();
   for (csize i = 0; i < numClauses; ++i)
   {
-    if (theClauses[i]->get_kind() == flwor_clause::order_clause)
+    if (theClauses[i]->get_kind() == flwor_clause::orderby_clause)
       return reinterpret_cast<orderby_clause*>(theClauses[i]);
   }
 
@@ -995,22 +1004,34 @@
 /*******************************************************************************
 
 ********************************************************************************/
-long flwor_expr::defines_variable(const var_expr* v) const
+bool flwor_expr::defines_var(const var_expr* v) const
+{
+  return v->get_flwor_clause()->get_flwor_expr() == this;
+}
+
+
+/*******************************************************************************
+
+********************************************************************************/
+bool flwor_expr::get_var_pos(const var_expr* v, csize& pos) const
 {
   const flwor_clause* varClause = v->get_flwor_clause();
 
   if (varClause == NULL)
-    return -1;
+    return false;
 
   csize numClauses = theClauses.size();
 
   for (csize i = 0; i < numClauses; ++i)
   {
     if (theClauses[i] == varClause)
-      return i;
+    {
+      pos = i;
+      return true;
+    }
   }
 
-  return -1;
+  return false;
 }
 
 
@@ -1115,9 +1136,9 @@
 ********************************************************************************/
 void flwor_expr::compute_scripting_kind()
 {
-  ulong numClauses = num_clauses();
+  csize numClauses = num_clauses();
 
-  for (ulong i = 0; i < numClauses; ++i)
+  for (csize i = 0; i < numClauses; ++i)
   {
     const flwor_clause* c = theClauses[i];
     flwor_clause::ClauseKind k = c->get_kind();
@@ -1163,5 +1184,82 @@
 }
 
 
+/*******************************************************************************
+
+********************************************************************************/
+bool flwor_expr::compute_is_general()
+{
+  bool has_where = false;
+  bool has_order = false;
+  bool has_group = false;
+
+  csize numClauses = num_clauses();
+
+  for (csize i = 0; i < numClauses; ++i)
+  {
+    const flwor_clause* c = get_clause(i);
+
+    switch (c->get_kind())
+    {
+    case flwor_clause::for_clause:
+    case flwor_clause::let_clause:
+    {
+      if (has_group || has_where || has_order)
+        return true;
+
+      const forlet_clause* flc = static_cast<const forlet_clause*>(c);
+
+      if (flc->is_allowing_empty())
+        return true;
+
+      break;
+    }
+    case flwor_clause::window_clause:
+    {
+      return true;
+    }
+    case flwor_clause::where_clause:
+    {
+      if (has_where || has_group || has_order)
+        return true;
+
+      has_where = true;
+      break;
+    }
+    case flwor_clause::orderby_clause:
+    {
+      if (has_order)
+        return true;
+
+      has_order = true;
+      break;
+    }
+    case flwor_clause::groupby_clause:
+    {
+      if (has_group || has_order)
+        return true;
+
+      has_group = true;
+      break;
+    }
+    case flwor_clause::count_clause:
+    {
+      return true;
+    }
+    case flwor_clause::materialize_clause:
+    {
+      break;
+    }
+    default:
+    {
+      ZORBA_ASSERT(false);
+    }
+    }
+  }
+
+  return false;
+}
+
+
 } // namespace zorba
 /* vim:set et sw=2 ts=2: */

=== modified file 'src/compiler/expression/flwor_expr.h'
--- src/compiler/expression/flwor_expr.h	2013-01-12 22:10:40 +0000
+++ src/compiler/expression/flwor_expr.h	2013-02-28 21:53:23 +0000
@@ -56,7 +56,7 @@
     let_clause,
     window_clause,
     groupby_clause,
-    order_clause,
+    orderby_clause,
     count_clause,
     where_clause,
     materialize_clause
@@ -669,6 +669,8 @@
 
   void set_general(bool v) { theKind = (v ? gflwor_expr_kind : flwor_expr_kind); }
 
+  bool compute_is_general();
+
   expr* get_return_expr() const { return theReturnExpr; }
 
   expr** get_return_expr_ref() { return &theReturnExpr; }
@@ -693,13 +695,17 @@
 
   void remove_clause(csize pos);
 
+  void remove_clause(flwor_clause* c, csize posHint);
+
   flwor_clause* get_clause(csize i) const;
 
   clause_list_t::const_iterator clause_begin() const { return theClauses.begin(); }
 
   clause_list_t::const_iterator clause_end() const { return theClauses.end(); }
 
-  long defines_variable(const var_expr* v) const;
+  bool defines_var(const var_expr* v) const;
+
+  bool get_var_pos(const var_expr* v, csize& pos) const;
 
   void get_vars(std::vector<var_expr*>& vars) const;
 
@@ -707,7 +713,6 @@
   // removed eventually.
   expr* get_where() const;
   void set_where(expr* e);
-  void remove_where_clause();
   groupby_clause* get_group_clause() const;
   orderby_clause* get_order_clause() const;
   csize num_forlet_clauses();

=== modified file 'src/compiler/expression/script_exprs.cpp'
--- src/compiler/expression/script_exprs.cpp	2012-12-06 22:49:35 +0000
+++ src/compiler/expression/script_exprs.cpp	2013-02-28 21:53:23 +0000
@@ -65,7 +65,7 @@
 }
 
 
-void block_expr::add_at(csize pos, expr* arg)
+void block_expr::add(csize pos, expr* arg)
 {
   assert(arg->get_expr_kind() != var_set_expr_kind);
 
@@ -102,6 +102,8 @@
 
       var_expr* varExpr = varDeclExpr->get_var_expr();
 
+      varExpr->set_block_expr(this);
+
       expr* initExpr = varDeclExpr->get_init_expr();
 
       if (initExpr != NULL)
@@ -167,6 +169,27 @@
 }
 
 
+bool block_expr::get_var_pos(const var_expr* v, csize& pos) const
+{
+  csize numArgs = theArgs.size();
+  for (csize i = 0; i < numArgs; ++i)
+  {
+    if (theArgs[i]->get_expr_kind() == var_decl_expr_kind)
+    {
+      var_decl_expr* decl = static_cast<var_decl_expr*>(theArgs[i]);
+
+      if (decl->get_var_expr() == v)
+      {
+        pos = i;
+        return true;
+      }
+    }
+  }
+
+  return false;
+}
+
+
 /*******************************************************************************
 
 ********************************************************************************/

=== modified file 'src/compiler/expression/script_exprs.h'
--- src/compiler/expression/script_exprs.h	2012-12-06 22:49:35 +0000
+++ src/compiler/expression/script_exprs.h	2013-02-28 21:53:23 +0000
@@ -123,7 +123,7 @@
 public:
   ~block_expr();
 
-  void add_at(csize pos, expr* arg);
+  void add(csize pos, expr* arg);
 
   csize size() const { return theArgs.size(); }
 
@@ -131,6 +131,8 @@
 
   expr* operator[](csize i) { return theArgs[i]; }
 
+  bool get_var_pos(const var_expr* v, csize& pos) const;
+
   void accept(expr_visitor&);
 
   std::ostream& put(std::ostream&) const;

=== modified file 'src/compiler/expression/var_expr.h'
--- src/compiler/expression/var_expr.h	2012-12-12 16:15:04 +0000
+++ src/compiler/expression/var_expr.h	2013-02-28 21:53:23 +0000
@@ -29,6 +29,7 @@
 class var_expr;
 class VarInfo;
 class var_set_expr;
+class block_expr;
 
 
 /******************************************************************************
@@ -74,6 +75,12 @@
   ----------------
   The type, if any, specified in the declaration of the variable
 
+  theBlockExpr:
+  -------------
+  If this is a prolog or a local var, theBlockExpr points to the block expr
+  where the var is declared at (prolog and local vars are always declared 
+  in block exprs).
+
   theFlworClause:
   ---------------
   If this is a var declared in flwor clause, theFlworClause points to the
@@ -159,6 +166,8 @@
 
   xqtref_t              theDeclaredType;
 
+  block_expr          * theBlockExpr;
+
   flwor_clause        * theFlworClause;
 
   copy_clause         * theCopyClause;
@@ -228,6 +237,10 @@
 
   void set_type(xqtref_t t);
 
+  void set_block_expr(const block_expr* b) { theBlockExpr = const_cast<block_expr*>(b); }
+
+  block_expr* get_block_expr() const { return theBlockExpr; }
+
   void set_flwor_clause(flwor_clause* c) { theFlworClause = c; }
 
   flwor_clause* get_flwor_clause() const { return theFlworClause; }

=== modified file 'src/compiler/parsetree/parsenodes.cpp'
--- src/compiler/parsetree/parsenodes.cpp	2013-01-19 20:47:55 +0000
+++ src/compiler/parsetree/parsenodes.cpp	2013-02-28 21:53:23 +0000
@@ -1317,15 +1317,15 @@
 
 
 FLWORExpr::FLWORExpr(
-    const QueryLoc& loc_,
+    const QueryLoc& loc,
     rchandle<FLWORClauseList> clauses_,
     rchandle<exprnode> ret_,
     const QueryLoc& return_loc_,
     bool force_general)
   :
-  exprnode (loc_),
-  clauses (clauses_),
-  return_val_h (ret_),
+  exprnode(loc),
+  clauses(clauses_),
+  return_val_h(ret_),
   return_location(return_loc_)
 {
   for (unsigned i = 0; i < clauses->size (); i++)
@@ -1342,7 +1342,7 @@
     }
   }
 
-  compute_general ();
+  compute_general();
 
   if (force_general)
     general = true;
@@ -1435,7 +1435,8 @@
     }
     else if (typeid (*cp) == typeid (GroupByClause))
     {
-      if (has_group || has_order) non_10 = general = true;
+      non_10 = true;
+      if (has_group || has_order) general = true;
       has_group = true;
     }
     else if (typeid (*cp) == typeid (CountClause))

=== modified file 'src/compiler/parsetree/parsenodes.h'
--- src/compiler/parsetree/parsenodes.h	2013-01-19 20:47:55 +0000
+++ src/compiler/parsetree/parsenodes.h	2013-02-28 21:53:23 +0000
@@ -1895,9 +1895,9 @@
         const QueryLoc& return_loc_,
         bool force_general = false);
 
-  bool is_general () const { return general; }
+  bool is_general() const { return general; }
 
-  bool is_non_10 () const { return non_10; }
+  bool is_non_10() const { return non_10; }
 
   const QueryLoc& get_return_location() const { return return_location; }
 
@@ -1914,7 +1914,7 @@
   void accept(parsenode_visitor&) const;
 
 protected:
-  void compute_general ();
+  void compute_general();
 };
 
 

=== modified file 'src/compiler/rewriter/framework/rewriter_context.cpp'
--- src/compiler/rewriter/framework/rewriter_context.cpp	2012-10-26 18:24:00 +0000
+++ src/compiler/rewriter/framework/rewriter_context.cpp	2013-02-28 21:53:23 +0000
@@ -44,10 +44,7 @@
   theUDF(udf),
   theMessage(msg),
   m_tempvarCounter(0),
-  theIsInOrderedMode(orderedMode),
-  theVarIdMap(NULL),
-  theIdVarMap(NULL),
-  theExprVarsMap(NULL)
+  theIsInOrderedMode(orderedMode)
 {
   theForSerializationOnly = theCCB->theConfig.for_serialization_only;
 
@@ -64,11 +61,6 @@
 
 RewriterContext::~RewriterContext()
 {
-  delete theVarIdMap;
-
-  delete theIdVarMap;
-
-  delete theExprVarsMap;
 }
 
 

=== modified file 'src/compiler/rewriter/framework/rewriter_context.h'
--- src/compiler/rewriter/framework/rewriter_context.h	2012-10-26 18:24:00 +0000
+++ src/compiler/rewriter/framework/rewriter_context.h	2013-02-28 21:53:23 +0000
@@ -32,10 +32,6 @@
 
 class user_function;
 
-typedef std::map<var_expr *, ulong> VarIdMap;
-typedef std::vector<var_expr*> IdVarMap;
-typedef std::map<const expr *, DynamicBitset> ExprVarsMap;
-
 
 /*******************************************************************************
 
@@ -43,32 +39,6 @@
   --------
   The root node of the expr DAG that is going to be optimized using this context.
 
-  theVarIdMap:
-  ------------
-  Maps a var_expr to its unique "prefix" id. The "prefix" id has the following
-  property: for 2 vars v1 and v2, v1 is defined before v2 if and only if
-  prefix-id(v1) < prefix-id(v2). See index_flwor_vars() function in
-  tools/expr_tools.cpp for more details.
-
-  theIdVarMap:
-  ------------
-  This is the reverse mapping of theVarIdMap.
-
-  theExprVarsMap:
-  ---------------
-  An entry into this map maps an expr to the variables that are referenced by
-  that expr and/or its sub-exprs. (Note: given that the domain expr of a var
-  $x is not considered a sub-expr of $x, if $x is referenced by an expr E and
-  the domain expr of $x references another var $y, $y is NOT considered to be
-  referenced by E). Only variables that have been assigned a prolog id (i.e.,
-  the ones that appear in theVarIdMap) are considered. The set of vars referenced
-  by an expr is implemented by a bitset that is indexed by prolog var ids and
-  whose size (in number of bits) is equal to the size of theVarIdMap.
-
-  theFlworStack:
-  --------------
-  The current "in-scope" flwor exprs, ie., flwor exprs that the rule has
-  entered but not exited yet.
 ********************************************************************************/
 class RewriterContext
 {
@@ -89,12 +59,6 @@
 
   bool                         theIsInOrderedMode;
 
-  VarIdMap                   * theVarIdMap;
-  IdVarMap                   * theIdVarMap;
-  ExprVarsMap                * theExprVarsMap;
-  std::vector<expr*>           theFlworStack;
-  std::vector<bool>            theInReturnClause;
-
 public:
   RewriterContext(
       CompilerCB* cb,

=== modified file 'src/compiler/rewriter/rewriters/default_optimizer.cpp'
--- src/compiler/rewriter/rewriters/default_optimizer.cpp	2012-12-14 18:01:41 +0000
+++ src/compiler/rewriter/rewriters/default_optimizer.cpp	2013-02-28 21:53:23 +0000
@@ -19,11 +19,13 @@
 #include "compiler/rewriter/rules/ruleset.h"
 #include "compiler/rewriter/rules/fold_rules.h"
 #include "compiler/rewriter/rules/index_matching_rule.h"
+#include "compiler/rewriter/rules/index_join_rule.h"
 #include "compiler/rewriter/rewriters/common_rewriter.h"
 #include "compiler/rewriter/rewriters/default_optimizer.h"
 #include "compiler/rewriter/tools/expr_tools.h"
+
 #include "compiler/xqddf/value_index.h"
-//#include "compiler/rewriter/tools/udf_graph.h"
+
 #include "compiler/api/compilercb.h"
 
 #include "functions/udf.h"
@@ -32,7 +34,6 @@
 
 #include "context/static_context.h"
 
-//#include "store/api/store.h"
 
 
 namespace zorba
@@ -225,31 +226,12 @@
   // Index Joins
   if (Properties::instance()->inferJoins())
   {
-    IndexJoinRule rule;
     bool local_modified = false;
 
-    rCtx.theVarIdMap = new VarIdMap;
-    rCtx.theIdVarMap = new IdVarMap;
-    rCtx.theExprVarsMap = new ExprVarsMap;
-
-    ulong numVars = 0;
-    expr_tools::index_flwor_vars(rCtx.getRoot(),
-                                 numVars,
-                                 *rCtx.theVarIdMap,
-                                 rCtx.theIdVarMap);
+    IndexJoinRule rule(&rCtx);
 
     do
     {
-      assert(rCtx.theFlworStack.empty());
-
-      rCtx.theExprVarsMap->clear();
-
-      DynamicBitset freeset(numVars);
-      expr_tools::build_expr_to_vars_map(rCtx.getRoot(),
-                                         *rCtx.theVarIdMap,
-                                         freeset,
-                                         *rCtx.theExprVarsMap);
-
       local_modified = false;
 
       expr* e = rule.apply(rCtx, rCtx.getRoot(), local_modified);

=== modified file 'src/compiler/rewriter/rules/flwor_rules.cpp'
--- src/compiler/rewriter/rules/flwor_rules.cpp	2012-12-29 20:25:12 +0000
+++ src/compiler/rewriter/rules/flwor_rules.cpp	2013-02-28 21:53:23 +0000
@@ -102,6 +102,10 @@
     theVarExpr(var),
     theSubstExpr(subst)
   {
+    while (theSubstExpr->get_expr_kind() == wrapper_expr_kind)
+    {
+      theSubstExpr = static_cast<wrapper_expr*>(theSubstExpr)->get_input();
+    }
   }
 
   expr* apply(RewriterContext& rCtx, expr* node, bool& modified);
@@ -116,7 +120,9 @@
 
   while (!iter.done())
   {
-    if (**iter == theVarExpr)
+    expr* childExpr = **iter;
+
+    if (childExpr == theVarExpr)
     {
 #if 0
       std::vector<expr*>::iterator ite = thePath.begin();
@@ -134,7 +140,13 @@
     }
     else
     {
-      apply(rCtx, **iter, modified);
+      apply(rCtx, childExpr, modified);
+    }
+
+    if (childExpr->isNonDiscardable() && !node->isNonDiscardable())
+    {
+      node->setNonDiscardable(ANNOTATION_TRUE);
+      modified = true;
     }
 
     iter.next();
@@ -452,7 +464,7 @@
       theFlwor->remove_clause(0);
       continue;
     }
-    else if (clause->get_kind() == flwor_clause::order_clause)
+    else if (clause->get_kind() == flwor_clause::orderby_clause)
     {
       theFlwor->remove_clause(0);
       continue;
@@ -670,7 +682,7 @@
 
         break;
       }
-      case flwor_clause::order_clause:
+      case flwor_clause::orderby_clause:
       {
         orderby_clause* cl = static_cast<orderby_clause*>(clause);
 
@@ -1518,18 +1530,18 @@
                               *rtm.INTEGER_TYPE_QUESTION,
                               posExpr->get_loc()))
       {
-        VarIdMap varidMap;
-        ulong numFlworVars = 0;
+        expr_tools::VarIdMap varidMap;
+        csize numFlworVars = 0;
         expr_tools::index_flwor_vars(flworExpr, numFlworVars, varidMap, NULL);
         
         DynamicBitset varset(numFlworVars);
-        ExprVarsMap exprVarMap;
+        expr_tools::ExprVarsMap exprVarMap;
         expr_tools::build_expr_to_vars_map(posExpr, varidMap, varset, exprVarMap);
         
         var_expr* forVar = forClause->get_var();
         ulong forVarId = varidMap[forVar];
 
-        std::vector<ulong> posExprVarIds;
+        std::vector<csize> posExprVarIds;
         exprVarMap[posExpr].getSet(posExprVarIds);
 
         csize numPosExprVars = posExprVarIds.size();
@@ -1588,7 +1600,7 @@
           
           if (c->get_kind() == flwor_clause::where_clause ||
               c->get_kind() == flwor_clause::groupby_clause ||
-              c->get_kind() == flwor_clause::order_clause)
+              c->get_kind() == flwor_clause::orderby_clause)
           {
             goto next1;
           }
@@ -1602,7 +1614,7 @@
         const flwor_clause* c = returnFlwor->get_clause(i);
         
         if (c->get_kind() == flwor_clause::groupby_clause ||
-            c->get_kind() == flwor_clause::order_clause)
+            c->get_kind() == flwor_clause::orderby_clause)
         {
           goto next1;
         }
@@ -1634,7 +1646,7 @@
       
       if (c->get_kind() == flwor_clause::let_clause)
       {
-        expr* domainExpr = static_cast<let_clause*>(c)->get_expr();
+        expr* domainExpr = static_cast<let_clause*>(c)->get_expr()->skip_wrappers();
         
         if (domainExpr->get_expr_kind() == flwor_expr_kind &&
             !domainExpr->is_sequential())
@@ -1670,7 +1682,7 @@
       else if (c->get_kind() == flwor_clause::for_clause &&
                static_cast<for_clause*>(c)->get_pos_var() == NULL)
       {
-        expr* domainExpr = static_cast<for_clause*>(c)->get_expr();
+        expr* domainExpr = static_cast<for_clause*>(c)->get_expr()->skip_wrappers();
 
         if (domainExpr->get_expr_kind() == flwor_expr_kind &&
             !domainExpr->is_sequential())

=== modified file 'src/compiler/rewriter/rules/fold_rules.cpp'
--- src/compiler/rewriter/rules/fold_rules.cpp	2013-01-08 11:07:41 +0000
+++ src/compiler/rewriter/rules/fold_rules.cpp	2013-02-28 21:53:23 +0000
@@ -161,8 +161,9 @@
         
     if (!f->isUdf())
     {
-      if (FunctionConsts::FN_ERROR_0 <= f->getKind() &&
-          f->getKind() <= FunctionConsts::FN_TRACE_2)
+      if (f->getKind() == FunctionConsts::OP_CREATE_INTERNAL_INDEX_2 ||
+          (FunctionConsts::FN_ERROR_0 <= f->getKind() &&
+           f->getKind() <= FunctionConsts::FN_TRACE_2))
       {
         curNonDiscardable = ANNOTATION_TRUE_FIXED;
         curUnfoldable = ANNOTATION_TRUE_FIXED;
@@ -562,7 +563,7 @@
   - Replace empty(E) with true if the return type of E is the emtpy sequence, or
     false if the return type of E has QUANT_ONE or QUANT_PLUS.
   - Replace exists(E) with false if the return type of E is the emtpy sequence, or
-    truee if the return type of E has QUANT_ONE or QUANT_PLUS.
+    true if the return type of E has QUANT_ONE or QUANT_PLUS.
 
   Replace EBV(E) with true if the return type of E is subtype on node()+ and E
   is not NONDISCARDABLE.

=== modified file 'src/compiler/rewriter/rules/hoist_rules.cpp'
--- src/compiler/rewriter/rules/hoist_rules.cpp	2012-12-12 16:15:04 +0000
+++ src/compiler/rewriter/rules/hoist_rules.cpp	2013-02-28 21:53:23 +0000
@@ -41,15 +41,15 @@
 static bool hoist_expressions(
     RewriterContext&,
     expr*,
-    const VarIdMap&,
-    const ExprVarsMap&,
+    const expr_tools::VarIdMap&,
+    const expr_tools::ExprVarsMap&,
     struct PathHolder*);
 
 static expr* try_hoisting(
     RewriterContext&,
     expr*,
-    const VarIdMap&,
-    const ExprVarsMap&,
+    const expr_tools::VarIdMap&,
+    const expr_tools::ExprVarsMap&,
     struct PathHolder*);
 
 static bool non_hoistable (const expr*);
@@ -58,7 +58,7 @@
 
 static bool contains_var(
     var_expr*,
-    const VarIdMap&,
+    const expr_tools::VarIdMap&,
     const DynamicBitset&);
 
 static bool is_enclosed_expr(const expr*);
@@ -98,13 +98,22 @@
 {
   assert(node == rCtx.getRoot());
 
-  ulong numVars = 0;
-  VarIdMap varmap;
+  csize numVars = 0;
+  expr_tools::VarIdMap varmap;
 
   expr_tools::index_flwor_vars(node, numVars, varmap, NULL);
 
-  ExprVarsMap freevarMap;
-  DynamicBitset freeset(numVars);
+  /*
+  expr_tools::VarIdMap::const_iterator ite = varmap.begin();
+  expr_tools::VarIdMap::const_iterator end = varmap.end();
+  for (; ite != end; ++ite)
+  {
+    std::cerr << "[ " << ite->first << " --> " << ite->second << "]" << std::endl;
+  }
+  */
+
+  expr_tools::ExprVarsMap freevarMap;
+  DynamicBitset freeset(numVars+1);
   expr_tools::build_expr_to_vars_map(node, varmap, freeset, freevarMap);
 
   PathHolder root;
@@ -128,8 +137,8 @@
 static bool hoist_expressions(
     RewriterContext& rCtx,
     expr* e,
-    const VarIdMap& varmap,
-    const ExprVarsMap& freevarMap,
+    const expr_tools::VarIdMap& varmap,
+    const expr_tools::ExprVarsMap& freevarMap,
     struct PathHolder* path)
 {
   bool status = false;
@@ -349,8 +358,8 @@
 static expr* try_hoisting(
     RewriterContext& rCtx,
     expr* e,
-    const VarIdMap& varmap,
-    const ExprVarsMap& freevarMap,
+    const expr_tools::VarIdMap& varmap,
+    const expr_tools::ExprVarsMap& freevarMap,
     struct PathHolder* path)
 {
   if (non_hoistable(e) || e->constructsNodes())
@@ -364,7 +373,7 @@
 
   assert(udf == rCtx.theUDF);
 
-  std::map<const expr*, DynamicBitset>::const_iterator fvme = freevarMap.find(e);
+  expr_tools::ExprVarsMap::const_iterator fvme = freevarMap.find(e);
   ZORBA_ASSERT(fvme != freevarMap.end());
   const DynamicBitset& varset = fvme->second;
 
@@ -534,7 +543,7 @@
 ********************************************************************************/
 static bool contains_var(
     var_expr* v,
-    const VarIdMap& varmap,
+    const expr_tools::VarIdMap& varmap,
     const DynamicBitset& varset)
 {
   if (v == NULL)
@@ -542,7 +551,7 @@
     return false;
   }
 
-  VarIdMap::const_iterator i = varmap.find(v);
+  expr_tools::VarIdMap::const_iterator i = varmap.find(v);
   if (i == varmap.end())
   {
     return false;

=== modified file 'src/compiler/rewriter/rules/index_join_rule.cpp'
--- src/compiler/rewriter/rules/index_join_rule.cpp	2012-12-07 01:31:50 +0000
+++ src/compiler/rewriter/rules/index_join_rule.cpp	2013-02-28 21:53:23 +0000
@@ -16,7 +16,7 @@
 #include "stdafx.h"
 
 #include "compiler/api/compilercb.h"
-#include "compiler/rewriter/rules/ruleset.h"
+#include "compiler/rewriter/rules/index_join_rule.h"
 #include "compiler/expression/flwor_expr.h"
 #include "compiler/expression/script_exprs.h"
 #include "compiler/expression/expr.h"
@@ -37,163 +37,307 @@
 namespace zorba
 {
 
-struct PredicateInfo;
-
-static bool isIndexJoinPredicate(RewriterContext&, PredicateInfo&);
-
-static void rewriteJoin(RewriterContext&, PredicateInfo&, bool&);
-
-static var_expr* findForVar(static_context*, RewriterContext&, const expr*, ulong&);
-
-static bool checkVarDependency(RewriterContext&, expr*, ulong);
-
-static bool expandVars(RewriterContext&, expr*, ulong, long&);
-
-static bool findFlworForVar(RewriterContext&, const var_expr*, flwor_expr*&, ulong&, ulong&);
-
-
 struct PredicateInfo
 {
   flwor_expr  * theFlworExpr;
+  csize         theWherePos;
   expr        * thePredicate;
+
   expr        * theOuterOp;
   var_expr    * theOuterVar;
-  ulong         theOuterVarId;
+  csize         theOuterVarId;
+
   expr        * theInnerOp;
   var_expr    * theInnerVar;
+  csize         theInnerVarId;
+
   bool          theIsGeneral;
+
+  csize         theStackPos;
+  csize         theChildPos;
 };
 
 
 /*******************************************************************************
-  This rule analyzes the where clause of flwor exprs to deterimne whether any
-  predicate in the clause is a join predicate and whether the associated join
-  can be converted into a hashjoin using an index that is built on-th-fly.
+
+********************************************************************************/
+IndexJoinRule::IndexJoinRule(RewriterContext* rctx)
+  :
+  RewriteRule(RewriteRule::IndexJoin, "IndexJoin"),
+  theRCtx(rctx),
+  theRootExpr(rctx->getRoot())
+{
+  theVarIdMap = new expr_tools::VarIdMap;
+  theIdVarMap = new expr_tools::IdVarMap;
+  theExprVarsMap = new expr_tools::ExprVarsMap;
+
+  theIdVarMap->reserve(32);
+  theIdVarMap->push_back(0); // because varids are starting from 1
+
+  csize numVars = 0;
+  expr_tools::index_flwor_vars(theRootExpr, numVars, *theVarIdMap, theIdVarMap);
+
+  DynamicBitset freeset(numVars+1);
+  expr_tools::build_expr_to_vars_map(theRootExpr,
+                                     *theVarIdMap,
+                                     freeset,
+                                     *theExprVarsMap);
+}
+
+
+/*******************************************************************************
+
+********************************************************************************/
+IndexJoinRule::~IndexJoinRule()
+{
+  assert(theVarDefExprs.empty());
+  assert(theChildPositions.empty());
+
+  delete theVarIdMap;
+  delete theIdVarMap;
+  delete theExprVarsMap;
+}
+
+
+/*******************************************************************************
+  This rule analyzes the where clauses of flwor exprs to deterimne whether any
+  predicate in a clause is a join predicate and whether the associated join
+  can be converted into a hashjoin using an index that is built on-the-fly.
 ********************************************************************************/
 expr* IndexJoinRule::apply(RewriterContext& rCtx, expr* node, bool& modified)
 {
+  assert(node != theRootExpr || theVarDefExprs.empty());
+  assert(node != theRootExpr || theChildPositions.empty());
+
   flwor_expr* flworExpr = NULL;
 
-  if (node->get_expr_kind() == gflwor_expr_kind)
-    return node;
+  expr_kind_t nodeKind = node->get_expr_kind();
 
-  if (node->get_expr_kind() == trycatch_expr_kind)
-  {
-    rCtx.theFlworStack.push_back(node);
-  }
-  else if (node->get_expr_kind() == flwor_expr_kind)
+  if (nodeKind == flwor_expr_kind || nodeKind == gflwor_expr_kind)
   {
     flworExpr = static_cast<flwor_expr *>(node);
 
     // Push the flwor expr into the stack
-    rCtx.theFlworStack.push_back(flworExpr);
-    rCtx.theInReturnClause.push_back(false);
-
-    expr* whereExpr = flworExpr->get_where();
-
-    if (whereExpr == NULL)
-      goto drilldown;
-
-    PredicateInfo predInfo;
-    predInfo.theFlworExpr = flworExpr;
-    predInfo.thePredicate = whereExpr;
-
-    if (isIndexJoinPredicate(rCtx, predInfo))
+    theVarDefExprs.push_back(flworExpr);
+    theChildPositions.push_back(0);
+
+    csize numClauses = flworExpr->num_clauses();
+    for (csize i = 0; i < numClauses; ++i)
     {
-      rewriteJoin(rCtx, predInfo, modified);
-
-      if (modified)
+      flwor_clause* c = flworExpr->get_clause(i);
+
+      if (c->get_kind() != flwor_clause::where_clause)
+        continue;
+
+      const where_clause* wc = static_cast<const where_clause *>(c);
+      expr* whereExpr = wc->get_expr();
+
+      PredicateInfo predInfo;
+      predInfo.theFlworExpr = flworExpr;
+      predInfo.theWherePos = i;
+      predInfo.thePredicate = whereExpr;
+
+      if (isIndexJoinPredicate(predInfo))
       {
-        predInfo.theFlworExpr->remove_where_clause();
-
-        expr* e = rCtx.theFlworStack.back();
-
-        ZORBA_ASSERT(e == node || e->get_expr_kind() == block_expr_kind);
-
-        rCtx.theFlworStack.pop_back();
-        rCtx.theInReturnClause.pop_back();
+        rewriteJoin(predInfo);
+        modified = true;
+
+        predInfo.theFlworExpr->remove_clause(c, i);
+
+        expr* e = theVarDefExprs.back();
+        ZORBA_ASSERT(e == node);
+        theVarDefExprs.pop_back();
+        theChildPositions.pop_back();
         return e;
       }
-    }
-    else if (whereExpr->get_function_kind() == FunctionConsts::OP_AND_N)
-    {
-      // TODO: consider multi-key indices
-      ExprIterator iter(whereExpr);
-      while (!iter.done())
+      else if (whereExpr->get_function_kind() == FunctionConsts::OP_AND_N)
       {
-        PredicateInfo predInfo;
-        predInfo.theFlworExpr = flworExpr;
-        predInfo.thePredicate = (**iter);
-
-        if (isIndexJoinPredicate(rCtx, predInfo))
+        // TODO: consider multi-key indices
+        ExprIterator iter(whereExpr);
+        while (!iter.done())
         {
-          rewriteJoin(rCtx, predInfo, modified);
-
-          if (modified)
+          PredicateInfo predInfo;
+          predInfo.theFlworExpr = flworExpr;
+          predInfo.theWherePos = i;
+          predInfo.thePredicate = (**iter);
+          
+          if (isIndexJoinPredicate(predInfo))
           {
+            rewriteJoin(predInfo);
+            
+            modified = true;
+
+            // TODO: just remove the pred instead of replacing it with true.
             expr* trueExpr = rCtx.theEM->
             create_const_expr(flworExpr->get_sctx(),
                               flworExpr->get_udf(),
                               flworExpr->get_loc(),
                               true);
             (**iter) = trueExpr;
-
-            expr* e = rCtx.theFlworStack.back();
-
-            ZORBA_ASSERT(e == node || e->get_expr_kind() == block_expr_kind);
-
-            rCtx.theFlworStack.pop_back();
-            rCtx.theInReturnClause.pop_back();
+              
+            expr* e = theVarDefExprs.back();
+            ZORBA_ASSERT(e == node);
+            theVarDefExprs.pop_back();
+            theChildPositions.pop_back();
             return e;
           }
-        }
+
+          iter.next();
+        }
+      }
+    }
+  }
+
+  // No index-join rewrite done, so drill down.
+
+  if (nodeKind == flwor_expr_kind || nodeKind == gflwor_expr_kind)
+  {
+    csize numClauses = flworExpr->num_clauses();
+    csize clausePos = 0;
+    csize numClauseChildren;
+
+    ExprIterator iter(node);
+    while (!iter.done() && !modified)
+    {
+      theChildPositions.back() = clausePos;
+
+      if (clausePos == numClauses)
+      {
+        numClauseChildren = 1;
+      }
+      else
+      {
+        flwor_clause* c = flworExpr->get_clause(clausePos);
+
+        switch (c->get_kind())
+        {
+        case flwor_clause::for_clause:
+        case flwor_clause::let_clause:
+        case flwor_clause::where_clause:
+        {
+          numClauseChildren = 1;
+          break;
+        }
+        case flwor_clause::window_clause:
+        {
+          window_clause* wc = static_cast<window_clause *>(c);
+          numClauseChildren = 1;
+          if (wc->get_win_start()) ++numClauseChildren;
+          if (wc->get_win_stop()) ++numClauseChildren;
+          break;
+        }
+        case flwor_clause::groupby_clause:
+        {
+          groupby_clause* gc = static_cast<groupby_clause *>(c);
+          numClauseChildren = gc->numGroupingVars() + gc->numNonGroupingVars();
+          break;
+        }
+        case flwor_clause::orderby_clause:
+        {
+          orderby_clause* oc = static_cast<orderby_clause *>(c);
+          numClauseChildren = oc->num_columns();
+          break;
+        }
+        case flwor_clause::count_clause:
+        case flwor_clause::materialize_clause:
+        {
+          numClauseChildren = 0;
+          break;
+        }
+        default:
+          ZORBA_ASSERT(false);
+        }
+      }
+
+      for (csize j = 0; j < numClauseChildren; ++j)
+      {
+        expr* currChild = **iter;
+        expr* newChild = apply(rCtx, currChild, modified);
+        ZORBA_ASSERT(currChild == newChild);
+
+        if (modified)
+          break;
 
         iter.next();
       }
-    }
-  }
-
- drilldown:
-
-  ExprIterator iter(node);
-  while (!iter.done())
-  {
-    expr* currChild = **iter;
-
-    if (flworExpr != NULL && currChild == flworExpr->get_return_expr())
-      rCtx.theInReturnClause.back() = true;
-
-    expr* newChild = apply(rCtx, currChild, modified);
-
-    if (currChild != newChild)
-    {
-      assert(modified);
-      **iter = newChild;
-    }
-
-    if (modified)
-      break;
-
-    iter.next();
-  }
-
-  if (node->get_expr_kind() == trycatch_expr_kind)
-  {
-    rCtx.theFlworStack.pop_back();
-    return node;
-  }
-  else if (node->get_expr_kind() == flwor_expr_kind)
-  {
-    expr* e = rCtx.theFlworStack.back();
-
-    ZORBA_ASSERT(e == node || e->get_expr_kind() == block_expr_kind);
-
-    rCtx.theFlworStack.pop_back();
-    rCtx.theInReturnClause.pop_back();
+
+      ++clausePos;
+    }
+
+    // drill out
+    expr* e = theVarDefExprs.back();
+    ZORBA_ASSERT(e == node);
+    theVarDefExprs.pop_back();
+    theChildPositions.pop_back();
     return e;
   }
+  else if (nodeKind == trycatch_expr_kind)
+  {
+    // drill in
+    theVarDefExprs.push_back(node);
+    theChildPositions.push_back(0);
+
+    ExprIterator iter(node);
+    while (!iter.done())
+    {
+      expr* currChild = **iter;
+      expr* newChild = apply(rCtx, currChild, modified);
+      ZORBA_ASSERT(currChild == newChild);
+
+      if (modified)
+        break;
+
+      iter.next();
+    }
+
+    // drill out
+    theVarDefExprs.pop_back();
+    theChildPositions.pop_back();
+
+    return node;
+  }
+  else if (nodeKind == block_expr_kind)
+  {
+    // drill in
+    theVarDefExprs.push_back(node);
+    theChildPositions.push_back(0);
+
+    block_expr* block = static_cast<block_expr*>(node);
+    csize numArgs = block->size();
+    for (csize i = 0; i < numArgs; ++i)
+    {
+      theChildPositions.back() = i;
+
+      expr* currChild = (*block)[i];
+      expr* newChild = apply(rCtx, currChild, modified);
+      ZORBA_ASSERT(currChild == newChild);
+
+      if (modified)
+        break;
+    }
+
+    // drill out
+    ZORBA_ASSERT(theVarDefExprs.back() == node);
+    theVarDefExprs.pop_back();
+    theChildPositions.pop_back();
+
+    return node;
+  }
   else
   {
+    ExprIterator iter(node);
+    while (!iter.done())
+    {
+      expr* currChild = **iter;
+      expr* newChild = apply(rCtx, currChild, modified);
+      ZORBA_ASSERT(currChild == newChild);
+      
+      if (modified)
+        break;
+
+      iter.next();
+    }
+
     return node;
   }
 }
@@ -203,7 +347,7 @@
   Check whether the given predicate is a join predicate that can be converted
   to a hashjoin.
 ********************************************************************************/
-static bool isIndexJoinPredicate(RewriterContext& rCtx, PredicateInfo& predInfo)
+bool IndexJoinRule::isIndexJoinPredicate(PredicateInfo& predInfo)
 {
   const fo_expr* foExpr = NULL;
   const function* fn;
@@ -232,22 +376,20 @@
   if (opKind != CompareConsts::VALUE_EQUAL && opKind != CompareConsts::GENERAL_EQUAL)
     return false;
 
-  static_context* sctx = predExpr->get_sctx();
-
   predInfo.theIsGeneral = (opKind == CompareConsts::GENERAL_EQUAL);
 
   expr* op1 = foExpr->get_arg(0);
   expr* op2 = foExpr->get_arg(1);
 
-  // Analyze each operand of the eq to see if it depends on a single for
+  // Analyze each operand of the eq to see if it depends on a single LOOP
   // variable. If that is not true, we reject this predicate.
-  ulong var1id;
-  var_expr* var1 = findForVar(sctx, rCtx, op1, var1id);
+  csize var1id;
+  var_expr* var1 = findLoopVar(op1, var1id);
   if (var1 == NULL)
     return false;
 
-  ulong var2id;
-  var_expr* var2 = findForVar(sctx, rCtx, op2, var2id);
+  csize var2id;
+  var_expr* var2 = findLoopVar(op2, var2id);
   if (var2 == NULL)
     return false;
 
@@ -255,8 +397,6 @@
     return false;
 
   // Determine the outer and inner side of the join
-  ulong outerVarId;
-
   if (var1id < var2id)
   {
     predInfo.theOuterOp = op1;
@@ -264,7 +404,7 @@
     predInfo.theOuterVarId = var1id;
     predInfo.theInnerOp = op2;
     predInfo.theInnerVar = var2;
-    outerVarId = var1id;
+    predInfo.theInnerVarId = var2id;
   }
   else
   {
@@ -273,26 +413,84 @@
     predInfo.theOuterVarId = var2id;
     predInfo.theInnerOp = op1;
     predInfo.theInnerVar = var1;
-    outerVarId = var2id;
+    predInfo.theInnerVarId = var1id;
   }
 
+  static_context* sctx = predExpr->get_sctx();
   TypeManager* tm = sctx->get_typemanager();
   RootTypeManager& rtm = GENV_TYPESYSTEM;
 
-  // The domain of the outer var must contain more than 1 item.
-  xqtref_t outerDomainType = predInfo.theOuterVar->get_domain_expr()->get_return_type();
-
-  if (outerDomainType->max_card() < 2)
-    return false;
-
-  // The expr that defines the inner var must not depend on the outer var.
-  expr* innerDomainExpr = predInfo.theInnerVar->get_forlet_clause()->get_expr();
-  if (checkVarDependency(rCtx, innerDomainExpr, outerVarId))
+  // Make sure that the inner loop var has a domain expression
+  if (predInfo.theInnerVar->get_kind() != var_expr::for_var &&
+      predInfo.theInnerVar->get_kind() != var_expr::pos_var)
+    return false;
+
+  if (predInfo.theInnerVar->get_flwor_clause() ==
+      predInfo.theOuterVar->get_flwor_clause())
+    return false;
+
+  forlet_clause* innerVarClause = predInfo.theInnerVar->get_forlet_clause();
+
+  // The inner var must not be an outer FOR var
+  if (innerVarClause->is_allowing_empty())
     return false;
 
   // The predicate must be in the same flwor that defines the inner var (this
   // way we can be sure that the pred acts as a filter over the inner var).
-  if (predInfo.theFlworExpr->defines_variable(predInfo.theInnerVar) < 0)
+  if (!predInfo.theFlworExpr->defines_var(predInfo.theInnerVar))
+    return false;
+
+  // There should be no COUNT clause and no sequential clauses between the
+  // WHERE clause and the inner var clause.
+  for (csize i = predInfo.theWherePos; i > 0; --i)
+  {
+    flwor_clause* c = predInfo.theFlworExpr->get_clause(i-1);
+
+    switch (c->get_kind())
+    {
+    case flwor_clause::for_clause:
+    case flwor_clause::let_clause:
+    case flwor_clause::window_clause:
+    {
+      forletwin_clause* flwc = static_cast<forletwin_clause*>(c);
+      if (flwc->get_expr()->is_sequential())
+        return false;
+
+      break;
+    }
+    case flwor_clause::count_clause:
+    {
+      return false;
+    }
+    case flwor_clause::orderby_clause:
+    case flwor_clause::where_clause:
+    {
+      break;
+    }
+    case flwor_clause::groupby_clause:
+    case flwor_clause::materialize_clause:
+    default:
+    {
+      ZORBA_ASSERT(false);
+    }
+    }
+    
+    if (c == innerVarClause)
+      break;
+  }
+
+  // Let OV be the outer LOOP var and BV be the inner-most non-LET variable that
+  // the index expression depends on. The index must be created somewhere between
+  // BV anmd OV. The checkVarDeps method finds BV and checks that BV is defined
+  // before OV.
+  expr* innerDomainExpr = innerVarClause->get_expr();
+  csize boundaryVarId = 0;
+  if (checkVarDeps(innerDomainExpr, predInfo.theOuterVarId, boundaryVarId))
+    return false;
+
+  // Check whether it is possible to place the index creation somewhere between
+  // BV and OV, and if so, determine the exact location.
+  if (!findIndexPos(predInfo, boundaryVarId))
     return false;
 
   // Type checks
@@ -309,9 +507,9 @@
   {
     // Normally, other rewrite rules should have added the necessary casting
     // to the eq operands so that their static types have quantifiers ONE
-    // or QUESTION and the associated prime types are not xs:anyAtomicType.
-    // But just in case those rules have been disabled, we check again here
-    // and reject the hashjoin rewrite if these condition are violated.
+    // or QUESTION. But just in case those rules have been disabled, we
+    // check again here and reject the hashjoin rewrite if these conditions
+    // are violated.
 
     if (innerQuant != TypeConstants::QUANT_ONE &&
         innerQuant != TypeConstants::QUANT_QUESTION)
@@ -321,15 +519,9 @@
         outerQuant != TypeConstants::QUANT_QUESTION)
       return false;
 
-    // The type of the outer/inner operands in the join predicate must not be
-    // xs:untypedAtomic or xs:anyAtomic.
-    /*
-    if (TypeOps::is_equal(tm, *primeOuterType, *rtm.UNTYPED_ATOMIC_TYPE_ONE, outerLoc) ||
-        TypeOps::is_equal(tm, *primeInnerType, *rtm.UNTYPED_ATOMIC_TYPE_ONE, innerLoc))
-      return false;
-    */
-    if (TypeOps::is_equal(tm, *primeOuterType, *rtm.ANY_ATOMIC_TYPE_ONE, outerLoc) ||
-        TypeOps::is_equal(tm, *primeInnerType, *rtm.ANY_ATOMIC_TYPE_ONE, innerLoc))
+    // The type of the inner operand in the join predicate must not be
+    // xs:anyAtomic.
+    if (TypeOps::is_equal(tm, *primeInnerType, *rtm.ANY_ATOMIC_TYPE_ONE, innerLoc))
       return false;
 
     // The prime type of the outer operand in the join predicate must be a
@@ -356,22 +548,26 @@
 
 
 /*******************************************************************************
-  Check if "curExpr" references a single var and that var is a FOR var. If so,
-  return that FOR var and its prefix id; otherwise return NULL.
+  Check if "curExpr" references a single var and that var is a LOOP var, i.e.,
+  a GROUPBY var or a FOR var whose domain expr has a max cardinality > 1. If so,
+  return that LOOP var and its prefix id; otherwise return NULL. If "curExpr"
+  references a single LET var, the check is applied recurdively to the domain
+  expr of the LET var.
+
+  TODO: it is possible to relax the conditions to allow curExpr to reference
+  a single LOOP var plus local/global vars. But then we must make sure we
+  don't cross these additional var when we look for the place where to put the
+  index creation.  
 ********************************************************************************/
-static var_expr* findForVar(
-    static_context* sctx,
-    RewriterContext& rCtx,
-    const expr* curExpr,
-    ulong& varid)
+var_expr* IndexJoinRule::findLoopVar(expr* curExpr, csize& varid)
 {
   var_expr* var = NULL;
 
   while (true)
   {
-    std::vector<ulong> varidSet;
+    std::vector<csize> varidSet;
 
-    const DynamicBitset& bitset = (*rCtx.theExprVarsMap)[curExpr];
+    const DynamicBitset& bitset = (*theExprVarsMap)[curExpr];
 
     bitset.getSet(varidSet);
 
@@ -379,34 +575,52 @@
       return NULL;
 
     varid = varidSet[0];
-    var = (*rCtx.theIdVarMap)[varid];
-
-    if (var->get_kind() == var_expr::for_var)
-    {
-      curExpr = var->get_domain_expr();
-
-      if (curExpr->is_sequential())
-        return NULL;
-
-      xqtref_t domainType = var->get_domain_expr()->get_return_type();
-
-      if (domainType->get_quantifier() != TypeConstants::QUANT_ONE)
+    var = (*theIdVarMap)[varid];
+
+    switch (var->get_kind())
+    {
+    case var_expr::for_var:
+    case var_expr::pos_var:
+    {
+      for_clause* fc = var->get_forlet_clause();
+      curExpr = fc->get_expr();
+
+      xqtref_t domainType = curExpr->get_return_type();
+      TypeConstants::quantifier_t quant = domainType->get_quantifier();
+
+      if (quant == TypeConstants::QUANT_STAR || quant == TypeConstants::QUANT_PLUS)
       {
-        // found a real FOR var, so we return it.
         return var;
       }
-    }
-    else if (var->get_kind() == var_expr::let_var)
+      else if (quant == TypeConstants::QUANT_ONE)
+      {
+        // this FOR var is equivalent to a LET var, so we drill into its
+        // domain expr
+        continue;
+      }
+      else
+      {
+        return false;
+      }
+    }
+    case var_expr::groupby_var:
+    case var_expr::wincond_out_var:
+    case var_expr::wincond_out_pos_var:
+    case var_expr::wincond_in_var:
+    case var_expr::wincond_in_pos_var:
+    {
+      return var;
+    }
+    case var_expr::let_var:
     {
       curExpr = var->get_domain_expr();
-
-      if (curExpr->is_sequential())
-        return NULL;
+      break;
     }
-    else
+    default:
     {
       return NULL;
     }
+    }
   }
 
   return var;
@@ -414,269 +628,374 @@
 
 
 /*******************************************************************************
-  Check if "curExpr" depends on the var V with the given prefix id. The method
-  returns true if "curExpr" references V or references another var whose domain
-  expr depends on V.
-********************************************************************************/
-static bool checkVarDependency(
-    RewriterContext& rCtx,
-    expr* curExpr,
-    ulong searchVarId)
-{
-  const DynamicBitset& bitset = (*rCtx.theExprVarsMap)[curExpr];
-
-  if (bitset.get(searchVarId))
-    return true;
-
-  std::vector<ulong> varidSet;
-  bitset.getSet(varidSet);
-
-  csize numVars = varidSet.size();
-  for (csize i = 0; i < numVars; ++i)
-  {
-    const var_expr* var = (*rCtx.theIdVarMap)[varidSet[i]];
-    curExpr = var->get_forletwin_clause()->get_expr();
-
-    if (checkVarDependency(rCtx, curExpr, searchVarId))
+  Return true iff "idxExpr" references the outer loop var OV or any other var
+  that is defined after OV. Also find and return the id of the "boundary" var
+  BV that is the inner-most variable that the indExp depends on.
+
+  NOTE: for the purposes of this method, LET vars are "transparent", i.e., the
+  method is applied recursively on the domain expr of each LET var that is
+  referenced by idxExpr. 
+********************************************************************************/
+bool IndexJoinRule::checkVarDeps(
+    expr* idxExpr,
+    csize outerVarId,
+    csize& boundVarId)
+{
+  const DynamicBitset& bitset = (*theExprVarsMap)[idxExpr];
+
+  csize numVars = bitset.size();
+  for (csize varid = 0; varid < numVars; ++varid)
+  {
+    if (!bitset.get(varid))
+      continue;
+
+    if (varid >= outerVarId)
+    {
+      if (varid == outerVarId)
+        return true;
+
+      const var_expr* var = (*theIdVarMap)[varid];
+
+      if (var->get_kind() == var_expr::let_var)
+      {
+        expr* domExpr = var->get_forlet_clause()->get_expr();
+        if (checkVarDeps(domExpr, outerVarId, boundVarId))
+          return true;
+      }
+      else
+      {
+        return true;
+      }
+    }
+    else if (varid > boundVarId)
+    {
+      boundVarId = varid;
+    }
+  }
+
+  return false;
+}
+
+
+/*******************************************************************************
+
+********************************************************************************/
+bool IndexJoinRule::findIndexPos(PredicateInfo& predInfo, csize boundVarId)
+{
+  csize numStackedExprs = theVarDefExprs.size();
+
+  //
+  // Find the position of the outer loop var
+  //
+  var_expr* outerVar = predInfo.theOuterVar;
+  flwor_expr* outerFlwor = outerVar->get_flwor_clause()->get_flwor_expr();
+  csize outerStackPos = numStackedExprs;
+  csize outerVarPos; 
+
+  bool found = outerFlwor->get_var_pos(outerVar, outerVarPos);
+  ZORBA_ASSERT(found);
+
+  for (csize i = 0; i < numStackedExprs; ++i)
+  {
+    if (theVarDefExprs[i] == outerFlwor)
+    {
+      outerStackPos = i;
+      break;
+    }
+  }
+
+  assert(outerStackPos < numStackedExprs);
+
+  //
+  // Find the position of the inner loop var
+  //
+  var_expr* innerVar = predInfo.theInnerVar;
+  flwor_expr* innerFlwor = innerVar->get_flwor_clause()->get_flwor_expr();
+  csize innerStackPos = numStackedExprs - 1;
+  csize innerVarPos; 
+
+  found = innerFlwor->get_var_pos(innerVar, innerVarPos);
+  ZORBA_ASSERT(found);
+
+  //
+  // Make sure that there are no sequential clauses/exprs and no trycatch exprs
+  // in the path between the inner and the outer loop vars.
+  //
+  theChildPositions[numStackedExprs-1] = innerVarPos;
+
+  for (csize i = outerStackPos; i <= innerStackPos; ++i)
+  {
+    expr* currExpr = theVarDefExprs[i];
+    expr_kind_t currKind = currExpr->get_expr_kind();
+
+    if (currKind == flwor_expr_kind || currKind == gflwor_expr_kind)
+    {
+      flwor_expr* flwor = static_cast<flwor_expr*>(currExpr);
+      csize numClauses = flwor->num_clauses();
+      csize startClause;
+      csize stopClause;
+      bool checkReturn = false;
+
+      if (outerStackPos == innerStackPos)
+      {
+        startClause = outerVarPos;
+        stopClause = innerVarPos;
+      }
+      else if (i == outerStackPos)
+      {
+        startClause = outerVarPos;
+        stopClause = numClauses - 1;
+        checkReturn = true;
+      }
+      else
+      {
+        startClause = 0;
+        stopClause = theChildPositions[i];
+        if (stopClause == numClauses)
+        {
+          --stopClause;
+          checkReturn = true;
+        }
+      }
+
+      for (csize j = startClause; j <= stopClause; ++j)
+      {
+        flwor_clause* c = flwor->get_clause(j);
+
+        switch (c->get_kind())
+        {
+        case flwor_clause::for_clause:
+        case flwor_clause::let_clause:
+        case flwor_clause::window_clause:
+        {
+          forletwin_clause* flwc = static_cast<forletwin_clause*>(c);
+
+          if (flwc->get_expr()->is_sequential())
+            return false;
+
+          break;
+        }
+        default:
+          break;
+        }
+      }
+
+      if (checkReturn && flwor->get_return_expr()->is_sequential())
+        return false;
+    }
+    else if (currKind == trycatch_expr_kind)
+    {
+      return false;
+    }
+    else if (currKind == block_expr_kind)
+    {
+      block_expr* block = static_cast<block_expr*>(currExpr);
+
+      for (csize j = 0; j <= theChildPositions[i]; ++j)
+      {
+        if ((*block)[j]->is_sequential())
+          return false;
+      } 
+    }
+  }
+
+  //
+  // Find the position of the boundary var
+  //
+  var_expr* boundVar = NULL; 
+  expr* boundExpr;
+  csize boundStackPos;
+  csize boundVarPos; 
+
+  if (boundVarId == 0)
+  {
+    boundStackPos = 0;
+    boundVarPos = 0;
+  }
+  else
+  {
+    boundVar = (*theIdVarMap)[boundVarId];
+    ZORBA_ASSERT(boundVar);
+
+    if (boundVar->get_kind() == var_expr::catch_var)
+    {
+      // Not worth bothering about index exprs accessing catch vars
+      return false; 
+    }
+    else if (boundVar->get_kind() == var_expr::prolog_var ||
+             boundVar->get_kind() == var_expr::local_var)
+    {
+      block_expr* blockExpr = boundVar->get_block_expr();
+      ZORBA_ASSERT(blockExpr);
+      boundExpr = blockExpr;
+
+      found = blockExpr->get_var_pos(boundVar, boundVarPos);
+      ZORBA_ASSERT(found);
+    }
+    else
+    {
+      flwor_clause* boundClause = boundVar->get_flwor_clause();
+      ZORBA_ASSERT(boundClause);
+      flwor_expr* boundFlwor = boundClause->get_flwor_expr();
+      boundExpr = boundFlwor;
+      
+      found = boundFlwor->get_var_pos(boundVar, boundVarPos);
+      ZORBA_ASSERT(found);
+    }
+    
+    for (csize i = 0; i < numStackedExprs; ++i)
+    {
+      if (theVarDefExprs[i] == boundExpr)
+      {
+        boundStackPos = i;
+        break;
+      }
+    }
+  }
+
+  ZORBA_ASSERT(boundStackPos < numStackedExprs);
+
+  //
+  // Try to place the index creation as early as possible, but without crossing
+  // any sequential expr or trycatch expr or the definition of the boundary var.
+  //
+  theChildPositions[outerStackPos] = outerVarPos;
+
+  for (csize i = outerStackPos; i >= boundStackPos; --i)
+  {
+    expr* currExpr = theVarDefExprs[i];
+    expr_kind_t currKind = currExpr->get_expr_kind();
+
+    if (currKind == flwor_expr_kind || currKind == gflwor_expr_kind)
+    {
+      flwor_expr* flwor = static_cast<flwor_expr*>(currExpr);
+      csize numClauses = flwor->num_clauses();
+      csize stopClause = 0;
+      csize startClause = theChildPositions[i];
+      bool checkReturn = false;
+
+      if (startClause == numClauses)
+      {
+        checkReturn = true;
+        --startClause;
+      }
+
+      if (checkReturn && flwor->get_return_expr()->is_sequential())
+      {
+        predInfo.theStackPos = i+1;
+        predInfo.theChildPos = 0;
+        return true;
+      }
+
+      for (csize j = startClause+1; j > stopClause; --j)
+      {
+        if (i == boundStackPos && j-1 == boundVarPos)
+        {
+          predInfo.theStackPos = i;
+          predInfo.theChildPos = (boundVarId == 0 ? j-1 : j);
+          return true;
+        }
+
+        flwor_clause* c = flwor->get_clause(j-1);
+
+        switch (c->get_kind())
+        {
+        case flwor_clause::for_clause:
+        case flwor_clause::let_clause:
+        case flwor_clause::window_clause:
+        {
+          forletwin_clause* flwc = static_cast<forletwin_clause*>(c);
+
+          if (flwc->get_expr()->is_sequential())
+          {
+            predInfo.theStackPos = i;
+            predInfo.theChildPos = j;
+            return true;
+          }
+          break;
+        }
+        default:
+          break;
+        }
+      }
+    }
+    else if (currKind == trycatch_expr_kind)
+    {
+      predInfo.theStackPos = i+1;
+      predInfo.theChildPos = 0;
       return true;
+    }
+    else if (currKind == block_expr_kind)
+    {
+      block_expr* block = static_cast<block_expr*>(currExpr);
+      csize stopArg = 0;
+      csize startArg = theChildPositions[i];
+
+      for (csize j = startArg+1; j > stopArg; --j)
+      {
+        csize arg = j-1;
+
+        if ((*block)[arg]->is_sequential() ||
+            (i == boundStackPos && arg == boundVarPos))
+        {
+          predInfo.theStackPos = i;
+          predInfo.theChildPos = j;
+          return true;
+        }
+      }
+    }
+
+    if (i == 0)
+      break;
   }
 
-  return false;
+  predInfo.theStackPos = 0;
+  predInfo.theChildPos = 0;
+  return true;
 }
 
 
 /*******************************************************************************
 
 ********************************************************************************/
-static void rewriteJoin(
-    RewriterContext& rCtx,
-    PredicateInfo& predInfo,
-    bool& modified)
+void IndexJoinRule::rewriteJoin(PredicateInfo& predInfo)
 {
-  // std::cout << "!!!!! Found Join Index Predicate !!!!!" << std::endl << std::endl;
-
-  const QueryLoc& loc = predInfo.thePredicate->get_loc();
+  CompilerCB* ccb = theRootExpr->get_ccb();
+  ExprManager* em = ccb->getExprManager();
   static_context* sctx = predInfo.thePredicate->get_sctx();
   user_function* udf = predInfo.thePredicate->get_udf();
-  for_clause* fc = predInfo.theInnerVar->get_forlet_clause();
-
-  assert(udf == rCtx.theUDF);
-
-  long maxInnerVarId = -1;
-
-  // The index domain expr is the expr that defines the inner var, expanded, if
-  // possible, so that it does not reference any variables defined after the outer
-  // var (because the index must be built ouside the loop of the outer FOR var).
-  // Note: must clone fc->get_expr() because expandVars modifies its input, but
-  // fc->get_expr should not be modified, because we may discover later that the
-  // rewrite is not possible after all,
-  expr::substitution_t subst;
-  expr* domainExpr = fc->get_expr()->clone(udf, subst);
-
-  if (!expandVars(rCtx, domainExpr, predInfo.theOuterVarId, maxInnerVarId))
-    return;
-
-  //
-  // Create the index name and the "create-index()" expr
+  const QueryLoc& loc = predInfo.theInnerVar->get_loc();
+
+  for_clause* innerClause = predInfo.theInnerVar->get_forlet_clause();
+
+  //
+  // The index domain expr is the expr that defines the inner var, expanded so
+  // that it does not reference any LET variables defined after the outer var
+  // 
+  expr* indexDomainExpr = innerClause->get_expr();
+  expandVars(indexDomainExpr, predInfo.theOuterVarId, predInfo.theInnerVarId);
+
+  //
+  // Create the index declaration
   //
   std::ostringstream os;
-  os << "tempIndex" << rCtx.theCCB->theTempIndexCounter++;
+  os << "tempIndex" << ccb->theTempIndexCounter++;
 
   store::Item_t qname;
   GENV_ITEMFACTORY->createQName(qname, "", "", os.str().c_str());
 
-  expr* qnameExpr(rCtx.theEM->create_const_expr(sctx, udf, loc, qname));
-  expr* buildExpr = NULL;
-
-  fo_expr* createExpr = rCtx.theEM->
-  create_fo_expr(sctx,
-                 udf,
-                 loc,
-                 BUILTIN_FUNC(OP_CREATE_INTERNAL_INDEX_2),
-                 qnameExpr,
-                 buildExpr);
-
-  //
-  // Find where to place the create-index expr
-  //
-  if (maxInnerVarId >= 0)
-  {
-    // The domain expr depends on some flwor var that is defined before the outer
-    // var. In this case, we find the flwor expr defining the inner-most var V
-    // referenced by the domain expr of the index. Let F be this flwor expr. If
-    // F does not define the outer var as well, then we create the index in the
-    // return expr of F. Otherwise, we first break up F by creating a sub-flwor
-    // expr (subF) and moving all clauses of F that appear after V's defining
-    // clause into subF, making the return expr of f be the return expr of subF,
-    // and setting subF as the return expr of F. Then, we create the index in
-    // the return expr of F.
-
-    const var_expr* mostInnerVar = (*rCtx.theIdVarMap)[maxInnerVarId];
-    flwor_clause* mostInnerVarClause = mostInnerVar->get_flwor_clause();
-
-    flwor_expr* innerFlwor = NULL;
-    ulong innerPosInStack = 0;
-    ulong mostInnerVarPos = 0;
-
-    if (!findFlworForVar(rCtx,
-                         mostInnerVar,
-                         innerFlwor,
-                         mostInnerVarPos,
-                         innerPosInStack))
-      return;
-
-    csize numClauses = innerFlwor->num_clauses();
-
-    if (innerFlwor->defines_variable(predInfo.theOuterVar) >= 0 ||
-        mostInnerVarPos < numClauses-1)
-    {
-      ZORBA_ASSERT(mostInnerVarPos < numClauses-1);
-
-      const QueryLoc& nestedLoc = mostInnerVarClause->get_loc();
-
-      flwor_expr* nestedFlwor = rCtx.theEM->
-      create_flwor_expr(sctx, udf, nestedLoc, false);
-
-      for (csize i = mostInnerVarPos+1; i < numClauses; ++i)
-      {
-        nestedFlwor->add_clause(innerFlwor->get_clause(i));
-      }
-
-      for (csize i = numClauses - 1; i > mostInnerVarPos; --i)
-      {
-        innerFlwor->remove_clause(i);
-      }
-
-      nestedFlwor->set_return_expr(innerFlwor->get_return_expr());
-
-      std::vector<expr*> args(2);
-      args[0] = createExpr;
-      args[1] = nestedFlwor;
-
-      block_expr* seqExpr = rCtx.theEM->
-      create_block_expr(sctx, udf, loc, false, args, NULL);
-
-      innerFlwor->set_return_expr(seqExpr);
-
-      if (predInfo.theFlworExpr == innerFlwor)
-      {
-        // The where clause has moved to the nested flwor.
-        predInfo.theFlworExpr = nestedFlwor;
-      }
-    }
-    else
-    {
-      expr* returnExpr = innerFlwor->get_return_expr();
-
-      if (returnExpr->get_expr_kind() == block_expr_kind)
-      {
-        block_expr* seqExpr = static_cast<block_expr*>(returnExpr);
-
-        csize numArgs = seqExpr->size();
-        csize arg;
-        for (arg = 0; arg < numArgs; ++arg)
-        {
-          if ((*seqExpr)[arg]->get_function_kind() !=
-              FunctionConsts::OP_CREATE_INTERNAL_INDEX_2)
-          {
-            break;
-          }
-        }
-
-        seqExpr->add_at(arg, createExpr);
-      }
-      else
-      {
-        std::vector<expr*> args(2);
-        args[0] = createExpr;
-        args[1] = returnExpr;
-
-        block_expr* seqExpr = rCtx.theEM->
-        create_block_expr(sctx, udf, loc, false, args, NULL);
-
-        innerFlwor->set_return_expr(seqExpr);
-      }
-    }
-  }
-  else
-  {
-    // The inner domain expr does not reference any flwor vars. In this case,
-    // find the flwor expr defining the outer var and create the index just
-    // before this flwor.
-    flwor_expr* outerFlworExpr = NULL;
-    ulong outerPosInStack = 0;
-    ulong dummy = 0;
-
-    if (!findFlworForVar(rCtx,
-                         predInfo.theOuterVar,
-                         outerFlworExpr,
-                         dummy,
-                         outerPosInStack))
-      return;
-
-    //  Build outer sequential expr
-    std::vector<expr*> args(2);
-    args[0] = createExpr;
-    args[1] = outerFlworExpr;
-
-    block_expr* seqExpr = rCtx.theEM->
-    create_block_expr(sctx, udf, loc, false, args, NULL);
-
-    rCtx.theFlworStack[outerPosInStack] = seqExpr;
-  }
-
-  modified = true;
-
-  //
-  // Replace the expr defining the inner var with an index probe.
-  //
-  fo_expr* probeExpr = NULL;
-
-  if (predInfo.theIsGeneral)
-  {
-    probeExpr = rCtx.theEM->
-    create_fo_expr(sctx,
-                   udf,
-                   loc,
-                   BUILTIN_FUNC(FN_ZORBA_XQDDF_PROBE_INDEX_POINT_GENERAL_N),
-                   qnameExpr,
-                   const_cast<expr*>(predInfo.theOuterOp));
-    
-    probeExpr = rCtx.theEM->
-    create_fo_expr(sctx,
-                   udf,
-                   loc,
-                   BUILTIN_FUNC(OP_SORT_DISTINCT_NODES_ASC_1),
-                   probeExpr);
-  }
-  else
-  {
-    probeExpr = rCtx.theEM->
-    create_fo_expr(sctx,
-                   udf,
-                   loc,
-                   BUILTIN_FUNC(FN_ZORBA_XQDDF_PROBE_INDEX_POINT_VALUE_N),
-                   qnameExpr,
-                   const_cast<expr*>(predInfo.theOuterOp));
-  }
-
-  fc->set_expr(probeExpr);
-
-  //
-  // Create the IndexDecl obj
-  //
-  IndexDecl_t idx = new IndexDecl(sctx, rCtx.theCCB, loc, qname);
+  IndexDecl_t idx = new IndexDecl(sctx, ccb, loc, qname);
 
   if (predInfo.theIsGeneral)
     idx->setGeneral(true);
 
   idx->setTemp(true);
 
-  idx->setDomainExpr(domainExpr);
-
-  idx->setDomainVariable(rCtx.createTempVar(sctx, loc, var_expr::for_var));
-
-  idx->setDomainPositionVariable(rCtx.createTempVar(sctx, loc, var_expr::pos_var));
+  idx->setDomainExpr(indexDomainExpr);
+
+  idx->setDomainVariable(theRCtx->createTempVar(sctx, loc, var_expr::for_var));
+
+  idx->setDomainPositionVariable(theRCtx->createTempVar(sctx, loc, var_expr::pos_var));
 
   std::vector<expr*> columnExprs(1);
   std::vector<xqtref_t> columnTypes(1);
@@ -700,20 +1019,114 @@
 
   idx->setOrderModifiers(modifiers);
 
+  //
+  // Create the "create-index()" expr
+  //
+  expr* qnameExpr = em->create_const_expr(sctx, udf, loc, qname);
+
+  expr* buildExpr = idx->getBuildExpr(loc);
+
+  function* f = BUILTIN_FUNC(OP_CREATE_INTERNAL_INDEX_2);
+  fo_expr* createExpr = em->create_fo_expr(sctx, udf, loc, f, qnameExpr, buildExpr);
+
+  DynamicBitset indexVars;
+  expr_tools::build_expr_to_vars_map(createExpr,
+                                     *theVarIdMap,
+                                     indexVars, 
+                                     *theExprVarsMap);
+
+  //
+  // Place the create-index expr to its target position
+  //
+  expr* targetExpr = theVarDefExprs[predInfo.theStackPos];
+  csize targetPos = predInfo.theChildPos;
+  expr_kind_t targetKind = targetExpr->get_expr_kind();
+
+  if (targetKind == block_expr_kind)
+  {
+    block_expr* block = static_cast<block_expr*>(targetExpr);
+
+    block->add(targetPos, createExpr);
+  }
+  else if (targetKind == flwor_expr_kind || targetKind == gflwor_expr_kind)
+  {
+    flwor_expr* flwor = static_cast<flwor_expr*>(targetExpr);
+    csize numClauses = flwor->num_clauses();
+
+    if (targetPos == numClauses)
+    {
+      std::vector<expr*> args(2);
+      args[0] = createExpr;
+      args[1] = flwor->get_return_expr();
+
+      block_expr* block = em->create_block_expr(sctx, udf, loc, false, args, NULL);
+
+      DynamicBitset blockVars = indexVars;
+      blockVars.set_union((*theExprVarsMap)[flwor->get_return_expr()]);
+      (*theExprVarsMap)[block] = blockVars;
+
+      flwor->set_return_expr(block);
+    }
+    else
+    {
+      var_expr* dummyVar = theRCtx->createTempVar(sctx, loc, var_expr::let_var);
+
+      let_clause* createClause = 
+      em->create_let_clause(sctx, loc, dummyVar, createExpr, false);
+
+      flwor->add_clause(targetPos, createClause);
+    }
+  }
+  else
+  {
+    ZORBA_ASSERT(false);
+  }
+
+  //
+  // Replace the expr defining the inner var with an index probe.
+  //
+  fo_expr* probeExpr = NULL;
+
+  if (predInfo.theIsGeneral)
+  {
+    probeExpr = em->
+    create_fo_expr(sctx,
+                   udf,
+                   loc,
+                   BUILTIN_FUNC(FN_ZORBA_XQDDF_PROBE_INDEX_POINT_GENERAL_N),
+                   qnameExpr,
+                   const_cast<expr*>(predInfo.theOuterOp));
+    
+    probeExpr = em->
+    create_fo_expr(sctx,
+                   udf,
+                   loc,
+                   BUILTIN_FUNC(OP_SORT_DISTINCT_NODES_ASC_1),
+                   probeExpr);
+  }
+  else
+  {
+    probeExpr = em->
+    create_fo_expr(sctx,
+                   udf,
+                   loc,
+                   BUILTIN_FUNC(FN_ZORBA_XQDDF_PROBE_INDEX_POINT_VALUE_N),
+                   qnameExpr,
+                   const_cast<expr*>(predInfo.theOuterOp));
+  }
+
+  DynamicBitset probeVars = (*theExprVarsMap)[predInfo.theOuterOp];
+  (*theExprVarsMap)[probeExpr] = probeVars;
+
+  innerClause->set_expr(probeExpr);
+
   sctx->bind_index(idx, loc);
 
-  buildExpr = idx->getBuildExpr(loc);
-
-  createExpr->set_arg(1, buildExpr);
-
   if (Properties::instance()->printIntermediateOpt())
   {
     std::cout << std::endl << idx->toString() << std::endl;
   }
 
-  //idx->setDomainExpr(NULL);
-  //idx->setDomainVariable(NULL);
-
 #if 0
   if (predInfo.theIsGeneral)
     std::cout << "!!!!! Applied General Hash Join !!!!!" << std::endl << std::endl;
@@ -723,133 +1136,64 @@
 }
 
 
-
 /*******************************************************************************
 
 ********************************************************************************/
-static bool expandVars(
-    RewriterContext& rCtx,
-    expr* subExpr,
-    ulong outerVarId,
-    long& maxVarId)
+void IndexJoinRule::expandVars(expr* e, csize outerVarId, csize innerVarId)
 {
-  if (subExpr->get_expr_kind() == wrapper_expr_kind)
+  if (e->get_expr_kind() == wrapper_expr_kind)
   {
-    wrapper_expr* wrapper = reinterpret_cast<wrapper_expr*>(subExpr);
+    wrapper_expr* wrapper = static_cast<wrapper_expr*>(e);
 
     if (wrapper->get_input()->get_expr_kind() == var_expr_kind)
     {
-      var_expr* var = reinterpret_cast<var_expr*>(wrapper->get_input());
-      long varid = -1;
+      var_expr* var = static_cast<var_expr*>(wrapper->get_input());
+      csize varid = 0;
 
-      if (rCtx.theVarIdMap->find(var) != rCtx.theVarIdMap->end())
-      {
-        varid = (*rCtx.theVarIdMap)[var];
-      }
-      else if (var->get_kind() == var_expr::local_var)
-      {
-        // TODO: allow index domain exprs to reference local vars
-        return false;
-      }
+      ZORBA_ASSERT(theVarIdMap->find(var) != theVarIdMap->end());
+      
+      varid = (*theVarIdMap)[var];
 
       // If it is a variable that is defined after the outer var
-      if (varid > (long)outerVarId)
+      if (outerVarId <= varid && varid < innerVarId)
       {
         if (var->get_kind() == var_expr::let_var)
         {
           wrapper->set_expr(var->get_forlet_clause()->get_expr());
 
-          return expandVars(rCtx, wrapper, outerVarId, maxVarId);
+          return expandVars(wrapper, outerVarId, innerVarId);
         }
         else if (var->get_kind() == var_expr::for_var)
         {
 #if 1
-          return false;
+          ZORBA_ASSERT(false);
 #else
           // TODO: to expand a FOR var, we must make sure that the expr is a
           // map w.r.t. that var.
           wrapper->set_expr(var->get_forlet_clause()->get_expr());
-
-          return expandVars(rCtx, wrapper, outerVarId, maxVarId);
+          return expandVars(wrapper, outerVarId, maxVarId);
 #endif
         }
         else
         {
-          return false;
+          ZORBA_ASSERT(false);
         }
       }
-      else
-      {
-        if (varid > maxVarId)
-          maxVarId = varid;
-
-        return true;
-      }
     }
   }
-  else if (subExpr->get_expr_kind() == var_expr_kind)
+  else if (e->get_expr_kind() == var_expr_kind)
   {
     ZORBA_ASSERT(false);
   }
-
-  ExprIterator iter(subExpr);
-  while (!iter.done())
-  {
-    if (!expandVars(rCtx, **iter, outerVarId, maxVarId))
-      return false;
-
-    iter.next();
-  }
-
-  return true;
-}
-
-
-/*******************************************************************************
-  Find the flwor expr defining the given var.
-********************************************************************************/
-static bool findFlworForVar(
-    RewriterContext& rCtx,
-    const var_expr* var,
-    flwor_expr*& flworExpr,
-    ulong& varPos,
-    ulong& posInStack)
-{
-  flworExpr = NULL;
-
-  long numFlwors = (long)rCtx.theFlworStack.size();
-
-  for (long i = numFlwors - 1; i >= 0; --i)
-  {
-    if (rCtx.theFlworStack[i]->get_expr_kind() == trycatch_expr_kind)
-      return false;
-
-    assert(rCtx.theFlworStack[i]->get_expr_kind() == flwor_expr_kind);
-
-    flworExpr = static_cast<flwor_expr*>(rCtx.theFlworStack[i]);
-
-    if (i < numFlwors - 1 &&
-        rCtx.theInReturnClause[i] == true &&
-        flworExpr->is_sequential())
-      return false;
-
-    // This condition is rather conservative and can be relaxed. TODO
-    if (flworExpr->has_sequential_clauses())
-      return false;
-
-    long pos;
-    if ((pos = flworExpr->defines_variable(var)) >= 0)
+  else
+  {
+    ExprIterator iter(e);
+    while (!iter.done())
     {
-      varPos = pos;
-      posInStack = i;
-      break;
+      expandVars(**iter, outerVarId, innerVarId);
+      iter.next();
     }
-
-    flworExpr = NULL;
   }
-
-  assert(flworExpr != NULL);
-  return true;
 }
 
 

=== added file 'src/compiler/rewriter/rules/index_join_rule.h'
--- src/compiler/rewriter/rules/index_join_rule.h	1970-01-01 00:00:00 +0000
+++ src/compiler/rewriter/rules/index_join_rule.h	2013-02-28 21:53:23 +0000
@@ -0,0 +1,96 @@
+/*
+ * Copyright 2006-2008 The FLWOR Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+#ifndef ZORBA_COMPILER_REWRITER_INDEX_JOINS_H
+#define ZORBA_COMPILER_REWRITER_INDEX_JOINS_H
+
+
+#include "compiler/rewriter/rules/rule_base.h"
+#include "compiler/rewriter/tools/expr_tools.h"
+
+namespace zorba
+{
+
+class DynamicBitset;
+struct PredicateInfo;
+
+
+/*******************************************************************************
+
+  theVarDefExprs:
+  ---------------
+  A stack Keeping track of "in-scope" variable-defining exprs. Var-defining 
+  exprs are flwors, trycatch, and block exprs (TOD: include transform exprs).
+  "in-scope" means that the rule has entered the expr but not exited it yet.
+
+  theClauses:
+  -----------
+
+  theVarIdMap:
+  ------------
+  Maps a var_expr to its unique "prefix" id.  See tools/expr_tools/h for more
+  details.
+
+  theIdVarMap:
+  ------------
+  This is the reverse mapping of theVarIdMap.
+
+  theExprVarsMap:
+  ---------------
+  An entry into this map maps an expr to the variables that are referenced by
+  that expr and/or its sub-exprs. See tools/expr_tools/h for more details.
+
+********************************************************************************/
+class IndexJoinRule : public RewriteRule
+{
+protected:
+  RewriterContext          * theRCtx;
+
+  expr                     * theRootExpr;
+
+  std::vector<expr*>         theVarDefExprs;
+  std::vector<csize>         theChildPositions;
+
+  expr_tools::VarIdMap     * theVarIdMap;
+  expr_tools::IdVarMap     * theIdVarMap;
+  expr_tools::ExprVarsMap  * theExprVarsMap;
+
+public:
+  IndexJoinRule(RewriterContext* rctx);
+
+  ~IndexJoinRule();
+
+  expr* apply(RewriterContext& rCtx, expr* node, bool& modified);
+
+protected:
+  bool isIndexJoinPredicate(PredicateInfo& predInfo);
+
+  var_expr* findLoopVar(expr* predExpr, csize& loopVarId);
+
+  bool checkVarDeps(expr* idxExpr, csize outerVarId, csize& boundVarId);
+
+  bool findIndexPos(PredicateInfo& predInfo, csize boundVarId);
+
+  void rewriteJoin(PredicateInfo& predInfo);
+
+  void expandVars(expr* idxExpr, csize outerVarId, csize innerVarId);
+};
+
+
+}
+
+#endif
+/* vim:set et sw=2 ts=2: */

=== modified file 'src/compiler/rewriter/rules/index_matching_rule.cpp'
--- src/compiler/rewriter/rules/index_matching_rule.cpp	2013-02-22 00:08:45 +0000
+++ src/compiler/rewriter/rules/index_matching_rule.cpp	2013-02-28 21:53:23 +0000
@@ -256,7 +256,7 @@
           break;
         }
         case flwor_clause::window_clause:
-        case flwor_clause::order_clause:
+        case flwor_clause::orderby_clause:
         {
           if (firstMatchedFOR != NULL)
           {
@@ -321,7 +321,7 @@
       getWherePreds(qi, static_cast<where_clause*>(qc), theUnmatchedQPreds);
       break;
     }
-    case flwor_clause::order_clause:
+    case flwor_clause::orderby_clause:
     {
       if (firstOrderByPos == 0)
         firstOrderByPos = qi;
@@ -415,7 +415,7 @@
       return false;
     }
     case flwor_clause::where_clause:
-    case flwor_clause::order_clause:
+    case flwor_clause::orderby_clause:
     {
       break;
     }
@@ -554,7 +554,7 @@
 
       break;
     }
-    case flwor_clause::order_clause:
+    case flwor_clause::orderby_clause:
     {
       orderby_clause* oc = static_cast<orderby_clause*>(c);
 

=== modified file 'src/compiler/rewriter/rules/nodeid_rules.cpp'
--- src/compiler/rewriter/rules/nodeid_rules.cpp	2013-01-11 22:58:12 +0000
+++ src/compiler/rewriter/rules/nodeid_rules.cpp	2013-02-28 21:53:23 +0000
@@ -250,7 +250,7 @@
         where_clause* wc = static_cast<where_clause*>(clause);
         apply(rCtx, wc->get_expr(), modified);
       }
-      else if (clause->get_kind() == flwor_clause::order_clause)
+      else if (clause->get_kind() == flwor_clause::orderby_clause)
       {
         // apply the rule recursively on the orderby exprs
         orderby_clause* oc = static_cast<orderby_clause*>(clause);

=== modified file 'src/compiler/rewriter/rules/ruleset.h'
--- src/compiler/rewriter/rules/ruleset.h	2012-12-12 16:15:04 +0000
+++ src/compiler/rewriter/rules/ruleset.h	2013-02-28 21:53:23 +0000
@@ -236,18 +236,6 @@
 
 
 
-/*******************************************************************************
-
-********************************************************************************/
-class IndexJoinRule : public RewriteRule
-{
-public:
-  IndexJoinRule() : RewriteRule(RewriteRule::IndexJoin, "IndexJoin") {}
-
-  expr* apply(RewriterContext& rCtx, expr* node, bool& modified);
-};
-
-
 }
 
 #endif /* ZORBA_REWRITE_RULE_H */

=== modified file 'src/compiler/rewriter/tools/dataflow_annotations.cpp'
--- src/compiler/rewriter/tools/dataflow_annotations.cpp	2013-01-11 22:58:12 +0000
+++ src/compiler/rewriter/tools/dataflow_annotations.cpp	2013-02-28 21:53:23 +0000
@@ -407,7 +407,7 @@
       {
         break;
       }
-      case flwor_clause::order_clause:
+      case flwor_clause::orderby_clause:
       case flwor_clause::groupby_clause:
       {
         return;

=== modified file 'src/compiler/rewriter/tools/expr_tools.cpp'
--- src/compiler/rewriter/tools/expr_tools.cpp	2013-02-27 13:13:26 +0000
+++ src/compiler/rewriter/tools/expr_tools.cpp	2013-02-28 21:53:23 +0000
@@ -16,7 +16,6 @@
 #include "stdafx.h"
 
 #include "compiler/rewriter/tools/expr_tools.h"
-#include "compiler/rewriter/framework/rewriter_context.h"
 #include "compiler/expression/flwor_expr.h"
 #include "compiler/expression/expr.h"
 #include "compiler/expression/path_expr.h"
@@ -38,9 +37,9 @@
 namespace expr_tools
 {
 
-static void add_wincond_vars(const flwor_wincond*, ulong&, VarIdMap&, IdVarMap*);
+static void add_wincond_vars(const flwor_wincond*, csize&, VarIdMap&, IdVarMap*);
 
-static void add_var(var_expr*, ulong&, VarIdMap&, IdVarMap*);
+static void add_var(var_expr*, csize&, VarIdMap&, IdVarMap*);
 
 static void remove_wincond_vars(const flwor_wincond*, const VarIdMap&, DynamicBitset&);
 
@@ -718,18 +717,19 @@
 
 
 /*******************************************************************************
-  Let FLWOR(e) be the set of flwor exprs within the expr tree rooted at expr e.
-  Let FV(e) be the set of variables defined in any of the flwor exprs in 
-  FLWOR(e). This method assigns a prefix id to each variable in FV(e) and
-  stores the mapping between var_expr and prefix id in "varidmap" and the
-  reverse mapping in "idvapmap". It also returns the number of vars in FV(e).
+  Let VDE(e) be the set of variable-defining exprs (i.e., flwor, trycatch, or 
+  vardecl exprs) within the expr tree rooted at expr e. Let V(e) be the set of
+  variables defined in any of the exprs in VDE(e). This method assigns a prefix
+  id to each variable in V(e) and stores the mapping between var_expr and prefix
+  id in "varidmap" and the reverse mapping in "idvapmap". It also returns the
+  number of vars in V(e).
 
-  Given 2 vars v1 and v2 in FV(e), their prefix ids allows to check if v1 is
+  Given 2 vars v1 and v2 in V(e), their prefix ids allows to check if v1 is
   defined before v2: v1 is defined before v2 iff id(v1) < id(v2).
 ********************************************************************************/
 void index_flwor_vars(
     const expr* e,
-    ulong& numVars,
+    csize& numVars,
     VarIdMap& varidmap,
     IdVarMap* idvarmap)
 {
@@ -738,32 +738,28 @@
   {
     const flwor_expr* flwor = static_cast<const flwor_expr *>(e);
 
-    for (flwor_expr::clause_list_t::const_iterator i = flwor->clause_begin();
-         i != flwor->clause_end();
-         ++i)
+    flwor_expr::clause_list_t::const_iterator i = flwor->clause_begin();
+    flwor_expr::clause_list_t::const_iterator end = flwor->clause_end();
+    for (; i != end; ++i)
     {
       const flwor_clause* c = *i;
 
-      if (c->get_kind() == flwor_clause::for_clause)
-      {
-        const for_clause* fc = static_cast<const for_clause *>(c);
-
-        add_var(fc->get_var(), numVars, varidmap, idvarmap);
-        add_var(fc->get_pos_var(), numVars, varidmap, idvarmap);
-        add_var(fc->get_score_var(), numVars, varidmap, idvarmap);
-
-        index_flwor_vars(fc->get_expr(), numVars, varidmap, idvarmap);
-      }
-      else if (c->get_kind() == flwor_clause::let_clause)
-      {
-        const let_clause* lc = static_cast<const let_clause *>(c);
-
-        add_var(lc->get_var(), numVars, varidmap, idvarmap);
-        add_var(lc->get_score_var(), numVars, varidmap, idvarmap);
-
-        index_flwor_vars(lc->get_expr(), numVars, varidmap, idvarmap);
-      }
-      else if (c->get_kind() == flwor_clause::window_clause)
+      switch (c->get_kind())
+      {
+      case flwor_clause::for_clause:
+      case flwor_clause::let_clause:
+      {
+        const forlet_clause* flc = static_cast<const forlet_clause *>(c);
+
+        add_var(flc->get_var(), numVars, varidmap, idvarmap);
+        add_var(flc->get_pos_var(), numVars, varidmap, idvarmap);
+        add_var(flc->get_score_var(), numVars, varidmap, idvarmap);
+
+        index_flwor_vars(flc->get_expr(), numVars, varidmap, idvarmap);
+
+        break;
+      }
+      case flwor_clause::window_clause:
       {
         const window_clause* wc = static_cast<const window_clause *>(c);
 
@@ -779,8 +775,10 @@
 
         if (stopCond != NULL)
           add_wincond_vars(stopCond, numVars, varidmap, idvarmap);
+
+        break;
       }
-      else if (c->get_kind() == flwor_clause::groupby_clause)
+      case flwor_clause::groupby_clause:
       {
         const groupby_clause* gc = static_cast<const groupby_clause *>(c);
 
@@ -808,20 +806,26 @@
         {
           add_var(ngvars[i].second, numVars, varidmap, idvarmap);
         }
+
+        break;
       }
-      else if (c->get_kind() == flwor_clause::count_clause)
+      case flwor_clause::count_clause:
       {
         const count_clause* cc = static_cast<const count_clause *>(c);
 
         add_var(cc->get_var(), numVars, varidmap, idvarmap);
+
+        break;
       }
-      else if (c->get_kind() == flwor_clause::where_clause)
+      case flwor_clause::where_clause:
       {
         const where_clause* wc = static_cast<const where_clause *>(c);
 
         index_flwor_vars(wc->get_expr(), numVars, varidmap, idvarmap);
+
+        break;
       }
-      else if (c->get_kind() == flwor_clause::order_clause)
+      case flwor_clause::orderby_clause:
       {
         const orderby_clause* obc = static_cast<const orderby_clause *>(c);
         csize numExprs = obc->num_columns();
@@ -829,6 +833,17 @@
         {
           index_flwor_vars(obc->get_column_expr(i), numVars, varidmap, idvarmap);
         }
+
+        break;
+      }
+      case flwor_clause::materialize_clause:
+      {
+        break;
+      }
+      default:
+      {
+        ZORBA_ASSERT(false);
+      }
       }
     }
 
@@ -847,7 +862,7 @@
       const catch_clause* clause = (*trycatch)[i];
 
       catch_clause::var_map_t& trycatchVars =
-        const_cast<catch_clause*>(clause)->get_vars();
+      const_cast<catch_clause*>(clause)->get_vars();
 
       catch_clause::var_map_t::const_iterator ite = trycatchVars.begin();
       catch_clause::var_map_t::const_iterator end = trycatchVars.end();
@@ -860,6 +875,14 @@
       index_flwor_vars(trycatch->get_catch_expr(i), numVars, varidmap, idvarmap);
     }
   }
+  else if (e->get_expr_kind() == var_decl_expr_kind)
+  {
+    const var_decl_expr* vdecl = static_cast<const var_decl_expr*>(e);
+    add_var(vdecl->get_var_expr(), numVars, varidmap, idvarmap);
+
+    if (vdecl->get_init_expr())
+      index_flwor_vars(vdecl->get_init_expr(), numVars, varidmap, idvarmap);
+  }
   else
   {
     ExprConstIterator iter(e);
@@ -881,7 +904,7 @@
 ********************************************************************************/
 static void add_wincond_vars(
     const flwor_wincond* cond,
-    ulong& numVars,
+    csize& numVars,
     VarIdMap& varidmap,
     IdVarMap* idvarmap)
 {
@@ -907,13 +930,13 @@
 ********************************************************************************/
 static void add_var(
     var_expr* v,
-    ulong& numVars,
+    csize& numVars,
     VarIdMap& varidmap,
     IdVarMap* idvarmap)
 {
   if (v != NULL)
   {
-    varidmap[v] = numVars++;
+    varidmap[v] = ++numVars;
     if (idvarmap)
       idvarmap->push_back(v);
   }
@@ -922,9 +945,9 @@
 
 /*******************************************************************************
   For each expr E in the expr tree rooted at "e", this method computes the set
-  of variables that belong to FV(e) and are referenced by E. Let V(E) be this
+  of variables that belong to V(e) and are referenced by E. Let V(E) be this
   set. V(E) is implemented as a bitset ("freeset") whose size is equal to the
-  size of FV(e) and whose i-th bit is on iff the var with prefix id i belongs
+  size of V(e) and whose i-th bit is on iff the var with prefix id i belongs
   to V(E). The mapping between E and V(E) is stored in "freevarMap".
 ********************************************************************************/
 void build_expr_to_vars_map(
@@ -935,7 +958,7 @@
 {
   if (e->get_expr_kind() == var_expr_kind)
   {
-    set_bit(static_cast<var_expr *>(e), varmap, freeset, true);
+    set_bit(static_cast<var_expr*>(e), varmap, freeset, true);
     freevarMap[e] = freeset;
     return;
   }
@@ -964,28 +987,26 @@
   {
     flwor_expr* flwor = static_cast<flwor_expr *>(e);
 
-    for(flwor_expr::clause_list_t::const_iterator i = flwor->clause_begin();
-        i != flwor->clause_end();
-        ++i)
+    for (flwor_expr::clause_list_t::const_iterator i = flwor->clause_begin();
+         i != flwor->clause_end();
+         ++i)
     {
       const flwor_clause* c = *i;
 
-      if (c->get_kind() == flwor_clause::for_clause)
-      {
-        const for_clause* fc = static_cast<const for_clause *>(c);
-
-        set_bit(fc->get_var(), varmap, freeset, false);
-        set_bit(fc->get_pos_var(), varmap, freeset, false);
-        set_bit(fc->get_score_var(), varmap, freeset, false);
-      }
-      else if (c->get_kind() == flwor_clause::let_clause)
-      {
-        const let_clause* lc = static_cast<const let_clause *>(c);
-
-        set_bit(lc->get_var(), varmap, freeset, false);
-        set_bit(lc->get_score_var(), varmap, freeset, false);
-      }
-      else if (c->get_kind() == flwor_clause::window_clause)
+      switch (c->get_kind())
+      {
+      case flwor_clause::for_clause:
+      case flwor_clause::let_clause:
+      {
+        const forlet_clause* flc = static_cast<const forlet_clause *>(c);
+
+        set_bit(flc->get_var(), varmap, freeset, false);
+        set_bit(flc->get_pos_var(), varmap, freeset, false);
+        set_bit(flc->get_score_var(), varmap, freeset, false);
+
+        break;
+      }
+      case flwor_clause::window_clause:
       {
         const window_clause* wc = static_cast<const window_clause *>(c);
 
@@ -999,8 +1020,10 @@
 
         if (stopCond != NULL)
           remove_wincond_vars(stopCond, varmap, freeset);
+
+        break;
       }
-      else if (c->get_kind() == flwor_clause::groupby_clause)
+      case flwor_clause::groupby_clause:
       {
         const groupby_clause* gc = static_cast<const groupby_clause *>(c);
 
@@ -1019,12 +1042,27 @@
         {
           set_bit(ngvars[i].second, varmap, freeset, false);
         }
+
+        break;
       }
-      else if (c->get_kind() == flwor_clause::count_clause)
+      case flwor_clause::count_clause:
       {
         const count_clause* cc = static_cast<const count_clause *>(c);
 
         set_bit(cc->get_var(), varmap, freeset, false);
+        
+        break;
+      }
+      case flwor_clause::where_clause:
+      case flwor_clause::orderby_clause:
+      case flwor_clause::materialize_clause:
+      {
+        break;
+      }
+      default:
+      {
+        ZORBA_ASSERT(false);
+      }
       }
     }
   }
@@ -1039,17 +1077,32 @@
       const catch_clause* clause = (*trycatch)[i];
 
       catch_clause::var_map_t& trycatchVars =
-        const_cast<catch_clause*>(clause)->get_vars();
+      const_cast<catch_clause*>(clause)->get_vars();
 
       catch_clause::var_map_t::const_iterator ite = trycatchVars.begin();
       catch_clause::var_map_t::const_iterator end = trycatchVars.end();
       for (; ite != end; ++ite)
       {
-        var_expr* trycatchVar = (*ite).second;
-        set_bit(trycatchVar, varmap, freeset, false);
+        var_expr* v = (*ite).second;
+        set_bit(v, varmap, freeset, false);
       }
     }
   }
+  else if (e->get_expr_kind() == block_expr_kind)
+  {
+    block_expr* block = static_cast<block_expr*>(e);
+
+    csize numArgs = block->size();
+
+    for (csize i = 0; i < numArgs; ++i)
+    {
+      if ((*block)[i]->get_expr_kind() != var_decl_expr_kind)
+        continue;
+
+      var_expr* v = static_cast<var_decl_expr*>((*block)[i])->get_var_expr();
+      set_bit(v, varmap, freeset, false);
+    }
+  }
 
   freevarMap[e] = freeset;
 }

=== modified file 'src/compiler/rewriter/tools/expr_tools.h'
--- src/compiler/rewriter/tools/expr_tools.h	2012-11-30 13:46:07 +0000
+++ src/compiler/rewriter/tools/expr_tools.h	2013-02-28 21:53:23 +0000
@@ -48,12 +48,7 @@
 
 bool match_exact(expr* query, expr* ast, expr::substitution_t& subst);
 
-
-int count_variable_uses(
-    expr* root,
-    var_expr* var,
-    int limit,
-    std::vector<expr**>* path);
+int count_variable_uses(expr* root, var_expr* v, int limit, std::vector<expr**>* path);
 
 
 /*******************************************************************************
@@ -63,10 +58,39 @@
 
 
 /*******************************************************************************
+  Each entry of VarIdMap maps a var_expr to its unique "prefix" id. The "prefix"
+  id has the following property: for 2 vars v1 and v2, v1 is defined before v2
+  if and only if prefix-id(v1) < prefix-id(v2). See the index_flwor_vars()
+  function in expr_tools.cpp for more details.
+********************************************************************************/
+typedef std::map<var_expr *, ulong> VarIdMap;
+
+
+/*******************************************************************************
+  This is the reverse mapping of VarIdMap.
+********************************************************************************/
+typedef std::vector<var_expr*> IdVarMap;
+
+
+/*******************************************************************************
+  An entry into this map maps an expr to the variables that are referenced by
+  that expr and/or its sub-exprs. (Note: in this context, the domain expr of a
+  var $x is not considered a sub-expr of $x, and as a result, if $x is 
+  referenced by an expr E and the domain expr of $x references another var $y,
+  $y is NOT considered to be referenced by E). Only variables that have been
+  assigned a prefix id (i.e., vars that appear in an associated VarIdMap) are
+  considered. The set of vars referenced by an expr is implemented by a bitset
+  that is indexed by prefix var id and whose size (in number of bits) is equal
+  to the size of the associated VarIdMap.
+********************************************************************************/
+typedef std::map<const expr *, DynamicBitset> ExprVarsMap;
+
+
+/*******************************************************************************
   Util functions used by rules: HoistExprsOutOfLoops and IndexJoin.
 ********************************************************************************/
 
-void index_flwor_vars(const expr*, ulong&, VarIdMap&, IdVarMap*);
+void index_flwor_vars(const expr*, csize&, VarIdMap&, IdVarMap*);
 
 void build_expr_to_vars_map(expr*, const VarIdMap&, DynamicBitset&, ExprVarsMap&);
 

=== modified file 'src/compiler/translator/translator.cpp'
--- src/compiler/translator/translator.cpp	2013-02-26 22:03:23 +0000
+++ src/compiler/translator/translator.cpp	2013-02-28 21:53:23 +0000
@@ -1730,23 +1730,27 @@
 
   if (withContextSize)
   {
-    // create a LET var equal to the seq returned by the input epxr
+    // create a LET var whose domain expr is the input epxr
     let_clause* lcInputSeq = wrap_in_letclause(inputExpr);
+    var_expr* lcInputVar = lcInputSeq->get_var();
 
     // compute the size of the input seq
-    fo_expr* countExpr = theExprManager->
-    create_fo_expr(theRootSctx,
-                   theUDF,
-                   loc,
-                   BUILTIN_FUNC(FN_COUNT_1),
-                   lcInputSeq->get_var());
+    expr* varWrapper = CREATE(wrapper)(theRootSctx, theUDF, loc, lcInputVar);
+
+    fo_expr* countExpr = CREATE(fo)(theRootSctx,
+                                    theUDF,
+                                    loc,
+                                    BUILTIN_FUNC(FN_COUNT_1),
+                                    varWrapper);
 
     normalize_fo(countExpr);
 
     forlet_clause* lcLast = wrap_in_letclause(countExpr, loc, LAST_IDX_VARNAME);
 
     // Iterate over the input seq
-    for_clause* fcDot = wrap_in_forclause(lcInputSeq->get_var(),
+    varWrapper = CREATE(wrapper)(theRootSctx, theUDF, loc, lcInputVar);
+
+    for_clause* fcDot = wrap_in_forclause(varWrapper,
                                           loc,
                                           DOT_VARNAME,
                                           DOT_POS_VARNAME);
@@ -6304,7 +6308,7 @@
       pop_scope();
       break;
     }
-    case flwor_clause::order_clause:
+    case flwor_clause::orderby_clause:
     case flwor_clause::where_clause:
     case flwor_clause::count_clause:
     {
@@ -6621,9 +6625,10 @@
 {
   TRACE_VISIT_OUT();
 
-  window_clause* windowClause =
-    dynamic_cast<window_clause*>(theFlworClausesStack.back());
-  assert(windowClause != NULL);
+  assert(theFlworClausesStack.back()->get_kind() == flwor_clause::window_clause);
+
+  window_clause* windowClause = 
+  static_cast<window_clause*>(theFlworClausesStack.back());
 
   // Pop the window var and associate it with this window clause
   var_expr* windowVarExpr = pop_nodestack_var();
@@ -6672,8 +6677,7 @@
       rchandle<WindowVars> vars = cond->get_winvars();
       pop_wincond_vars(vars, inputCondVarExprs[i]);
 
-      conds[i] = theExprManager->create_flwor_wincond(
-                                                      theSctx,
+      conds[i] = theExprManager->create_flwor_wincond(theSctx,
                                                       cond->is_only(),
                                                       inputCondVarExprs[i],
                                                       outputCondVarExprs[i],

=== modified file 'src/runtime/core/path_iterators.cpp'
--- src/runtime/core/path_iterators.cpp	2012-10-12 21:07:51 +0000
+++ src/runtime/core/path_iterators.cpp	2013-02-28 21:53:23 +0000
@@ -1134,12 +1134,12 @@
     do
     {
       if (!consumeNext(state->theContextNode, theChild.getp(), planState))
-        return false;
+        goto done;
       
       if (!state->theContextNode->isNode())
       {
         assert(false);
-        throw XQUERY_EXCEPTION( err::XPTY0020, ERROR_LOC( loc ) );
+        throw XQUERY_EXCEPTION(err::XPTY0020, ERROR_LOC(loc));
       }
     }
     while (!isElementOrDocumentNode(state->theContextNode.getp()));
@@ -1172,7 +1172,8 @@
     state->theChildren->close();
   }
 
-  STACK_END (state);
+ done:
+  STACK_END(state);
 }
 
 
@@ -1245,8 +1246,7 @@
     {
       if (!consumeNext(state->theContextNode, theChild.getp(), planState))
       {
-        state->reset(planState);
-        return false;
+        goto done;
       }
 
       if (!state->theContextNode->isNode())
@@ -1309,6 +1309,7 @@
     state->clear();
   }
 
+ done:
   STACK_END(state);
 }
 
@@ -1340,7 +1341,7 @@
     do
     {
       if (!consumeNext(state->theContextNode, theChild.getp(), planState))
-        return false;
+        goto done;
 
       if (!state->theContextNode->isNode())
       {
@@ -1409,7 +1410,8 @@
     state->clear();
   }
 
-  STACK_END (state);
+ done:
+  STACK_END(state);
 }
 
 
@@ -1573,7 +1575,7 @@
     state->clear();
   }
 
-  STACK_END (state);
+  STACK_END(state);
 }
 
 

=== modified file 'src/util/dynamic_bitset.cpp'
--- src/util/dynamic_bitset.cpp	2012-12-07 01:30:55 +0000
+++ src/util/dynamic_bitset.cpp	2013-02-28 21:53:23 +0000
@@ -25,7 +25,7 @@
 std::ostream& operator <<(std::ostream& s, const DynamicBitset& set)
 {
   s << "BitSet[" << set.size() << "] = {";
-  for(std::size_t i = 0; i < set.size(); ++i) 
+  for(csize i = 0; i < set.size(); ++i) 
   {
     if (set.get(i)) 
     {

=== modified file 'src/util/dynamic_bitset.h'
--- src/util/dynamic_bitset.h	2012-12-07 01:30:55 +0000
+++ src/util/dynamic_bitset.h	2013-02-28 21:53:23 +0000
@@ -22,6 +22,9 @@
 #include "zorbamisc/config/stdint.h"
 #include "zorbamisc/config/platform.h"
 
+#include "store/api/shared_types.h"
+
+
 namespace zorba {
 
 class DynamicBitset 
@@ -30,13 +33,13 @@
   typedef std::vector<uint8_t> bits_t;
 
 private:
-  ulong    m_num_bits;
+  csize    m_num_bits;
   bits_t   m_bits;
 
 public:
-  DynamicBitset() { }
+  DynamicBitset() : m_num_bits(0) { }
 
-  DynamicBitset(ulong size)
+  DynamicBitset(csize size)
     :
     m_num_bits(size),
     m_bits((size >> 3) + ((size & 7) ? 1 : 0)) 
@@ -45,20 +48,20 @@
 
   ~DynamicBitset() { }
 
-  ulong size() const
+  csize size() const
   {
     return m_num_bits;
   }
 
-  void getSet(std::vector<ulong>& set) const
+  void getSet(std::vector<csize>& set) const
   {
-    ulong numBytes = (ulong)m_bits.size();
-    for (ulong i = 0; i < numBytes; ++i)
+    csize numBytes = m_bits.size();
+    for (csize i = 0; i < numBytes; ++i)
     {
       if (m_bits[i] == 0)
         continue;
 
-      for (ulong j = 0; j < 8; ++j) 
+      for (csize j = 0; j < 8; ++j) 
       {
         if ((m_bits[i] & (128 >> j)) != 0) // 128 = 1000 0000
           set.push_back(i * 8 + j);
@@ -77,16 +80,16 @@
     return (byte & (128 >> getBitWithinByte(bit))) != 0;
   }
 
-  void set(ulong bit, bool value)
+  void set(csize bit, bool value)
   {
-    ulong off = getByteIndex(bit);
+    csize off = getByteIndex(bit);
     if (!value && m_bits.size() <= (unsigned)off)
       return;
 
     if (m_bits.size() <= (unsigned)off)
       m_bits.resize(off + 1);
  
-    ulong bitnum = getBitWithinByte(bit);
+    csize bitnum = getBitWithinByte(bit);
     if (value) 
     {
       m_bits[off] |= (128 >> bitnum);
@@ -108,7 +111,7 @@
   
   DynamicBitset& set_union(const DynamicBitset& rhs)
   {
-    ulong idx = 0;
+    csize idx = 0;
     while(idx < m_bits.size() && idx < rhs.m_bits.size()) 
     {
       m_bits[idx] |= rhs.m_bits[idx];
@@ -128,7 +131,7 @@
   
   DynamicBitset& set_intersect(const DynamicBitset& rhs)
   {
-    ulong idx = 0;
+    csize idx = 0;
     while(idx < m_bits.size() && idx < rhs.m_bits.size()) 
     {
       m_bits[idx] &= rhs.m_bits[idx];
@@ -143,12 +146,12 @@
 
 
 private:
-  static ulong getByteIndex(ulong bit)
+  static csize getByteIndex(csize bit)
   {
     return bit >> 3;
   }
   
-  static ulong getBitWithinByte(ulong bit)
+  static csize getBitWithinByte(csize bit)
   {
     // bit = 8 ==> bit & 7 = 0,
     // bit = 9 ==> bit & 7 = 1, etc

=== modified file 'src/zorbaserialization/archiver.cpp'
--- src/zorbaserialization/archiver.cpp	2012-09-19 21:16:15 +0000
+++ src/zorbaserialization/archiver.cpp	2013-02-28 21:53:23 +0000
@@ -67,7 +67,9 @@
   theOnlyForEval(only_for_eval),
   theAllowDelay2(allow_delay)
 {
-  assert(theKind == ARCHIVE_FIELD_NORMAL);
+  assert(theKind == ARCHIVE_FIELD_NORMAL || 
+         (theKind == ARCHIVE_FIELD_PTR && 
+          (type == TYPE_COLLATOR || type == TYPE_STD_STRING)));
 
   theValue.uint64v = 0;
 

=== modified file 'test/fots_driver/fots-driver.xq'
--- test/fots_driver/fots-driver.xq	2013-02-26 06:30:28 +0000
+++ test/fots_driver/fots-driver.xq	2013-02-28 21:53:23 +0000
@@ -582,7 +582,6 @@
 };
 
 
-
 (:~
  : A helper functrion to process a specified subset of all test-cases and
  : report the outcome for each such test case.

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9067.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9067.iter	2012-12-18 15:09:02 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9067.iter	2013-02-28 21:53:23 +0000
@@ -8,60 +8,56 @@
       </OpToIterator>
     </HoistIterator>
   </LetVariable>
-  <ReturnClause>
-    <SequentialIterator>
-      <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
-        <flwor::FLWORIterator>
-          <ForVariable name="$$opt_temp_2">
-            <UnhoistIterator>
-              <LetVarIterator varname="$$opt_temp_0"/>
-            </UnhoistIterator>
-          </ForVariable>
-          <ReturnClause>
-            <ValueIndexEntryBuilderIterator>
-              <ForVarIterator varname="$$opt_temp_2"/>
-              <ForVarIterator varname="$$opt_temp_2"/>
-            </ValueIndexEntryBuilderIterator>
-          </ReturnClause>
-        </flwor::FLWORIterator>
-      </CreateInternalIndexIterator>
+  <LetVariable name="$$opt_temp_4" materialize="true">
+    <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
       <flwor::FLWORIterator>
-        <ForVariable name="b">
-          <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
-            <FnDocIterator>
-              <SingletonIterator value="xs:string(books.xml)"/>
-            </FnDocIterator>
-          </DescendantAxisIterator>
-        </ForVariable>
-        <ForVariable name="$$opt_temp_1">
-          <HoistIterator>
-            <FnCountIterator>
-              <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                <ForVarIterator varname="b"/>
-              </ChildAxisIterator>
-            </FnCountIterator>
-          </HoistIterator>
-        </ForVariable>
-        <ForVariable name="er">
-          <ProbeIndexPointValueIterator>
-            <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-            <UnhoistIterator>
-              <ForVarIterator varname="$$opt_temp_1"/>
-            </UnhoistIterator>
-          </ProbeIndexPointValueIterator>
+        <ForVariable name="$$opt_temp_2">
+          <UnhoistIterator>
+            <LetVarIterator varname="$$opt_temp_0"/>
+          </UnhoistIterator>
         </ForVariable>
         <ReturnClause>
-          <ElementIterator>
-            <SingletonIterator value="xs:QName(,,gruppe)"/>
-            <AttributeIterator qname="xs:QName(,,anzahl)">
-              <EnclosedIterator attr_cont="true">
-                <ForVarIterator varname="er"/>
-              </EnclosedIterator>
-            </AttributeIterator>
-          </ElementIterator>
+          <ValueIndexEntryBuilderIterator>
+            <ForVarIterator varname="$$opt_temp_2"/>
+            <ForVarIterator varname="$$opt_temp_2"/>
+          </ValueIndexEntryBuilderIterator>
         </ReturnClause>
       </flwor::FLWORIterator>
-    </SequentialIterator>
+    </CreateInternalIndexIterator>
+  </LetVariable>
+  <ForVariable name="b">
+    <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
+      <FnDocIterator>
+        <SingletonIterator value="xs:string(books.xml)"/>
+      </FnDocIterator>
+    </DescendantAxisIterator>
+  </ForVariable>
+  <ForVariable name="$$opt_temp_1">
+    <HoistIterator>
+      <FnCountIterator>
+        <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+          <ForVarIterator varname="b"/>
+        </ChildAxisIterator>
+      </FnCountIterator>
+    </HoistIterator>
+  </ForVariable>
+  <ForVariable name="er">
+    <ProbeIndexPointValueIterator>
+      <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+      <UnhoistIterator>
+        <ForVarIterator varname="$$opt_temp_1"/>
+      </UnhoistIterator>
+    </ProbeIndexPointValueIterator>
+  </ForVariable>
+  <ReturnClause>
+    <ElementIterator>
+      <SingletonIterator value="xs:QName(,,gruppe)"/>
+      <AttributeIterator qname="xs:QName(,,anzahl)">
+        <EnclosedIterator attr_cont="true">
+          <ForVarIterator varname="er"/>
+        </EnclosedIterator>
+      </AttributeIterator>
+    </ElementIterator>
   </ReturnClause>
 </flwor::FLWORIterator>
 

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9068.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9068.iter	2012-12-18 15:09:02 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9068.iter	2013-02-28 21:53:23 +0000
@@ -8,72 +8,68 @@
       </OpToIterator>
     </HoistIterator>
   </LetVariable>
-  <ReturnClause>
-    <SequentialIterator>
-      <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
-        <flwor::FLWORIterator>
-          <ForVariable name="$$opt_temp_3">
-            <UnhoistIterator>
-              <LetVarIterator varname="$$opt_temp_0"/>
-            </UnhoistIterator>
-          </ForVariable>
-          <ReturnClause>
-            <ValueIndexEntryBuilderIterator>
-              <ForVarIterator varname="$$opt_temp_3"/>
-              <ForVarIterator varname="$$opt_temp_3"/>
-            </ValueIndexEntryBuilderIterator>
-          </ReturnClause>
-        </flwor::FLWORIterator>
-      </CreateInternalIndexIterator>
+  <LetVariable name="$$opt_temp_5" materialize="true">
+    <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
       <flwor::FLWORIterator>
-        <ForVariable name="b">
-          <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
-            <FnDocIterator>
-              <SingletonIterator value="xs:string(books.xml)"/>
-            </FnDocIterator>
-          </DescendantAxisIterator>
-        </ForVariable>
-        <LetVariable name="$$opt_temp_2" materialize="true">
-          <HoistIterator>
-            <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,title)" typename="*" nill allowed="0">
-              <ForVarIterator varname="b"/>
-            </ChildAxisIterator>
-          </HoistIterator>
-        </LetVariable>
-        <ForVariable name="$$opt_temp_1">
-          <HoistIterator>
-            <FnCountIterator>
-              <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                <ForVarIterator varname="b"/>
-              </ChildAxisIterator>
-            </FnCountIterator>
-          </HoistIterator>
-        </ForVariable>
-        <ForVariable name="er">
-          <ProbeIndexPointValueIterator>
-            <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-            <UnhoistIterator>
-              <ForVarIterator varname="$$opt_temp_1"/>
-            </UnhoistIterator>
-          </ProbeIndexPointValueIterator>
+        <ForVariable name="$$opt_temp_3">
+          <UnhoistIterator>
+            <LetVarIterator varname="$$opt_temp_0"/>
+          </UnhoistIterator>
         </ForVariable>
         <ReturnClause>
-          <ElementIterator>
-            <SingletonIterator value="xs:QName(,,gruppe)"/>
-            <AttributeIterator qname="xs:QName(,,anzahl)">
-              <EnclosedIterator attr_cont="true">
-                <ForVarIterator varname="er"/>
-              </EnclosedIterator>
-            </AttributeIterator>
-            <EnclosedIterator attr_cont="false">
-              <UnhoistIterator>
-                <LetVarIterator varname="$$opt_temp_2"/>
-              </UnhoistIterator>
-            </EnclosedIterator>
-          </ElementIterator>
+          <ValueIndexEntryBuilderIterator>
+            <ForVarIterator varname="$$opt_temp_3"/>
+            <ForVarIterator varname="$$opt_temp_3"/>
+          </ValueIndexEntryBuilderIterator>
         </ReturnClause>
       </flwor::FLWORIterator>
-    </SequentialIterator>
+    </CreateInternalIndexIterator>
+  </LetVariable>
+  <ForVariable name="b">
+    <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
+      <FnDocIterator>
+        <SingletonIterator value="xs:string(books.xml)"/>
+      </FnDocIterator>
+    </DescendantAxisIterator>
+  </ForVariable>
+  <LetVariable name="$$opt_temp_2" materialize="true">
+    <HoistIterator>
+      <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,title)" typename="*" nill allowed="0">
+        <ForVarIterator varname="b"/>
+      </ChildAxisIterator>
+    </HoistIterator>
+  </LetVariable>
+  <ForVariable name="$$opt_temp_1">
+    <HoistIterator>
+      <FnCountIterator>
+        <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+          <ForVarIterator varname="b"/>
+        </ChildAxisIterator>
+      </FnCountIterator>
+    </HoistIterator>
+  </ForVariable>
+  <ForVariable name="er">
+    <ProbeIndexPointValueIterator>
+      <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+      <UnhoistIterator>
+        <ForVarIterator varname="$$opt_temp_1"/>
+      </UnhoistIterator>
+    </ProbeIndexPointValueIterator>
+  </ForVariable>
+  <ReturnClause>
+    <ElementIterator>
+      <SingletonIterator value="xs:QName(,,gruppe)"/>
+      <AttributeIterator qname="xs:QName(,,anzahl)">
+        <EnclosedIterator attr_cont="true">
+          <ForVarIterator varname="er"/>
+        </EnclosedIterator>
+      </AttributeIterator>
+      <EnclosedIterator attr_cont="false">
+        <UnhoistIterator>
+          <LetVarIterator varname="$$opt_temp_2"/>
+        </UnhoistIterator>
+      </EnclosedIterator>
+    </ElementIterator>
   </ReturnClause>
 </flwor::FLWORIterator>
 

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9197.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9197.iter	2012-12-18 15:09:02 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9197.iter	2013-02-28 21:53:23 +0000
@@ -11,60 +11,56 @@
           </OpToIterator>
         </HoistIterator>
       </LetVariable>
-      <ReturnClause>
-        <SequentialIterator>
-          <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
-            <flwor::FLWORIterator>
-              <ForVariable name="$$opt_temp_2">
-                <UnhoistIterator>
-                  <LetVarIterator varname="$$opt_temp_0"/>
-                </UnhoistIterator>
-              </ForVariable>
-              <ReturnClause>
-                <ValueIndexEntryBuilderIterator>
-                  <ForVarIterator varname="$$opt_temp_2"/>
-                  <ForVarIterator varname="$$opt_temp_2"/>
-                </ValueIndexEntryBuilderIterator>
-              </ReturnClause>
-            </flwor::FLWORIterator>
-          </CreateInternalIndexIterator>
+      <LetVariable name="$$opt_temp_4" materialize="true">
+        <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
           <flwor::FLWORIterator>
-            <ForVariable name="book">
-              <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
-                <FnDocIterator>
-                  <SingletonIterator value="xs:string(books.xml)"/>
-                </FnDocIterator>
-              </DescendantAxisIterator>
-            </ForVariable>
-            <ForVariable name="$$opt_temp_1">
-              <HoistIterator>
-                <FnCountIterator>
-                  <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                    <ForVarIterator varname="book"/>
-                  </ChildAxisIterator>
-                </FnCountIterator>
-              </HoistIterator>
-            </ForVariable>
-            <ForVariable name="anzahl">
-              <ProbeIndexPointValueIterator>
-                <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-                <UnhoistIterator>
-                  <ForVarIterator varname="$$opt_temp_1"/>
-                </UnhoistIterator>
-              </ProbeIndexPointValueIterator>
+            <ForVariable name="$$opt_temp_2">
+              <UnhoistIterator>
+                <LetVarIterator varname="$$opt_temp_0"/>
+              </UnhoistIterator>
             </ForVariable>
             <ReturnClause>
-              <ElementIterator>
-                <SingletonIterator value="xs:QName(,,gruppe)"/>
-                <AttributeIterator qname="xs:QName(,,anzahl)">
-                  <EnclosedIterator attr_cont="true">
-                    <ForVarIterator varname="anzahl"/>
-                  </EnclosedIterator>
-                </AttributeIterator>
-              </ElementIterator>
+              <ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="$$opt_temp_2"/>
+                <ForVarIterator varname="$$opt_temp_2"/>
+              </ValueIndexEntryBuilderIterator>
             </ReturnClause>
           </flwor::FLWORIterator>
-        </SequentialIterator>
+        </CreateInternalIndexIterator>
+      </LetVariable>
+      <ForVariable name="book">
+        <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
+          <FnDocIterator>
+            <SingletonIterator value="xs:string(books.xml)"/>
+          </FnDocIterator>
+        </DescendantAxisIterator>
+      </ForVariable>
+      <ForVariable name="$$opt_temp_1">
+        <HoistIterator>
+          <FnCountIterator>
+            <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+              <ForVarIterator varname="book"/>
+            </ChildAxisIterator>
+          </FnCountIterator>
+        </HoistIterator>
+      </ForVariable>
+      <ForVariable name="anzahl">
+        <ProbeIndexPointValueIterator>
+          <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+          <UnhoistIterator>
+            <ForVarIterator varname="$$opt_temp_1"/>
+          </UnhoistIterator>
+        </ProbeIndexPointValueIterator>
+      </ForVariable>
+      <ReturnClause>
+        <ElementIterator>
+          <SingletonIterator value="xs:QName(,,gruppe)"/>
+          <AttributeIterator qname="xs:QName(,,anzahl)">
+            <EnclosedIterator attr_cont="true">
+              <ForVarIterator varname="anzahl"/>
+            </EnclosedIterator>
+          </AttributeIterator>
+        </ElementIterator>
       </ReturnClause>
     </flwor::FLWORIterator>
   </EnclosedIterator>

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9198.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9198.iter	2012-12-18 15:09:02 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9198.iter	2013-02-28 21:53:23 +0000
@@ -11,63 +11,59 @@
           </OpToIterator>
         </HoistIterator>
       </LetVariable>
-      <ReturnClause>
-        <SequentialIterator>
-          <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
-            <flwor::FLWORIterator>
-              <ForVariable name="$$opt_temp_2">
-                <UnhoistIterator>
-                  <LetVarIterator varname="$$opt_temp_0"/>
-                </UnhoistIterator>
-              </ForVariable>
-              <ReturnClause>
-                <ValueIndexEntryBuilderIterator>
-                  <ForVarIterator varname="$$opt_temp_2"/>
-                  <ForVarIterator varname="$$opt_temp_2"/>
-                </ValueIndexEntryBuilderIterator>
-              </ReturnClause>
-            </flwor::FLWORIterator>
-          </CreateInternalIndexIterator>
+      <LetVariable name="$$opt_temp_4" materialize="true">
+        <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
           <flwor::FLWORIterator>
-            <ForVariable name="book">
-              <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
-                <FnDocIterator>
-                  <SingletonIterator value="xs:string(books.xml)"/>
-                </FnDocIterator>
-              </DescendantAxisIterator>
-            </ForVariable>
-            <ForVariable name="$$opt_temp_1">
-              <HoistIterator>
-                <FnCountIterator>
-                  <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                    <ForVarIterator varname="book"/>
-                  </ChildAxisIterator>
-                </FnCountIterator>
-              </HoistIterator>
-            </ForVariable>
-            <ForVariable name="anzahl">
-              <ProbeIndexPointValueIterator>
-                <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-                <UnhoistIterator>
-                  <ForVarIterator varname="$$opt_temp_1"/>
-                </UnhoistIterator>
-              </ProbeIndexPointValueIterator>
+            <ForVariable name="$$opt_temp_2">
+              <UnhoistIterator>
+                <LetVarIterator varname="$$opt_temp_0"/>
+              </UnhoistIterator>
             </ForVariable>
             <ReturnClause>
-              <ElementIterator>
-                <SingletonIterator value="xs:QName(,,gruppe)"/>
-                <AttributeIterator qname="xs:QName(,,anzahl)">
-                  <EnclosedIterator attr_cont="true">
-                    <ForVarIterator varname="anzahl"/>
-                  </EnclosedIterator>
-                </AttributeIterator>
-                <EnclosedIterator attr_cont="false">
-                  <ForVarIterator varname="book"/>
-                </EnclosedIterator>
-              </ElementIterator>
+              <ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="$$opt_temp_2"/>
+                <ForVarIterator varname="$$opt_temp_2"/>
+              </ValueIndexEntryBuilderIterator>
             </ReturnClause>
           </flwor::FLWORIterator>
-        </SequentialIterator>
+        </CreateInternalIndexIterator>
+      </LetVariable>
+      <ForVariable name="book">
+        <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
+          <FnDocIterator>
+            <SingletonIterator value="xs:string(books.xml)"/>
+          </FnDocIterator>
+        </DescendantAxisIterator>
+      </ForVariable>
+      <ForVariable name="$$opt_temp_1">
+        <HoistIterator>
+          <FnCountIterator>
+            <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+              <ForVarIterator varname="book"/>
+            </ChildAxisIterator>
+          </FnCountIterator>
+        </HoistIterator>
+      </ForVariable>
+      <ForVariable name="anzahl">
+        <ProbeIndexPointValueIterator>
+          <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+          <UnhoistIterator>
+            <ForVarIterator varname="$$opt_temp_1"/>
+          </UnhoistIterator>
+        </ProbeIndexPointValueIterator>
+      </ForVariable>
+      <ReturnClause>
+        <ElementIterator>
+          <SingletonIterator value="xs:QName(,,gruppe)"/>
+          <AttributeIterator qname="xs:QName(,,anzahl)">
+            <EnclosedIterator attr_cont="true">
+              <ForVarIterator varname="anzahl"/>
+            </EnclosedIterator>
+          </AttributeIterator>
+          <EnclosedIterator attr_cont="false">
+            <ForVarIterator varname="book"/>
+          </EnclosedIterator>
+        </ElementIterator>
       </ReturnClause>
     </flwor::FLWORIterator>
   </EnclosedIterator>

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9199.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9199.iter	2012-12-18 15:09:02 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9199.iter	2013-02-28 21:53:23 +0000
@@ -11,63 +11,59 @@
           </OpToIterator>
         </HoistIterator>
       </LetVariable>
-      <ReturnClause>
-        <SequentialIterator>
-          <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
-            <flwor::FLWORIterator>
-              <ForVariable name="$$opt_temp_2">
-                <UnhoistIterator>
-                  <LetVarIterator varname="$$opt_temp_0"/>
-                </UnhoistIterator>
-              </ForVariable>
-              <ReturnClause>
-                <ValueIndexEntryBuilderIterator>
-                  <ForVarIterator varname="$$opt_temp_2"/>
-                  <ForVarIterator varname="$$opt_temp_2"/>
-                </ValueIndexEntryBuilderIterator>
-              </ReturnClause>
-            </flwor::FLWORIterator>
-          </CreateInternalIndexIterator>
+      <LetVariable name="$$opt_temp_4" materialize="true">
+        <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
           <flwor::FLWORIterator>
-            <ForVariable name="book">
-              <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
-                <FnDocIterator>
-                  <SingletonIterator value="xs:string(books.xml)"/>
-                </FnDocIterator>
-              </DescendantAxisIterator>
-            </ForVariable>
-            <ForVariable name="$$opt_temp_1">
-              <HoistIterator>
-                <FnCountIterator>
-                  <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                    <ForVarIterator varname="book"/>
-                  </ChildAxisIterator>
-                </FnCountIterator>
-              </HoistIterator>
-            </ForVariable>
-            <ForVariable name="anzahl">
-              <ProbeIndexPointValueIterator>
-                <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-                <UnhoistIterator>
-                  <ForVarIterator varname="$$opt_temp_1"/>
-                </UnhoistIterator>
-              </ProbeIndexPointValueIterator>
+            <ForVariable name="$$opt_temp_2">
+              <UnhoistIterator>
+                <LetVarIterator varname="$$opt_temp_0"/>
+              </UnhoistIterator>
             </ForVariable>
             <ReturnClause>
-              <ElementIterator>
-                <SingletonIterator value="xs:QName(,,gruppe)"/>
-                <AttributeIterator qname="xs:QName(,,anzahl)">
-                  <EnclosedIterator attr_cont="true">
-                    <ForVarIterator varname="anzahl"/>
-                  </EnclosedIterator>
-                </AttributeIterator>
-                <EnclosedIterator attr_cont="false">
-                  <ForVarIterator varname="book"/>
-                </EnclosedIterator>
-              </ElementIterator>
+              <ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="$$opt_temp_2"/>
+                <ForVarIterator varname="$$opt_temp_2"/>
+              </ValueIndexEntryBuilderIterator>
             </ReturnClause>
           </flwor::FLWORIterator>
-        </SequentialIterator>
+        </CreateInternalIndexIterator>
+      </LetVariable>
+      <ForVariable name="book">
+        <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
+          <FnDocIterator>
+            <SingletonIterator value="xs:string(books.xml)"/>
+          </FnDocIterator>
+        </DescendantAxisIterator>
+      </ForVariable>
+      <ForVariable name="$$opt_temp_1">
+        <HoistIterator>
+          <FnCountIterator>
+            <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+              <ForVarIterator varname="book"/>
+            </ChildAxisIterator>
+          </FnCountIterator>
+        </HoistIterator>
+      </ForVariable>
+      <ForVariable name="anzahl">
+        <ProbeIndexPointValueIterator>
+          <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+          <UnhoistIterator>
+            <ForVarIterator varname="$$opt_temp_1"/>
+          </UnhoistIterator>
+        </ProbeIndexPointValueIterator>
+      </ForVariable>
+      <ReturnClause>
+        <ElementIterator>
+          <SingletonIterator value="xs:QName(,,gruppe)"/>
+          <AttributeIterator qname="xs:QName(,,anzahl)">
+            <EnclosedIterator attr_cont="true">
+              <ForVarIterator varname="anzahl"/>
+            </EnclosedIterator>
+          </AttributeIterator>
+          <EnclosedIterator attr_cont="false">
+            <ForVarIterator varname="book"/>
+          </EnclosedIterator>
+        </ElementIterator>
       </ReturnClause>
     </flwor::FLWORIterator>
   </EnclosedIterator>

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9204.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9204.iter	2012-12-18 15:09:02 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9204.iter	2013-02-28 21:53:23 +0000
@@ -8,72 +8,68 @@
       </OpToIterator>
     </HoistIterator>
   </LetVariable>
+  <LetVariable name="$$opt_temp_4" materialize="true">
+    <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
+      <flwor::FLWORIterator>
+        <ForVariable name="$$opt_temp_2">
+          <UnhoistIterator>
+            <LetVarIterator varname="$$opt_temp_0"/>
+          </UnhoistIterator>
+        </ForVariable>
+        <ReturnClause>
+          <ValueIndexEntryBuilderIterator>
+            <ForVarIterator varname="$$opt_temp_2"/>
+            <ForVarIterator varname="$$opt_temp_2"/>
+          </ValueIndexEntryBuilderIterator>
+        </ReturnClause>
+      </flwor::FLWORIterator>
+    </CreateInternalIndexIterator>
+  </LetVariable>
+  <ForVariable name="book">
+    <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
+      <FnDocIterator>
+        <SingletonIterator value="xs:string(books.xml)"/>
+      </FnDocIterator>
+    </DescendantAxisIterator>
+  </ForVariable>
+  <ForVariable name="$$opt_temp_1">
+    <HoistIterator>
+      <FnCountIterator>
+        <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+          <ForVarIterator varname="book"/>
+        </ChildAxisIterator>
+      </FnCountIterator>
+    </HoistIterator>
+  </ForVariable>
   <ReturnClause>
-    <SequentialIterator>
-      <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
+    <ElementIterator>
+      <SingletonIterator value="xs:QName(,,karteikasten)"/>
+      <EnclosedIterator attr_cont="false">
         <flwor::FLWORIterator>
-          <ForVariable name="$$opt_temp_2">
-            <UnhoistIterator>
-              <LetVarIterator varname="$$opt_temp_0"/>
-            </UnhoistIterator>
+          <ForVariable name="anzahl">
+            <ProbeIndexPointValueIterator>
+              <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+              <UnhoistIterator>
+                <ForVarIterator varname="$$opt_temp_1"/>
+              </UnhoistIterator>
+            </ProbeIndexPointValueIterator>
           </ForVariable>
           <ReturnClause>
-            <ValueIndexEntryBuilderIterator>
-              <ForVarIterator varname="$$opt_temp_2"/>
-              <ForVarIterator varname="$$opt_temp_2"/>
-            </ValueIndexEntryBuilderIterator>
+            <ElementIterator>
+              <SingletonIterator value="xs:QName(,,gruppe)"/>
+              <AttributeIterator qname="xs:QName(,,anzahl)">
+                <EnclosedIterator attr_cont="true">
+                  <ForVarIterator varname="anzahl"/>
+                </EnclosedIterator>
+              </AttributeIterator>
+              <EnclosedIterator attr_cont="false">
+                <ForVarIterator varname="book"/>
+              </EnclosedIterator>
+            </ElementIterator>
           </ReturnClause>
         </flwor::FLWORIterator>
-      </CreateInternalIndexIterator>
-      <flwor::FLWORIterator>
-        <ForVariable name="book">
-          <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
-            <FnDocIterator>
-              <SingletonIterator value="xs:string(books.xml)"/>
-            </FnDocIterator>
-          </DescendantAxisIterator>
-        </ForVariable>
-        <ForVariable name="$$opt_temp_1">
-          <HoistIterator>
-            <FnCountIterator>
-              <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                <ForVarIterator varname="book"/>
-              </ChildAxisIterator>
-            </FnCountIterator>
-          </HoistIterator>
-        </ForVariable>
-        <ReturnClause>
-          <ElementIterator>
-            <SingletonIterator value="xs:QName(,,karteikasten)"/>
-            <EnclosedIterator attr_cont="false">
-              <flwor::FLWORIterator>
-                <ForVariable name="anzahl">
-                  <ProbeIndexPointValueIterator>
-                    <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-                    <UnhoistIterator>
-                      <ForVarIterator varname="$$opt_temp_1"/>
-                    </UnhoistIterator>
-                  </ProbeIndexPointValueIterator>
-                </ForVariable>
-                <ReturnClause>
-                  <ElementIterator>
-                    <SingletonIterator value="xs:QName(,,gruppe)"/>
-                    <AttributeIterator qname="xs:QName(,,anzahl)">
-                      <EnclosedIterator attr_cont="true">
-                        <ForVarIterator varname="anzahl"/>
-                      </EnclosedIterator>
-                    </AttributeIterator>
-                    <EnclosedIterator attr_cont="false">
-                      <ForVarIterator varname="book"/>
-                    </EnclosedIterator>
-                  </ElementIterator>
-                </ReturnClause>
-              </flwor::FLWORIterator>
-            </EnclosedIterator>
-          </ElementIterator>
-        </ReturnClause>
-      </flwor::FLWORIterator>
-    </SequentialIterator>
+      </EnclosedIterator>
+    </ElementIterator>
   </ReturnClause>
 </flwor::FLWORIterator>
 

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9206.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9206.iter	2012-09-19 21:16:15 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9206.iter	2013-02-28 21:53:23 +0000
@@ -12,55 +12,51 @@
           </DescendantAxisIterator>
         </HoistIterator>
       </LetVariable>
-      <ReturnClause>
-        <SequentialIterator>
-          <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
-            <flwor::FLWORIterator>
-              <ForVariable name="$$opt_temp_1">
-                <UnhoistIterator>
-                  <LetVarIterator varname="$$opt_temp_0"/>
-                </UnhoistIterator>
-              </ForVariable>
-              <ReturnClause>
-                <ValueIndexEntryBuilderIterator>
-                  <ForVarIterator varname="$$opt_temp_1"/>
-                  <FnCountIterator>
-                    <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                      <ForVarIterator varname="$$opt_temp_1"/>
-                    </ChildAxisIterator>
-                  </FnCountIterator>
-                </ValueIndexEntryBuilderIterator>
-              </ReturnClause>
-            </flwor::FLWORIterator>
-          </CreateInternalIndexIterator>
+      <LetVariable name="$$opt_temp_3" materialize="true">
+        <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
           <flwor::FLWORIterator>
-            <ForVariable name="anzahl">
-              <OpToIterator>
-                <SingletonIterator value="xs:integer(1)"/>
-                <SingletonIterator value="xs:integer(2)"/>
-              </OpToIterator>
-            </ForVariable>
-            <ForVariable name="book">
-              <ProbeIndexPointValueIterator>
-                <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-                <ForVarIterator varname="anzahl"/>
-              </ProbeIndexPointValueIterator>
+            <ForVariable name="$$opt_temp_1">
+              <UnhoistIterator>
+                <LetVarIterator varname="$$opt_temp_0"/>
+              </UnhoistIterator>
             </ForVariable>
             <ReturnClause>
-              <ElementIterator>
-                <SingletonIterator value="xs:QName(,,gruppe)"/>
-                <AttributeIterator qname="xs:QName(,,anzahl)">
-                  <EnclosedIterator attr_cont="true">
-                    <ForVarIterator varname="anzahl"/>
-                  </EnclosedIterator>
-                </AttributeIterator>
-                <EnclosedIterator attr_cont="false">
-                  <ForVarIterator varname="book"/>
-                </EnclosedIterator>
-              </ElementIterator>
+              <ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="$$opt_temp_1"/>
+                <FnCountIterator>
+                  <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+                    <ForVarIterator varname="$$opt_temp_1"/>
+                  </ChildAxisIterator>
+                </FnCountIterator>
+              </ValueIndexEntryBuilderIterator>
             </ReturnClause>
           </flwor::FLWORIterator>
-        </SequentialIterator>
+        </CreateInternalIndexIterator>
+      </LetVariable>
+      <ForVariable name="anzahl">
+        <OpToIterator>
+          <SingletonIterator value="xs:integer(1)"/>
+          <SingletonIterator value="xs:integer(2)"/>
+        </OpToIterator>
+      </ForVariable>
+      <ForVariable name="book">
+        <ProbeIndexPointValueIterator>
+          <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+          <ForVarIterator varname="anzahl"/>
+        </ProbeIndexPointValueIterator>
+      </ForVariable>
+      <ReturnClause>
+        <ElementIterator>
+          <SingletonIterator value="xs:QName(,,gruppe)"/>
+          <AttributeIterator qname="xs:QName(,,anzahl)">
+            <EnclosedIterator attr_cont="true">
+              <ForVarIterator varname="anzahl"/>
+            </EnclosedIterator>
+          </AttributeIterator>
+          <EnclosedIterator attr_cont="false">
+            <ForVarIterator varname="book"/>
+          </EnclosedIterator>
+        </ElementIterator>
       </ReturnClause>
     </flwor::FLWORIterator>
   </EnclosedIterator>

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9207.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9207.iter	2012-12-18 15:09:02 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9207.iter	2013-02-28 21:53:23 +0000
@@ -11,63 +11,59 @@
           </OpToIterator>
         </HoistIterator>
       </LetVariable>
-      <ReturnClause>
-        <SequentialIterator>
-          <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
-            <flwor::FLWORIterator>
-              <ForVariable name="$$opt_temp_2">
-                <UnhoistIterator>
-                  <LetVarIterator varname="$$opt_temp_0"/>
-                </UnhoistIterator>
-              </ForVariable>
-              <ReturnClause>
-                <ValueIndexEntryBuilderIterator>
-                  <ForVarIterator varname="$$opt_temp_2"/>
-                  <ForVarIterator varname="$$opt_temp_2"/>
-                </ValueIndexEntryBuilderIterator>
-              </ReturnClause>
-            </flwor::FLWORIterator>
-          </CreateInternalIndexIterator>
+      <LetVariable name="$$opt_temp_4" materialize="true">
+        <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
           <flwor::FLWORIterator>
-            <ForVariable name="book">
-              <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
-                <FnDocIterator>
-                  <SingletonIterator value="xs:string(books.xml)"/>
-                </FnDocIterator>
-              </DescendantAxisIterator>
-            </ForVariable>
-            <ForVariable name="$$opt_temp_1">
-              <HoistIterator>
-                <FnCountIterator>
-                  <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                    <ForVarIterator varname="book"/>
-                  </ChildAxisIterator>
-                </FnCountIterator>
-              </HoistIterator>
-            </ForVariable>
-            <ForVariable name="anzahl">
-              <ProbeIndexPointValueIterator>
-                <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-                <UnhoistIterator>
-                  <ForVarIterator varname="$$opt_temp_1"/>
-                </UnhoistIterator>
-              </ProbeIndexPointValueIterator>
+            <ForVariable name="$$opt_temp_2">
+              <UnhoistIterator>
+                <LetVarIterator varname="$$opt_temp_0"/>
+              </UnhoistIterator>
             </ForVariable>
             <ReturnClause>
-              <ElementIterator>
-                <SingletonIterator value="xs:QName(,,gruppe)"/>
-                <AttributeIterator qname="xs:QName(,,anzahl)">
-                  <EnclosedIterator attr_cont="true">
-                    <ForVarIterator varname="anzahl"/>
-                  </EnclosedIterator>
-                </AttributeIterator>
-                <EnclosedIterator attr_cont="false">
-                  <ForVarIterator varname="book"/>
-                </EnclosedIterator>
-              </ElementIterator>
+              <ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="$$opt_temp_2"/>
+                <ForVarIterator varname="$$opt_temp_2"/>
+              </ValueIndexEntryBuilderIterator>
             </ReturnClause>
           </flwor::FLWORIterator>
-        </SequentialIterator>
+        </CreateInternalIndexIterator>
+      </LetVariable>
+      <ForVariable name="book">
+        <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
+          <FnDocIterator>
+            <SingletonIterator value="xs:string(books.xml)"/>
+          </FnDocIterator>
+        </DescendantAxisIterator>
+      </ForVariable>
+      <ForVariable name="$$opt_temp_1">
+        <HoistIterator>
+          <FnCountIterator>
+            <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+              <ForVarIterator varname="book"/>
+            </ChildAxisIterator>
+          </FnCountIterator>
+        </HoistIterator>
+      </ForVariable>
+      <ForVariable name="anzahl">
+        <ProbeIndexPointValueIterator>
+          <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+          <UnhoistIterator>
+            <ForVarIterator varname="$$opt_temp_1"/>
+          </UnhoistIterator>
+        </ProbeIndexPointValueIterator>
+      </ForVariable>
+      <ReturnClause>
+        <ElementIterator>
+          <SingletonIterator value="xs:QName(,,gruppe)"/>
+          <AttributeIterator qname="xs:QName(,,anzahl)">
+            <EnclosedIterator attr_cont="true">
+              <ForVarIterator varname="anzahl"/>
+            </EnclosedIterator>
+          </AttributeIterator>
+          <EnclosedIterator attr_cont="false">
+            <ForVarIterator varname="book"/>
+          </EnclosedIterator>
+        </ElementIterator>
       </ReturnClause>
     </flwor::FLWORIterator>
   </EnclosedIterator>

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9210.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9210.iter	2012-09-19 21:16:15 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9210.iter	2013-02-28 21:53:23 +0000
@@ -12,59 +12,55 @@
           </DescendantAxisIterator>
         </HoistIterator>
       </LetVariable>
+      <LetVariable name="$$opt_temp_3" materialize="true">
+        <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
+          <flwor::FLWORIterator>
+            <ForVariable name="$$opt_temp_1">
+              <UnhoistIterator>
+                <LetVarIterator varname="$$opt_temp_0"/>
+              </UnhoistIterator>
+            </ForVariable>
+            <ReturnClause>
+              <ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="$$opt_temp_1"/>
+                <FnCountIterator>
+                  <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+                    <ForVarIterator varname="$$opt_temp_1"/>
+                  </ChildAxisIterator>
+                </FnCountIterator>
+              </ValueIndexEntryBuilderIterator>
+            </ReturnClause>
+          </flwor::FLWORIterator>
+        </CreateInternalIndexIterator>
+      </LetVariable>
+      <ForVariable name="anzahl">
+        <OpToIterator>
+          <SingletonIterator value="xs:integer(1)"/>
+          <SingletonIterator value="xs:integer(2)"/>
+        </OpToIterator>
+      </ForVariable>
       <ReturnClause>
-        <SequentialIterator>
-          <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
+        <ElementIterator>
+          <SingletonIterator value="xs:QName(,,gruppe)"/>
+          <AttributeIterator qname="xs:QName(,,anzahl)">
+            <EnclosedIterator attr_cont="true">
+              <ForVarIterator varname="anzahl"/>
+            </EnclosedIterator>
+          </AttributeIterator>
+          <EnclosedIterator attr_cont="false">
             <flwor::FLWORIterator>
-              <ForVariable name="$$opt_temp_1">
-                <UnhoistIterator>
-                  <LetVarIterator varname="$$opt_temp_0"/>
-                </UnhoistIterator>
+              <ForVariable name="book">
+                <ProbeIndexPointValueIterator>
+                  <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+                  <ForVarIterator varname="anzahl"/>
+                </ProbeIndexPointValueIterator>
               </ForVariable>
               <ReturnClause>
-                <ValueIndexEntryBuilderIterator>
-                  <ForVarIterator varname="$$opt_temp_1"/>
-                  <FnCountIterator>
-                    <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                      <ForVarIterator varname="$$opt_temp_1"/>
-                    </ChildAxisIterator>
-                  </FnCountIterator>
-                </ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="book"/>
               </ReturnClause>
             </flwor::FLWORIterator>
-          </CreateInternalIndexIterator>
-          <flwor::FLWORIterator>
-            <ForVariable name="anzahl">
-              <OpToIterator>
-                <SingletonIterator value="xs:integer(1)"/>
-                <SingletonIterator value="xs:integer(2)"/>
-              </OpToIterator>
-            </ForVariable>
-            <ReturnClause>
-              <ElementIterator>
-                <SingletonIterator value="xs:QName(,,gruppe)"/>
-                <AttributeIterator qname="xs:QName(,,anzahl)">
-                  <EnclosedIterator attr_cont="true">
-                    <ForVarIterator varname="anzahl"/>
-                  </EnclosedIterator>
-                </AttributeIterator>
-                <EnclosedIterator attr_cont="false">
-                  <flwor::FLWORIterator>
-                    <ForVariable name="book">
-                      <ProbeIndexPointValueIterator>
-                        <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-                        <ForVarIterator varname="anzahl"/>
-                      </ProbeIndexPointValueIterator>
-                    </ForVariable>
-                    <ReturnClause>
-                      <ForVarIterator varname="book"/>
-                    </ReturnClause>
-                  </flwor::FLWORIterator>
-                </EnclosedIterator>
-              </ElementIterator>
-            </ReturnClause>
-          </flwor::FLWORIterator>
-        </SequentialIterator>
+          </EnclosedIterator>
+        </ElementIterator>
       </ReturnClause>
     </flwor::FLWORIterator>
   </EnclosedIterator>

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9211.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9211.iter	2012-09-19 21:16:15 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9211.iter	2013-02-28 21:53:23 +0000
@@ -12,62 +12,58 @@
           </DescendantAxisIterator>
         </HoistIterator>
       </LetVariable>
+      <LetVariable name="$$opt_temp_3" materialize="true">
+        <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
+          <flwor::FLWORIterator>
+            <ForVariable name="$$opt_temp_1">
+              <UnhoistIterator>
+                <LetVarIterator varname="$$opt_temp_0"/>
+              </UnhoistIterator>
+            </ForVariable>
+            <ReturnClause>
+              <ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="$$opt_temp_1"/>
+                <FnCountIterator>
+                  <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+                    <ForVarIterator varname="$$opt_temp_1"/>
+                  </ChildAxisIterator>
+                </FnCountIterator>
+              </ValueIndexEntryBuilderIterator>
+            </ReturnClause>
+          </flwor::FLWORIterator>
+        </CreateInternalIndexIterator>
+      </LetVariable>
+      <ForVariable name="anzahl">
+        <OpToIterator>
+          <SingletonIterator value="xs:integer(1)"/>
+          <SingletonIterator value="xs:integer(2)"/>
+        </OpToIterator>
+      </ForVariable>
+      <OrderBySpec>
+        <ForVarIterator varname="anzahl"/>
+      </OrderBySpec>
       <ReturnClause>
-        <SequentialIterator>
-          <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
+        <ElementIterator>
+          <SingletonIterator value="xs:QName(,,gruppe)"/>
+          <AttributeIterator qname="xs:QName(,,anzahl)">
+            <EnclosedIterator attr_cont="true">
+              <ForVarIterator varname="anzahl"/>
+            </EnclosedIterator>
+          </AttributeIterator>
+          <EnclosedIterator attr_cont="false">
             <flwor::FLWORIterator>
-              <ForVariable name="$$opt_temp_1">
-                <UnhoistIterator>
-                  <LetVarIterator varname="$$opt_temp_0"/>
-                </UnhoistIterator>
+              <ForVariable name="book">
+                <ProbeIndexPointValueIterator>
+                  <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+                  <ForVarIterator varname="anzahl"/>
+                </ProbeIndexPointValueIterator>
               </ForVariable>
               <ReturnClause>
-                <ValueIndexEntryBuilderIterator>
-                  <ForVarIterator varname="$$opt_temp_1"/>
-                  <FnCountIterator>
-                    <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                      <ForVarIterator varname="$$opt_temp_1"/>
-                    </ChildAxisIterator>
-                  </FnCountIterator>
-                </ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="book"/>
               </ReturnClause>
             </flwor::FLWORIterator>
-          </CreateInternalIndexIterator>
-          <flwor::FLWORIterator>
-            <ForVariable name="anzahl">
-              <OpToIterator>
-                <SingletonIterator value="xs:integer(1)"/>
-                <SingletonIterator value="xs:integer(2)"/>
-              </OpToIterator>
-            </ForVariable>
-            <OrderBySpec>
-              <ForVarIterator varname="anzahl"/>
-            </OrderBySpec>
-            <ReturnClause>
-              <ElementIterator>
-                <SingletonIterator value="xs:QName(,,gruppe)"/>
-                <AttributeIterator qname="xs:QName(,,anzahl)">
-                  <EnclosedIterator attr_cont="true">
-                    <ForVarIterator varname="anzahl"/>
-                  </EnclosedIterator>
-                </AttributeIterator>
-                <EnclosedIterator attr_cont="false">
-                  <flwor::FLWORIterator>
-                    <ForVariable name="book">
-                      <ProbeIndexPointValueIterator>
-                        <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-                        <ForVarIterator varname="anzahl"/>
-                      </ProbeIndexPointValueIterator>
-                    </ForVariable>
-                    <ReturnClause>
-                      <ForVarIterator varname="book"/>
-                    </ReturnClause>
-                  </flwor::FLWORIterator>
-                </EnclosedIterator>
-              </ElementIterator>
-            </ReturnClause>
-          </flwor::FLWORIterator>
-        </SequentialIterator>
+          </EnclosedIterator>
+        </ElementIterator>
       </ReturnClause>
     </flwor::FLWORIterator>
   </EnclosedIterator>

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9212.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9212.iter	2012-09-19 21:16:15 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9212.iter	2013-02-28 21:53:23 +0000
@@ -12,70 +12,66 @@
           </DescendantAxisIterator>
         </HoistIterator>
       </LetVariable>
+      <LetVariable name="$$opt_temp_3" materialize="true">
+        <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
+          <flwor::FLWORIterator>
+            <ForVariable name="$$opt_temp_1">
+              <UnhoistIterator>
+                <LetVarIterator varname="$$opt_temp_0"/>
+              </UnhoistIterator>
+            </ForVariable>
+            <ReturnClause>
+              <ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="$$opt_temp_1"/>
+                <FnCountIterator>
+                  <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+                    <ForVarIterator varname="$$opt_temp_1"/>
+                  </ChildAxisIterator>
+                </FnCountIterator>
+              </ValueIndexEntryBuilderIterator>
+            </ReturnClause>
+          </flwor::FLWORIterator>
+        </CreateInternalIndexIterator>
+      </LetVariable>
+      <ForVariable name="book">
+        <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
+          <FnDocIterator>
+            <SingletonIterator value="xs:string(books.xml)"/>
+          </FnDocIterator>
+        </DescendantAxisIterator>
+      </ForVariable>
+      <ForVariable name="anzahl">
+        <FnCountIterator>
+          <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
+            <ForVarIterator varname="book"/>
+          </ChildAxisIterator>
+        </FnCountIterator>
+      </ForVariable>
+      <OrderBySpec>
+        <ForVarIterator varname="anzahl"/>
+      </OrderBySpec>
       <ReturnClause>
-        <SequentialIterator>
-          <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
+        <ElementIterator>
+          <SingletonIterator value="xs:QName(,,gruppe)"/>
+          <AttributeIterator qname="xs:QName(,,anzahl)">
+            <EnclosedIterator attr_cont="true">
+              <ForVarIterator varname="anzahl"/>
+            </EnclosedIterator>
+          </AttributeIterator>
+          <EnclosedIterator attr_cont="false">
             <flwor::FLWORIterator>
-              <ForVariable name="$$opt_temp_1">
-                <UnhoistIterator>
-                  <LetVarIterator varname="$$opt_temp_0"/>
-                </UnhoistIterator>
+              <ForVariable name="book">
+                <ProbeIndexPointValueIterator>
+                  <SingletonIterator value="xs:QName(,,tempIndex0)"/>
+                  <ForVarIterator varname="anzahl"/>
+                </ProbeIndexPointValueIterator>
               </ForVariable>
               <ReturnClause>
-                <ValueIndexEntryBuilderIterator>
-                  <ForVarIterator varname="$$opt_temp_1"/>
-                  <FnCountIterator>
-                    <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                      <ForVarIterator varname="$$opt_temp_1"/>
-                    </ChildAxisIterator>
-                  </FnCountIterator>
-                </ValueIndexEntryBuilderIterator>
+                <ForVarIterator varname="book"/>
               </ReturnClause>
             </flwor::FLWORIterator>
-          </CreateInternalIndexIterator>
-          <flwor::FLWORIterator>
-            <ForVariable name="book">
-              <DescendantAxisIterator test kind="match_name_test" qname="xs:QName(,,book)" typename="*" nill allowed="0">
-                <FnDocIterator>
-                  <SingletonIterator value="xs:string(books.xml)"/>
-                </FnDocIterator>
-              </DescendantAxisIterator>
-            </ForVariable>
-            <ForVariable name="anzahl">
-              <FnCountIterator>
-                <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                  <ForVarIterator varname="book"/>
-                </ChildAxisIterator>
-              </FnCountIterator>
-            </ForVariable>
-            <OrderBySpec>
-              <ForVarIterator varname="anzahl"/>
-            </OrderBySpec>
-            <ReturnClause>
-              <ElementIterator>
-                <SingletonIterator value="xs:QName(,,gruppe)"/>
-                <AttributeIterator qname="xs:QName(,,anzahl)">
-                  <EnclosedIterator attr_cont="true">
-                    <ForVarIterator varname="anzahl"/>
-                  </EnclosedIterator>
-                </AttributeIterator>
-                <EnclosedIterator attr_cont="false">
-                  <flwor::FLWORIterator>
-                    <ForVariable name="book">
-                      <ProbeIndexPointValueIterator>
-                        <SingletonIterator value="xs:QName(,,tempIndex0)"/>
-                        <ForVarIterator varname="anzahl"/>
-                      </ProbeIndexPointValueIterator>
-                    </ForVariable>
-                    <ReturnClause>
-                      <ForVarIterator varname="book"/>
-                    </ReturnClause>
-                  </flwor::FLWORIterator>
-                </EnclosedIterator>
-              </ElementIterator>
-            </ReturnClause>
-          </flwor::FLWORIterator>
-        </SequentialIterator>
+          </EnclosedIterator>
+        </ElementIterator>
       </ReturnClause>
     </flwor::FLWORIterator>
   </EnclosedIterator>

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9389.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9389.iter	2012-09-19 21:16:15 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/9389.iter	2013-02-28 21:53:23 +0000
@@ -12,57 +12,53 @@
           </DescendantAxisIterator>
         </HoistIterator>
       </LetVariable>
-      <ReturnClause>
-        <SequentialIterator>
-          <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
-            <flwor::FLWORIterator>
-              <ForVariable name="$$opt_temp_1">
-                <UnhoistIterator>
-                  <LetVarIterator varname="$$opt_temp_0"/>
-                </UnhoistIterator>
-              </ForVariable>
-              <ReturnClause>
-                <ValueIndexEntryBuilderIterator>
-                  <ForVarIterator varname="$$opt_temp_1"/>
-                  <FnCountIterator>
-                    <ChildAxisIterator test kind="match_name_test" qname="xs:QName(,,author)" typename="*" nill allowed="0">
-                      <ForVarIterator varname="$$opt_temp_1"/>
-                    </ChildAxisIterator>
-                  </FnCountIterator>
-                </ValueIndexEntryBuilderIterator>
-              </ReturnClause>
-            </flwor::FLWORIterator>
-          </CreateInternalIndexIterator>
+      <LetVariable name="$$opt_temp_3" materialize="true">
+        <CreateInternalIndexIterator name="xs:QName(,,tempIndex0)">
           <flwor::FLWORIterator>
-            <ForVariable name="anzahl">
-              <OpToIterator>
-                <SingletonIterator value="xs:integer(1)"/>
-                <SingletonIterator value="xs:integer(5)"/>
-              </OpToIterator>
+            <ForVariable name="$$opt_temp_1">
+              <UnhoistIterator>
+                <LetVarIterator varname="$$opt_temp_0"/>
+              </UnhoistIterator>