Merge lp:~zorba-coders/zorba/markos-scratch into lp:zorba

[Markos Zaharioudakis]

More aggressive eliminationof unused LET variables

More aggressive eliminationof unused LET variables

[Markos Zaharioudakis (markos-za)]

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for the following merge proposals:
https://code.launchpad.net/~zorba-coders/zorba/markos-scratch/+merge/185014

Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue

[Zorba Build Bot (zorba-buildbot)]

Validation queue result for https://code.launchpad.net/~zorba-coders/zorba/markos-scratch/+merge/185014

Stage "TestZorbaUbuntu" failed.
5 tests failed (8425 total tests run).

Check test results at http://jenkins.lambda.nu/job/TestZorbaUbuntu/280/testReport/ to view the results.

[lp:~zorba-coders/zorba/markos-scratch]

11220. By Markos Zaharioudakis on 2013-09-11

More aggressive eliminationof unused LET variables

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for the following merge proposals:
https://code.launchpad.net/~zorba-coders/zorba/markos-scratch/+merge/185014

Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue

[Zorba Build Bot (zorba-buildbot)]

Validation queue result for https://code.launchpad.net/~zorba-coders/zorba/markos-scratch/+merge/185014

Stage "TestZorbaUbuntu" failed.
4 tests failed (8425 total tests run).

Check test results at http://jenkins.lambda.nu/job/TestZorbaUbuntu/281/testReport/ to view the results.

[lp:~zorba-coders/zorba/markos-scratch]

11221. By Markos Zaharioudakis on 2013-09-12

More aggressive eliminationof unused LET variables

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for the following merge proposals:
https://code.launchpad.net/~zorba-coders/zorba/markos-scratch/+merge/185014

Progress dashboard at http://jenkins.lambda.nu/view/ValidationQueue

[Zorba Build Bot (zorba-buildbot)]

Validation queue succeeded - proposal merged!

=== modified file 'ChangeLog'
--- ChangeLog	2013-09-11 01:43:33 +0000
+++ ChangeLog	2013-09-12 07:54:27 +0000
@@ -36,6 +36,7 @@
   * Optimized implementation of function caching and removed the restriction
     on the return type.
   * No node ordering/distinct required for self axis.
+  * More aggressive eliminationof unused LET variables
 
 Bug Fixes/Other Changes:
   * Fixed bug #1117952 (Improve XML error output format)

=== modified file 'src/compiler/expression/expr_base.h'
--- src/compiler/expression/expr_base.h	2013-05-29 04:17:01 +0000
+++ src/compiler/expression/expr_base.h	2013-09-12 07:54:27 +0000
@@ -234,6 +234,8 @@
 
   unsigned short get_scripting_detail() const { return theScriptingKind; }
 
+  void set_scripting_detail(unsigned short k) { theScriptingKind = k; }
+
   bool is_updating() const;
 
   bool is_sequential() const;
@@ -246,8 +248,6 @@
 
   void set_not_exiting();
 
-  void adjustSequential();
-
   void checkScriptingKind() const;
 
   void compute_return_type(bool deep, bool* modified);

=== modified file 'src/compiler/expression/var_expr.cpp'
--- src/compiler/expression/var_expr.cpp	2013-04-24 01:35:58 +0000
+++ src/compiler/expression/var_expr.cpp	2013-09-12 07:54:27 +0000
@@ -85,6 +85,7 @@
   theCopyClause(NULL),
   theParamPos(0),
   theVarInfo(NULL),
+  theNumRefs(0),
   theIsExternal(false),
   theIsPrivate(false),
   theIsMutable(true),
@@ -95,7 +96,7 @@
 
 
 /*******************************************************************************
-
+  Used when cloning FLWOR clauses that define variables.
 ********************************************************************************/
 var_expr::var_expr(user_function* udf, const var_expr& source)
   :
@@ -108,6 +109,7 @@
   theCopyClause(NULL),
   theParamPos(source.theParamPos),
   theVarInfo(NULL),
+  theNumRefs(source.theNumRefs),
   theIsExternal(source.theIsExternal),
   theIsPrivate(source.theIsPrivate),
   theIsMutable(source.theIsMutable),

=== modified file 'src/compiler/expression/var_expr.h'
--- src/compiler/expression/var_expr.h	2013-04-24 01:35:58 +0000
+++ src/compiler/expression/var_expr.h	2013-09-12 07:54:27 +0000
@@ -178,6 +178,8 @@
 
   VarInfo             * theVarInfo;
 
+  csize                 theNumRefs;
+
   bool                  theIsExternal;
 
   bool                  theIsPrivate;
@@ -217,6 +219,10 @@
 
   void set_kind(var_kind k) { theVarKind = k; }
 
+  csize get_num_refs() const { return theNumRefs; }
+
+  void add_ref() { ++theNumRefs; }
+
   bool is_private() const { return theIsPrivate; }
 
   void set_private(bool v) { theIsPrivate = v; }

=== modified file 'src/compiler/rewriter/rules/flwor_rules.cpp'
--- src/compiler/rewriter/rules/flwor_rules.cpp	2013-08-09 10:23:10 +0000
+++ src/compiler/rewriter/rules/flwor_rules.cpp	2013-09-12 07:54:27 +0000
@@ -566,8 +566,16 @@
 
   if (safe && numRefs == 0)
   {
-    if (varDomExpr->isNonDiscardable() || !isSafeVar)
-    {
+    if (varDomExpr->get_function_kind() == FunctionConsts::OP_CREATE_INTERNAL_INDEX_2 ||
+        !isSafeVar)
+    {
+      return false;
+    }
+    else if (varDomExpr->isNonDiscardable())
+    {
+      if (var->get_kind() == var_expr::let_var && var->get_num_refs() == 0)
+        return true;
+
       return false;
     }
     else

=== modified file 'src/compiler/rewriter/rules/fold_rules.cpp'
--- src/compiler/rewriter/rules/fold_rules.cpp	2013-08-14 15:16:42 +0000
+++ src/compiler/rewriter/rules/fold_rules.cpp	2013-09-12 07:54:27 +0000
@@ -216,7 +216,8 @@
 
   case var_expr_kind:
   {
-    var_expr::var_kind varKind = static_cast<var_expr *>(node)->get_kind();
+    var_expr* var = static_cast<var_expr *>(node);
+    var_expr::var_kind varKind = var->get_kind();
 
     if (varKind == var_expr::prolog_var || varKind == var_expr::local_var)
       curUnfoldable = ANNOTATION_TRUE_FIXED;

=== modified file 'src/compiler/translator/translator.cpp'
--- src/compiler/translator/translator.cpp	2013-08-30 14:50:21 +0000
+++ src/compiler/translator/translator.cpp	2013-09-12 07:54:27 +0000
@@ -1669,10 +1669,13 @@
     argVars.push_back(argVar);
 
     expr* arg = CREATE(wrapper)(theRootSctx, theUDF, loc, argVar);
+    argVar->add_ref();
     arg = normalize_fo_arg(i, arg, inlineUDF.get(), loc);
     arguments.push_back(arg);
   }
 
+  fiSubstVar->add_ref();
+
   expr* body = 
   CREATE(dynamic_function_invocation)(theRootSctx,
                                       theUDF,
@@ -1935,6 +1938,8 @@
     // compute the size of the input seq
     expr* varWrapper = CREATE(wrapper)(theRootSctx, theUDF, loc, lcInputVar);
 
+    lcInputVar->add_ref();
+
     fo_expr* countExpr = CREATE(fo)(theRootSctx,
                                     theUDF,
                                     loc,
@@ -6605,6 +6610,8 @@
     ERROR_PARAMS(ve->get_name()->getStringValue()));
   }
 
+  ve->add_ref();
+
   xqtref_t varType = ve->get_type();
 
   expr* valueExpr = pop_nodestack();
@@ -7629,6 +7636,8 @@
 
       inputExpr = CREATE(wrapper)(theRootSctx, theUDF, specLoc, inputExpr);
 
+      var->getVar()->add_ref();
+
       inputExpr = wrap_in_atomization(inputExpr);
 
       inputExpr = wrap_in_type_match(inputExpr,
@@ -7681,6 +7690,8 @@
     expr* inputExpr =
     theExprManager->create_wrapper_expr(theRootSctx, theUDF, loc, inputVar);
 
+    inputVar->add_ref();
+
     nongrouping_rebind.push_back(std::pair<expr*, var_expr*>(inputExpr, ngVar));
   }
 
@@ -10343,8 +10354,11 @@
   //
   // The flworExpr as well as the $$predInput varExpr are pushed to the nodestack.
   const for_clause* fcOuterDot = static_cast<const for_clause*>(flworExpr->get_clause(0));
-  relpath_expr* predPathExpr = theExprManager->create_relpath_expr(theRootSctx, theUDF, loc);
-  predPathExpr->add_back(theExprManager->create_wrapper_expr(theRootSctx, theUDF, loc, fcOuterDot->get_var()));
+  relpath_expr* predPathExpr = CREATE(relpath)(theRootSctx, theUDF, loc);
+  predPathExpr->add_back(CREATE(wrapper)(theRootSctx, theUDF, loc, fcOuterDot->get_var()));
+
+  fcOuterDot->get_var()->add_ref();
+
   predPathExpr->add_back(axisExpr);
 
   expr* predInputExpr = predPathExpr;
@@ -11317,6 +11331,8 @@
     throw;
   }
 
+  ve->add_ref();
+
   if (ve->get_kind() == var_expr::prolog_var)
   {
     TypeManager* tm = CTX_TM;
@@ -11389,6 +11405,8 @@
     }
   }
 
+  dotVar->add_ref();
+
   return CREATE(wrapper)(theRootSctx, theUDF, loc, dotVar);
 }
 
@@ -11408,6 +11426,8 @@
     }
   }
 
+  posVar->add_ref();
+
   return CREATE(wrapper)(theRootSctx, theUDF, loc, posVar);
 }
 
@@ -11427,6 +11447,8 @@
     }
   }
 
+  sizeVar->add_ref();
+
   return CREATE(wrapper)(theRootSctx, theUDF, loc, sizeVar);
 }
 
@@ -11445,17 +11467,7 @@
 {
   TRACE_VISIT_OUT();
   
-  var_expr* ve = lookup_ctx_var(getDotItemVarName(), loc);
-
-  if (ve->get_kind() == var_expr::prolog_var)
-  {
-    if (!theCurrentPrologVFDecl.isNull())
-    {
-      thePrologGraph.addEdge(theCurrentPrologVFDecl, ve);
-    }
-  }
-
-  push_nodestack(CREATE(wrapper)(theRootSctx, theUDF, loc, ve));
+  push_nodestack(dotRef(loc));
 }
 
 
@@ -11992,18 +12004,7 @@
   }
   case FunctionConsts::FN_ZORBA_CONTEXT_ITEM_0:
   { 
-    // copy+pasted from the ContextItemExpr
-    var_expr* ve = lookup_ctx_var(getDotItemVarName(), loc);
-      
-    if (ve->get_kind() == var_expr::prolog_var)
-    {
-      if (!theCurrentPrologVFDecl.isNull())
-      {
-        thePrologGraph.addEdge(theCurrentPrologVFDecl, ve);
-      }
-    }
-    
-    resultExpr = CREATE(wrapper)(theRootSctx, theUDF, loc, ve);
+    resultExpr = dotRef(loc);
     break;        
   }
   case FunctionConsts::FN_STRING_LENGTH_0:
@@ -12352,6 +12353,8 @@
     std::vector<expr*> fncall_args;
     fncall_args.push_back(CREATE(wrapper)(theRootSctx, theUDF, loc, seq_fc->get_var()));
 
+    seq_fc->get_var()->add_ref();
+
     expr* dynamic_fncall = 
     CREATE(dynamic_function_invocation)(theRootSctx, theUDF, loc,
                                         arguments[1],
@@ -12381,6 +12384,8 @@
     std::vector<expr*> fncall_args;
     fncall_args.push_back(CREATE(wrapper)(theRootSctx, theUDF, loc, seq_fc->get_var()));
 
+    seq_fc->get_var()->add_ref();
+
     expr* dynamic_fncall =
     CREATE(dynamic_function_invocation)(theRootSctx, theUDF, loc,
                                         arguments[1],
@@ -12563,6 +12568,8 @@
 
     sourceExpr = CREATE(wrapper)(theRootSctx, theUDF, loc, fc->get_var());
 
+    fc->get_var()->add_ref();
+
     expr* dynFuncInvocation =
     CREATE(dynamic_function_invocation)(theRootSctx, theUDF, loc,
                                         sourceExpr,
@@ -12825,6 +12832,8 @@
         flworBody->add_clause(lc);
         foArgs[1] = CREATE(wrapper)(theRootSctx, theUDF, loc, lc->get_var());
 
+        lc->get_var()->add_ref();
+
         flworBody->set_return_expr(generate_fn_body(func, foArgs, loc));
         body = flworBody;
         break;

=== modified file 'src/compiler/xqddf/value_index.cpp'
--- src/compiler/xqddf/value_index.cpp	2013-05-31 01:47:24 +0000
+++ src/compiler/xqddf/value_index.cpp	2013-09-12 07:54:27 +0000
@@ -569,7 +569,7 @@
 
     expr* keyClone = theKeyExprs[i]->clone(udf, subst);
 
-    keyClone->setNonDiscardable(ANNOTATION_TRUE_FIXED);
+    keyClone->set_scripting_detail(SEQUENTIAL_FUNC_EXPR);
 
     const QueryLoc& keyloc = keyClone->get_loc();
 

=== modified file 'src/functions/func_index_ddl.h'
--- src/functions/func_index_ddl.h	2013-02-07 17:24:36 +0000
+++ src/functions/func_index_ddl.h	2013-09-12 07:54:27 +0000
@@ -45,10 +45,15 @@
   {
   }
 
-  unsigned short getScriptingKind() const { return SIMPLE_EXPR; }
-
   bool accessesDynCtx() const { return true; }
 
+  unsigned short getScriptingKind() const 
+  {
+    // Although the index creation will be applied immediately, we should
+    // NOT mark this function as sequential.
+    return SIMPLE_EXPR;
+  }
+
   bool mustCopyInputNodes(expr* fo, csize input) const { return false; }
 
   BoolAnnotationValue ignoresSortedNodes(expr* fo, csize input) const 

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/idx5.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/idx5.iter	2013-02-07 17:24:36 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/hashjoins/idx5.iter	2013-09-12 07:54:27 +0000
@@ -19,40 +19,6 @@
       <SingletonIterator value="xs:QName(,,tests)"/>
     </ElementIterator>
   </CtxVarDeclareIterator>
-  <flwor::FLWORIterator>
-    <LetVariable name="test_old" materialize="true">
-      <flwor::FLWORIterator>
-        <ForVariable name="$$context-item">
-          <CtxVarIterator varid="4" varname="tests" varkind="global"/>
-        </ForVariable>
-        <WhereClause>
-          <FnBooleanIterator>
-            <TypedValueCompareIterator_STRING>
-              <PromoteIterator type="xs:anyAtomicType">
-                <FnDataIterator>
-                  <ChildAxisIterator test kind="match_text_test" qname="*" typename="*" nill allowed="0">
-                    <ForVarIterator varname="$$context-item"/>
-                  </ChildAxisIterator>
-                </FnDataIterator>
-              </PromoteIterator>
-              <PromoteIterator type="xs:anyAtomicType">
-                <FnDataIterator>
-                  <ChildAxisIterator test kind="match_text_test" qname="*" typename="*" nill allowed="0">
-                    <CtxVarIterator varid="4" varname="tests" varkind="global"/>
-                  </ChildAxisIterator>
-                </FnDataIterator>
-              </PromoteIterator>
-            </TypedValueCompareIterator_STRING>
-          </FnBooleanIterator>
-        </WhereClause>
-        <ReturnClause>
-          <ForVarIterator varname="$$context-item"/>
-        </ReturnClause>
-      </flwor::FLWORIterator>
-    </LetVariable>
-    <ReturnClause>
-      <SingletonIterator value="xs:integer(1)"/>
-    </ReturnClause>
-  </flwor::FLWORIterator>
+  <SingletonIterator value="xs:integer(1)"/>
 </SequentialIterator>
 

=== modified file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/optim/hoist01.iter'
--- test/rbkt/ExpCompilerResults/IterPlan/zorba/optim/hoist01.iter	2013-03-04 21:00:58 +0000
+++ test/rbkt/ExpCompilerResults/IterPlan/zorba/optim/hoist01.iter	2013-09-12 07:54:27 +0000
@@ -9,42 +9,6 @@
 </OrIterator>
 
 Iterator tree for const-folded expr:
-<HoistIterator>
-  <FnBooleanIterator>
-    <IfThenElseIterator>
-      <FnBooleanIterator>
-        <FnConcatIterator/>
-      </FnBooleanIterator>
-      <FnConcatIterator/>
-      <FnConcatIterator/>
-    </IfThenElseIterator>
-  </FnBooleanIterator>
-</HoistIterator>
-
-Iterator tree for const-folded expr:
-<HoistIterator>
-  <FnBooleanIterator>
-    <IfThenElseIterator>
-      <FnBooleanIterator>
-        <FnConcatIterator/>
-      </FnBooleanIterator>
-      <FnConcatIterator/>
-      <FnConcatIterator/>
-    </IfThenElseIterator>
-  </FnBooleanIterator>
-</HoistIterator>
-
-Iterator tree for const-folded expr:
-<UnhoistIterator>
-  <SingletonIterator value="xs:boolean(false)"/>
-</UnhoistIterator>
-
-Iterator tree for const-folded expr:
-<UnhoistIterator>
-  <SingletonIterator value="xs:boolean(false)"/>
-</UnhoistIterator>
-
-Iterator tree for const-folded expr:
 <UDFunctionCallIterator function="testlib:searchUser">
   <SingletonIterator value="xs:integer(0)"/>
 </UDFunctionCallIterator>