Merge lp:~zorba-coders/zorba/caching into lp:zorba
- caching
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Markos Zaharioudakis |
Approved revision: | 10542 |
Merged at revision: | 10582 |
Proposed branch: | lp:~zorba-coders/zorba/caching |
Merge into: | lp:zorba |
Diff against target: |
1275 lines (+677/-106) 25 files modified
ChangeLog (+4/-0) doc/zorba/options.dox (+35/-0) include/zorba/pregenerated/diagnostic_list.h (+6/-0) modules/com/zorba-xquery/www/modules/pregenerated/errors.xq (+4/-0) modules/com/zorba-xquery/www/modules/pregenerated/warnings.xq (+20/-1) src/annotations/annotations.cpp (+7/-1) src/annotations/annotations.h (+2/-0) src/compiler/codegen/plan_visitor.cpp (+5/-37) src/diagnostics/diagnostic_en.xml (+35/-0) src/diagnostics/pregenerated/diagnostic_list.cpp (+9/-0) src/diagnostics/pregenerated/dict_en.cpp (+7/-0) src/functions/udf.cpp (+189/-5) src/functions/udf.h (+37/-0) src/runtime/core/fncall_iterator.cpp (+168/-24) src/runtime/core/fncall_iterator.h (+39/-6) src/runtime/visitors/printer_visitor_impl.cpp (+17/-1) src/store/api/index.h (+12/-0) src/store/naive/simple_index_general.cpp (+14/-0) src/store/naive/simple_index_general.h (+2/-0) src/store/naive/simple_store.cpp (+3/-0) src/store/naive/simple_temp_seq.cpp (+6/-26) src/store/naive/simple_temp_seq.h (+5/-5) test/rbkt/ExpCompilerResults/IterPlan/zorba/udf/udf-fib-rec.iter (+42/-0) test/rbkt/ExpQueryResults/zorba/udf/udf-fib-rec.xml.res (+1/-0) test/rbkt/Queries/zorba/udf/udf-fib-rec.xq (+8/-0) |
To merge this branch: | bzr merge lp:~zorba-coders/zorba/caching |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Markos Zaharioudakis | Approve | ||
Matthias Brantner | Approve | ||
Review via email: mp+85423@code.launchpad.net |
This proposal supersedes a proposal from 2011-12-12.
Commit message
- automatic caching of recursive, non-sequential, and deterministic functions with atomic parameter and return types
- %ann:cache and %ann:no-cache for controlling function result caching
Description of the change
- automatic caching of recursive, non-sequential, and deterministic functions with atomic parameter and return types
- %ann:cache and %ann:no-cache for controlling function result caching
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
The attempt to merge lp:~matthias-brantner/zorba/caching into lp:zorba failed. Below is the output from the failed tests.
CMake Error at /home/ceej/
Validation queue job caching-
final status was:
31 tests did not succeed - changes not commited.
Error in read script: /home/ceej/
Markos Zaharioudakis (markos-za) wrote : Posted in a previous version of this proposal | # |
Here are a few problems I can see so far:
1. I don't think we can do function caching for variadic functions. The current implementation of user_function:
2. The user_function:
3. In UDFunctionCallI
lSpec.
is not correct if the type of the param is a user-defined type (because the store will not understand its name). You must call getBaseBuiltinT
4. It's probably worthwhile to do something so that we don't have to recompute the args in case of a cache miss. What we have to do is create a temp sequence for each arg, and fill it with the corresponding arg value; then bind the arg reference to that temp seq, instead the arg wrapper.
5. It's probably worthwhile to allow subtypes of xs:anyAtomicType? as param types. Index creation already supports "null" keys, but we will need a new probe function. This we should probably do as a phase-2 task.
Matthias Brantner (matthias-brantner) wrote : Posted in a previous version of this proposal | # |
Adressed comments 1-4.
Matthias Brantner (matthias-brantner) : Posted in a previous version of this proposal | # |
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
The attempt to merge lp:~matthias-brantner/zorba/caching into lp:zorba failed. Below is the output from the failed tests.
CMake Error at /home/ceej/
Validation queue job caching-
final status was:
3 tests did not succeed - changes not commited.
Error in read script: /home/ceej/
Markos Zaharioudakis (markos-za) wrote : Posted in a previous version of this proposal | # |
Matthias, can you change the ownership to zorba-coders so that I can do some small changes (documentation and style)?
Markos Zaharioudakis (markos-za) wrote : Posted in a previous version of this proposal | # |
I think there is a bug in user_function:
Matthias Brantner (matthias-brantner) : Posted in a previous version of this proposal | # |
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Attempt to merge into lp:zorba failed due to conflicts:
text conflict in ChangeLog
text conflict in src/context/
Matthias Brantner (matthias-brantner) : Posted in a previous version of this proposal | # |
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Attempt to merge into lp:zorba failed due to conflicts:
text conflict in ChangeLog
text conflict in src/context/
Matthias Brantner (matthias-brantner) : Posted in a previous version of this proposal | # |
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Validation queue job caching-
All tests succeeded!
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1. Got: 1 Approve, 2 Pending.
Markos Zaharioudakis (markos-za) : Posted in a previous version of this proposal | # |
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Attempt to merge into lp:zorba failed due to conflicts:
text conflict in ChangeLog
Matthias Brantner (matthias-brantner) : Posted in a previous version of this proposal | # |
Markos Zaharioudakis (markos-za) : Posted in a previous version of this proposal | # |
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Attempt to merge into lp:zorba failed due to conflicts:
text conflict in ChangeLog
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Attempt to merge into lp:zorba failed due to conflicts:
text conflict in ChangeLog
Matthias Brantner (matthias-brantner) : | # |
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue job caching-
All tests succeeded!
Zorba Build Bot (zorba-buildbot) wrote : | # |
Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1. Got: 1 Approve, 2 Pending.
Markos Zaharioudakis (markos-za) : | # |
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue job caching-
All tests succeeded!
Preview Diff
1 | === modified file 'ChangeLog' |
2 | --- ChangeLog 2011-12-10 00:10:52 +0000 |
3 | +++ ChangeLog 2011-12-13 03:43:26 +0000 |
4 | @@ -1,6 +1,9 @@ |
5 | Zorba - The XQuery Processor |
6 | |
7 | version 2.2 |
8 | + |
9 | + * Caching of results for recursive functions with atomic parameter and return types. |
10 | + * Added %ann:cache and %ann:no-cache to enable or disable caching of results of functions with atomic parameter and return types. |
11 | * Added index management function to the C++ api's StaticCollectionManager. |
12 | |
13 | version 2.1 |
14 | @@ -77,6 +80,7 @@ |
15 | * General index cannot be declared as unique if the type of its key is |
16 | xs:anyAtomicType or xs:untypedAtomic. |
17 | * Added undo for node revalidation |
18 | + * Optimization for count(collection()) expressions |
19 | * Fixed bug #867133 (SWIG PHP build failure on Mac OSX) |
20 | * Fixed bug #872796 (validate-in-place can interfere with other update primitives) |
21 | * Fixed bug #872799 (validate-in-place can set incorrect types) |
22 | |
23 | === modified file 'doc/zorba/options.dox' |
24 | --- doc/zorba/options.dox 2011-09-14 06:15:19 +0000 |
25 | +++ doc/zorba/options.dox 2011-12-13 03:43:26 +0000 |
26 | @@ -278,6 +278,41 @@ |
27 | In order to be able to use the value twice, the <tt>string:materialize</tt> function must be used to materialize the entire contents of the file <tt>myfile.txt</tt> in memory. |
28 | Otherwise, the error zerr:ZSTR0055 is raised. |
29 | |
30 | + |
31 | +\paragraph caching_annotation Caching Results of Functions |
32 | + |
33 | +Caching of function results may improve performance when computationally |
34 | +expensive functions are invoked multiple times with the same arguments. |
35 | + |
36 | +If the optimization level is O1 or higher, Zorba automatically caches results |
37 | +of recursive, deterministic, and non-sequential functions whose parameter and |
38 | +return types are subtypes of xs:anyAtomicType. Specifically, if such a function |
39 | +is called more than once with the same arguments, the result of the first call |
40 | +will be cached and subsequent calls will return the cached value without |
41 | +re-evaluating the function. |
42 | + |
43 | +For example, in the following recursive function computing a fibonacci number, |
44 | +each result is automatically cached and, hence, dramatically improves performance. |
45 | + |
46 | +\include zorba/udf/udf-fib-rec.xq |
47 | + |
48 | +Specifically, this optimization reduces the complexity of the function from |
49 | +O(1.6^n) to O(n). |
50 | + |
51 | +In order to explicitly disable function caching, the user can specify the |
52 | +<tt>%ann:no-cache</tt> annotation. |
53 | + |
54 | +In addition, the user can use the <tt>%ann:cache</tt> annotation to cache the |
55 | +results of functions other than the ones that are automatically cached. However, |
56 | +this will only work if the function is not updating and its parameter and return |
57 | +types are subtypes of xs:anyAtomicType; otherwise, Zorba will raise a warning |
58 | +(zwarn:ZWST0005) and simply ignore the %ann:cache annotation. |
59 | + |
60 | +Please note, that explicitly enforcing caching for sequential or nondeterministic |
61 | +functions might not give the intended result. In such cases, Zorba will raise a |
62 | +warning (zwarn:ZWST0006) but obey the %ann:cache annotation. |
63 | + |
64 | + |
65 | \paragraph collection_index_annotations Annotations on Collections and Indexes |
66 | |
67 | The \ref xqddf uses annotations to assign properties to collections and indexes. |
68 | |
69 | === modified file 'include/zorba/pregenerated/diagnostic_list.h' |
70 | --- include/zorba/pregenerated/diagnostic_list.h 2011-11-15 08:23:20 +0000 |
71 | +++ include/zorba/pregenerated/diagnostic_list.h 2011-12-13 03:43:26 +0000 |
72 | @@ -600,6 +600,8 @@ |
73 | |
74 | extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZDDY0034_INDEX_RANGE_VALUE_PROBE_BAD_KEY_TYPES; |
75 | |
76 | +extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZDDY0035_INDEX_GENERAL_INSERT; |
77 | + |
78 | extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZDDY0031_IC_NOT_DECLARED; |
79 | |
80 | extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZDDY0032_IC_NOT_ACTIVATED; |
81 | @@ -752,6 +754,10 @@ |
82 | |
83 | extern ZORBA_DLL_PUBLIC ZorbaWarningCode ZWST0004_AMBIGUOUS_SEQUENTIAL_FLWOR; |
84 | |
85 | +extern ZORBA_DLL_PUBLIC ZorbaWarningCode ZWST0005_CACHING_NOT_POSSIBLE; |
86 | + |
87 | +extern ZORBA_DLL_PUBLIC ZorbaWarningCode ZWST0006_CACHING_MIGHT_NOT_BE_INTENDED; |
88 | + |
89 | } // namespace zwarn |
90 | } // namespace zorba |
91 | #endif /* ZORBA_DIAGNOSTIC_LIST_API_H */ |
92 | |
93 | === modified file 'modules/com/zorba-xquery/www/modules/pregenerated/errors.xq' |
94 | --- modules/com/zorba-xquery/www/modules/pregenerated/errors.xq 2011-11-15 08:23:20 +0000 |
95 | +++ modules/com/zorba-xquery/www/modules/pregenerated/errors.xq 2011-12-13 03:43:26 +0000 |
96 | @@ -501,6 +501,10 @@ |
97 | |
98 | (:~ |
99 | :) |
100 | +declare variable $zerr:ZDDY0035 as xs:QName := fn:QName($zerr:NS, "zerr:ZDDY0035"); |
101 | + |
102 | +(:~ |
103 | +:) |
104 | declare variable $zerr:ZDDY0031 as xs:QName := fn:QName($zerr:NS, "zerr:ZDDY0031"); |
105 | |
106 | (:~ |
107 | |
108 | === modified file 'modules/com/zorba-xquery/www/modules/pregenerated/warnings.xq' |
109 | --- modules/com/zorba-xquery/www/modules/pregenerated/warnings.xq 2011-11-15 08:10:49 +0000 |
110 | +++ modules/com/zorba-xquery/www/modules/pregenerated/warnings.xq 2011-12-13 03:43:26 +0000 |
111 | @@ -53,4 +53,23 @@ |
112 | |
113 | (:~ |
114 | :) |
115 | -declare variable $zwarn:ZWST0004 as xs:QName := fn:QName($zwarn:NS, "zwarn:ZWST0004"); |
116 | \ No newline at end of file |
117 | +declare variable $zwarn:ZWST0004 as xs:QName := fn:QName($zwarn:NS, "zwarn:ZWST0004"); |
118 | + |
119 | +(:~ |
120 | + : |
121 | + : This warning is raised if the user explicitly enables caching |
122 | + : of function results (using the %ann:cache annotation) but the function |
123 | + : is updating or its parameter and return types are not subtypes of |
124 | + : xs:anyAtomicType. |
125 | + : |
126 | +:) |
127 | +declare variable $zwarn:ZWST0005 as xs:QName := fn:QName($zwarn:NS, "zwarn:ZWST0005"); |
128 | + |
129 | +(:~ |
130 | + : |
131 | + : This warning is raised if the user explicitly enables caching |
132 | + : of function results (using the %ann:cache annotation) and the function |
133 | + : is annotated as sequential or nondeterministic. |
134 | + : |
135 | +:) |
136 | +declare variable $zwarn:ZWST0006 as xs:QName := fn:QName($zwarn:NS, "zwarn:ZWST0006"); |
137 | \ No newline at end of file |
138 | |
139 | === modified file 'src/annotations/annotations.cpp' |
140 | --- src/annotations/annotations.cpp 2011-11-07 06:32:00 +0000 |
141 | +++ src/annotations/annotations.cpp 2011-12-13 03:43:26 +0000 |
142 | @@ -100,6 +100,9 @@ |
143 | |
144 | ZANN(streamable, streamable); |
145 | |
146 | + ZANN(cache, cache); |
147 | + ZANN(no-cache, nocache); |
148 | + |
149 | // |
150 | // Zorba annotations - xqddf |
151 | // |
152 | @@ -168,6 +171,10 @@ |
153 | ZANN(zann_nonsequential)); |
154 | |
155 | theRuleSet.push_back( |
156 | + ZANN(zann_cache) | |
157 | + ZANN(zann_nocache)); |
158 | + |
159 | + theRuleSet.push_back( |
160 | ZANN(fn_private) | |
161 | ZANN(fn_public)); |
162 | |
163 | @@ -433,6 +440,5 @@ |
164 | } |
165 | |
166 | |
167 | - |
168 | } /* namespace zorba */ |
169 | /* vim:set et sw=2 ts=2: */ |
170 | |
171 | === modified file 'src/annotations/annotations.h' |
172 | --- src/annotations/annotations.h 2011-11-01 13:47:10 +0000 |
173 | +++ src/annotations/annotations.h 2011-12-13 03:43:26 +0000 |
174 | @@ -55,6 +55,8 @@ |
175 | zann_nonassignable, |
176 | zann_sequential, |
177 | zann_nonsequential, |
178 | + zann_cache, |
179 | + zann_nocache, |
180 | zann_variadic, |
181 | zann_streamable, |
182 | zann_unique, |
183 | |
184 | === modified file 'src/compiler/codegen/plan_visitor.cpp' |
185 | --- src/compiler/codegen/plan_visitor.cpp 2011-12-07 20:46:23 +0000 |
186 | +++ src/compiler/codegen/plan_visitor.cpp 2011-12-13 03:43:26 +0000 |
187 | @@ -98,6 +98,7 @@ |
188 | #endif |
189 | |
190 | #include "functions/function.h" |
191 | +#include "functions/udf.h" |
192 | #include "functions/library.h" |
193 | |
194 | #include "types/typeops.h" |
195 | @@ -2276,7 +2277,7 @@ |
196 | { |
197 | CODEGEN_TRACE_OUT(""); |
198 | |
199 | - const function* func = v.get_func(); |
200 | + function* func = v.get_func(); |
201 | |
202 | std::vector<PlanIter_t> argv; |
203 | |
204 | @@ -2320,45 +2321,12 @@ |
205 | dynamic_cast<EnclosedIterator*>(iter.getp())->setInUpdateExpr(); |
206 | } |
207 | } |
208 | -#if 0 |
209 | else if (func->isUdf()) |
210 | { |
211 | - const user_function* udf = static_cast<const user_function*>(func); |
212 | - |
213 | - if (udf->isExiting()) |
214 | - { |
215 | - TypeManager* tm = v.get_type_manager(); |
216 | - |
217 | - const xqtref_t& udfType = udf->getSignature().returnType(); |
218 | - |
219 | - expr* body = udf->getBody(); |
220 | - |
221 | - std::vector<expr*> exitExprs; |
222 | - ulong numExitExprs; |
223 | - ulong i; |
224 | - |
225 | - body->get_exprs_of_kind(exit_expr_kind, exitExprs); |
226 | - |
227 | - for (i = 0; i < numExitExprs; ++i) |
228 | - { |
229 | - if (!TypeOps::is_subtype(tm, |
230 | - *exitExprs[i]->get_return_type(), |
231 | - *udfType, |
232 | - loc)) |
233 | - break; |
234 | - } |
235 | - |
236 | - if (i < numExitExprs) |
237 | - { |
238 | - UDFunctionCallIterator* udfIter = |
239 | - dynamic_cast<UDFunctionCallIterator*>(iter.getp()); |
240 | - |
241 | - ZORBA_ASSERT(udfIter != NULL); |
242 | - udfIter->setCheckType(); |
243 | - } |
244 | - } |
245 | + // need to computeResultCaching here for iterprint to work |
246 | + user_function* udf = static_cast<user_function*>(func); |
247 | + udf->computeResultCaching(theCCB->theXQueryDiagnostics); |
248 | } |
249 | -#endif |
250 | } |
251 | else |
252 | { |
253 | |
254 | === modified file 'src/diagnostics/diagnostic_en.xml' |
255 | --- src/diagnostics/diagnostic_en.xml 2011-12-07 20:46:23 +0000 |
256 | +++ src/diagnostics/diagnostic_en.xml 2011-12-13 03:43:26 +0000 |
257 | @@ -2013,6 +2013,10 @@ |
258 | <value>"$1": index range-value probe has search keys with incompatible types</value> |
259 | </diagnostic> |
260 | |
261 | + <diagnostic code="ZDDY0035" name="INDEX_GENERAL_INSERT"> |
262 | + <value>"$1": index inserting more than one key not allowed for general index</value> |
263 | + </diagnostic> |
264 | + |
265 | <diagnostic code="ZDDY0031" name="IC_NOT_DECLARED"> |
266 | <value>"$1": integrity constraint is not declared</value> |
267 | </diagnostic> |
268 | @@ -2323,6 +2327,37 @@ |
269 | <value>Sequential FLWOR expr may not have the semantics you expect</value> |
270 | </diagnostic> |
271 | |
272 | + <diagnostic code="ZWST0005" name="CACHING_NOT_POSSIBLE"> |
273 | + <comment> |
274 | + This warning is raised if the user explicitly enables caching |
275 | + of function results (using the %ann:cache annotation) but the function |
276 | + is updating or its parameter and return types are not subtypes of |
277 | + xs:anyAtomicType. |
278 | + </comment> |
279 | + <value>"$1": function caching not possible; $2</value> |
280 | + <entry key="RETURN_TYPE"> |
281 | + <value>return type ($3) is not subtype of xs:anyAtomicType</value> |
282 | + </entry> |
283 | + <entry key="PARAM_TYPE"> |
284 | + <value>type of parameter $3 is $4 which is not a subtype of xs:anyAtomicType</value> |
285 | + </entry> |
286 | + <entry key="UPDATING"> |
287 | + <value>function is updating</value> |
288 | + </entry> |
289 | + <entry key="VARIADIC"> |
290 | + <value>function is variadic</value> |
291 | + </entry> |
292 | + </diagnostic> |
293 | + |
294 | + <diagnostic code="ZWST0006" name="CACHING_MIGHT_NOT_BE_INTENDED"> |
295 | + <comment> |
296 | + This warning is raised if the user explicitly enables caching |
297 | + of function results (using the %ann:cache annotation) and the function |
298 | + is annotated as sequential or nondeterministic. |
299 | + </comment> |
300 | + <value>"$1": function caching might not give the intended result because the function is declared as $2</value> |
301 | + </diagnostic> |
302 | + |
303 | </namespace> |
304 | |
305 | <!--////////// Subvalues /////////////////////////////////////////////////--> |
306 | |
307 | === modified file 'src/diagnostics/pregenerated/diagnostic_list.cpp' |
308 | --- src/diagnostics/pregenerated/diagnostic_list.cpp 2011-11-15 08:23:20 +0000 |
309 | +++ src/diagnostics/pregenerated/diagnostic_list.cpp 2011-12-13 03:43:26 +0000 |
310 | @@ -879,6 +879,9 @@ |
311 | ZorbaErrorCode ZDDY0034_INDEX_RANGE_VALUE_PROBE_BAD_KEY_TYPES( "ZDDY0034" ); |
312 | |
313 | |
314 | +ZorbaErrorCode ZDDY0035_INDEX_GENERAL_INSERT( "ZDDY0035" ); |
315 | + |
316 | + |
317 | ZorbaErrorCode ZDDY0031_IC_NOT_DECLARED( "ZDDY0031" ); |
318 | |
319 | |
320 | @@ -1104,6 +1107,12 @@ |
321 | ZorbaWarningCode ZWST0004_AMBIGUOUS_SEQUENTIAL_FLWOR( "ZWST0004" ); |
322 | |
323 | |
324 | +ZorbaWarningCode ZWST0005_CACHING_NOT_POSSIBLE( "ZWST0005" ); |
325 | + |
326 | + |
327 | +ZorbaWarningCode ZWST0006_CACHING_MIGHT_NOT_BE_INTENDED( "ZWST0006" ); |
328 | + |
329 | + |
330 | } // namespace zwarn |
331 | |
332 | } // namespace zorba |
333 | |
334 | === modified file 'src/diagnostics/pregenerated/dict_en.cpp' |
335 | --- src/diagnostics/pregenerated/dict_en.cpp 2011-12-01 16:19:52 +0000 |
336 | +++ src/diagnostics/pregenerated/dict_en.cpp 2011-12-13 03:43:26 +0000 |
337 | @@ -298,6 +298,7 @@ |
338 | { "ZDDY0032", "\"$1\": integrity constraint is not activated" }, |
339 | { "ZDDY0033", "\"$1\": integrity constraint not met for collection \"$2\"" }, |
340 | { "ZDDY0034", "\"$1\": index range-value probe has search keys with incompatible types" }, |
341 | + { "ZDDY0035", "\"$1\": index inserting more than one key not allowed for general index" }, |
342 | { "ZDST0001", "\"$1\": collection already declared" }, |
343 | { "ZDST0002", "\"$1\": collection already imported into module \"$2\"" }, |
344 | { "ZDST0003", "\"$1\": collection declaration not allowed in main module" }, |
345 | @@ -360,6 +361,8 @@ |
346 | { "ZWST0002", "\"$1\": unknown or unsupported annotation" }, |
347 | { "ZWST0003", "\"$1\": function declared sequential, but has non-sequential body" }, |
348 | { "ZWST0004", "Sequential FLWOR expr may not have the semantics you expect" }, |
349 | + { "ZWST0005", "\"$1\": function caching not possible; $2" }, |
350 | + { "ZWST0006", "\"$1\": function caching might not give the intended result because the function is declared as $2" }, |
351 | { "ZXQD0001", "\"$1\": prefix not declared when calling function \"$2\" from $3" }, |
352 | { "ZXQD0002", "\"$1\": $2" }, |
353 | { "ZXQD0003", "inconsistent options to the parse-xml-fragment() function: $1" }, |
354 | @@ -664,6 +667,10 @@ |
355 | { "~XUST0002_UDF_2", "\"$2\": function declared updating but body is not updating or vacuous" }, |
356 | { "~ZDST0060_unknown_localname", "unknown localname ($3)" }, |
357 | { "~ZDST0060_unknown_namespace", "unknown namespace ($3)" }, |
358 | + { "~ZWST0005_PARAM_TYPE", "type of parameter $3 is $4 which is not a subtype of xs:anyAtomicType" }, |
359 | + { "~ZWST0005_RETURN_TYPE", "return type ($3) is not subtype of xs:anyAtomicType" }, |
360 | + { "~ZWST0005_UPDATING", "function is updating" }, |
361 | + { "~ZWST0005_VARIADIC", "function is variadic" }, |
362 | { "~ZXQD0004_NON_NEGATIVE", "given value must be non-negative ($2)" }, |
363 | { "~ZXQD0004_NOT_WITHIN_RANGE", "not within allowed range ($2)" }, |
364 | { "~ZXQP0004_TypeOps_is_in_scope_ForFunctionItemTypes", "TypeOps::is_in_scope() for function-item types" }, |
365 | |
366 | === modified file 'src/functions/udf.cpp' |
367 | --- src/functions/udf.cpp 2011-10-18 17:37:41 +0000 |
368 | +++ src/functions/udf.cpp 2011-12-13 03:43:26 +0000 |
369 | @@ -26,10 +26,15 @@ |
370 | #include "compiler/rewriter/framework/rewriter.h" |
371 | |
372 | #include "functions/udf.h" |
373 | +#include "annotations/annotations.h" |
374 | #include "functions/function_impl.h" |
375 | |
376 | +#include "diagnostics/xquery_warning.h" |
377 | + |
378 | #include "types/typeops.h" |
379 | |
380 | +#include "store/api/index.h" // needed for destruction of the cache |
381 | + |
382 | |
383 | namespace zorba |
384 | { |
385 | @@ -54,7 +59,10 @@ |
386 | theIsExiting(false), |
387 | theIsLeaf(true), |
388 | theIsOptimized(false), |
389 | - thePlanStateSize(0) |
390 | + thePlanStateSize(0), |
391 | + theCache(0), |
392 | + theCacheResults(false), |
393 | + theCacheComputed(false) |
394 | { |
395 | setFlag(FunctionConsts::isUDF); |
396 | resetFlag(FunctionConsts::isBuiltin); |
397 | @@ -318,7 +326,7 @@ |
398 | /******************************************************************************* |
399 | |
400 | ********************************************************************************/ |
401 | - PlanIter_t user_function::getPlan(CompilerCB* ccb, uint32_t& planStateSize) |
402 | +PlanIter_t user_function::getPlan(CompilerCB* ccb, uint32_t& planStateSize) |
403 | { |
404 | if (thePlan == NULL) |
405 | { |
406 | @@ -372,9 +380,185 @@ |
407 | /******************************************************************************* |
408 | |
409 | ********************************************************************************/ |
410 | -CODEGEN_DEF(user_function) |
411 | -{ |
412 | - return new UDFunctionCallIterator(aSctx, aLoc, aArgs, this); |
413 | +store::Index* user_function::getCache() const |
414 | +{ |
415 | + return theCache.getp(); |
416 | +} |
417 | + |
418 | + |
419 | +/******************************************************************************* |
420 | + |
421 | +********************************************************************************/ |
422 | +void user_function::setCache(store::Index* aCache) |
423 | +{ |
424 | + theCache = aCache; |
425 | +} |
426 | + |
427 | + |
428 | +/******************************************************************************* |
429 | + |
430 | +********************************************************************************/ |
431 | +bool user_function::cacheResults() const |
432 | +{ |
433 | + return theCacheResults; |
434 | +} |
435 | + |
436 | + |
437 | +/******************************************************************************* |
438 | + only cache recursive (non-sequential, non-updating, deterministic) |
439 | + functions with singleton atomic input and output |
440 | +********************************************************************************/ |
441 | +void user_function::computeResultCaching(XQueryDiagnostics* diag) |
442 | +{ |
443 | + if (theCacheComputed) |
444 | + { |
445 | + return; |
446 | + } |
447 | + |
448 | + struct OnExit |
449 | + { |
450 | + private: |
451 | + bool& theResult; |
452 | + bool& theCacheComputed; |
453 | + |
454 | + public: |
455 | + OnExit(bool& aResult, bool& aCacheComputed) |
456 | + : |
457 | + theResult(aResult), |
458 | + theCacheComputed(aCacheComputed) {} |
459 | + |
460 | + void cache() { theResult = true; } |
461 | + |
462 | + ~OnExit() |
463 | + { |
464 | + theCacheComputed = true; |
465 | + } |
466 | + }; |
467 | + |
468 | + // will be destroyed when the function is exited |
469 | + // set caching to true if cache() is called |
470 | + OnExit lExit(theCacheResults, theCacheComputed); |
471 | + |
472 | + // check necessary conditions |
473 | + // %ann:cache or not %ann:no-cache |
474 | + if (theAnnotationList && |
475 | + theAnnotationList->contains(AnnotationInternal::zann_nocache)) |
476 | + { |
477 | + return; |
478 | + } |
479 | + |
480 | + // was the %ann:cache annotation given explicitly by the user |
481 | + bool lExplicitCacheRequest = |
482 | + (theAnnotationList ? |
483 | + theAnnotationList->contains(AnnotationInternal::zann_cache) : |
484 | + false); |
485 | + |
486 | + if (isVariadic()) |
487 | + { |
488 | + if (lExplicitCacheRequest) |
489 | + { |
490 | + diag->add_warning( |
491 | + NEW_XQUERY_WARNING(zwarn::ZWST0005_CACHING_NOT_POSSIBLE, |
492 | + WARN_PARAMS(getName()->getStringValue(), ZED(ZWST0005_VARIADIC)), |
493 | + WARN_LOC(theLoc))); |
494 | + } |
495 | + return; |
496 | + } |
497 | + |
498 | + // parameter and return types are subtype of xs:anyAtomicType? |
499 | + const xqtref_t& lRes = theSignature.returnType(); |
500 | + TypeManager* tm = lRes->get_manager(); |
501 | + |
502 | + if (!TypeOps::is_subtype(tm, |
503 | + *lRes, |
504 | + *GENV_TYPESYSTEM.ANY_ATOMIC_TYPE_ONE, |
505 | + theLoc)) |
506 | + { |
507 | + if (lExplicitCacheRequest) |
508 | + { |
509 | + diag->add_warning( |
510 | + NEW_XQUERY_WARNING(zwarn::ZWST0005_CACHING_NOT_POSSIBLE, |
511 | + WARN_PARAMS(getName()->getStringValue(), |
512 | + ZED(ZWST0005_RETURN_TYPE), |
513 | + lRes->toString()), |
514 | + WARN_LOC(theLoc))); |
515 | + } |
516 | + return; |
517 | + } |
518 | + |
519 | + csize lArity = theSignature.paramCount(); |
520 | + for (csize i = 0; i < lArity; ++i) |
521 | + { |
522 | + const xqtref_t& lArg = theSignature[i]; |
523 | + if (!TypeOps::is_subtype(tm, |
524 | + *lArg, |
525 | + *GENV_TYPESYSTEM.ANY_ATOMIC_TYPE_ONE, |
526 | + theLoc)) |
527 | + { |
528 | + if (lExplicitCacheRequest) |
529 | + { |
530 | + diag->add_warning( |
531 | + NEW_XQUERY_WARNING(zwarn::ZWST0005_CACHING_NOT_POSSIBLE, |
532 | + WARN_PARAMS(getName()->getStringValue(), |
533 | + ZED(ZWST0005_PARAM_TYPE), |
534 | + i+1, |
535 | + lArg->toString()), |
536 | + WARN_LOC(theLoc))); |
537 | + } |
538 | + return; |
539 | + } |
540 | + } |
541 | + |
542 | + // function updating? |
543 | + if (isUpdating()) |
544 | + { |
545 | + if (lExplicitCacheRequest) |
546 | + { |
547 | + diag->add_warning( |
548 | + NEW_XQUERY_WARNING(zwarn::ZWST0005_CACHING_NOT_POSSIBLE, |
549 | + WARN_PARAMS(getName()->getStringValue(), ZED(ZWST0005_UPDATING)), |
550 | + WARN_LOC(theLoc))); |
551 | + } |
552 | + return; |
553 | + } |
554 | + |
555 | + if (isSequential() || !isDeterministic()) |
556 | + { |
557 | + if (lExplicitCacheRequest) |
558 | + { |
559 | + diag->add_warning( |
560 | + NEW_XQUERY_WARNING(zwarn::ZWST0006_CACHING_MIGHT_NOT_BE_INTENDED, |
561 | + WARN_PARAMS(getName()->getStringValue(), |
562 | + (isSequential()?"sequential":"non-deterministic")), |
563 | + WARN_LOC(theLoc))); |
564 | + |
565 | + lExit.cache(); |
566 | + } |
567 | + return; |
568 | + } |
569 | + |
570 | + |
571 | + // optimization is prerequisite before invoking isRecursive |
572 | + if (!lExplicitCacheRequest && isOptimized() && !isRecursive()) |
573 | + { |
574 | + return; |
575 | + } |
576 | + |
577 | + lExit.cache(); |
578 | +} |
579 | + |
580 | + |
581 | +/******************************************************************************* |
582 | + |
583 | +********************************************************************************/ |
584 | +PlanIter_t user_function::codegen( |
585 | + CompilerCB* cb, |
586 | + static_context* sctx, |
587 | + const QueryLoc& loc, |
588 | + std::vector<PlanIter_t>& argv, |
589 | + AnnotationHolder& ann) const |
590 | +{ |
591 | + return new UDFunctionCallIterator(sctx, loc, argv, this); |
592 | } |
593 | |
594 | |
595 | |
596 | === modified file 'src/functions/udf.h' |
597 | --- src/functions/udf.h 2011-10-18 17:37:41 +0000 |
598 | +++ src/functions/udf.h 2011-12-13 03:43:26 +0000 |
599 | @@ -25,6 +25,12 @@ |
600 | namespace zorba |
601 | { |
602 | |
603 | + namespace store |
604 | + { |
605 | + class Index; |
606 | + typedef rchandle<Index> Index_t; |
607 | + } |
608 | + |
609 | |
610 | /******************************************************************************* |
611 | A udf with params $x1, $x2, ..., $xn and a body_expr is translated into a |
612 | @@ -78,6 +84,25 @@ |
613 | references of an arg var, these references are "mutually exclusive", ie, |
614 | at most one of the references will actually be reached during each particular |
615 | execution of the body. |
616 | + |
617 | + theCache: |
618 | + --------- |
619 | + Maps the arg values of an invocation to the result of that invocation. |
620 | + If an invocation uses the same arg values as a previous invocation, the cached |
621 | + result is simply returned without re-evaluating the udf. |
622 | + |
623 | + theCacheResults: |
624 | + ---------------- |
625 | + Tells whether caching should be done for this udf or not. |
626 | + |
627 | + theCacheComputed: |
628 | + ----------------- |
629 | + Tells whether theCacheResults has been computed already or not. |
630 | + theCacheResults is computed by the computeResultCaching() method, which is |
631 | + invoked during codegen every time a udf call is encountered. The same udf may |
632 | + be invoked multiple times, but the computation of theCacheResults needs to |
633 | + be done only once. So, during the 1st invcocation of computeResultCaching(), |
634 | + theCacheComputed is set to true, and subsequent invocations are noops. |
635 | ********************************************************************************/ |
636 | class user_function : public function |
637 | { |
638 | @@ -102,6 +127,10 @@ |
639 | uint32_t thePlanStateSize; |
640 | std::vector<ArgVarRefs> theArgVarsRefs; |
641 | |
642 | + store::Index_t theCache; |
643 | + bool theCacheResults; |
644 | + bool theCacheComputed; |
645 | + |
646 | public: |
647 | SERIALIZABLE_CLASS(user_function) |
648 | user_function(::zorba::serialization::Archiver& ar); |
649 | @@ -160,6 +189,14 @@ |
650 | |
651 | const std::vector<ArgVarRefs>& getArgVarsRefs() const; |
652 | |
653 | + store::Index* getCache() const; |
654 | + |
655 | + void setCache(store::Index* aCache); |
656 | + |
657 | + bool cacheResults() const; |
658 | + |
659 | + void computeResultCaching(XQueryDiagnostics*); |
660 | + |
661 | PlanIter_t codegen( |
662 | CompilerCB* cb, |
663 | static_context* sctx, |
664 | |
665 | === modified file 'src/runtime/core/fncall_iterator.cpp' |
666 | --- src/runtime/core/fncall_iterator.cpp 2011-10-30 00:18:34 +0000 |
667 | +++ src/runtime/core/fncall_iterator.cpp 2011-12-13 03:43:26 +0000 |
668 | @@ -48,6 +48,11 @@ |
669 | |
670 | #include "util/string_util.h" |
671 | |
672 | +#include "store/api/index.h" |
673 | +#include "store/api/store.h" |
674 | +#include "store/api/iterator_factory.h" |
675 | +#include "store/api/temp_seq.h" |
676 | + |
677 | #ifdef ZORBA_WITH_DEBUGGER |
678 | #include "debugger/debugger_commons.h" |
679 | |
680 | @@ -93,7 +98,8 @@ |
681 | thePlanState(NULL), |
682 | thePlanStateSize(0), |
683 | theLocalDCtx(NULL), |
684 | - thePlanOpen(false) |
685 | + thePlanOpen(false), |
686 | + theCache(0) |
687 | { |
688 | } |
689 | |
690 | @@ -153,6 +159,8 @@ |
691 | { |
692 | thePlan->reset(*thePlanState); |
693 | } |
694 | + |
695 | + theArgValues.clear(); |
696 | } |
697 | |
698 | |
699 | @@ -198,6 +206,113 @@ |
700 | /******************************************************************************* |
701 | |
702 | ********************************************************************************/ |
703 | +bool UDFunctionCallIterator::isCached() const |
704 | +{ |
705 | + return theUDF->cacheResults(); |
706 | +} |
707 | + |
708 | + |
709 | +/******************************************************************************* |
710 | + |
711 | +********************************************************************************/ |
712 | +void UDFunctionCallIterator::createCache( |
713 | + PlanState& planState, |
714 | + UDFunctionCallIteratorState* state) |
715 | +{ |
716 | + store::Index_t lIndex = theUDF->getCache(); |
717 | + |
718 | + if (!lIndex && theUDF->cacheResults()) |
719 | + { |
720 | + const signature& sig = theUDF->getSignature(); |
721 | + |
722 | + csize numArgs = theChildren.size(); |
723 | + |
724 | + store::IndexSpecification lSpec; |
725 | + lSpec.theNumKeyColumns = numArgs; |
726 | + lSpec.theKeyTypes.resize(numArgs); |
727 | + lSpec.theCollations.resize(numArgs); |
728 | + lSpec.theIsTemp = true; |
729 | + lSpec.theIsUnique = true; |
730 | + |
731 | + for (csize i = 0; i < numArgs; ++i) |
732 | + { |
733 | + lSpec.theKeyTypes[i] = sig[i]->getBaseBuiltinType()->get_qname().getp(); |
734 | + } |
735 | + |
736 | + lIndex = GENV_STORE.createIndex(theUDF->getName(), lSpec, 0); |
737 | + |
738 | + theUDF->setCache(lIndex.getp()); // cache the cache in the function itself |
739 | + |
740 | + state->theArgValues.reserve(numArgs); |
741 | + } |
742 | + |
743 | + state->theCache = lIndex.getp(); |
744 | +} |
745 | + |
746 | + |
747 | +/******************************************************************************* |
748 | + |
749 | +********************************************************************************/ |
750 | +bool UDFunctionCallIterator::probeCache( |
751 | + PlanState& planState, |
752 | + UDFunctionCallIteratorState* state, |
753 | + store::Item_t& result, |
754 | + std::vector<store::Item_t>& aKey) const |
755 | +{ |
756 | + if (!state->theCache) |
757 | + return false; |
758 | + |
759 | + store::IndexCondition_t lCond = |
760 | + state->theCache->createCondition(store::IndexCondition::POINT_VALUE); |
761 | + |
762 | + std::vector<store::Iterator_t>::iterator lIter = state->theArgWrappers.begin(); |
763 | + |
764 | + for (; lIter != state->theArgWrappers.end(); ++lIter) |
765 | + { |
766 | + store::Iterator_t& argWrapper = (*lIter); |
767 | + store::Item_t lArg; |
768 | + if (argWrapper) // might be 0 if argument is not used |
769 | + { |
770 | + argWrapper->next(lArg); // guaranteed to have exactly one result |
771 | + } |
772 | + aKey.push_back(lArg); |
773 | + lCond->pushItem(lArg); |
774 | + } |
775 | + |
776 | + store::IndexProbeIterator_t probeIte = |
777 | + GENV_STORE.getIteratorFactory()->createIndexProbeIterator(state->theCache); |
778 | + |
779 | + probeIte->init(lCond); |
780 | + probeIte->open(); |
781 | + return probeIte->next(result); |
782 | +} |
783 | + |
784 | + |
785 | +/******************************************************************************* |
786 | + |
787 | +********************************************************************************/ |
788 | +void UDFunctionCallIterator::insertCacheEntry( |
789 | + UDFunctionCallIteratorState* state, |
790 | + std::vector<store::Item_t>& aKey, |
791 | + store::Item_t& aValue) const |
792 | +{ |
793 | + if (state->theCache) |
794 | + { |
795 | + std::auto_ptr<store::IndexKey> k(new store::IndexKey()); |
796 | + store::IndexKey* k2 = k.get(); |
797 | + k->theItems = aKey; |
798 | + store::Item_t lTmp = aValue; // insert will eventually transfer the Item_t |
799 | + if (!state->theCache->insert(k2, lTmp)) |
800 | + { |
801 | + k.release(); |
802 | + } |
803 | + } |
804 | +} |
805 | + |
806 | + |
807 | +/******************************************************************************* |
808 | + |
809 | +********************************************************************************/ |
810 | void UDFunctionCallIterator::openImpl(PlanState& planState, uint32_t& offset) |
811 | { |
812 | UDFunctionCallIteratorState* state; |
813 | @@ -229,6 +344,11 @@ |
814 | // the plan state (but not the state block) and dynamic context. |
815 | state->open(planState, theUDF); |
816 | |
817 | + // if the results of the function should be cached (prereq: atomic in and out) |
818 | + // this functions stores an index in the dynamic context that contains |
819 | + // the cached results. The name of the index is the name of the function. |
820 | + createCache(planState, state); |
821 | + |
822 | // Create a wrapper over each subplan that computes an argument expr, if the |
823 | // associated param is actually used anywhere in the function body. |
824 | csize numArgs = theChildren.size(); |
825 | @@ -301,6 +421,9 @@ |
826 | { |
827 | try |
828 | { |
829 | + std::vector<store::Item_t> lKey; |
830 | + bool lCacheHit; |
831 | + |
832 | UDFunctionCallIteratorState* state; |
833 | DEFAULT_STACK_INIT(UDFunctionCallIteratorState, state, planState); |
834 | |
835 | @@ -314,19 +437,31 @@ |
836 | state->thePlanOpen = true; |
837 | } |
838 | |
839 | - // Bind the args. |
840 | + // check if there is a cache and the result is already in the cache |
841 | + lCacheHit = probeCache(planState, state, result, lKey); |
842 | + |
843 | + // if not in the cache, we bind the arguments to the function |
844 | + if (!lCacheHit) |
845 | { |
846 | const std::vector<ArgVarRefs>& argsRefs = theUDF->getArgVarsRefs(); |
847 | - std::vector<ArgVarRefs>::const_iterator argsRefsIte = argsRefs.begin(); |
848 | - std::vector<ArgVarRefs>::const_iterator argsRefsEnd = argsRefs.end(); |
849 | - |
850 | - std::vector<store::Iterator_t>::iterator argWrapsIte = |
851 | - state->theArgWrappers.begin(); |
852 | - |
853 | - for (; argsRefsIte != argsRefsEnd; ++argsRefsIte, ++argWrapsIte) |
854 | + const std::vector<store::Iterator_t>& argWraps = state->theArgWrappers; |
855 | + |
856 | + for (size_t i = 0; i < argsRefs.size(); ++i) |
857 | { |
858 | - store::Iterator_t& argWrapper = (*argWrapsIte); |
859 | - const ArgVarRefs& argVarRefs = (*argsRefsIte); |
860 | + const ArgVarRefs& argVarRefs = argsRefs[i]; |
861 | + store::Iterator_t argWrapper; |
862 | + if (state->theCache) |
863 | + { |
864 | + std::vector<store::Item_t> lParam(1, lKey[i]); |
865 | + state->theArgValues.push_back(GENV_STORE.createTempSeq(lParam)); |
866 | + argWrapper = state->theArgValues.back()->getIterator(); |
867 | + argWrapper->open(); |
868 | + } |
869 | + else |
870 | + { |
871 | + argWrapper = argWraps[i]; |
872 | + } |
873 | + |
874 | ArgVarRefs::const_iterator argVarRefsIte = argVarRefs.begin(); |
875 | ArgVarRefs::const_iterator argVarRefsEnd = argVarRefs.end(); |
876 | |
877 | @@ -343,26 +478,35 @@ |
878 | } |
879 | } |
880 | |
881 | -#ifdef ZORBA_WITH_DEBUGGER |
882 | - DEBUGGER_PUSH_FRAME; |
883 | -#endif |
884 | - |
885 | - while (consumeNext(result, state->thePlan, *state->thePlanState)) |
886 | + if (lCacheHit) |
887 | { |
888 | -#ifdef ZORBA_WITH_DEBUGGER |
889 | - DEBUGGER_POP_FRAME; |
890 | -#endif |
891 | STACK_PUSH(true, state); |
892 | - |
893 | + } |
894 | + else |
895 | + { |
896 | #ifdef ZORBA_WITH_DEBUGGER |
897 | DEBUGGER_PUSH_FRAME; |
898 | #endif |
899 | + while (consumeNext(result, state->thePlan, *state->thePlanState)) |
900 | + { |
901 | +#ifdef ZORBA_WITH_DEBUGGER |
902 | + DEBUGGER_POP_FRAME; |
903 | +#endif |
904 | + |
905 | + insertCacheEntry(state, lKey, result); |
906 | + STACK_PUSH(true, state); |
907 | + |
908 | +#ifdef ZORBA_WITH_DEBUGGER |
909 | + DEBUGGER_PUSH_FRAME; |
910 | +#endif |
911 | + } |
912 | + |
913 | +#ifdef ZORBA_WITH_DEBUGGER |
914 | + DEBUGGER_POP_FRAME; |
915 | +#endif |
916 | + |
917 | } |
918 | |
919 | -#ifdef ZORBA_WITH_DEBUGGER |
920 | - DEBUGGER_POP_FRAME; |
921 | -#endif |
922 | - |
923 | STACK_END(state); |
924 | |
925 | } |
926 | |
927 | === modified file 'src/runtime/core/fncall_iterator.h' |
928 | --- src/runtime/core/fncall_iterator.h 2011-10-30 00:18:34 +0000 |
929 | +++ src/runtime/core/fncall_iterator.h 2011-12-13 03:43:26 +0000 |
930 | @@ -69,16 +69,31 @@ |
931 | the body. So, it is never the case that the arg expr will have more than one |
932 | consumers, and as a result we can bind all those V references to the same arg |
933 | wrapper. |
934 | + |
935 | + theCache: |
936 | + --------- |
937 | + Is an Index which is set in the state if caching for the invoked function |
938 | + should be done. The cache is owned by the UDF itself and shared across |
939 | + all function invocations. |
940 | + |
941 | + theCacheHits: |
942 | + ------------- |
943 | + If caching is used, this vector contains the results of all arguments |
944 | + of the function evaluation. It's used to bind the variables if the |
945 | + cache didn't give a result in order to avoid duplicate evaluation of |
946 | + the arguments. |
947 | ********************************************************************************/ |
948 | class UDFunctionCallIteratorState : public PlanIteratorState |
949 | { |
950 | public: |
951 | - PlanIterator * thePlan; |
952 | - PlanState * thePlanState; |
953 | - uint32_t thePlanStateSize; |
954 | - dynamic_context * theLocalDCtx; |
955 | - bool thePlanOpen; |
956 | - std::vector<store::Iterator_t> theArgWrappers; |
957 | + PlanIterator * thePlan; |
958 | + PlanState * thePlanState; |
959 | + uint32_t thePlanStateSize; |
960 | + dynamic_context * theLocalDCtx; |
961 | + bool thePlanOpen; |
962 | + std::vector<store::Iterator_t> theArgWrappers; |
963 | + store::Index * theCache; |
964 | + std::vector<store::TempSeq_t> theArgValues; |
965 | |
966 | UDFunctionCallIteratorState(); |
967 | |
968 | @@ -130,6 +145,8 @@ |
969 | |
970 | void setDynamic() { theIsDynamic = true; } |
971 | |
972 | + bool isCached() const; |
973 | + |
974 | void accept(PlanIterVisitor& v) const; |
975 | |
976 | void openImpl(PlanState& planState, uint32_t& offset); |
977 | @@ -139,6 +156,22 @@ |
978 | void closeImpl(PlanState& planState); |
979 | |
980 | bool nextImpl(store::Item_t& result, PlanState& planState) const; |
981 | + |
982 | +protected: |
983 | + void createCache( |
984 | + PlanState& planState, |
985 | + UDFunctionCallIteratorState* state); |
986 | + |
987 | + bool probeCache( |
988 | + PlanState& planState, |
989 | + UDFunctionCallIteratorState* state, |
990 | + store::Item_t& result, |
991 | + std::vector<store::Item_t>& aKey) const; |
992 | + |
993 | + void insertCacheEntry( |
994 | + UDFunctionCallIteratorState* state, |
995 | + std::vector<store::Item_t>& aKey, |
996 | + store::Item_t& aValue) const; |
997 | }; |
998 | |
999 | |
1000 | |
1001 | === modified file 'src/runtime/visitors/printer_visitor_impl.cpp' |
1002 | --- src/runtime/visitors/printer_visitor_impl.cpp 2011-12-01 11:02:25 +0000 |
1003 | +++ src/runtime/visitors/printer_visitor_impl.cpp 2011-12-13 03:43:26 +0000 |
1004 | @@ -1209,7 +1209,23 @@ |
1005 | |
1006 | #undef TYPED_VAL_CMP |
1007 | |
1008 | - PRINTER_VISITOR_DEFINITION (UDFunctionCallIterator) |
1009 | + void PrinterVisitor::beginVisit ( const UDFunctionCallIterator& a ) |
1010 | + { |
1011 | + thePrinter.startBeginVisit("UDFunctionCallIterator", ++theId); |
1012 | + printCommons( &a, theId ); |
1013 | + if (a.isCached()) |
1014 | + { |
1015 | + thePrinter.addAttribute("cached", "true"); |
1016 | + } |
1017 | + thePrinter.endBeginVisit( theId); |
1018 | + } |
1019 | + |
1020 | + void PrinterVisitor::endVisit ( const UDFunctionCallIterator& ) |
1021 | + { |
1022 | + thePrinter.startEndVisit(); |
1023 | + thePrinter.endEndVisit(); |
1024 | + } |
1025 | + |
1026 | PRINTER_VISITOR_DEFINITION (ExtFunctionCallIterator) |
1027 | PRINTER_VISITOR_DEFINITION (FnBooleanIterator) |
1028 | PRINTER_VISITOR_DEFINITION (OrIterator) |
1029 | |
1030 | === modified file 'src/store/api/index.h' |
1031 | --- src/store/api/index.h 2011-10-11 12:11:48 +0000 |
1032 | +++ src/store/api/index.h 2011-12-13 03:43:26 +0000 |
1033 | @@ -415,6 +415,18 @@ |
1034 | * Returns all keys stored in this index |
1035 | */ |
1036 | virtual KeyIterator_t keys() const = 0; |
1037 | + |
1038 | + /** |
1039 | + * Insert the given item in the value set of the given key. If the key is not |
1040 | + * in the index already, then the key itself is inserted as well. Return true |
1041 | + * if the key was already in the index, false otherwise |
1042 | + * The index wil take the ownership of the key if it was not already in the |
1043 | + * index. |
1044 | + * |
1045 | + * @error ZDDY0035 if a key with more than one item is inserted into |
1046 | + * a general index |
1047 | + */ |
1048 | + virtual bool insert(store::IndexKey*& key, store::Item_t& item) = 0; |
1049 | }; |
1050 | |
1051 | |
1052 | |
1053 | === modified file 'src/store/naive/simple_index_general.cpp' |
1054 | --- src/store/naive/simple_index_general.cpp 2011-11-15 18:48:54 +0000 |
1055 | +++ src/store/naive/simple_index_general.cpp 2011-12-13 03:43:26 +0000 |
1056 | @@ -236,6 +236,20 @@ |
1057 | /****************************************************************************** |
1058 | |
1059 | *******************************************************************************/ |
1060 | +bool GeneralIndex::insert(store::IndexKey*& key, store::Item_t& value) |
1061 | +{ |
1062 | + if (key->size() != 1) |
1063 | + { |
1064 | + RAISE_ERROR_NO_LOC(zerr::ZDDY0035_INDEX_GENERAL_INSERT, |
1065 | + ERROR_PARAMS(getName()->getStringValue())); |
1066 | + } |
1067 | + return insert((*key)[0], value); |
1068 | +} |
1069 | + |
1070 | + |
1071 | +/****************************************************************************** |
1072 | + |
1073 | +*******************************************************************************/ |
1074 | bool GeneralIndex::insert(store::Item_t& key, store::Item_t& node) |
1075 | { |
1076 | bool lossy = false; |
1077 | |
1078 | === modified file 'src/store/naive/simple_index_general.h' |
1079 | --- src/store/naive/simple_index_general.h 2011-11-15 18:48:54 +0000 |
1080 | +++ src/store/naive/simple_index_general.h 2011-12-13 03:43:26 +0000 |
1081 | @@ -174,6 +174,8 @@ |
1082 | |
1083 | bool insert(store::Item_t& key, store::Item_t& node); |
1084 | |
1085 | + bool insert(store::IndexKey*& key, store::Item_t& value); |
1086 | + |
1087 | virtual bool remove(const store::Item_t& key, store::Item_t& item, bool all) = 0; |
1088 | }; |
1089 | |
1090 | |
1091 | === modified file 'src/store/naive/simple_store.cpp' |
1092 | --- src/store/naive/simple_store.cpp 2011-11-02 17:19:09 +0000 |
1093 | +++ src/store/naive/simple_store.cpp 2011-12-13 03:43:26 +0000 |
1094 | @@ -555,6 +555,9 @@ |
1095 | store::Iterator* aSourceIter, |
1096 | ulong aNumColumns) |
1097 | { |
1098 | + if (!aSourceIter) |
1099 | + return; |
1100 | + |
1101 | store::Item_t domainItem; |
1102 | store::IndexKey* key = NULL; |
1103 | |
1104 | |
1105 | === modified file 'src/store/naive/simple_temp_seq.cpp' |
1106 | --- src/store/naive/simple_temp_seq.cpp 2011-08-19 17:04:48 +0000 |
1107 | +++ src/store/naive/simple_temp_seq.cpp 2011-12-13 03:43:26 +0000 |
1108 | @@ -53,27 +53,6 @@ |
1109 | /******************************************************************************* |
1110 | |
1111 | ********************************************************************************/ |
1112 | -store::Item_t SimpleTempSeq::operator[](xs_integer aIndex) |
1113 | -{ |
1114 | - uint64_t lIndex; |
1115 | - try { |
1116 | - lIndex = to_xs_unsignedLong(aIndex); |
1117 | - } catch (std::range_error& e) |
1118 | - { |
1119 | - throw ZORBA_EXCEPTION( |
1120 | - zerr::ZSTR0060_RANGE_EXCEPTION, |
1121 | - ERROR_PARAMS( |
1122 | - BUILD_STRING("access out of bounds " << e.what() << ")") |
1123 | - ) |
1124 | - ); |
1125 | - } |
1126 | - return theItems[lIndex]; |
1127 | -} |
1128 | - |
1129 | - |
1130 | -/******************************************************************************* |
1131 | - |
1132 | -********************************************************************************/ |
1133 | xs_integer SimpleTempSeq::getSize() |
1134 | { |
1135 | return theItems.size(); |
1136 | @@ -312,14 +291,15 @@ |
1137 | case none: |
1138 | if ( theCurPos < to_xs_unsignedLong(theTempSeq->getSize()) ) |
1139 | { |
1140 | - result = (*theTempSeq)[theCurPos]; |
1141 | + result = theTempSeq->theItems[theCurPos]; |
1142 | return true; |
1143 | } |
1144 | break; |
1145 | + |
1146 | case startEnd: |
1147 | if ( theCurPos < theEndPos ) |
1148 | { |
1149 | - result = (*theTempSeq)[theCurPos]; |
1150 | + result = theTempSeq->theItems[theCurPos]; |
1151 | return true; |
1152 | } |
1153 | break; |
1154 | @@ -336,17 +316,17 @@ |
1155 | { |
1156 | case none: |
1157 | if ( theCurPos < to_xs_unsignedLong(theTempSeq->getSize()) ) |
1158 | - return (*theTempSeq)[theCurPos]; |
1159 | + return theTempSeq->theItems[theCurPos]; |
1160 | break; |
1161 | case startEnd: |
1162 | if ( theCurPos < theEndPos ) |
1163 | - return (*theTempSeq)[theCurPos]; |
1164 | + return theTempSeq->theItems[theCurPos]; |
1165 | break; |
1166 | } |
1167 | |
1168 | return NULL; |
1169 | } |
1170 | - |
1171 | + |
1172 | |
1173 | void SimpleTempSeqIter::reset() |
1174 | { |
1175 | |
1176 | === modified file 'src/store/naive/simple_temp_seq.h' |
1177 | --- src/store/naive/simple_temp_seq.h 2011-07-29 23:01:30 +0000 |
1178 | +++ src/store/naive/simple_temp_seq.h 2011-12-13 03:43:26 +0000 |
1179 | @@ -35,6 +35,8 @@ |
1180 | ********************************************************************************/ |
1181 | class SimpleTempSeq : public store::TempSeq |
1182 | { |
1183 | + friend class SimpleTempSeqIter; |
1184 | + |
1185 | private: |
1186 | std::vector<store::Item_t> theItems; |
1187 | |
1188 | @@ -55,8 +57,6 @@ |
1189 | |
1190 | void append(store::Iterator_t iter, bool copy); |
1191 | |
1192 | - store::Item_t operator[](xs_integer aIndex); |
1193 | - |
1194 | void getItem(xs_integer position, store::Item_t& res); |
1195 | |
1196 | bool containsItem(xs_integer position); |
1197 | @@ -102,9 +102,9 @@ |
1198 | SimpleTempSeq_t theTempSeq; |
1199 | BorderType theBorderType; |
1200 | |
1201 | - uint64_t theCurPos; |
1202 | - uint64_t theStartPos; |
1203 | - uint64_t theEndPos; |
1204 | + uint64_t theCurPos; |
1205 | + uint64_t theStartPos; |
1206 | + uint64_t theEndPos; |
1207 | |
1208 | public: |
1209 | SimpleTempSeqIter() {} |
1210 | |
1211 | === added file 'test/rbkt/ExpCompilerResults/IterPlan/zorba/udf/udf-fib-rec.iter' |
1212 | --- test/rbkt/ExpCompilerResults/IterPlan/zorba/udf/udf-fib-rec.iter 1970-01-01 00:00:00 +0000 |
1213 | +++ test/rbkt/ExpCompilerResults/IterPlan/zorba/udf/udf-fib-rec.iter 2011-12-13 03:43:26 +0000 |
1214 | @@ -0,0 +1,42 @@ |
1215 | +Iterator tree for main query: |
1216 | +<UDFunctionCallIterator cached="true"> |
1217 | + <SingletonIterator value="xs:integer(100)"/> |
1218 | +</UDFunctionCallIterator> |
1219 | + |
1220 | +Iterator tree for local:fib: |
1221 | +<flwor::FLWORIterator> |
1222 | + <ForVariable name="n"> |
1223 | + <LetVarIterator varname="n"/> |
1224 | + </ForVariable> |
1225 | + <ReturnClause> |
1226 | + <IfThenElseIterator> |
1227 | + <TypedValueCompareIterator_INTEGER> |
1228 | + <ForVarIterator varname="n"/> |
1229 | + <SingletonIterator value="xs:integer(0)"/> |
1230 | + </TypedValueCompareIterator_INTEGER> |
1231 | + <SingletonIterator value="xs:integer(0)"/> |
1232 | + <IfThenElseIterator> |
1233 | + <TypedValueCompareIterator_INTEGER> |
1234 | + <ForVarIterator varname="n"/> |
1235 | + <SingletonIterator value="xs:integer(1)"/> |
1236 | + </TypedValueCompareIterator_INTEGER> |
1237 | + <SingletonIterator value="xs:integer(1)"/> |
1238 | + <SpecificNumArithIterator_AddOperation_INTEGER> |
1239 | + <UDFunctionCallIterator cached="true"> |
1240 | + <SpecificNumArithIterator_SubtractOperation_INTEGER> |
1241 | + <ForVarIterator varname="n"/> |
1242 | + <SingletonIterator value="xs:integer(1)"/> |
1243 | + </SpecificNumArithIterator_SubtractOperation_INTEGER> |
1244 | + </UDFunctionCallIterator> |
1245 | + <UDFunctionCallIterator cached="true"> |
1246 | + <SpecificNumArithIterator_SubtractOperation_INTEGER> |
1247 | + <ForVarIterator varname="n"/> |
1248 | + <SingletonIterator value="xs:integer(2)"/> |
1249 | + </SpecificNumArithIterator_SubtractOperation_INTEGER> |
1250 | + </UDFunctionCallIterator> |
1251 | + </SpecificNumArithIterator_AddOperation_INTEGER> |
1252 | + </IfThenElseIterator> |
1253 | + </IfThenElseIterator> |
1254 | + </ReturnClause> |
1255 | +</flwor::FLWORIterator> |
1256 | + |
1257 | |
1258 | === added file 'test/rbkt/ExpQueryResults/zorba/udf/udf-fib-rec.xml.res' |
1259 | --- test/rbkt/ExpQueryResults/zorba/udf/udf-fib-rec.xml.res 1970-01-01 00:00:00 +0000 |
1260 | +++ test/rbkt/ExpQueryResults/zorba/udf/udf-fib-rec.xml.res 2011-12-13 03:43:26 +0000 |
1261 | @@ -0,0 +1,1 @@ |
1262 | +3736710778780434371 |
1263 | |
1264 | === added file 'test/rbkt/Queries/zorba/udf/udf-fib-rec.xq' |
1265 | --- test/rbkt/Queries/zorba/udf/udf-fib-rec.xq 1970-01-01 00:00:00 +0000 |
1266 | +++ test/rbkt/Queries/zorba/udf/udf-fib-rec.xq 2011-12-13 03:43:26 +0000 |
1267 | @@ -0,0 +1,8 @@ |
1268 | +declare function local:fib($n as xs:integer) as xs:integer |
1269 | +{ |
1270 | + if ($n eq 0) then 0 |
1271 | + else if ($n eq 1) then 1 |
1272 | + else local:fib($n - 1) + local:fib($n - 2) |
1273 | +}; |
1274 | + |
1275 | +local:fib(100) |
Validation queue starting for merge proposal. zorbatest. lambda. nu:8080/ remotequeue/ caching- 2011-11- 04T17-29- 08.827Z/ log.html
Log at: http://