Merge lp:~zorba-coders/zorba/bug-996084-reuse-stream into lp:zorba
- bug-996084-reuse-stream
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Chris Hillery | ||||
Approved revision: | 10826 | ||||
Merged at revision: | 10831 | ||||
Proposed branch: | lp:~zorba-coders/zorba/bug-996084-reuse-stream | ||||
Merge into: | lp:zorba | ||||
Diff against target: |
435 lines (+198/-22) 18 files modified
ChangeLog (+2/-0) modules/com/zorba-xquery/www/modules/converters/base64.xq (+21/-1) src/functions/pregenerated/func_base64.cpp (+13/-0) src/functions/pregenerated/function_enum.h (+1/-0) src/runtime/base64/base64_impl.cpp (+66/-11) src/runtime/spec/base64/base64.xml (+6/-1) src/store/api/item_factory.h (+13/-0) src/store/naive/atomic_items.cpp (+30/-0) src/store/naive/atomic_items.h (+6/-9) src/store/naive/simple_item_factory.cpp (+8/-0) src/store/naive/simple_item_factory.h (+4/-0) test/rbkt/ExpQueryResults/zorba/base64/decode_iso-8859-1.xml.res (+1/-0) test/rbkt/ExpQueryResults/zorba/base64/decode_iso-8859-1_file.xml.res (+1/-0) test/rbkt/ExpQueryResults/zorba/base64/reuse-stream.xml.res (+1/-0) test/rbkt/Queries/zorba/base64/decode_iso-8859-1.xq (+3/-0) test/rbkt/Queries/zorba/base64/decode_iso-8859-1_file.xq (+5/-0) test/rbkt/Queries/zorba/base64/iso-8859-1.txt (+1/-0) test/rbkt/Queries/zorba/base64/reuse-stream.xq (+16/-0) |
||||
To merge this branch: | bzr merge lp:~zorba-coders/zorba/bug-996084-reuse-stream | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Chris Hillery | Approve | ||
Dennis Knochenwefel | Approve | ||
William Candillon | Approve | ||
Matthias Brantner | Approve | ||
Review via email: mp+105025@code.launchpad.net |
This proposal supersedes a proposal from 2012-05-08.
Commit message
- Add the ability to create a StreamableStrin
- Add a base64:decode#2 function that also does transcoding to utf-8
Description of the change
Add the ability to create a StreamableStrin
Added a base64:decode#2 function that also does transcoding to utf-8
Chris Hillery (ceejatec) : Posted in a previous version of this proposal | # |
Chris Hillery (ceejatec) wrote : Posted in a previous version of this proposal | # |
Matthias Brantner (matthias-brantner) : | # |
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue job bug-996084-reuse-
All tests succeeded!
Zorba Build Bot (zorba-buildbot) wrote : | # |
Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 1 Approve, 3 Pending.
William Candillon (wcandillon) wrote : | # |
Should the implementation of base64:decode#1 be:
declare function base64:
?
Chris Hillery (ceejatec) wrote : | # |
I don't believe it can be done that way, because the base64 module is an internally-
William Candillon (wcandillon) wrote : | # |
yes correct, sorry I missed that.
On Tue, May 8, 2012 at 12:23 PM, Chris Hillery
<email address hidden> wrote:
> I don't believe it can be done that way, because the base64 module is an internally-
> --
> https:/
> You are reviewing the proposed merge of lp:~zorba-coders/zorba/bug-996084-reuse-stream into lp:zorba.
William Candillon (wcandillon) : | # |
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue job bug-996084-reuse-
All tests succeeded!
Zorba Build Bot (zorba-buildbot) wrote : | # |
Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 2 Approve, 2 Pending.
Matthias Brantner (matthias-brantner) wrote : | # |
> I don't believe it can be done that way, because the base64 module is an
> internally-
> won't even be parsed, so it cannot contain user-defined functions. Matthias,
> correct me if I'm wrong.
It could but it would require us to process the .xq file at runtime which I wanted to avoid given the fact that it doesn't make the C++ implementation too complex.
Dennis Knochenwefel (dennis-knochenwefel) wrote : | # |
I tested it and it works.
Chris Hillery (ceejatec) : | # |
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue job bug-996084-reuse-
All tests succeeded!
Preview Diff
1 | === modified file 'ChangeLog' | |||
2 | --- ChangeLog 2012-05-07 23:43:04 +0000 | |||
3 | +++ ChangeLog 2012-05-08 03:24:31 +0000 | |||
4 | @@ -18,6 +18,7 @@ | |||
5 | 18 | - fn:has-children#0 | 18 | - fn:has-children#0 |
6 | 19 | - fn:nilled#0 | 19 | - fn:nilled#0 |
7 | 20 | - fn:path | 20 | - fn:path |
8 | 21 | * Added base64:decode#2 function which also does transcoding | ||
9 | 21 | * Extended API for Python, Java, PHP and Ruby. | 22 | * Extended API for Python, Java, PHP and Ruby. |
10 | 22 | * Add jvm classpath to zorbacmd and to Zorba API. Tracked by #931816 | 23 | * Add jvm classpath to zorbacmd and to Zorba API. Tracked by #931816 |
11 | 23 | * Added full-text module. | 24 | * Added full-text module. |
12 | @@ -49,6 +50,7 @@ | |||
13 | 49 | * Fixed bug #912586, #912593 and #912722 (assertion failures with lax validation) | 50 | * Fixed bug #912586, #912593 and #912722 (assertion failures with lax validation) |
14 | 50 | * Fixed bug #921458 (file:read-text-lines() blocking) | 51 | * Fixed bug #921458 (file:read-text-lines() blocking) |
15 | 51 | * Fixed bug #981405 (do not hoist expr containing try-catch variables out of the associated try-catch expression) | 52 | * Fixed bug #981405 (do not hoist expr containing try-catch variables out of the associated try-catch expression) |
16 | 53 | * Fixed bug #996084 (crash in Streamable*Item with file module) | ||
17 | 52 | * Fixed bug #947627 (throw XQST0099 if more than one declarations of context item type in same module) | 54 | * Fixed bug #947627 (throw XQST0099 if more than one declarations of context item type in same module) |
18 | 53 | * Fixed bug #980526 (no-copy rule bug due to global var being set in "distant" udf) | 55 | * Fixed bug #980526 (no-copy rule bug due to global var being set in "distant" udf) |
19 | 54 | * Fixed bug #949910 (has-children may be invoked on all nodes). Internally, zorba::store::Item::getChildren() now returns NULL on node classes without offspring (instead of raising an error). | 56 | * Fixed bug #949910 (has-children may be invoked on all nodes). Internally, zorba::store::Item::getChildren() now returns NULL on node classes without offspring (instead of raising an error). |
20 | 55 | 57 | ||
21 | === modified file 'modules/com/zorba-xquery/www/modules/converters/base64.xq' | |||
22 | --- modules/com/zorba-xquery/www/modules/converters/base64.xq 2012-05-03 12:31:51 +0000 | |||
23 | +++ modules/com/zorba-xquery/www/modules/converters/base64.xq 2012-05-08 03:24:31 +0000 | |||
24 | @@ -25,6 +25,8 @@ | |||
25 | 25 | :) | 25 | :) |
26 | 26 | module namespace base64 = "http://www.zorba-xquery.com/modules/converters/base64"; | 26 | module namespace base64 = "http://www.zorba-xquery.com/modules/converters/base64"; |
27 | 27 | 27 | ||
28 | 28 | declare namespace zerr = "http://www.zorba-xquery.com/errors"; | ||
29 | 29 | |||
30 | 28 | declare namespace ver = "http://www.zorba-xquery.com/options/versioning"; | 30 | declare namespace ver = "http://www.zorba-xquery.com/options/versioning"; |
31 | 29 | declare option ver:module-version "2.0"; | 31 | declare option ver:module-version "2.0"; |
32 | 30 | 32 | ||
33 | @@ -32,7 +34,7 @@ | |||
34 | 32 | : Decode a xs:base64Binary. | 34 | : Decode a xs:base64Binary. |
35 | 33 | : | 35 | : |
36 | 34 | : The function assumes that the content after decoding is valid | 36 | : The function assumes that the content after decoding is valid |
38 | 35 | : UTF-8. | 37 | : UTF-8. |
39 | 36 | : | 38 | : |
40 | 37 | : @param $base64 The xs:base64Binary item to decode | 39 | : @param $base64 The xs:base64Binary item to decode |
41 | 38 | : @return the base64 decoded value as string | 40 | : @return the base64 decoded value as string |
42 | @@ -41,6 +43,24 @@ | |||
43 | 41 | as xs:string external; | 43 | as xs:string external; |
44 | 42 | 44 | ||
45 | 43 | (:~ | 45 | (:~ |
46 | 46 | : Decode a xs:base64Binary. | ||
47 | 47 | : | ||
48 | 48 | : The function assumes that the content after decoding has | ||
49 | 49 | : the given encoding. | ||
50 | 50 | : | ||
51 | 51 | : @param $base64 The xs:base64Binary item to decode | ||
52 | 52 | : @param $encoding The encoding of the string after base64 decoding it. | ||
53 | 53 | : | ||
54 | 54 | : @return the base64 decoded value as string | ||
55 | 55 | : | ||
56 | 56 | : @error zerr:ZXQP0006 if the given encoding is invalid or not supported. | ||
57 | 57 | :) | ||
58 | 58 | declare function base64:decode( | ||
59 | 59 | $base64 as xs:base64Binary, | ||
60 | 60 | $encoding as xs:string) | ||
61 | 61 | as xs:string external; | ||
62 | 62 | |||
63 | 63 | (:~ | ||
64 | 44 | : Encode a xs:string as xs:base64Binary. | 64 | : Encode a xs:string as xs:base64Binary. |
65 | 45 | : | 65 | : |
66 | 46 | : @param $string The item whose string-value should be encoded | 66 | : @param $string The item whose string-value should be encoded |
67 | 47 | 67 | ||
68 | === modified file 'src/functions/pregenerated/func_base64.cpp' | |||
69 | --- src/functions/pregenerated/func_base64.cpp 2012-05-03 12:31:51 +0000 | |||
70 | +++ src/functions/pregenerated/func_base64.cpp 2012-05-08 03:24:31 +0000 | |||
71 | @@ -68,6 +68,19 @@ | |||
72 | 68 | { | 68 | { |
73 | 69 | 69 | ||
74 | 70 | 70 | ||
75 | 71 | DECL_WITH_KIND(sctx, fn_zorba_base64_decode, | ||
76 | 72 | (createQName("http://www.zorba-xquery.com/modules/converters/base64","","decode"), | ||
77 | 73 | GENV_TYPESYSTEM.BASE64BINARY_TYPE_ONE, | ||
78 | 74 | GENV_TYPESYSTEM.STRING_TYPE_ONE, | ||
79 | 75 | GENV_TYPESYSTEM.STRING_TYPE_ONE), | ||
80 | 76 | FunctionConsts::FN_ZORBA_BASE64_DECODE_2); | ||
81 | 77 | |||
82 | 78 | } | ||
83 | 79 | |||
84 | 80 | |||
85 | 81 | { | ||
86 | 82 | |||
87 | 83 | |||
88 | 71 | DECL_WITH_KIND(sctx, fn_zorba_base64_encode, | 84 | DECL_WITH_KIND(sctx, fn_zorba_base64_encode, |
89 | 72 | (createQName("http://www.zorba-xquery.com/modules/converters/base64","","encode"), | 85 | (createQName("http://www.zorba-xquery.com/modules/converters/base64","","encode"), |
90 | 73 | GENV_TYPESYSTEM.STRING_TYPE_ONE, | 86 | GENV_TYPESYSTEM.STRING_TYPE_ONE, |
91 | 74 | 87 | ||
92 | === modified file 'src/functions/pregenerated/function_enum.h' | |||
93 | --- src/functions/pregenerated/function_enum.h 2012-05-03 12:31:51 +0000 | |||
94 | +++ src/functions/pregenerated/function_enum.h 2012-05-08 03:24:31 +0000 | |||
95 | @@ -36,6 +36,7 @@ | |||
96 | 36 | FN_RESOLVE_URI_1, | 36 | FN_RESOLVE_URI_1, |
97 | 37 | FN_RESOLVE_URI_2, | 37 | FN_RESOLVE_URI_2, |
98 | 38 | FN_ZORBA_BASE64_DECODE_1, | 38 | FN_ZORBA_BASE64_DECODE_1, |
99 | 39 | FN_ZORBA_BASE64_DECODE_2, | ||
100 | 39 | FN_ZORBA_BASE64_ENCODE_1, | 40 | FN_ZORBA_BASE64_ENCODE_1, |
101 | 40 | OP_IS_SAME_NODE_2, | 41 | OP_IS_SAME_NODE_2, |
102 | 41 | OP_NODE_BEFORE_2, | 42 | OP_NODE_BEFORE_2, |
103 | 42 | 43 | ||
104 | === modified file 'src/runtime/base64/base64_impl.cpp' | |||
105 | --- src/runtime/base64/base64_impl.cpp 2012-05-03 12:31:51 +0000 | |||
106 | +++ src/runtime/base64/base64_impl.cpp 2012-05-08 03:24:31 +0000 | |||
107 | @@ -15,6 +15,9 @@ | |||
108 | 15 | */ | 15 | */ |
109 | 16 | #include "stdafx.h" | 16 | #include "stdafx.h" |
110 | 17 | 17 | ||
111 | 18 | #include <sstream> | ||
112 | 19 | #include <zorba/transcode_stream.h> | ||
113 | 20 | |||
114 | 18 | #include "system/globalenv.h" | 21 | #include "system/globalenv.h" |
115 | 19 | 22 | ||
116 | 20 | #include "diagnostics/xquery_diagnostics.h" | 23 | #include "diagnostics/xquery_diagnostics.h" |
117 | @@ -33,6 +36,7 @@ | |||
118 | 33 | { | 36 | { |
119 | 34 | store::Item_t lItem; | 37 | store::Item_t lItem; |
120 | 35 | zstring lResultString; | 38 | zstring lResultString; |
121 | 39 | zstring lEncoding("UTF-8"); | ||
122 | 36 | const char* lContent; | 40 | const char* lContent; |
123 | 37 | size_t lSize; | 41 | size_t lSize; |
124 | 38 | result = NULL; | 42 | result = NULL; |
125 | @@ -42,21 +46,45 @@ | |||
126 | 42 | 46 | ||
127 | 43 | consumeNext(lItem, theChildren[0].getp(), planState); | 47 | consumeNext(lItem, theChildren[0].getp(), planState); |
128 | 44 | 48 | ||
129 | 49 | if (theChildren.size() == 2) | ||
130 | 50 | { | ||
131 | 51 | store::Item_t lEncodingItem; | ||
132 | 52 | consumeNext(lEncodingItem, theChildren[1].getp(), planState); | ||
133 | 53 | lEncoding = lEncodingItem->getStringValue(); | ||
134 | 54 | |||
135 | 55 | if (!transcode::is_supported(lEncoding.c_str())) | ||
136 | 56 | { | ||
137 | 57 | throw XQUERY_EXCEPTION( | ||
138 | 58 | zerr::ZXQP0006_UNKNOWN_ENCODING, | ||
139 | 59 | ERROR_PARAMS( lEncoding ), | ||
140 | 60 | ERROR_LOC( loc ) | ||
141 | 61 | ); | ||
142 | 62 | } | ||
143 | 63 | } | ||
144 | 64 | |||
145 | 45 | if (lItem->isStreamable()) | 65 | if (lItem->isStreamable()) |
146 | 46 | { | 66 | { |
147 | 47 | if (lItem->isEncoded()) | 67 | if (lItem->isEncoded()) |
148 | 48 | { | 68 | { |
149 | 49 | // decode and eventually transcode | ||
150 | 50 | lResultString = Base64::decode(lItem->getStream()); | 69 | lResultString = Base64::decode(lItem->getStream()); |
151 | 51 | } | 70 | } |
152 | 52 | else | 71 | else |
153 | 53 | { | 72 | { |
160 | 54 | // streamable string eventually transcoding | 73 | if (transcode::is_necessary(lEncoding.c_str())) |
161 | 55 | GENV_ITEMFACTORY->createStreamableString( | 74 | { |
162 | 56 | result, | 75 | transcode::attach(lItem->getStream(), lEncoding.c_str()); |
163 | 57 | lItem->getStream(), | 76 | GENV_ITEMFACTORY->createStreamableString( |
164 | 58 | lItem->getStreamReleaser(), | 77 | result, |
165 | 59 | lItem->isSeekable()); | 78 | lItem->getStream(), |
166 | 79 | lItem->getStreamReleaser(), | ||
167 | 80 | lItem->isSeekable()); | ||
168 | 81 | } | ||
169 | 82 | else | ||
170 | 83 | { | ||
171 | 84 | GENV_ITEMFACTORY->createSharedStreamableString( | ||
172 | 85 | result, | ||
173 | 86 | lItem); | ||
174 | 87 | } | ||
175 | 60 | } | 88 | } |
176 | 61 | } | 89 | } |
177 | 62 | else | 90 | else |
178 | @@ -74,10 +102,37 @@ | |||
179 | 74 | { | 102 | { |
180 | 75 | lResultString.insert(0, lContent, lSize); | 103 | lResultString.insert(0, lContent, lSize); |
181 | 76 | } | 104 | } |
186 | 77 | } | 105 | |
187 | 78 | if (!result) // otherwise it's a streamable string already | 106 | if (transcode::is_necessary(lEncoding.c_str())) |
188 | 79 | { | 107 | { |
189 | 80 | GENV_ITEMFACTORY->createString(result, lResultString); | 108 | try |
190 | 109 | { | ||
191 | 110 | zstring lTranscodedString; | ||
192 | 111 | transcode::stream<std::istringstream> lTranscoder( | ||
193 | 112 | lEncoding.c_str(), | ||
194 | 113 | lResultString.c_str() | ||
195 | 114 | ); | ||
196 | 115 | char buf[1024]; | ||
197 | 116 | while (lTranscoder.good()) | ||
198 | 117 | { | ||
199 | 118 | lTranscoder.read(buf, 1024); | ||
200 | 119 | lTranscodedString.append(buf, lTranscoder.gcount()); | ||
201 | 120 | } | ||
202 | 121 | GENV_ITEMFACTORY->createString(result, lTranscodedString); | ||
203 | 122 | } | ||
204 | 123 | catch (ZorbaException& e) | ||
205 | 124 | { | ||
206 | 125 | throw XQUERY_EXCEPTION( | ||
207 | 126 | zerr::ZOSE0006_TRANSCODING_ERROR, | ||
208 | 127 | ERROR_PARAMS( e.what() ), | ||
209 | 128 | ERROR_LOC( loc ) | ||
210 | 129 | ); | ||
211 | 130 | } | ||
212 | 131 | } | ||
213 | 132 | else | ||
214 | 133 | { | ||
215 | 134 | GENV_ITEMFACTORY->createString(result, lResultString); | ||
216 | 135 | } | ||
217 | 81 | } | 136 | } |
218 | 82 | STACK_PUSH (true, state); | 137 | STACK_PUSH (true, state); |
219 | 83 | 138 | ||
220 | 84 | 139 | ||
221 | === modified file 'src/runtime/spec/base64/base64.xml' | |||
222 | --- src/runtime/spec/base64/base64.xml 2012-05-03 12:31:51 +0000 | |||
223 | +++ src/runtime/spec/base64/base64.xml 2012-05-08 03:24:31 +0000 | |||
224 | @@ -25,8 +25,13 @@ | |||
225 | 25 | <zorba:param>xs:base64Binary</zorba:param> | 25 | <zorba:param>xs:base64Binary</zorba:param> |
226 | 26 | <zorba:output>xs:string</zorba:output> | 26 | <zorba:output>xs:string</zorba:output> |
227 | 27 | </zorba:signature> | 27 | </zorba:signature> |
228 | 28 | <zorba:signature localname="decode" prefix="fn-zorba-base64"> | ||
229 | 29 | <zorba:param>xs:base64Binary</zorba:param> | ||
230 | 30 | <zorba:param>xs:string</zorba:param> <!-- encoding --> | ||
231 | 31 | <zorba:output>xs:string</zorba:output> | ||
232 | 32 | </zorba:signature> | ||
233 | 28 | </zorba:function> | 33 | </zorba:function> |
235 | 29 | 34 | ||
236 | 30 | </zorba:iterator> | 35 | </zorba:iterator> |
237 | 31 | 36 | ||
238 | 32 | <!-- | 37 | <!-- |
239 | 33 | 38 | ||
240 | === modified file 'src/store/api/item_factory.h' | |||
241 | --- src/store/api/item_factory.h 2012-05-03 12:31:51 +0000 | |||
242 | +++ src/store/api/item_factory.h 2012-05-08 03:24:31 +0000 | |||
243 | @@ -97,6 +97,19 @@ | |||
244 | 97 | bool seekable = false) = 0; | 97 | bool seekable = false) = 0; |
245 | 98 | 98 | ||
246 | 99 | /** | 99 | /** |
247 | 100 | * Create a StreamableStringItem which re-uses the stream from another | ||
248 | 101 | * Streamable*Item. This will maintain a reference to the original | ||
249 | 102 | * item to ensure the stream is not cleaned up before we are done with it. | ||
250 | 103 | * | ||
251 | 104 | * It only makes sense to use this method if either (a) the dependent item's | ||
252 | 105 | * stream is seekable and hence re-usable, or (b) you are sure that the | ||
253 | 106 | * dependent item will not be utilized after this new item is created. | ||
254 | 107 | */ | ||
255 | 108 | virtual bool createSharedStreamableString( | ||
256 | 109 | Item_t& result, | ||
257 | 110 | Item_t& streamble_dependent) = 0; | ||
258 | 111 | |||
259 | 112 | /** | ||
260 | 100 | * Specification: [http://www.w3.org/TR/xmlschema-2/#normalizedString] | 113 | * Specification: [http://www.w3.org/TR/xmlschema-2/#normalizedString] |
261 | 101 | * @param value string representation of the value | 114 | * @param value string representation of the value |
262 | 102 | */ | 115 | */ |
263 | 103 | 116 | ||
264 | === modified file 'src/store/naive/atomic_items.cpp' | |||
265 | --- src/store/naive/atomic_items.cpp 2012-05-04 14:32:28 +0000 | |||
266 | +++ src/store/naive/atomic_items.cpp 2012-05-08 03:24:31 +0000 | |||
267 | @@ -1673,6 +1673,36 @@ | |||
268 | 1673 | /******************************************************************************* | 1673 | /******************************************************************************* |
269 | 1674 | class StreamableStringItem | 1674 | class StreamableStringItem |
270 | 1675 | ********************************************************************************/ | 1675 | ********************************************************************************/ |
271 | 1676 | StreamableStringItem::StreamableStringItem( | ||
272 | 1677 | std::istream& aStream, | ||
273 | 1678 | StreamReleaser streamReleaser, | ||
274 | 1679 | bool seekable) : | ||
275 | 1680 | theIstream(aStream), | ||
276 | 1681 | theIsMaterialized(false), | ||
277 | 1682 | theIsConsumed(false), | ||
278 | 1683 | theIsSeekable(seekable), | ||
279 | 1684 | theStreamReleaser(streamReleaser), | ||
280 | 1685 | theStreamableDependent(nullptr) | ||
281 | 1686 | { | ||
282 | 1687 | } | ||
283 | 1688 | |||
284 | 1689 | StreamableStringItem::StreamableStringItem( | ||
285 | 1690 | store::Item_t& aStreamableDependent) : | ||
286 | 1691 | theIstream(aStreamableDependent->getStream()), | ||
287 | 1692 | theIsMaterialized(false), | ||
288 | 1693 | theIsConsumed(false), | ||
289 | 1694 | theIsSeekable(aStreamableDependent->isSeekable()), | ||
290 | 1695 | theStreamReleaser(nullptr), | ||
291 | 1696 | theStreamableDependent(aStreamableDependent) | ||
292 | 1697 | { | ||
293 | 1698 | ZORBA_ASSERT(theStreamableDependent->isStreamable()); | ||
294 | 1699 | |||
295 | 1700 | // We copied the dependent item's stream and seekable flag in the initializer | ||
296 | 1701 | // above, but did NOT copy the StreamReleaser. The dependent item maintains | ||
297 | 1702 | // memory ownership of the stream in this way. | ||
298 | 1703 | } | ||
299 | 1704 | |||
300 | 1705 | |||
301 | 1676 | void StreamableStringItem::appendStringValue(zstring& aBuf) const | 1706 | void StreamableStringItem::appendStringValue(zstring& aBuf) const |
302 | 1677 | { | 1707 | { |
303 | 1678 | if (!theIsMaterialized) | 1708 | if (!theIsMaterialized) |
304 | 1679 | 1709 | ||
305 | === modified file 'src/store/naive/atomic_items.h' | |||
306 | --- src/store/naive/atomic_items.h 2012-05-03 12:31:51 +0000 | |||
307 | +++ src/store/naive/atomic_items.h 2012-05-08 03:24:31 +0000 | |||
308 | @@ -875,6 +875,8 @@ | |||
309 | 875 | 875 | ||
310 | 876 | StreamReleaser theStreamReleaser; | 876 | StreamReleaser theStreamReleaser; |
311 | 877 | 877 | ||
312 | 878 | store::Item_t theStreamableDependent; | ||
313 | 879 | |||
314 | 878 | public: | 880 | public: |
315 | 879 | bool equals( | 881 | bool equals( |
316 | 880 | store::Item const*, | 882 | store::Item const*, |
317 | @@ -922,15 +924,10 @@ | |||
318 | 922 | StreamableStringItem( | 924 | StreamableStringItem( |
319 | 923 | std::istream& aStream, | 925 | std::istream& aStream, |
320 | 924 | StreamReleaser streamReleaser, | 926 | StreamReleaser streamReleaser, |
330 | 925 | bool seekable = false) | 927 | bool seekable = false); |
331 | 926 | : | 928 | |
332 | 927 | theIstream(aStream), | 929 | StreamableStringItem( |
333 | 928 | theIsMaterialized(false), | 930 | store::Item_t& aStreamableDependent); |
325 | 929 | theIsConsumed(false), | ||
326 | 930 | theIsSeekable(seekable), | ||
327 | 931 | theStreamReleaser(streamReleaser) | ||
328 | 932 | { | ||
329 | 933 | } | ||
334 | 934 | 931 | ||
335 | 935 | void materialize() const; | 932 | void materialize() const; |
336 | 936 | }; | 933 | }; |
337 | 937 | 934 | ||
338 | === modified file 'src/store/naive/simple_item_factory.cpp' | |||
339 | --- src/store/naive/simple_item_factory.cpp 2012-05-03 12:31:51 +0000 | |||
340 | +++ src/store/naive/simple_item_factory.cpp 2012-05-08 03:24:31 +0000 | |||
341 | @@ -157,6 +157,14 @@ | |||
342 | 157 | return true; | 157 | return true; |
343 | 158 | } | 158 | } |
344 | 159 | 159 | ||
345 | 160 | bool BasicItemFactory::createSharedStreamableString( | ||
346 | 161 | store::Item_t &result, | ||
347 | 162 | store::Item_t &streamable_dependent) | ||
348 | 163 | { | ||
349 | 164 | result = new StreamableStringItem( streamable_dependent ); | ||
350 | 165 | return true; | ||
351 | 166 | } | ||
352 | 167 | |||
353 | 160 | 168 | ||
354 | 161 | bool BasicItemFactory::createNormalizedString(store::Item_t& result, zstring& value) | 169 | bool BasicItemFactory::createNormalizedString(store::Item_t& result, zstring& value) |
355 | 162 | { | 170 | { |
356 | 163 | 171 | ||
357 | === modified file 'src/store/naive/simple_item_factory.h' | |||
358 | --- src/store/naive/simple_item_factory.h 2012-05-03 12:31:51 +0000 | |||
359 | +++ src/store/naive/simple_item_factory.h 2012-05-08 03:24:31 +0000 | |||
360 | @@ -106,6 +106,10 @@ | |||
361 | 106 | StreamReleaser, | 106 | StreamReleaser, |
362 | 107 | bool seekable = false); | 107 | bool seekable = false); |
363 | 108 | 108 | ||
364 | 109 | bool createSharedStreamableString( | ||
365 | 110 | store::Item_t& result, | ||
366 | 111 | store::Item_t& streamable_dependent); | ||
367 | 112 | |||
368 | 109 | bool createBase64Binary(store::Item_t& result, xs_base64Binary value); | 113 | bool createBase64Binary(store::Item_t& result, xs_base64Binary value); |
369 | 110 | 114 | ||
370 | 111 | bool createBase64Binary( | 115 | bool createBase64Binary( |
371 | 112 | 116 | ||
372 | === added file 'test/rbkt/ExpQueryResults/zorba/base64/decode_iso-8859-1.xml.res' | |||
373 | --- test/rbkt/ExpQueryResults/zorba/base64/decode_iso-8859-1.xml.res 1970-01-01 00:00:00 +0000 | |||
374 | +++ test/rbkt/ExpQueryResults/zorba/base64/decode_iso-8859-1.xml.res 2012-05-08 03:24:31 +0000 | |||
375 | @@ -0,0 +1,1 @@ | |||
376 | 1 | äöü | ||
377 | 0 | 2 | ||
378 | === added file 'test/rbkt/ExpQueryResults/zorba/base64/decode_iso-8859-1_file.xml.res' | |||
379 | --- test/rbkt/ExpQueryResults/zorba/base64/decode_iso-8859-1_file.xml.res 1970-01-01 00:00:00 +0000 | |||
380 | +++ test/rbkt/ExpQueryResults/zorba/base64/decode_iso-8859-1_file.xml.res 2012-05-08 03:24:31 +0000 | |||
381 | @@ -0,0 +1,1 @@ | |||
382 | 1 | äöü | ||
383 | 0 | 2 | ||
384 | === added file 'test/rbkt/ExpQueryResults/zorba/base64/reuse-stream.xml.res' | |||
385 | --- test/rbkt/ExpQueryResults/zorba/base64/reuse-stream.xml.res 1970-01-01 00:00:00 +0000 | |||
386 | +++ test/rbkt/ExpQueryResults/zorba/base64/reuse-stream.xml.res 2012-05-08 03:24:31 +0000 | |||
387 | @@ -0,0 +1,1 @@ | |||
388 | 1 | <file><content>f0VMRgEBAQAAAAAAAAAAAAMAAwABAAAAIPxDADQAAAD0JTMHAAAAADQAIAAHACgAKAAlAAEAAAAAAAAAAAAAAAAAAAAv22gBL9toAQUAAAAAEAAAAQAAAJTbaAGU62gBlOtoAehnBwAwpgcABgAAAAAQAAACAAAAvKBuAbywbgG8sG4BGAEAABgBAAAGAAAABAAAAAQAAAAUAQAAFAEAABQBAAAkAAAAJAAAAAQAAAAEAAAAUOV0ZKyHJwGshycBrIcnAfyBCAD8gQgABAAAAAQAAABR5XRkAAAAAAAAAAAAAAAAAAAAAAAAAAAGAAAABAAAAFLldGSU22gBlOtoAZTraAFs5AUAbOQFAAQAAAABAAAABAAAABQAAAADAAAAR05VANMOFlHRDhyfmkw9H3+lYuAncwGhG0AAAAYDAAAAIAAAEgAAABBkRVIkEIgUIwhRukiBHLCOBAAEAgggoSCCg0CkED5YAAAGFAIgABkACJAAiQAwAFCQEwAQIABBAIIFYAIACAASAAZARACAABkTpAQB5EQIYEF0IBIAIgAhciYEoEICZGgAAQAABCAAkBACZAJAAQQUEhGAIIAAKBULIJYAAABRAKFV4AIWgQAEIYgAAIAAQECAMgBGiCIGiRAEkAABAMAIgEQAAAQAACgIJYAAAEAAAAGAAgtCpFwEAAZRQACAABAICAASiBBFgRhIAkEBMAApJchACQZDAIAAgABIAKQEgKEACGE05FgCAAAgACEAgEAAQoxkUAEAiBAGAAkCFiBoISRFFCBEBMV3gARBgBAosUJBQQAEEDIAUUCBAABAEAhgAEAAAAAAACApACIBBgHKkAWBQAAEAAAAAACDURggAkEEEIlEgIEADQZwAAATC5AQNYxCABABAAAAAAEAlQD0WGD3gAAAIEDKCCAQQMGNIAAQAAAACECCAADAbMAWABABaRAAUFQCgAAAgxJmDAEIAiIZCmIAIAIEgAESCSAjETEQIws2A8AIgAEAAAAAkGkAiEAKDEYEBRGJhFEAIACICAIBKMAGAAAHIAQBqIgAUAECgAAAAhoEER4BASMBBhEIIQGCAFbg8IiBYAgCAQgAgEBwCAAAIAEAQATGAj+QQaAIIQAIAAShAgg2AKQAAAQBkCoASAAAFAAgAYFqAQDCcoOEg0sAIhKAABQAAgpAAghAAIQgIBAkAAQBDA4gRAACAAIgAPBAUgBDARCAggB0GQBaEAg1Mk8CABEAAAQgBBgAECIAvEIhFCRAIBgAAEALyBEQAhACQoIKwAABkBiAVM4QAQlFgBAS4AASAIIkAZEkEgAsBMCABSCAKACAIBMAD4JSLEyAAGQBBqATAUECQUgACQGBGA==</content><md5>f027633a677f42caf1544f6913bc2dab</md5></file> | ||
389 | 0 | \ No newline at end of file | 2 | \ No newline at end of file |
390 | 1 | 3 | ||
391 | === added file 'test/rbkt/Queries/zorba/base64/decode_iso-8859-1.xq' | |||
392 | --- test/rbkt/Queries/zorba/base64/decode_iso-8859-1.xq 1970-01-01 00:00:00 +0000 | |||
393 | +++ test/rbkt/Queries/zorba/base64/decode_iso-8859-1.xq 2012-05-08 03:24:31 +0000 | |||
394 | @@ -0,0 +1,3 @@ | |||
395 | 1 | import module namespace b = "http://www.zorba-xquery.com/modules/converters/base64"; | ||
396 | 2 | |||
397 | 3 | b:decode(xs:base64Binary("5Pb8Cg=="), "ISO-8859-1") | ||
398 | 0 | 4 | ||
399 | === added file 'test/rbkt/Queries/zorba/base64/decode_iso-8859-1_file.xq' | |||
400 | --- test/rbkt/Queries/zorba/base64/decode_iso-8859-1_file.xq 1970-01-01 00:00:00 +0000 | |||
401 | +++ test/rbkt/Queries/zorba/base64/decode_iso-8859-1_file.xq 2012-05-08 03:24:31 +0000 | |||
402 | @@ -0,0 +1,5 @@ | |||
403 | 1 | import module namespace f = "http://expath.org/ns/file"; | ||
404 | 2 | import module namespace b = "http://www.zorba-xquery.com/modules/converters/base64"; | ||
405 | 3 | |||
406 | 4 | b:decode(xs:base64Binary(f:read-text(resolve-uri("iso-8859-1.txt"))), "ISO-8859-1") | ||
407 | 5 | |||
408 | 0 | 6 | ||
409 | === added file 'test/rbkt/Queries/zorba/base64/iso-8859-1.txt' | |||
410 | --- test/rbkt/Queries/zorba/base64/iso-8859-1.txt 1970-01-01 00:00:00 +0000 | |||
411 | +++ test/rbkt/Queries/zorba/base64/iso-8859-1.txt 2012-05-08 03:24:31 +0000 | |||
412 | @@ -0,0 +1,1 @@ | |||
413 | 1 | 5Pb8Cg== | ||
414 | 0 | \ No newline at end of file | 2 | \ No newline at end of file |
415 | 1 | 3 | ||
416 | === added file 'test/rbkt/Queries/zorba/base64/reuse-stream.xq' | |||
417 | --- test/rbkt/Queries/zorba/base64/reuse-stream.xq 1970-01-01 00:00:00 +0000 | |||
418 | +++ test/rbkt/Queries/zorba/base64/reuse-stream.xq 2012-05-08 03:24:31 +0000 | |||
419 | @@ -0,0 +1,16 @@ | |||
420 | 1 | (: From bug 996084. We need to ensure that a streamable base64 item can be re-used. :) | ||
421 | 2 | |||
422 | 3 | import module namespace base64="http://www.zorba-xquery.com/modules/converters/base64"; | ||
423 | 4 | import module namespace hash = "http://www.zorba-xquery.com/modules/cryptography/hash"; | ||
424 | 5 | import module namespace file = "http://expath.org/ns/file"; | ||
425 | 6 | |||
426 | 7 | declare variable $filename as xs:anyURI := resolve-uri("decoded"); | ||
427 | 8 | |||
428 | 9 | let $data as xs:base64Binary := file:read-binary ($filename) | ||
429 | 10 | let $md5 as xs:string := hash:md5(base64:decode($data)) | ||
430 | 11 | |||
431 | 12 | return | ||
432 | 13 | <file> | ||
433 | 14 | <content>{$data}</content> | ||
434 | 15 | <md5>{$md5}</md5> | ||
435 | 16 | </file> |
Two things I'm not totally sure/happy about:
1. The name createSharedStr eamableString( ). Probably createSharingSt reamableString( ) would be more apt, but it doesn't seem clear to me. Maybe I should have just stuck with overloading createStreamabl eString( ). Thoughts?
2. I had to initialize theIstream in the initializer block of the new StreamableStrin gItem constructor, because theIstream is a reference. Therefore, the ZORBA_ASSERT() in that constructor will probably never go off - if the streamable_ dependent isn't streamable, we'll have already thrown an exception in the call to getStream() in the initializer block. I can't see a way around this.