Merge lp:~zorba-coders/zorba/ft-base64Binary into lp:zorba
- ft-base64Binary
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Till Westmann | ||||
Approved revision: | 10674 | ||||
Merged at revision: | 10677 | ||||
Proposed branch: | lp:~zorba-coders/zorba/ft-base64Binary | ||||
Merge into: | lp:zorba | ||||
Diff against target: |
1259 lines (+739/-105) 31 files modified
ChangeLog (+1/-0) include/zorba/item.h (+23/-0) include/zorba/item_factory.h (+20/-1) modules/com/zorba-xquery/www/modules/converters/base64.xq (+6/-3) modules/org/expath/ns/file.xq.src/file.cpp (+8/-13) modules/org/expath/ns/file.xq.src/file_function.cpp (+20/-4) src/api/item.cpp (+23/-0) src/api/itemfactoryimpl.cpp (+20/-1) src/api/itemfactoryimpl.h (+7/-0) src/api/options.cpp (+1/-0) src/api/serialization/serializer.cpp (+86/-40) src/runtime/base64/base64_impl.cpp (+53/-21) src/store/api/item.h (+8/-1) src/store/api/item_factory.h (+25/-0) src/store/naive/atomic_items.cpp (+204/-6) src/store/naive/atomic_items.h (+97/-8) src/store/naive/item.cpp (+19/-1) src/store/naive/simple_item_factory.cpp (+30/-3) src/store/naive/simple_item_factory.h (+13/-0) src/types/casting.cpp (+12/-1) src/zorbaserialization/zorba_class_serializer.cpp (+25/-2) src/zorbatypes/binary.cpp (+10/-0) test/rbkt/ExpQueryResults/zorba/base64/binary_1.xml.res (+1/-0) test/rbkt/ExpQueryResults/zorba/base64/file_read_1.xml.res (+1/-0) test/rbkt/ExpQueryResults/zorba/base64/file_read_2.xml.res (+1/-0) test/rbkt/Queries/zorba/base64/binary_1.xq (+1/-0) test/rbkt/Queries/zorba/base64/decoded-text (+1/-0) test/rbkt/Queries/zorba/base64/encoded (+1/-0) test/rbkt/Queries/zorba/base64/encoded-text (+1/-0) test/rbkt/Queries/zorba/base64/file_read_1.xq (+10/-0) test/rbkt/Queries/zorba/base64/file_read_2.xq (+11/-0) |
||||
To merge this branch: | bzr merge lp:~zorba-coders/zorba/ft-base64Binary | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Till Westmann | Approve | ||
Matthias Brantner | Approve | ||
Review via email: mp+94494@code.launchpad.net |
This proposal supersedes a proposal from 2012-02-22.
Commit message
more efficient implementation of xs:base64Binary items
Description of the change
more efficient implementation of xs:base64Binary items
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
The attempt to merge lp:~zorba-coders/zorba/ft-base64Binary into lp:zorba failed. Below is the output from the failed tests.
CMake Error at /home/ceej/
Validation queue job ft-base64Binary
The final status was:
7 tests did not succeed - changes not commited.
Error in read script: /home/ceej/
Matthias Brantner (matthias-brantner) : Posted in a previous version of this proposal | # |
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Validation queue job ft-base64Binary
All tests succeeded!
Zorba Build Bot (zorba-buildbot) wrote : Posted in a previous version of this proposal | # |
Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1. Got: 1 Approve, 1 Pending.
Dennis Knochenwefel (dennis-knochenwefel) wrote : Posted in a previous version of this proposal | # |
this also fixes bug #933490
Till Westmann (tillw) wrote : Posted in a previous version of this proposal | # |
include/
getBase64Binary
include/
(http://
src/runtime/
consumeNext because the input can't be an empty sequence? If so, should
we add a comment?
Could we implement Base64BinaryIte
Base64BinaryIte
Does Base64BinaryIte
empty? It seems that theValue would be prepended to the current content
of val. Could we implement this method by emptying val and calling
Base64BinaryIte
StreamableBase6
- Is there a reason for the number 4048?
- In general those might be a lot of re-allocations, if the item is a
little larger (e.g. if the item is 1MB, we've got more than 250
re-allocations). I'm not sure where the right trade-off is, but it
seems that this parameter might become expensive (if someone
materializes a large item that should be streamed …)
I just hope that src/zorbaserial
Matthias Brantner (matthias-brantner) wrote : Posted in a previous version of this proposal | # |
> include/
> getBase64Binary
fixed
>
> include/
> (http://
fixed (although the comment was originally written by a native speaker ;-)
>
> src/runtime/
> consumeNext because the input can't be an empty sequence? If so, should
> we add a comment?
this is common practice in almost every iterator. It's not possible that
consumeNext returns false if the sequence-type of the function doesn't allow
the empty sequence.
>
> Could we implement Base64BinaryIte
> Base64BinaryIte
done
>
> Does Base64BinaryIte
> empty? It seems that theValue would be prepended to the current content
> of val. Could we implement this method by emptying val and calling
> Base64BinaryIte
fixed & done
> StreamableBase6
> - Is there a reason for the number 4048?
no - I have no idea what the best value would be
> - In general those might be a lot of re-allocations, if the item is a
> little larger (e.g. if the item is 1MB, we've got more than 250
> re-allocations). I'm not sure where the right trade-off is, but it
> seems that this parameter might become expensive (if someone
> materializes a large item that should be streamed …)
Yes, it's really expensive if the stream is not seekable. The situation
is much better if the stream is seekable but I have no idea how to improve
the materialize if not.
>
> I just hope that src/zorbaserial
me too
Matthias Brantner (matthias-brantner) : | # |
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue job ft-base64Binary
All tests succeeded!
Zorba Build Bot (zorba-buildbot) wrote : | # |
Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1. Got: 1 Approve, 1 Pending.
Till Westmann (tillw) : | # |
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue job ft-base64Binary
All tests succeeded!
Preview Diff
1 | === modified file 'ChangeLog' |
2 | --- ChangeLog 2012-02-23 16:41:10 +0000 |
3 | +++ ChangeLog 2012-02-24 04:03:18 +0000 |
4 | @@ -21,6 +21,7 @@ |
5 | * Fixed bug #911585 (management of variables during eval) |
6 | * Fixed bug #866423 (fn:empty and fn:exists iterators must reset their input in |
7 | case of early-out) |
8 | + * More efficient implementation for base64Binary items |
9 | * Added index management function to the C++ api's StaticCollectionManager. |
10 | * Fixed bug #872288 (reset recursive flag during node rename) |
11 | * Fixed bug #905041 (allow for the default element and function namespaces to be |
12 | |
13 | === modified file 'include/zorba/item.h' |
14 | --- include/zorba/item.h 2012-01-11 17:30:25 +0000 |
15 | +++ include/zorba/item.h 2012-02-24 04:03:18 +0000 |
16 | @@ -363,6 +363,29 @@ |
17 | std::istream& |
18 | getStream(); |
19 | |
20 | + /** |
21 | + * Returns true if the contents of a binary item is already encoded |
22 | + * |
23 | + * @return true if the content is already encoded, false otherwise |
24 | + */ |
25 | + bool |
26 | + isEncoded() const; |
27 | + |
28 | + /** |
29 | + * Returns the value and size of the given base64Binary item |
30 | + * |
31 | + * The value is a string which is base64 encoded if isEncoded() |
32 | + * returns true. Otherwise, it is the original unencoded binary |
33 | + * data. |
34 | + * |
35 | + * If the given item is streamable (i.e. isStreamable() returns true), |
36 | + * the stream returned by getStream() should to be used to retrieve |
37 | + * the value. Otherwise, the contents of the stream will be materialized |
38 | + * in main memory. |
39 | + */ |
40 | + const char* |
41 | + getBase64BinaryValue(size_t& s) const; |
42 | + |
43 | /** \brief Returns the name of the collection this node is stored in. |
44 | * |
45 | * @return The name of the collection or 0 if the given item is not |
46 | |
47 | === modified file 'include/zorba/item_factory.h' |
48 | --- include/zorba/item_factory.h 2012-01-11 17:30:25 +0000 |
49 | +++ include/zorba/item_factory.h 2012-02-24 04:03:18 +0000 |
50 | @@ -54,7 +54,7 @@ |
51 | /** \brief Creates a streamable String Item |
52 | * see [http://www.w3.org/TR/xmlschema-2/#string] |
53 | * |
54 | - * @param stream An istream whence to read the string's content. |
55 | + * @param stream An istream from where to read the string's content. |
56 | * @param streamReleaser A function pointer which is invoked once |
57 | * the StreamableStringItem is destroyed. Normally this function |
58 | * will delete the std::istream object passed to it. |
59 | @@ -149,6 +149,25 @@ |
60 | virtual Item |
61 | createBase64Binary(const unsigned char* aBinData, size_t aLength) = 0; |
62 | |
63 | + /** \brief Creates a streamable Base64Binary Item |
64 | + * see [http://www.w3.org/TR/xmlschema-2/#base64Binary] |
65 | + * |
66 | + * @param stream An istream from where to read the binary's content. |
67 | + * @param streamReleaser A function pointer which is invoked once |
68 | + * the StreamableBase64Binary is destroyed. Normally this function |
69 | + * will delete the std::istream object passed to it. |
70 | + * @param seekable is the given stream seekable |
71 | + * @param encoded is the contents of the given stream already base64 |
72 | + * encoded |
73 | + * @return The streamable String Item |
74 | + */ |
75 | + virtual Item |
76 | + createStreamableBase64Binary( |
77 | + std::istream &stream, |
78 | + StreamReleaser streamReleaser, |
79 | + bool seekable = false, |
80 | + bool encoded = false) = 0; |
81 | + |
82 | /** \brief Creates a Boolean Item |
83 | * see [http://www.w3.org/TR/xmlschema-2/#bool] |
84 | * |
85 | |
86 | === modified file 'modules/com/zorba-xquery/www/modules/converters/base64.xq' |
87 | --- modules/com/zorba-xquery/www/modules/converters/base64.xq 2011-08-01 10:18:53 +0000 |
88 | +++ modules/com/zorba-xquery/www/modules/converters/base64.xq 2012-02-24 04:03:18 +0000 |
89 | @@ -28,14 +28,17 @@ |
90 | declare namespace ver = "http://www.zorba-xquery.com/options/versioning"; |
91 | declare option ver:module-version "2.0"; |
92 | |
93 | - |
94 | (:~ |
95 | : Decode a xs:base64Binary. |
96 | : |
97 | + : The function assumes that the content after decoding is valid |
98 | + : UTF-8. |
99 | + : |
100 | : @param $base64 The xs:base64Binary item to decode |
101 | - : @return the decoded xs:base64Binary item as string |
102 | + : @return the base64 decoded value as string |
103 | :) |
104 | -declare function base64:decode($base64 as xs:base64Binary) as xs:string external; |
105 | +declare function base64:decode($base64 as xs:base64Binary) |
106 | +as xs:string external; |
107 | |
108 | (:~ |
109 | : Encode a xs:string as xs:base64Binary. |
110 | |
111 | === modified file 'modules/org/expath/ns/file.xq.src/file.cpp' |
112 | --- modules/org/expath/ns/file.xq.src/file.cpp 2012-02-16 14:11:02 +0000 |
113 | +++ modules/org/expath/ns/file.xq.src/file.cpp 2012-02-24 04:03:18 +0000 |
114 | @@ -144,19 +144,14 @@ |
115 | // actual read |
116 | Item lItem; |
117 | try { |
118 | - std::ifstream lInStream; |
119 | - lFile->openInputStream(lInStream, true, false); |
120 | - |
121 | - std::stringstream lStrStream; |
122 | - char lBuf[1024]; |
123 | - while (!lInStream.eof()) { |
124 | - lInStream.read(lBuf, 1024); |
125 | - lStrStream.write(lBuf, lInStream.gcount()); |
126 | - } |
127 | - |
128 | - String lContent(lStrStream.str()); |
129 | - String lEncodedContent = encoding::Base64::encode(lContent); |
130 | - lItem = theModule->getItemFactory()->createBase64Binary(lEncodedContent.data(), lEncodedContent.size()); |
131 | + std::unique_ptr<std::ifstream> lInStream; |
132 | + lInStream.reset( new std::ifstream() ); |
133 | + lFile->openInputStream(*lInStream.get(), true, false); |
134 | + |
135 | + lItem = theModule->getItemFactory()->createStreamableBase64Binary( |
136 | + *lInStream.release(), &FileModule::streamReleaser, true |
137 | + ); |
138 | + |
139 | } catch (ZorbaException& ze) { |
140 | std::stringstream lSs; |
141 | lSs << "An unknown error occured: " << ze.what() << "Can not read file"; |
142 | |
143 | === modified file 'modules/org/expath/ns/file.xq.src/file_function.cpp' |
144 | --- modules/org/expath/ns/file.xq.src/file_function.cpp 2012-02-16 14:11:02 +0000 |
145 | +++ modules/org/expath/ns/file.xq.src/file_function.cpp 2012-02-24 04:03:18 +0000 |
146 | @@ -26,6 +26,7 @@ |
147 | #include <zorba/user_exception.h> |
148 | #include <zorba/util/path.h> |
149 | #include <zorba/xquery_functions.h> |
150 | +#include <zorba/singleton_item_sequence.h> |
151 | #include <zorba/zorba.h> |
152 | |
153 | #include "file_module.h" |
154 | @@ -255,10 +256,25 @@ |
155 | |
156 | // if this is a binary write |
157 | if (lBinary) { |
158 | - Zorba_SerializerOptions lOptions; |
159 | - lOptions.ser_method = ZORBA_SERIALIZATION_METHOD_BINARY; |
160 | - Serializer_t lSerializer = Serializer::createSerializer(lOptions); |
161 | - lSerializer->serialize(aArgs[1], lOutStream); |
162 | + Item lBinaryItem; |
163 | + Iterator_t lContentSeq = aArgs[1]->getIterator(); |
164 | + lContentSeq->open(); |
165 | + while (lContentSeq->next(lBinaryItem)) |
166 | + { |
167 | + if (lBinaryItem.isStreamable() && !lBinaryItem.isEncoded()) |
168 | + { |
169 | + lOutStream << lBinaryItem.getStream().rdbuf(); |
170 | + } |
171 | + else |
172 | + { |
173 | + Zorba_SerializerOptions lOptions; |
174 | + lOptions.ser_method = ZORBA_SERIALIZATION_METHOD_BINARY; |
175 | + Serializer_t lSerializer = Serializer::createSerializer(lOptions); |
176 | + SingletonItemSequence lSeq(lBinaryItem); |
177 | + lSerializer->serialize(&lSeq, lOutStream); |
178 | + } |
179 | + |
180 | + } |
181 | } |
182 | // if we only write text |
183 | else { |
184 | |
185 | === modified file 'src/api/item.cpp' |
186 | --- src/api/item.cpp 2012-02-02 09:56:52 +0000 |
187 | +++ src/api/item.cpp 2012-02-24 04:03:18 +0000 |
188 | @@ -470,6 +470,29 @@ |
189 | // TODO: throw exception |
190 | } |
191 | |
192 | +bool |
193 | +Item::isEncoded() const |
194 | +{ |
195 | + ITEM_TRY |
196 | + SYNC_CODE(AutoLock lock(GENV_STORE.getGlobalLock(), Lock::READ);) |
197 | + |
198 | + return m_item->isEncoded(); |
199 | + ITEM_CATCH |
200 | + // TODO: throw exception |
201 | +} |
202 | + |
203 | +const char* |
204 | +Item::getBase64BinaryValue(size_t& s) const |
205 | +{ |
206 | + ITEM_TRY |
207 | + SYNC_CODE(AutoLock lock(GENV_STORE.getGlobalLock(), Lock::READ);) |
208 | + |
209 | + return m_item->getBase64BinaryValue(s); |
210 | + ITEM_CATCH |
211 | + // TODO: throw exception |
212 | +} |
213 | + |
214 | + |
215 | Item |
216 | Item::getCollectionName() const |
217 | { |
218 | |
219 | === modified file 'src/api/itemfactoryimpl.cpp' |
220 | --- src/api/itemfactoryimpl.cpp 2012-01-11 17:30:25 +0000 |
221 | +++ src/api/itemfactoryimpl.cpp 2012-02-24 04:03:18 +0000 |
222 | @@ -212,13 +212,32 @@ |
223 | std::stringstream lSs; |
224 | while (aEncodedStream.good()) |
225 | { |
226 | - lSs.put(aEncodedStream.get()); |
227 | + char c = aEncodedStream.get(); |
228 | + if (aEncodedStream.good()) |
229 | + { |
230 | + lSs.put(c); |
231 | + } |
232 | } |
233 | std::string lContent = lSs.str(); |
234 | return createBase64Binary(lContent.c_str(), lContent.size()); |
235 | } |
236 | |
237 | |
238 | +Item |
239 | +ItemFactoryImpl::createStreamableBase64Binary( |
240 | + std::istream &stream, |
241 | + StreamReleaser streamReleaser, |
242 | + bool seekable, |
243 | + bool encoded) |
244 | +{ |
245 | + store::Item_t lItem; |
246 | + theItemFactory->createStreamableBase64Binary( |
247 | + lItem, stream, streamReleaser, seekable, encoded |
248 | + ); |
249 | + return &*lItem; |
250 | +} |
251 | + |
252 | + |
253 | Item ItemFactoryImpl::createBoolean(bool aValue) |
254 | { |
255 | store::Item_t lItem; |
256 | |
257 | === modified file 'src/api/itemfactoryimpl.h' |
258 | --- src/api/itemfactoryimpl.h 2012-01-11 17:30:25 +0000 |
259 | +++ src/api/itemfactoryimpl.h 2012-02-24 04:03:18 +0000 |
260 | @@ -69,6 +69,13 @@ |
261 | virtual Item |
262 | createBase64Binary(const unsigned char* aBinData, size_t aLength); |
263 | |
264 | + virtual Item |
265 | + createStreamableBase64Binary( |
266 | + std::istream &stream, |
267 | + StreamReleaser streamReleaser, |
268 | + bool seekable = false, |
269 | + bool encoded = false); |
270 | + |
271 | virtual Item |
272 | createBoolean(bool aValue); |
273 | |
274 | |
275 | === modified file 'src/api/options.cpp' |
276 | --- src/api/options.cpp 2012-01-11 17:30:25 +0000 |
277 | +++ src/api/options.cpp 2012-02-24 04:03:18 +0000 |
278 | @@ -63,6 +63,7 @@ |
279 | else if (strcmp(value, "html") == 0) ser_method = ZORBA_SERIALIZATION_METHOD_HTML; |
280 | else if (strcmp(value, "xhtml") == 0) ser_method = ZORBA_SERIALIZATION_METHOD_XHTML; |
281 | else if (strcmp(value, "text") == 0) ser_method = ZORBA_SERIALIZATION_METHOD_TEXT; |
282 | + else if (strcmp(value, "binary") == 0) ser_method = ZORBA_SERIALIZATION_METHOD_BINARY; |
283 | else |
284 | { |
285 | ; // TODO signal errors for incorrect values? |
286 | |
287 | === modified file 'src/api/serialization/serializer.cpp' |
288 | --- src/api/serialization/serializer.cpp 2012-01-11 17:30:25 +0000 |
289 | +++ src/api/serialization/serializer.cpp 2012-02-24 04:03:18 +0000 |
290 | @@ -368,22 +368,50 @@ |
291 | void serializer::emitter::emit_streamable_item(store::Item* item) |
292 | { |
293 | // Streamable item |
294 | - char buffer[1024]; |
295 | - int rollover = 0; |
296 | - std::streambuf * pbuf; |
297 | - std::streamsize read_bytes; |
298 | - std::istream& is = item->getStream(); |
299 | - |
300 | - // read bytes and do string expansion |
301 | - do |
302 | - { |
303 | - //std::istream::read uses a try/catch internally so the Zorba_Exception is lost: that is why we are using std::streambuf::sgetn |
304 | - pbuf = is.rdbuf(); |
305 | - read_bytes = pbuf->sgetn(buffer + rollover, 1024 - rollover); |
306 | - rollover = emit_expanded_string(buffer, static_cast<zstring::size_type>(read_bytes + rollover)); |
307 | - memmove(buffer, buffer + 1024 - rollover, rollover); |
308 | - } |
309 | - while (read_bytes > 0); |
310 | + store::SchemaTypeCode lTypeCode = item->getTypeCode(); |
311 | + |
312 | + switch (lTypeCode) |
313 | + { |
314 | + case store::XS_STRING: |
315 | + { |
316 | + char buffer[1024]; |
317 | + int rollover = 0; |
318 | + std::streambuf * pbuf; |
319 | + std::streamsize read_bytes; |
320 | + std::istream& is = item->getStream(); |
321 | + |
322 | + // read bytes and do string expansion |
323 | + do |
324 | + { |
325 | + //std::istream::read uses a try/catch internally so the Zorba_Exception is lost: that is why we are using std::streambuf::sgetn |
326 | + pbuf = is.rdbuf(); |
327 | + read_bytes = pbuf->sgetn(buffer + rollover, 1024 - rollover); |
328 | + rollover = emit_expanded_string(buffer, static_cast<zstring::size_type>(read_bytes + rollover)); |
329 | + memmove(buffer, buffer + 1024 - rollover, rollover); |
330 | + } |
331 | + while (read_bytes > 0); |
332 | + break; |
333 | + } |
334 | + case store::XS_BASE64BINARY: |
335 | + { |
336 | + if (item->isEncoded()) |
337 | + { |
338 | + std::istream& is = item->getStream(); |
339 | + char buf[1024]; |
340 | + while (is.good()) |
341 | + { |
342 | + is.read(buf, 1024); |
343 | + tr.write(buf, is.gcount()); |
344 | + } |
345 | + } |
346 | + else |
347 | + { |
348 | + tr << item->getStringValue(); |
349 | + } |
350 | + break; |
351 | + } |
352 | + default: assert(false); |
353 | + } |
354 | |
355 | } |
356 | |
357 | @@ -1866,30 +1894,48 @@ |
358 | ********************************************************************************/ |
359 | void serializer::binary_emitter::emit_item(store::Item* item) |
360 | { |
361 | - xs_base64Binary lValue; |
362 | - |
363 | - // First assume the item is a base64Binary item and try to get its value. |
364 | - try |
365 | - { |
366 | - lValue = item->getBase64BinaryValue(); |
367 | - } |
368 | - catch (...) |
369 | - { |
370 | - // If this fails, then just get the string value of the item and convert |
371 | - // it to base64 |
372 | - zstring lStringValue; |
373 | - item->getStringValue2(lStringValue); |
374 | - Base64::encode(lStringValue, lValue); |
375 | - } |
376 | - |
377 | - std::vector<char> lDecodedData; |
378 | - lValue.decode(lDecodedData); |
379 | - |
380 | - for (std::vector<char>::const_iterator lIter = lDecodedData.begin(); |
381 | - lIter != lDecodedData.end(); |
382 | - ++lIter) |
383 | - { |
384 | - tr << *lIter; |
385 | + if (item->isStreamable()) |
386 | + { |
387 | + std::istream& stream = item->getStream(); |
388 | + if (item->isEncoded()) |
389 | + { |
390 | + tr << Base64::decode(stream); |
391 | + } |
392 | + else |
393 | + { |
394 | + char buf[1024]; |
395 | + while (!stream.eof()) |
396 | + { |
397 | + stream.read(buf, 1024); |
398 | + tr.write(buf, stream.gcount()); |
399 | + } |
400 | + } |
401 | + } |
402 | + else |
403 | + { |
404 | + if (!item->isNode() && |
405 | + item->getTypeCode() == store::XS_BASE64BINARY) |
406 | + { |
407 | + size_t len; |
408 | + const char* value = item->getBase64BinaryValue(len); |
409 | + |
410 | + if (item->isEncoded()) |
411 | + { |
412 | + std::stringstream tmp; |
413 | + tmp.write(value, len); |
414 | + tr << Base64::decode(tmp); |
415 | + } |
416 | + else |
417 | + { |
418 | + tr.write(value, len); |
419 | + } |
420 | + } |
421 | + else |
422 | + { |
423 | + zstring lStringValue; |
424 | + item->getStringValue2(lStringValue); |
425 | + tr << lStringValue; |
426 | + } |
427 | } |
428 | } |
429 | |
430 | |
431 | === modified file 'src/runtime/base64/base64_impl.cpp' |
432 | --- src/runtime/base64/base64_impl.cpp 2011-06-14 17:26:33 +0000 |
433 | +++ src/runtime/base64/base64_impl.cpp 2012-02-24 04:03:18 +0000 |
434 | @@ -22,37 +22,74 @@ |
435 | |
436 | #include "runtime/base64/base64.h" |
437 | |
438 | - |
439 | #include "store/api/item.h" |
440 | #include "store/api/item_factory.h" |
441 | |
442 | namespace zorba { |
443 | |
444 | -bool Base64DecodeIterator::nextImpl(store::Item_t& result, PlanState& planState) const |
445 | +bool Base64DecodeIterator::nextImpl( |
446 | + store::Item_t& result, |
447 | + PlanState& planState) const |
448 | { |
449 | store::Item_t lItem; |
450 | - Base64 lDecodedData; |
451 | zstring lResultString; |
452 | + const char* lContent; |
453 | + size_t lSize; |
454 | + result = NULL; |
455 | |
456 | PlanIteratorState *state; |
457 | DEFAULT_STACK_INIT(PlanIteratorState, state, planState); |
458 | |
459 | - if (consumeNext(lItem, theChildren[0].getp(), planState)) |
460 | - { |
461 | - lDecodedData = lItem->getBase64BinaryValue(); |
462 | - lResultString = lDecodedData.decode().str(); |
463 | + consumeNext(lItem, theChildren[0].getp(), planState); |
464 | + |
465 | + if (lItem->isStreamable()) |
466 | + { |
467 | + if (lItem->isEncoded()) |
468 | + { |
469 | + // decode and eventually transcode |
470 | + lResultString = Base64::decode(lItem->getStream()); |
471 | + } |
472 | + else |
473 | + { |
474 | + // streamable string eventually transcoding |
475 | + GENV_ITEMFACTORY->createStreamableString( |
476 | + result, |
477 | + lItem->getStream(), |
478 | + lItem->getStreamReleaser(), |
479 | + lItem->isSeekable()); |
480 | + } |
481 | + } |
482 | + else |
483 | + { |
484 | + lContent = lItem->getBase64BinaryValue(lSize); |
485 | + |
486 | + if (lItem->isEncoded()) |
487 | + { |
488 | + std::vector<char> encoded(lContent, lContent+lSize); |
489 | + std::vector<char> decoded; |
490 | + Base64::decode(encoded, decoded); |
491 | + lResultString.insert(0, &decoded[0], decoded.size()); |
492 | + } |
493 | + else |
494 | + { |
495 | + lResultString.insert(0, lContent, lSize); |
496 | + } |
497 | + } |
498 | + if (!result) // otherwise it's a streamable string already |
499 | + { |
500 | GENV_ITEMFACTORY->createString(result, lResultString); |
501 | - STACK_PUSH (true, state); |
502 | } |
503 | + STACK_PUSH (true, state); |
504 | |
505 | STACK_END (state); |
506 | } |
507 | |
508 | |
509 | -bool Base64EncodeIterator::nextImpl(store::Item_t& result, PlanState& planState) const |
510 | +bool Base64EncodeIterator::nextImpl( |
511 | + store::Item_t& result, |
512 | + PlanState& planState) const |
513 | { |
514 | store::Item_t lItem; |
515 | - Base64 lBase64; |
516 | zstring lTmpString; |
517 | |
518 | PlanIteratorState* state; |
519 | @@ -61,17 +98,12 @@ |
520 | if (consumeNext(lItem, theChildren[0].getp(), planState)) |
521 | { |
522 | lItem->getStringValue2(lTmpString); |
523 | - Base64::encode(lTmpString, lBase64); |
524 | - if (GENV_ITEMFACTORY->createBase64Binary(result, lBase64)) |
525 | - { |
526 | - STACK_PUSH (true, state); |
527 | - } |
528 | - else |
529 | - { |
530 | - throw XQUERY_EXCEPTION( |
531 | - zerr::ZXQP0025_ITEM_CREATION_FAILED, ERROR_LOC( loc ) |
532 | - ); |
533 | - } |
534 | + // create a base64Binary item |
535 | + // the content is the non-encoded string |
536 | + GENV_ITEMFACTORY->createBase64Binary( |
537 | + result, lTmpString.c_str(), lTmpString.size(), false |
538 | + ); |
539 | + STACK_PUSH (true, state); |
540 | } |
541 | STACK_END (state); |
542 | } |
543 | |
544 | === modified file 'src/store/api/item.h' |
545 | --- src/store/api/item.h 2012-02-07 15:53:23 +0000 |
546 | +++ src/store/api/item.h 2012-02-24 04:03:18 +0000 |
547 | @@ -274,7 +274,14 @@ |
548 | |
549 | /** Accessor for xs:base64Binary |
550 | */ |
551 | - virtual xs_base64Binary getBase64BinaryValue() const; |
552 | + virtual const char* getBase64BinaryValue(size_t& size) const; |
553 | + |
554 | + /** |
555 | + * Checks whether a base64 item's content is already encoded |
556 | + * |
557 | + * @return true only if it is. |
558 | + */ |
559 | + virtual bool isEncoded() const; |
560 | |
561 | /** Accessor for xs:boolean |
562 | */ |
563 | |
564 | === modified file 'src/store/api/item_factory.h' |
565 | --- src/store/api/item_factory.h 2011-12-21 14:40:33 +0000 |
566 | +++ src/store/api/item_factory.h 2012-02-24 04:03:18 +0000 |
567 | @@ -205,6 +205,31 @@ |
568 | virtual bool createBase64Binary(Item_t& result, xs_base64Binary value) = 0; |
569 | |
570 | /** |
571 | + * Specification: [http://www.w3.org/TR/xmlschema-2/#base64Binary] |
572 | + * creates a base64Binary item with the given content |
573 | + * the encoded flag specifies whether the given content is already |
574 | + * base64 encoded or not. |
575 | + */ |
576 | + virtual bool createBase64Binary( |
577 | + Item_t& result, |
578 | + const char* value, |
579 | + size_t size, |
580 | + bool encoded) = 0; |
581 | + |
582 | + /** |
583 | + * Specification: [http://www.w3.org/TR/xmlschema-2/#base64Binary] |
584 | + * the encoded flag specifies whether the given content is already |
585 | + * base64 encoded or not. |
586 | + */ |
587 | + virtual bool createStreamableBase64Binary( |
588 | + Item_t& result, |
589 | + std::istream&, |
590 | + StreamReleaser, |
591 | + bool seekable = false, |
592 | + bool encoded = false) = 0; |
593 | + |
594 | + |
595 | + /** |
596 | * Specification: [http://www.w3.org/TR/xmlschema-2/#bool] |
597 | * @param value |
598 | */ |
599 | |
600 | === modified file 'src/store/naive/atomic_items.cpp' |
601 | --- src/store/naive/atomic_items.cpp 2012-02-15 10:25:02 +0000 |
602 | +++ src/store/naive/atomic_items.cpp 2012-02-24 04:03:18 +0000 |
603 | @@ -3059,6 +3059,45 @@ |
604 | /******************************************************************************* |
605 | class Base64BinaryItem |
606 | ********************************************************************************/ |
607 | +bool |
608 | +Base64BinaryItem::equals( |
609 | + const store::Item* other, |
610 | + long timezone, |
611 | + const XQPCollator* aCollation) const |
612 | +{ |
613 | + if (isEncoded() == other->isEncoded()) |
614 | + { |
615 | + size_t this_size, other_size; |
616 | + const char* this_data = getBase64BinaryValue(this_size); |
617 | + const char* other_data = other->getBase64BinaryValue(other_size); |
618 | + return this_size == other_size && |
619 | + memcmp(this_data, other_data, this_size) == 0; |
620 | + } |
621 | + else |
622 | + { |
623 | + return getStringValue().compare(other->getStringValue()) == 0; |
624 | + } |
625 | +} |
626 | + |
627 | + |
628 | +uint32_t |
629 | +Base64BinaryItem::hash(long timezone, const XQPCollator* aCollation) const |
630 | +{ |
631 | + // always need to hash on the string-value because otherwise |
632 | + // a base64 item that is encoded would have a different hash-value |
633 | + // as a base64 item that is decoded but represents the same binary content |
634 | + return utf8::hash(getStringValue(), aCollation); |
635 | +} |
636 | + |
637 | + |
638 | +const char* |
639 | +Base64BinaryItem::getBase64BinaryValue(size_t& size) const |
640 | +{ |
641 | + size = theValue.size(); |
642 | + return &theValue[0]; |
643 | +} |
644 | + |
645 | + |
646 | store::Item* Base64BinaryItem::getType() const |
647 | { |
648 | return GET_STORE().theSchemaTypeNames[store::XS_BASE64BINARY]; |
649 | @@ -3067,19 +3106,32 @@ |
650 | |
651 | zstring Base64BinaryItem::getStringValue() const |
652 | { |
653 | - return theValue.str(); |
654 | + zstring lRes; |
655 | + getStringValue2(lRes); |
656 | + return lRes; |
657 | } |
658 | |
659 | |
660 | void Base64BinaryItem::getStringValue2(zstring& val) const |
661 | { |
662 | - val = theValue.str(); |
663 | + val.clear(); |
664 | + appendStringValue(val); |
665 | } |
666 | |
667 | |
668 | void Base64BinaryItem::appendStringValue(zstring& buf) const |
669 | { |
670 | - buf += theValue.str(); |
671 | + if (theIsEncoded) |
672 | + { |
673 | + buf.insert(buf.size(), &theValue[0], theValue.size()); |
674 | + } |
675 | + else |
676 | + { |
677 | + std::vector<char> encoded; |
678 | + encoded.reserve(theValue.size()); |
679 | + Base64::encode(theValue, encoded); |
680 | + buf.insert(buf.size(), &encoded[0], encoded.size()); |
681 | + } |
682 | } |
683 | |
684 | |
685 | @@ -3092,9 +3144,155 @@ |
686 | } |
687 | |
688 | |
689 | -uint32_t Base64BinaryItem::hash(long timezone, const XQPCollator* aCollation) const |
690 | -{ |
691 | - return theValue.hash(); |
692 | +/******************************************************************************* |
693 | + class StreamableStringItem |
694 | +********************************************************************************/ |
695 | +zstring StreamableBase64BinaryItem::getStringValue() const |
696 | +{ |
697 | + if (!theIsMaterialized) |
698 | + { |
699 | + materialize(); |
700 | + } |
701 | + return Base64BinaryItem::getStringValue(); |
702 | +} |
703 | + |
704 | + |
705 | +void StreamableBase64BinaryItem::getStringValue2(zstring& val) const |
706 | +{ |
707 | + if (!theIsMaterialized) |
708 | + { |
709 | + materialize(); |
710 | + } |
711 | + Base64BinaryItem::getStringValue2(val); |
712 | +} |
713 | + |
714 | + |
715 | +void StreamableBase64BinaryItem::appendStringValue(zstring& buf) const |
716 | +{ |
717 | + if (!theIsMaterialized) |
718 | + { |
719 | + materialize(); |
720 | + } |
721 | + Base64BinaryItem::appendStringValue(buf); |
722 | +} |
723 | + |
724 | + |
725 | +zstring StreamableBase64BinaryItem::show() const |
726 | +{ |
727 | + if (!theIsMaterialized) |
728 | + { |
729 | + materialize(); |
730 | + } |
731 | + zstring res("xs:base64Binary("); |
732 | + appendStringValue(res); |
733 | + res += ")"; |
734 | + return res; |
735 | +} |
736 | + |
737 | + |
738 | +uint32_t |
739 | +StreamableBase64BinaryItem::hash(long timezone, const XQPCollator* aCollation) const |
740 | +{ |
741 | + if (!theIsMaterialized) |
742 | + { |
743 | + materialize(); |
744 | + } |
745 | + return Base64BinaryItem::hash(timezone, aCollation); |
746 | +} |
747 | + |
748 | + |
749 | +const char* |
750 | +StreamableBase64BinaryItem::getBase64BinaryValue(size_t& s) const |
751 | +{ |
752 | + if (!theIsMaterialized) |
753 | + { |
754 | + materialize(); |
755 | + } |
756 | + return Base64BinaryItem::getBase64BinaryValue(s); |
757 | +} |
758 | + |
759 | + |
760 | +bool StreamableBase64BinaryItem::isStreamable() const |
761 | +{ |
762 | + return true; |
763 | +} |
764 | + |
765 | + |
766 | +bool StreamableBase64BinaryItem::isSeekable() const |
767 | +{ |
768 | + return theIsSeekable; |
769 | +} |
770 | + |
771 | + |
772 | +StreamReleaser StreamableBase64BinaryItem::getStreamReleaser() |
773 | +{ |
774 | + return theStreamReleaser; |
775 | +} |
776 | + |
777 | + |
778 | +void StreamableBase64BinaryItem::setStreamReleaser(StreamReleaser aReleaser) |
779 | +{ |
780 | + theStreamReleaser = aReleaser; |
781 | +} |
782 | + |
783 | + |
784 | +std::istream& StreamableBase64BinaryItem::getStream() |
785 | +{ |
786 | + // a non-seekable stream can only be consumed once |
787 | + // we raise an error if getStream is called twice |
788 | + // if a query requires a stream to be consumed more than once, |
789 | + // the query needs to make sure that the stream is explicitly |
790 | + // materialized before |
791 | + if (!theIsSeekable && theIsConsumed) |
792 | + { |
793 | + throw ZORBA_EXCEPTION( zerr::ZSTR0055_STREAMABLE_STRING_CONSUMED ); |
794 | + } |
795 | + else |
796 | + { |
797 | + // if the stream is seekable, we seek to the beginning. |
798 | + // We are not using theIstream.seekg because the USER_ERROR that is thrown |
799 | + // by Zorba is lost possibly in an internal try/catch of the seekg |
800 | + std::streambuf * pbuf; |
801 | + pbuf = theIstream.rdbuf(); |
802 | + pbuf->pubseekoff(0, std::ios::beg); |
803 | + } |
804 | + theIsConsumed = true; |
805 | + return theIstream; |
806 | +} |
807 | + |
808 | + |
809 | +void StreamableBase64BinaryItem::materialize() const |
810 | +{ |
811 | + StreamableBase64BinaryItem* const s |
812 | + = const_cast<StreamableBase64BinaryItem*>(this); |
813 | + std::istream& lStream = s->getStream(); |
814 | + |
815 | + s->theIsMaterialized = true; |
816 | + s->theIsConsumed = true; |
817 | + |
818 | + if (isSeekable()) |
819 | + { |
820 | + lStream.seekg(0, std::ios::end); |
821 | + size_t len = lStream.tellg(); |
822 | + lStream.seekg(0, std::ios::beg); |
823 | + s->theValue.reserve(len); |
824 | + char buf[1024]; |
825 | + while (lStream.good()) |
826 | + { |
827 | + lStream.read(buf, 1024); |
828 | + s->theValue.insert(s->theValue.end(), buf, buf+lStream.gcount()); |
829 | + } |
830 | + } |
831 | + else |
832 | + { |
833 | + char buf[4048]; |
834 | + while (lStream.good()) |
835 | + { |
836 | + lStream.read(buf, 4048); |
837 | + s->theValue.reserve(s->theValue.size() + lStream.gcount()); |
838 | + s->theValue.insert(s->theValue.end(), buf, buf+lStream.gcount()); |
839 | + } |
840 | + } |
841 | } |
842 | |
843 | |
844 | |
845 | === modified file 'src/store/naive/atomic_items.h' |
846 | --- src/store/naive/atomic_items.h 2012-01-26 19:56:14 +0000 |
847 | +++ src/store/naive/atomic_items.h 2012-02-24 04:03:18 +0000 |
848 | @@ -20,6 +20,7 @@ |
849 | #include <zorba/config.h> |
850 | #include <iostream> |
851 | #include <vector> |
852 | +#include <cstring> |
853 | |
854 | #include <zorba/streams.h> |
855 | #ifndef ZORBA_NO_FULL_TEXT |
856 | @@ -153,7 +154,10 @@ |
857 | |
858 | const zstring& getString() const { return theBaseItem->getString(); } |
859 | |
860 | - xs_base64Binary getBase64BinaryValue() const { return theBaseItem->getBase64BinaryValue(); } |
861 | + const char* getBase64BinaryValue(size_t& s) const |
862 | + { |
863 | + return theBaseItem->getBase64BinaryValue(s); |
864 | + } |
865 | |
866 | xs_hexBinary getHexBinaryValue() const { return theBaseItem->getHexBinaryValue(); } |
867 | |
868 | @@ -2218,30 +2222,115 @@ |
869 | friend class BasicItemFactory; |
870 | |
871 | protected: |
872 | - xs_base64Binary theValue; |
873 | + std::vector<char> theValue; |
874 | + bool theIsEncoded; |
875 | |
876 | protected: |
877 | - Base64BinaryItem(xs_base64Binary aValue) : theValue(aValue) {} |
878 | + Base64BinaryItem(bool aIsEncoded) |
879 | + : theIsEncoded(aIsEncoded) {} |
880 | |
881 | - Base64BinaryItem() {} |
882 | + Base64BinaryItem(const char* aValue, size_t aSize, bool aIsEncoded = true) |
883 | + : theIsEncoded(aIsEncoded) |
884 | + { |
885 | + theValue.reserve(aSize); |
886 | + theValue.insert(theValue.begin(), aValue, aValue + aSize); |
887 | + } |
888 | |
889 | public: |
890 | - xs_base64Binary getBase64BinaryValue() const { return theValue; } |
891 | + const char* getBase64BinaryValue(size_t& data) const; |
892 | |
893 | store::SchemaTypeCode getTypeCode() const { return store::XS_BASE64BINARY; } |
894 | |
895 | store::Item* getType() const; |
896 | |
897 | + bool isEncoded() const { return theIsEncoded; } |
898 | + |
899 | uint32_t hash(long timezone = 0, const XQPCollator* aCollation = 0) const; |
900 | |
901 | bool equals( |
902 | const store::Item* other, |
903 | long timezone = 0, |
904 | - const XQPCollator* aCollation = 0 ) const |
905 | - { |
906 | - return theValue.equal(other->getBase64BinaryValue()); |
907 | + const XQPCollator* aCollation = 0 ) const; |
908 | + |
909 | + zstring getStringValue() const; |
910 | + |
911 | + void getStringValue2(zstring& val) const; |
912 | + |
913 | + void appendStringValue(zstring& buf) const; |
914 | + |
915 | + zstring show() const; |
916 | + |
917 | +protected: |
918 | + // used in hash doing simple xor of the data |
919 | + struct hash_functor |
920 | + { |
921 | + uint32_t hash_value; |
922 | + |
923 | + void operator() (char c) |
924 | + { |
925 | + hash_value ^= (uint32_t) c; |
926 | + } |
927 | + }; |
928 | +}; |
929 | + |
930 | + |
931 | +/******************************************************************************* |
932 | + class StreamableBase64BinaryItem |
933 | +********************************************************************************/ |
934 | +class StreamableBase64BinaryItem : public Base64BinaryItem |
935 | +{ |
936 | + friend class BasicItemFactory; |
937 | + |
938 | +protected: |
939 | + std::istream & theIstream; |
940 | + |
941 | + bool theIsMaterialized; |
942 | + bool theIsConsumed; |
943 | + bool theIsSeekable; |
944 | + |
945 | + StreamReleaser theStreamReleaser; |
946 | + |
947 | +protected: |
948 | + StreamableBase64BinaryItem( |
949 | + std::istream& aStream, |
950 | + StreamReleaser streamReleaser, |
951 | + bool seekable = false, |
952 | + bool is_encoded = false) |
953 | + : Base64BinaryItem(is_encoded), |
954 | + theIstream(aStream), |
955 | + theIsMaterialized(false), |
956 | + theIsConsumed(false), |
957 | + theIsSeekable(seekable), |
958 | + theStreamReleaser(streamReleaser) |
959 | + {} |
960 | + |
961 | + void materialize() const; |
962 | + |
963 | +public: |
964 | + virtual ~StreamableBase64BinaryItem() |
965 | + { |
966 | + if (theStreamReleaser) |
967 | + { |
968 | + theStreamReleaser(&theIstream); |
969 | + } |
970 | } |
971 | |
972 | + bool isStreamable() const; |
973 | + |
974 | + bool isSeekable() const; |
975 | + |
976 | + std::istream& getStream(); |
977 | + |
978 | + StreamReleaser getStreamReleaser(); |
979 | + |
980 | + void setStreamReleaser(StreamReleaser aReleaser); |
981 | + |
982 | + const char* getBase64BinaryValue(size_t&) const; |
983 | + |
984 | + store::SchemaTypeCode getTypeCode() const { return store::XS_BASE64BINARY; } |
985 | + |
986 | + uint32_t hash(long timezone = 0, const XQPCollator* aCollation = 0) const; |
987 | + |
988 | zstring getStringValue() const; |
989 | |
990 | void getStringValue2(zstring& val) const; |
991 | |
992 | === modified file 'src/store/naive/item.cpp' |
993 | --- src/store/naive/item.cpp 2012-02-15 10:25:02 +0000 |
994 | +++ src/store/naive/item.cpp 2012-02-24 04:03:18 +0000 |
995 | @@ -430,7 +430,7 @@ |
996 | /** |
997 | * Accessor for xs:base64Binary |
998 | */ |
999 | -xs_base64Binary Item::getBase64BinaryValue() const |
1000 | +const char* Item::getBase64BinaryValue(size_t&) const |
1001 | { |
1002 | throw ZORBA_EXCEPTION( |
1003 | zerr::ZSTR0040_TYPE_ERROR, |
1004 | @@ -441,6 +441,24 @@ |
1005 | ); |
1006 | } |
1007 | |
1008 | + |
1009 | +/** |
1010 | + * Checks whether a base64 item's content is already encoded |
1011 | + * |
1012 | + * @return true only if it is. |
1013 | + */ |
1014 | +bool Item::isEncoded() const |
1015 | +{ |
1016 | + throw ZORBA_EXCEPTION( |
1017 | + zerr::ZSTR0040_TYPE_ERROR, |
1018 | + ERROR_PARAMS( |
1019 | + ZED( OperationNotDef_23 ), "Item::isEncoded()", |
1020 | + getType()->getStringValue() |
1021 | + ) |
1022 | + ); |
1023 | +} |
1024 | + |
1025 | + |
1026 | /** |
1027 | * Accessor for xs:boolean |
1028 | */ |
1029 | |
1030 | === modified file 'src/store/naive/simple_item_factory.cpp' |
1031 | --- src/store/naive/simple_item_factory.cpp 2012-02-15 10:25:02 +0000 |
1032 | +++ src/store/naive/simple_item_factory.cpp 2012-02-24 04:03:18 +0000 |
1033 | @@ -997,9 +997,36 @@ |
1034 | } |
1035 | |
1036 | |
1037 | -bool BasicItemFactory::createBase64Binary(store::Item_t& result, xs_base64Binary value) |
1038 | -{ |
1039 | - result = new Base64BinaryItem(value); |
1040 | +bool BasicItemFactory::createBase64Binary( |
1041 | + store::Item_t& result, |
1042 | + xs_base64Binary value) |
1043 | +{ |
1044 | + const std::vector<char>& data = value.getData(); |
1045 | + result = new Base64BinaryItem(&data[0], data.size(), true); |
1046 | + return true; |
1047 | +} |
1048 | + |
1049 | +bool BasicItemFactory::createBase64Binary( |
1050 | + store::Item_t& result, |
1051 | + const char* value, |
1052 | + size_t size, |
1053 | + bool encoded) |
1054 | +{ |
1055 | + result = new Base64BinaryItem(value, size, encoded); |
1056 | + return true; |
1057 | +} |
1058 | + |
1059 | + |
1060 | +bool BasicItemFactory::createStreamableBase64Binary( |
1061 | + store::Item_t& result, |
1062 | + std::istream& aStream, |
1063 | + StreamReleaser aReleaser, |
1064 | + bool seekable, |
1065 | + bool encoded) |
1066 | +{ |
1067 | + result = new StreamableBase64BinaryItem( |
1068 | + aStream, aReleaser, seekable, encoded |
1069 | + ); |
1070 | return true; |
1071 | } |
1072 | |
1073 | |
1074 | === modified file 'src/store/naive/simple_item_factory.h' |
1075 | --- src/store/naive/simple_item_factory.h 2011-12-21 14:40:33 +0000 |
1076 | +++ src/store/naive/simple_item_factory.h 2012-02-24 04:03:18 +0000 |
1077 | @@ -107,6 +107,19 @@ |
1078 | |
1079 | bool createBase64Binary(store::Item_t& result, xs_base64Binary value); |
1080 | |
1081 | + bool createBase64Binary( |
1082 | + store::Item_t& result, |
1083 | + const char* value, |
1084 | + size_t size, |
1085 | + bool encoded); |
1086 | + |
1087 | + bool createStreamableBase64Binary( |
1088 | + store::Item_t& result, |
1089 | + std::istream&, |
1090 | + StreamReleaser, |
1091 | + bool seekable = false, |
1092 | + bool encoded = false); |
1093 | + |
1094 | bool createBoolean(store::Item_t& result, xs_boolean value); |
1095 | |
1096 | |
1097 | |
1098 | === modified file 'src/types/casting.cpp' |
1099 | --- src/types/casting.cpp 2012-01-30 15:23:21 +0000 |
1100 | +++ src/types/casting.cpp 2012-02-24 04:03:18 +0000 |
1101 | @@ -1267,7 +1267,18 @@ |
1102 | |
1103 | T1_TO_T2(b64, hxB) |
1104 | { |
1105 | - return aFactory->createHexBinary(result, xs_hexBinary(aItem->getBase64BinaryValue())); |
1106 | + size_t s; |
1107 | + const char* c = aItem->getBase64BinaryValue(s); |
1108 | + Base64 tmp; |
1109 | + if (aItem->isEncoded()) |
1110 | + { |
1111 | + Base64::parseString(c, s, tmp); |
1112 | + } |
1113 | + else |
1114 | + { |
1115 | + Base64::encode((const unsigned char*)c, s, tmp); |
1116 | + } |
1117 | + return aFactory->createHexBinary(result, xs_hexBinary(tmp)); |
1118 | } |
1119 | |
1120 | |
1121 | |
1122 | === modified file 'src/zorbaserialization/zorba_class_serializer.cpp' |
1123 | --- src/zorbaserialization/zorba_class_serializer.cpp 2012-01-11 17:30:25 +0000 |
1124 | +++ src/zorbaserialization/zorba_class_serializer.cpp 2012-02-24 04:03:18 +0000 |
1125 | @@ -673,8 +673,31 @@ |
1126 | |
1127 | else if(name_of_type == "base64Binary") |
1128 | { |
1129 | - SERIALIZE_REF_FIELD(xs_base64Binary, value, getBase64BinaryValue()); |
1130 | - FINALIZE_SERIALIZE(createBase64Binary, (result, value_in)); |
1131 | + if (ar.is_serializing_out()) |
1132 | + { |
1133 | + size_t s; |
1134 | + const char* c = obj->getBase64BinaryValue(s); |
1135 | + if (obj->isEncoded()) |
1136 | + { |
1137 | + Base64 tmp; |
1138 | + Base64::parseString(c, s, tmp); |
1139 | + ar.dont_allow_delay(); |
1140 | + ar & tmp; |
1141 | + } |
1142 | + else |
1143 | + { |
1144 | + Base64 tmp((const unsigned char*)c, s); |
1145 | + ar.dont_allow_delay(); |
1146 | + ar & tmp; |
1147 | + } |
1148 | + } |
1149 | + else |
1150 | + { |
1151 | + ar.dont_allow_delay(); |
1152 | + Base64 tmp; |
1153 | + ar & tmp; |
1154 | + FINALIZE_SERIALIZE(createBase64Binary, (result, tmp)); |
1155 | + } |
1156 | } |
1157 | else if(name_of_type == "hexBinary") |
1158 | { |
1159 | |
1160 | === modified file 'src/zorbatypes/binary.cpp' |
1161 | --- src/zorbatypes/binary.cpp 2011-06-14 17:26:33 +0000 |
1162 | +++ src/zorbatypes/binary.cpp 2012-02-24 04:03:18 +0000 |
1163 | @@ -306,6 +306,16 @@ |
1164 | } |
1165 | |
1166 | |
1167 | +Base64::Base64(const unsigned char *bin_data, size_t len) |
1168 | +{ |
1169 | + std::vector<char> tmp; |
1170 | + tmp.reserve(len); |
1171 | + tmp.insert(tmp.begin(), (const char*)bin_data, ((const char*)bin_data) + len); |
1172 | + theData.reserve(len); |
1173 | + encode(tmp, theData); |
1174 | +} |
1175 | + |
1176 | + |
1177 | void Base64::serialize(::zorba::serialization::Archiver& ar) |
1178 | { |
1179 | ar & theData; |
1180 | |
1181 | === added directory 'test/rbkt/ExpQueryResults/zorba/base64' |
1182 | === added file 'test/rbkt/ExpQueryResults/zorba/base64/binary_1.xml.res' |
1183 | --- test/rbkt/ExpQueryResults/zorba/base64/binary_1.xml.res 1970-01-01 00:00:00 +0000 |
1184 | +++ test/rbkt/ExpQueryResults/zorba/base64/binary_1.xml.res 2012-02-24 04:03:18 +0000 |
1185 | @@ -0,0 +1,1 @@ |
1186 | +true |
1187 | |
1188 | === added file 'test/rbkt/ExpQueryResults/zorba/base64/file_read_1.xml.res' |
1189 | --- test/rbkt/ExpQueryResults/zorba/base64/file_read_1.xml.res 1970-01-01 00:00:00 +0000 |
1190 | +++ test/rbkt/ExpQueryResults/zorba/base64/file_read_1.xml.res 2012-02-24 04:03:18 +0000 |
1191 | @@ -0,0 +1,1 @@ |
1192 | +true true f0VMRgEBAQAAAAAAAAAAAAMAAwABAAAAIPxDADQAAAD0JTMHAAAAADQAIAAHACgAKAAlAAEAAAAAAAAAAAAAAAAAAAAv22gBL9toAQUAAAAAEAAAAQAAAJTbaAGU62gBlOtoAehnBwAwpgcABgAAAAAQAAACAAAAvKBuAbywbgG8sG4BGAEAABgBAAAGAAAABAAAAAQAAAAUAQAAFAEAABQBAAAkAAAAJAAAAAQAAAAEAAAAUOV0ZKyHJwGshycBrIcnAfyBCAD8gQgABAAAAAQAAABR5XRkAAAAAAAAAAAAAAAAAAAAAAAAAAAGAAAABAAAAFLldGSU22gBlOtoAZTraAFs5AUAbOQFAAQAAAABAAAABAAAABQAAAADAAAAR05VANMOFlHRDhyfmkw9H3+lYuAncwGhG0AAAAYDAAAAIAAAEgAAABBkRVIkEIgUIwhRukiBHLCOBAAEAgggoSCCg0CkED5YAAAGFAIgABkACJAAiQAwAFCQEwAQIABBAIIFYAIACAASAAZARACAABkTpAQB5EQIYEF0IBIAIgAhciYEoEICZGgAAQAABCAAkBACZAJAAQQUEhGAIIAAKBULIJYAAABRAKFV4AIWgQAEIYgAAIAAQECAMgBGiCIGiRAEkAABAMAIgEQAAAQAACgIJYAAAEAAAAGAAgtCpFwEAAZRQACAABAICAASiBBFgRhIAkEBMAApJchACQZDAIAAgABIAKQEgKEACGE05FgCAAAgACEAgEAAQoxkUAEAiBAGAAkCFiBoISRFFCBEBMV3gARBgBAosUJBQQAEEDIAUUCBAABAEAhgAEAAAAAAACApACIBBgHKkAWBQAAEAAAAAACDURggAkEEEIlEgIEADQZwAAATC5AQNYxCABABAAAAAAEAlQD0WGD3gAAAIEDKCCAQQMGNIAAQAAAACECCAADAbMAWABABaRAAUFQCgAAAgxJmDAEIAiIZCmIAIAIEgAESCSAjETEQIws2A8AIgAEAAAAAkGkAiEAKDEYEBRGJhFEAIACICAIBKMAGAAAHIAQBqIgAUAECgAAAAhoEER4BASMBBhEIIQGCAFbg8IiBYAgCAQgAgEBwCAAAIAEAQATGAj+QQaAIIQAIAAShAgg2AKQAAAQBkCoASAAAFAAgAYFqAQDCcoOEg0sAIhKAABQAAgpAAghAAIQgIBAkAAQBDA4gRAACAAIgAPBAUgBDARCAggB0GQBaEAg1Mk8CABEAAAQgBBgAECIAvEIhFCRAIBgAAEALyBEQAhACQoIKwAABkBiAVM4QAQlFgBAS4AASAIIkAZEkEgAsBMCABSCAKACAIBMAD4JSLEyAAGQBBqATAUECQUgACQGBGA== |
1193 | |
1194 | === added file 'test/rbkt/ExpQueryResults/zorba/base64/file_read_2.xml.res' |
1195 | --- test/rbkt/ExpQueryResults/zorba/base64/file_read_2.xml.res 1970-01-01 00:00:00 +0000 |
1196 | +++ test/rbkt/ExpQueryResults/zorba/base64/file_read_2.xml.res 2012-02-24 04:03:18 +0000 |
1197 | @@ -0,0 +1,1 @@ |
1198 | +true true true |
1199 | |
1200 | === added directory 'test/rbkt/Queries/zorba/base64' |
1201 | === added file 'test/rbkt/Queries/zorba/base64/binary_1.xq' |
1202 | --- test/rbkt/Queries/zorba/base64/binary_1.xq 1970-01-01 00:00:00 +0000 |
1203 | +++ test/rbkt/Queries/zorba/base64/binary_1.xq 2012-02-24 04:03:18 +0000 |
1204 | @@ -0,0 +1,1 @@ |
1205 | +xs:string(xs:base64Binary("Wm9yYmEgaXMgR3JlYXQhIMOkw7bDvA==")) eq "Wm9yYmEgaXMgR3JlYXQhIMOkw7bDvA==" |
1206 | |
1207 | === added file 'test/rbkt/Queries/zorba/base64/decoded' |
1208 | Binary files test/rbkt/Queries/zorba/base64/decoded 1970-01-01 00:00:00 +0000 and test/rbkt/Queries/zorba/base64/decoded 2012-02-24 04:03:18 +0000 differ |
1209 | === added file 'test/rbkt/Queries/zorba/base64/decoded-text' |
1210 | --- test/rbkt/Queries/zorba/base64/decoded-text 1970-01-01 00:00:00 +0000 |
1211 | +++ test/rbkt/Queries/zorba/base64/decoded-text 2012-02-24 04:03:18 +0000 |
1212 | @@ -0,0 +1,1 @@ |
1213 | +Zorba is Great! äöü |
1214 | \ No newline at end of file |
1215 | |
1216 | === added file 'test/rbkt/Queries/zorba/base64/encoded' |
1217 | --- test/rbkt/Queries/zorba/base64/encoded 1970-01-01 00:00:00 +0000 |
1218 | +++ test/rbkt/Queries/zorba/base64/encoded 2012-02-24 04:03:18 +0000 |
1219 | @@ -0,0 +1,1 @@ |
1220 | +f0VMRgEBAQAAAAAAAAAAAAMAAwABAAAAIPxDADQAAAD0JTMHAAAAADQAIAAHACgAKAAlAAEAAAAAAAAAAAAAAAAAAAAv22gBL9toAQUAAAAAEAAAAQAAAJTbaAGU62gBlOtoAehnBwAwpgcABgAAAAAQAAACAAAAvKBuAbywbgG8sG4BGAEAABgBAAAGAAAABAAAAAQAAAAUAQAAFAEAABQBAAAkAAAAJAAAAAQAAAAEAAAAUOV0ZKyHJwGshycBrIcnAfyBCAD8gQgABAAAAAQAAABR5XRkAAAAAAAAAAAAAAAAAAAAAAAAAAAGAAAABAAAAFLldGSU22gBlOtoAZTraAFs5AUAbOQFAAQAAAABAAAABAAAABQAAAADAAAAR05VANMOFlHRDhyfmkw9H3+lYuAncwGhG0AAAAYDAAAAIAAAEgAAABBkRVIkEIgUIwhRukiBHLCOBAAEAgggoSCCg0CkED5YAAAGFAIgABkACJAAiQAwAFCQEwAQIABBAIIFYAIACAASAAZARACAABkTpAQB5EQIYEF0IBIAIgAhciYEoEICZGgAAQAABCAAkBACZAJAAQQUEhGAIIAAKBULIJYAAABRAKFV4AIWgQAEIYgAAIAAQECAMgBGiCIGiRAEkAABAMAIgEQAAAQAACgIJYAAAEAAAAGAAgtCpFwEAAZRQACAABAICAASiBBFgRhIAkEBMAApJchACQZDAIAAgABIAKQEgKEACGE05FgCAAAgACEAgEAAQoxkUAEAiBAGAAkCFiBoISRFFCBEBMV3gARBgBAosUJBQQAEEDIAUUCBAABAEAhgAEAAAAAAACApACIBBgHKkAWBQAAEAAAAAACDURggAkEEEIlEgIEADQZwAAATC5AQNYxCABABAAAAAAEAlQD0WGD3gAAAIEDKCCAQQMGNIAAQAAAACECCAADAbMAWABABaRAAUFQCgAAAgxJmDAEIAiIZCmIAIAIEgAESCSAjETEQIws2A8AIgAEAAAAAkGkAiEAKDEYEBRGJhFEAIACICAIBKMAGAAAHIAQBqIgAUAECgAAAAhoEER4BASMBBhEIIQGCAFbg8IiBYAgCAQgAgEBwCAAAIAEAQATGAj+QQaAIIQAIAAShAgg2AKQAAAQBkCoASAAAFAAgAYFqAQDCcoOEg0sAIhKAABQAAgpAAghAAIQgIBAkAAQBDA4gRAACAAIgAPBAUgBDARCAggB0GQBaEAg1Mk8CABEAAAQgBBgAECIAvEIhFCRAIBgAAEALyBEQAhACQoIKwAABkBiAVM4QAQlFgBAS4AASAIIkAZEkEgAsBMCABSCAKACAIBMAD4JSLEyAAGQBBqATAUECQUgACQGBGA== |
1221 | \ No newline at end of file |
1222 | |
1223 | === added file 'test/rbkt/Queries/zorba/base64/encoded-text' |
1224 | --- test/rbkt/Queries/zorba/base64/encoded-text 1970-01-01 00:00:00 +0000 |
1225 | +++ test/rbkt/Queries/zorba/base64/encoded-text 2012-02-24 04:03:18 +0000 |
1226 | @@ -0,0 +1,1 @@ |
1227 | +Wm9yYmEgaXMgR3JlYXQhIMOkw7bDvA== |
1228 | \ No newline at end of file |
1229 | |
1230 | === added file 'test/rbkt/Queries/zorba/base64/file_read_1.xq' |
1231 | --- test/rbkt/Queries/zorba/base64/file_read_1.xq 1970-01-01 00:00:00 +0000 |
1232 | +++ test/rbkt/Queries/zorba/base64/file_read_1.xq 2012-02-24 04:03:18 +0000 |
1233 | @@ -0,0 +1,10 @@ |
1234 | +import module namespace f = "http://expath.org/ns/file"; |
1235 | + |
1236 | +variable $enc-file-name := resolve-uri("encoded"); |
1237 | +variable $dec-file-name := resolve-uri("decoded"); |
1238 | +variable $base64-1 := f:read-binary($dec-file-name); |
1239 | +variable $base64-2 := f:read-binary($dec-file-name); |
1240 | +variable $ref-result := f:read-text($enc-file-name); |
1241 | + |
1242 | +xs:string($base64-1) eq $ref-result, $base64-1 eq $base64-2, |
1243 | +fn:serialize($base64-1) |
1244 | |
1245 | === added file 'test/rbkt/Queries/zorba/base64/file_read_2.xq' |
1246 | --- test/rbkt/Queries/zorba/base64/file_read_2.xq 1970-01-01 00:00:00 +0000 |
1247 | +++ test/rbkt/Queries/zorba/base64/file_read_2.xq 2012-02-24 04:03:18 +0000 |
1248 | @@ -0,0 +1,11 @@ |
1249 | +import module namespace f = "http://expath.org/ns/file"; |
1250 | +import module namespace b = "http://www.zorba-xquery.com/modules/converters/base64"; |
1251 | + |
1252 | +variable $enc-file-name := resolve-uri("encoded-text"); |
1253 | +variable $dec-file-name := resolve-uri("decoded-text"); |
1254 | +variable $encoded := f:read-text($enc-file-name); |
1255 | +variable $decoded := f:read-text($dec-file-name); |
1256 | + |
1257 | +$encoded eq xs:string(b:encode($decoded)), |
1258 | +xs:base64Binary($encoded) eq b:encode($decoded), |
1259 | +b:decode(xs:base64Binary($encoded)) eq $decoded |
Validation queue starting for merge proposal. zorbatest. lambda. nu:8080/ remotequeue/ ft-base64Binary -2012-02- 22T01-41- 08.742Z/ log.html
Log at: http://