Merge lp:~zorba-coders/zorba/bug935639 into lp:zorba

Proposed by Rodolfo Ochoa on 2012-02-28
Status: Merged
Approved by: Rodolfo Ochoa on 2012-03-12
Approved revision: 10692
Merged at revision: 10717
Proposed branch: lp:~zorba-coders/zorba/bug935639
Merge into: lp:zorba
Diff against target: 274 lines (+165/-7)
9 files modified
doc/zorba/indexpage.dox.in (+2/-0)
doc/zorba/memory_leaks.dox (+141/-0)
include/zorba/pregenerated/diagnostic_list.h (+2/-0)
modules/com/zorba-xquery/www/modules/pregenerated/errors.xq (+4/-0)
src/diagnostics/diagnostic_en.xml (+4/-0)
src/diagnostics/pregenerated/diagnostic_list.cpp (+3/-0)
src/diagnostics/pregenerated/dict_en.cpp (+1/-0)
src/store/naive/node_items.cpp (+1/-1)
src/store/naive/string_pool.cpp (+7/-6)
To merge this branch: bzr merge lp:~zorba-coders/zorba/bug935639
Reviewer Review Type Date Requested Status
Matthias Brantner 2012-02-28 Approve on 2012-03-07
Markos Zaharioudakis 2012-03-07 Approve on 2012-03-07
Cezar Andrei 2012-02-28 Approve on 2012-02-29
Review via email: mp+94900@code.launchpad.net

Commit message

This fixes the error:
n strings remain in the pool.
and change it to a more comprehensive error with reference to document that helps to prevent the error.

Description of the change

The problem can't be solved by technical means, the developer have to take care of resource usage, as in C++ the user must release all objects before shutdown the store, the same thing applies for XQJ.
So. the best workaround this problem is to show a more friendly message around the Strings Pool and create documentation that explain this issue. At the end, this issue affects every API usage from our clients.
Please check my changes and help me with English usage in the new document I created.

To post a comment you must log in.
review: Approve

The Zorba XQuery Processor manage => The Zorba XQuery Processor manages

automaticall => automatically

review: Needs Fixing
review: Approve
Zorba Build Bot (zorba-buildbot) wrote :

There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.

review: Approve
Zorba Build Bot (zorba-buildbot) wrote :

The attempt to merge lp:~zorba-coders/zorba/bug935639 into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job bug935639-2012-03-07T22-33-06.388Z is finished. The
  final status was:

  No tests were run - build or configure step must have failed.

  Not commiting changes.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

Zorba Build Bot (zorba-buildbot) wrote :

There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.

Zorba Build Bot (zorba-buildbot) wrote :

The attempt to merge lp:~zorba-coders/zorba/bug935639 into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job bug935639-2012-03-11T01-05-09.922Z is finished. The
  final status was:

  3 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

Zorba Build Bot (zorba-buildbot) wrote :

The attempt to merge lp:~zorba-coders/zorba/bug935639 into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job bug935639-2012-03-12T01-12-03.818Z is finished. The
  final status was:

  1 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

lp:~zorba-coders/zorba/bug935639 updated on 2012-03-12
10691. By Rodolfo Ochoa on 2012-03-12

fix for errors.xq in CTESTs

10692. By Rodolfo Ochoa on 2012-03-12

bug from the editor

Zorba Build Bot (zorba-buildbot) wrote :

Validation queue job bug935639-2012-03-12T23-14-08.66Z is finished. The final status was:

All tests succeeded!

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'doc/zorba/indexpage.dox.in'
2--- doc/zorba/indexpage.dox.in 2012-03-07 17:13:34 +0000
3+++ doc/zorba/indexpage.dox.in 2012-03-12 23:11:19 +0000
4@@ -28,6 +28,8 @@
5 - \ref ft_intro
6 - \ref errors_warnings
7 - \ref impl_dep_features
8+ - \ref memory_management
9+ - \ref memory_leaks
10
11 \section xqlib XQuery Modules
12 - <a href="../xqdoc/xhtml/index.html">XQuery Module Library</a>
13
14=== added file 'doc/zorba/memory_leaks.dox'
15--- doc/zorba/memory_leaks.dox 1970-01-01 00:00:00 +0000
16+++ doc/zorba/memory_leaks.dox 2012-03-12 23:11:19 +0000
17@@ -0,0 +1,141 @@
18+/** \page memory_leaks Memory Leaks
19+
20+\section memory_leaks_intro Introduction
21+
22+The Zorba XQuery Processor manages automatically its own memory, but still, there are cases where you may receive the following message:
23+
24+\code
25+Zorba did not close properly, objects may still in memory while shutdown the store.
26+[n] referenced URIs remain in the string pool.
27+For help avoiding this message please refer to http://www.zorba-xquery.com/html/documentation in section General Architecture -> Memory Leaks.
28+\endcode
29+
30+This message means that not all the resources (e.g. XQuery objects)
31+were released before Zorba and its store were shutdown.
32+While this might be a bug/memory leak in Zorba itself, it's also quite likely that the leak is in the application.
33+For the application, the rule is that before shutting down the store
34+all resources must have been released.
35+
36+So, strictly speaking, your program is leaking resources.
37+
38+This problem might occur even though the application's code (C++ or other languages) looks good. Here is an example:
39+
40+\section memory_leaks_example_c Example in C++
41+In the following example, the StaticContext and XQuery objects are not destroyed before Zorba and the store are shutdown.
42+
43+\code
44+int main(int argc, char* argv[])
45+{
46+ void* lStore = zorba::StoreManager::getStore();
47+ Zorba* lZorba = Zorba::getInstance(lStore);
48+
49+ StaticContext_t lContext = lZorba->createStaticContext();
50+ lContext->addNamespace("foo", "http://www.foo.com");
51+ XQuery_t lQuery = lZorba->compileQuery("<foo:e/>", lContext);
52+ lQuery->execute();
53+
54+ lZorba->shutdown();
55+ zorba::StoreManager::shutdownStore(lStore);
56+ return 0;
57+}
58+\endcode
59+
60+So, for this scenario, the ideal way to solve the problem is through scopes,
61+where you implement certain part of code in methods and release them when the objects loose their reference.
62+Alternatively, you can explicitly set the variable to null, in which case the object will automatically be released.
63+
64+Solution example:
65+\code
66+int main(int argc, char* argv[])
67+{
68+ void* lStore = zorba::StoreManager::getStore();
69+ Zorba* lZorba = Zorba::getInstance(lStore);
70+
71+ { // Use a scope
72+
73+ StaticContext_t lContext = lZorba->createStaticContext();
74+ lContext->addNamespace("foo", "http://www.foo.com");
75+ XQuery_t lQuery = lZorba->compileQuery("<foo:e/>", lContext);
76+ lQuery->execute();
77+
78+ lContext = NULL; // or explicitely set free the resources
79+ lQuery = NULL;
80+ }
81+ lZorba->shutdown();
82+ zorba::StoreManager::shutdownStore(lStore);
83+ return 0;
84+}
85+\endcode
86+
87+
88+\section memory_leaks_example_java Example in Java
89+This particular error is specially notorious when a memory managed language
90+shows the error because you expect the language frees all memory, here is an example:
91+
92+\code
93+public static void main ( String argv[] )
94+{
95+ InMemoryStore store = InMemoryStore.getInstance();
96+ Zorba zorba = Zorba.getInstance ( store );
97+
98+ StaticContext context = zorba.createStaticContext();
99+ context.addNamespace("foo", "http://www.foo.com");
100+ XQuery query = zorba.compileQuery("<foo:e/>", context);
101+ String result = query.execute();
102+
103+ zorba.shutdown();
104+ InMemoryStore.shutdown ( store );
105+
106+}
107+\endcode
108+
109+In this example, and for the rest of the languages because Zorba is created in C++,
110+we have created in the Zorba API the method \c destroy() \e that will set free the object
111+that could be pointing to any resource from the store, this method is in every object that need to be released.
112+
113+*Java Note: Because Java is a garbage collected language you cannot predict when (or even if) an object will be destroyed.
114+Hence there is no direct equivalent of a destructor.
115+There is an inherited method called finalize, but this is called entirely at the discretion of the garbage collector.
116+So, destroy() is the best practice for any language including Java.
117+
118+Solution example:
119+\code
120+public static void main ( String argv[] )
121+{
122+ InMemoryStore store = InMemoryStore.getInstance();
123+ Zorba zorba = Zorba.getInstance ( store );
124+
125+ StaticContext context = zorba.createStaticContext();
126+ context.addNamespace("foo", "http://www.foo.com");
127+ XQuery query = zorba.compileQuery("<foo:e/>", context);
128+ String result = query.execute();
129+
130+ query.destroy(); // Release memory for this XQuery
131+ context.destroy(); // Release memory for this StaticContext
132+
133+ zorba.shutdown();
134+ InMemoryStore.shutdown ( store );
135+
136+}
137+\endcode
138+
139+
140+\section memory_leaks_example_xqj Example in XQJ
141+
142+XQJ standard provides specific \c close() \e methods for this specific purpose:
143+\code
144+ XQDataSource xqdatasource = new XQDataSource();
145+ XQConnection xqconnection = xqdatasource.getConnection();
146+ XQStaticContext staticContext = xqconnection.getStaticContext();
147+ staticContext.declareNamespace("foo", "http://www.foo.com");
148+ xqconnection.setStaticContext(staticContext);
149+ XQExpression xqexpression = xqconnection.createExpression();
150+ XQSequence xqsequence = xqexpression.executeQuery("<foo:e/>");
151+ // code to show the output
152+
153+ xqconnection.close(); // Closing connection frees all related resources
154+\endcode
155+
156+
157+*/
158+/* vim:set et sw=2 ts=2: */
159
160=== modified file 'include/zorba/pregenerated/diagnostic_list.h'
161--- include/zorba/pregenerated/diagnostic_list.h 2012-03-08 18:51:25 +0000
162+++ include/zorba/pregenerated/diagnostic_list.h 2012-03-12 23:11:19 +0000
163@@ -734,6 +734,8 @@
164
165 extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZSTR0060_RANGE_EXCEPTION;
166
167+extern ZORBA_DLL_PUBLIC ZorbaErrorCode ZSTR0065_STRINGS_IN_POOL;
168+
169 extern ZORBA_DLL_PUBLIC ZorbaErrorCode XSST0001;
170
171 extern ZORBA_DLL_PUBLIC ZorbaErrorCode XSST0002;
172
173=== modified file 'modules/com/zorba-xquery/www/modules/pregenerated/errors.xq'
174--- modules/com/zorba-xquery/www/modules/pregenerated/errors.xq 2012-03-08 18:51:25 +0000
175+++ modules/com/zorba-xquery/www/modules/pregenerated/errors.xq 2012-03-12 23:11:19 +0000
176@@ -764,6 +764,10 @@
177
178 (:~
179 :)
180+declare variable $zerr:ZSTR0065 as xs:QName := fn:QName($zerr:NS, "zerr:ZSTR0065");
181+
182+(:~
183+:)
184 declare variable $zerr:XSST0001 as xs:QName := fn:QName($zerr:NS, "zerr:XSST0001");
185
186 (:~
187
188=== modified file 'src/diagnostics/diagnostic_en.xml'
189--- src/diagnostics/diagnostic_en.xml 2012-03-08 18:51:25 +0000
190+++ src/diagnostics/diagnostic_en.xml 2012-03-12 23:11:19 +0000
191@@ -2334,6 +2334,10 @@
192 <value>out of range: $1</value>
193 </diagnostic>
194
195+ <diagnostic code="ZSTR0065" name="STRINGS_IN_POOL">
196+ <value>Zorba did not close properly, objects may still in memory.\n$1 referenced URI(s) remain in the string pool.\nFor help avoiding this message please refer to http://www.zorba-xquery.com/html/documentation in section General Architecture -> Memory Leaks.</value>
197+ </diagnostic>
198+
199 <!--////////// XQuery Scripting ////////////////////////////////////////-->
200
201 <diagnostic code="XSST0001">
202
203=== modified file 'src/diagnostics/pregenerated/diagnostic_list.cpp'
204--- src/diagnostics/pregenerated/diagnostic_list.cpp 2012-03-08 18:51:25 +0000
205+++ src/diagnostics/pregenerated/diagnostic_list.cpp 2012-03-12 23:11:19 +0000
206@@ -1079,6 +1079,9 @@
207 ZorbaErrorCode ZSTR0060_RANGE_EXCEPTION( "ZSTR0060" );
208
209
210+ZorbaErrorCode ZSTR0065_STRINGS_IN_POOL( "ZSTR0065" );
211+
212+
213 ZorbaErrorCode XSST0001( "XSST0001" );
214
215
216
217=== modified file 'src/diagnostics/pregenerated/dict_en.cpp'
218--- src/diagnostics/pregenerated/dict_en.cpp 2012-03-08 18:51:25 +0000
219+++ src/diagnostics/pregenerated/dict_en.cpp 2012-03-12 23:11:19 +0000
220@@ -378,6 +378,7 @@
221 { "ZSTR0050", "\"$1\" not implemented for item type \"$2\"" },
222 { "ZSTR0055", "streamable string has already been consumed" },
223 { "ZSTR0060", "out of range: $1" },
224+ { "ZSTR0065", "Zorba did not close properly, objects may still in memory.\n$1 referenced URI(s) remain in the string pool.\nFor help avoiding this message please refer to http://www.zorba-xquery.com/html/documentation in section General Architecture -> Memory Leaks." },
225 { "ZWST0002", "\"$1\": unknown or unsupported annotation" },
226 { "ZWST0003", "\"$1\": function declared sequential, but has non-sequential body" },
227 { "ZWST0004", "Sequential FLWOR expr may not have the semantics you expect" },
228
229=== modified file 'src/store/naive/node_items.cpp'
230--- src/store/naive/node_items.cpp 2012-03-07 14:22:29 +0000
231+++ src/store/naive/node_items.cpp 2012-03-12 23:11:19 +0000
232@@ -722,7 +722,7 @@
233 }
234 catch (...)
235 {
236- ZORBA_FATAL(false, "Unexpectd exception");
237+ ZORBA_FATAL(false, "Unexpected exception");
238 }
239 }
240
241
242=== modified file 'src/store/naive/string_pool.cpp'
243--- src/store/naive/string_pool.cpp 2012-02-15 10:25:02 +0000
244+++ src/store/naive/string_pool.cpp 2012-03-12 23:11:19 +0000
245@@ -15,9 +15,10 @@
246 */
247 #include "stdafx.h"
248
249-#include "zorbautils/fatal.h"
250 #include "util/string_util.h"
251 #include "string_pool.h"
252+#include "zorba/zorba_exception.h"
253+#include "diagnostics/xquery_diagnostics.h"
254
255 namespace zorba { namespace simplestore {
256
257@@ -32,14 +33,14 @@
258 {
259 if (theHashTab[i].theItem.is_shared())
260 {
261- std::cerr << "i = " << i << " String " << theHashTab[i].theItem
262- << " is still in the pool" << std::endl;
263+ //std::cerr << "ID: " << i << " Referenced URI: " << theHashTab[i].theItem << std::endl;
264 //delete theHashTab[i].theString.getp();
265- count++;
266+ count ++;
267 }
268 }
269-
270- ZORBA_FATAL(count == 0, count << " strings remain in the string pool");
271+ if (count>0) {
272+ throw ZORBA_EXCEPTION(zerr::ZSTR0065_STRINGS_IN_POOL, ERROR_PARAMS(count));
273+ }
274 }
275
276

Subscribers

People subscribed via source and target branches