Merge lp:~davidagraf/zorba/trace_without_debug_info into lp:zorba

[David Graf]

Removed internal debug info from fn:trace out by using zorba serializer instead of the internal show function.

[David Graf (davidagraf)]

Hello Chris
Here is the current work I've done to remove the debug output in fn:trace (as discussed in the email with markos). It's working fine. But I need some advice to make it better. I am doing this via a merge request. I think that's easier for you because I can see the diff.
In the current implementation, I am doing two nasty things:
1) The code to print an attribute is redundant. That's bad. Would it make sense to add an option to the serializer to enable printing of all nodes? I am not sure. Especially because it is not so easy to do. All the emmitter need to be changed. Do you see a nicer solution?
2) If the node is not an attribute, I create a serializer in each iteration! Although I always need the same. But I didn't find a nice solution to use just one serializer. If I do a singleton, I need to create the serializer on the heap. If I put the seralizer as a member in the state or the iterator, I need to import the serializer in the header file.
Best
David

[Matthias Brantner (matthias-brantner)]

It would be great if the usage of API classes such as SingletonItemSequence, zorba::Item, or zorba::Iterator_t could be avoided in the runtime.

[Chris Hillery (ceejatec)]

Matthias: why is that desirable? Especially for SingletonItemSequence, is there any internal-runtime class with equivalent functionality?

[Matthias Brantner (matthias-brantner)]

> Matthias: why is that desirable? Especially for SingletonItemSequence, is
> there any internal-runtime class with equivalent functionality?
It's one level of indirection which is not necessary and it bloats includes and dependencies.

[Chris Hillery (ceejatec)]

David:

1. I agree, the duplication of code to output attributes is not pretty. I don't think it would be too difficult to add support for directly serializing attributes to the serializer itself. And it wouldn't have to be changed in every emitter, only whichever emitter is used for basic XML serialization.

2. I think adding the serializer to the either the state or the iterator class is the right answer. I'm not quite clear enough on how the whole state/STACK_PUSH stuff works in the runtime to say whether the state or the iterator is the better place for it.

I don't immediately see any problem with adding the #include of serializer to the corresponding header file. If you're desperate to avoid this, however, you could have a forward reference to Serializer and put a Serializer* on the appropriate class. Yes, that means the Serializer would need to be heap-allocated and you'd have to worry about de-allocating it in the destructor.

I wonder if you could have a forward reference to Serializer and then have a data member that was an auto_ptr<Serializer> or unique_ptr<Serializer>?

[Chris Hillery (ceejatec)]

> > Matthias: why is that desirable? Especially for SingletonItemSequence, is
> > there any internal-runtime class with equivalent functionality?
> It's one level of indirection which is not necessary and it bloats includes
> and dependencies.

In that case, can you answer my other question: Is there equivalent functionality he could use in the runtime?

[Matthias Brantner (matthias-brantner)]

> > > Matthias: why is that desirable? Especially for SingletonItemSequence, is
> > > there any internal-runtime class with equivalent functionality?
> > It's one level of indirection which is not necessary and it bloats includes
> > and dependencies.
>
> In that case, can you answer my other question: Is there equivalent
> functionality he could use in the runtime?
store::TempSeq_t Store::createTempSeq(store::Item_t&) could be used.

[David Graf (davidagraf)]

> David:
>
> 1. I agree, the duplication of code to output attributes is not pretty. I
> don't think it would be too difficult to add support for directly serializing
> attributes to the serializer itself. And it wouldn't have to be changed in
> every emitter, only whichever emitter is used for basic XML serialization.
>
Hello Chris
The only solution I see is to pass an additional parameter to emitter::emit_item to tell the function if it should throw an error or not if an attribute is passed. But this implies changing the emit_item function of all emitters. Do you see a nicer solution?

[Chris Hillery (ceejatec)]

Ah, I see what you're saying. Another option, though, would be to have the flag on the xml_emitter constructor; then you just need to pass it from setup(), and wouldn't need to change the emit_item() signature. I haven't looked deeply to see whether it would be possible to get the option value easily at setup() time or not, though.

[David Graf (davidagraf)]

- Extended serializer to print attributes
- storing serializer in the trace state object

[David Graf (davidagraf)]

Chris, can you review the change again and give some feedback?
Thanks

[Chris Hillery (ceejatec)]

The changes all look good.

Is there any way you can add any test cases for this? I don't know if testdriver captures fn:trace() output, but if it does (or can do so easily) I'd feel better if we had at least one or two tests showing the functionality.

[David Graf (davidagraf)]

Hey Chris

There is a quick solution: I could just pipe the trace stuff into the result file of a test. But I think this is not clean. The query result file should not contain trace stuff.

But implementing a clean solution would take quite some time. I don't know if it is worth it. I think I would need to introduce fn:trace output files in the ExpQueryResults directory. Something like test/rbkt/ExpQueryResults/zorba/misc/baseuri.fntrace.res for example. And compare the resulting fn:trace output with those files.

Do you think this is necessary?

[David Graf (davidagraf)]

review: Approve

[Chris Hillery (ceejatec)]

Ok, yeah, I see your point. I'd prefer it if we could test this SOMEhow, but it's not too critical. For now, I've tested your changes locally and the work fine, so let's run with it.

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/trace_without_debug_info-2012-07-13T05-56-14.18Z/log.html

[Zorba Build Bot (zorba-buildbot)]

The attempt to merge lp:~davidagraf/zorba/trace_without_debug_info into lp:zorba failed. Below is the output from the failed tests.

CMake Error at /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake:274 (message):
  Validation queue job trace_without_debug_info-2012-07-13T05-56-14.18Z is
  finished. The final status was:

  1 tests did not succeed - changes not commited.

Error in read script: /home/ceej/zo/testing/zorbatest/tester/TarmacLander.cmake

[lp:~davidagraf/zorba/trace_without_debug_info]

10882. By David Graf on 2012-07-13

merged trunk

10883. By David Graf on 2012-07-13

changed trace comparison in a test

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/trace_without_debug_info-2012-07-13T08-33-57.278Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job trace_without_debug_info-2012-07-13T08-33-57.278Z is finished. The final status was:

All tests succeeded!

=== modified file 'doc/cxx/examples/context.cpp'
--- doc/cxx/examples/context.cpp	2012-07-12 17:29:55 +0000
+++ doc/cxx/examples/context.cpp	2012-07-13 08:35:21 +0000
@@ -462,9 +462,10 @@
 
     // check if the trace was successful
     std::string lTraceString = lTraceStream.str();
-    if (lTraceString.compare("foo [0]: xs:integer(1)\n"
-          "foo [1]: xs:integer(2)\n"
-          "foo [2]: xs:integer(3)\n") != 0) {
+    std::cout << lTraceString << std::endl;
+    if (lTraceString.compare("foo [0]: 1\n"
+          "foo [1]: 2\n"
+          "foo [2]: 3\n") != 0) {
       return false;
     }
     std::cout << lTraceString << std::endl;

=== modified file 'src/api/serialization/serializer.cpp'
--- src/api/serialization/serializer.cpp	2012-07-12 17:29:55 +0000
+++ src/api/serialization/serializer.cpp	2012-07-13 08:35:21 +0000
@@ -112,14 +112,18 @@
 /*******************************************************************************
 
 ********************************************************************************/
-serializer::emitter::emitter(serializer* the_serializer, transcoder& the_transcoder)
+serializer::emitter::emitter(
+    serializer* the_serializer, 
+    transcoder& the_transcoder,
+    bool aEmitAttributes)
   :
   ser(the_serializer),
   tr(the_transcoder),
   previous_item(INVALID_ITEM),
   theChildIters(8),
   theFirstFreeChildIter(0),
-  isFirstElementNode(true)
+  isFirstElementNode(true),
+  theEmitAttributes(aEmitAttributes)
 {
   for (ulong i = 0; i < 8; i++)
     theChildIters[i] = GENV_ITERATOR_FACTORY->createChildrenIterator();
@@ -442,7 +446,8 @@
 
     previous_item = PREVIOUS_ITEM_WAS_TEXT;
   }
-  else if (item->getNodeKind() == store::StoreConsts::attributeNode)
+  else if (!theEmitAttributes 
+        && item->getNodeKind() == store::StoreConsts::attributeNode)
   {
     throw XQUERY_EXCEPTION(err::SENR0001,
     ERROR_PARAMS(item->getStringValue(), ZED(AttributeNode)));
@@ -855,9 +860,10 @@
 ********************************************************************************/
 serializer::xml_emitter::xml_emitter(
   serializer* the_serializer,
-  transcoder& the_transcoder)
+  transcoder& the_transcoder,
+  bool aEmitAttributes)
   :
-  emitter(the_serializer, the_transcoder)
+  emitter(the_serializer, the_transcoder, aEmitAttributes)
 {
 }
 
@@ -2664,14 +2670,14 @@
 /*******************************************************************************
 
 ********************************************************************************/
-bool serializer::setup(std::ostream& os)
+bool serializer::setup(std::ostream& os, bool aEmitAttributes)
 {
   tr = create_transcoder(os);
   if (!tr) {
     return false;
   }
   if (method == PARAMETER_VALUE_XML)
-    e = new xml_emitter(this, *tr);
+    e = new xml_emitter(this, *tr, aEmitAttributes);
   else if (method == PARAMETER_VALUE_HTML)
     e = new html_emitter(this, *tr);
   else if (method == PARAMETER_VALUE_XHTML)
@@ -2728,9 +2734,10 @@
 ********************************************************************************/
 void serializer::serialize(
     store::Iterator_t    aObject,
-    std::ostream& aOStream)
+    std::ostream& aOStream,
+    bool aEmitAttributes)
 {
-  serialize(aObject, aOStream, 0);
+  serialize(aObject, aOStream, 0, aEmitAttributes);
 }
 
 
@@ -2741,13 +2748,14 @@
 serializer::serialize(
   store::Iterator_t     aObject,
   std::ostream&         aOStream,
-  SAX2_ContentHandler*  aHandler)
+  SAX2_ContentHandler*  aHandler,
+  bool                  aEmitAttributes)
 {
   std::stringstream temp_sstream; // used to temporarily hold expanded strings for the SAX serializer
 
   validate_parameters();
 
-  if (!setup(aOStream))
+  if (!setup(aOStream, aEmitAttributes))
   {
     return;
   }

=== modified file 'src/api/serialization/serializer.h'
--- src/api/serialization/serializer.h	2012-07-12 17:29:55 +0000
+++ src/api/serialization/serializer.h	2012-07-13 08:35:21 +0000
@@ -141,8 +141,12 @@
    * @param object The serializable object that provides a sequence
    *        to be serialized.
    * @param stream The stream to serialize to.
+   * @param aEmitAttributeValue If true, attributes are emitted.
    */
-  void serialize(store::Iterator_t object, std::ostream& stream);
+  void serialize(
+      store::Iterator_t object, 
+      std::ostream& stream, 
+      bool aEmitAttributes = false);
 
   void serialize(
         store::Iterator_t object,
@@ -158,11 +162,13 @@
    *        to be serialized.
    * @param stream The stream to serialize to.
    * @param handler The SAX handler.
+   * @param aEmitAttributes If true, attributes are emitted.
    */
   void serialize(
         store::Iterator_t     object,
         std::ostream&         stream,
-        SAX2_ContentHandler*  handler);
+        SAX2_ContentHandler*  handler,
+        bool aEmitAttributes = false);
 
   /**
    * Set the serializer's parameters. The list of handled parameters
@@ -185,7 +191,7 @@
 
   void validate_parameters();
 
-  bool setup(std::ostream& os);
+  bool setup(std::ostream& os, bool aEmitAttributes = false);
 
   transcoder* create_transcoder(std::ostream& os);
 
@@ -205,8 +211,12 @@
      *
      * @param the_serializer The parent serializer object.
      * @param output_stream Target output stream.
+     * @param aEmitAttributes If true, attributes are emitted.
      */
-    emitter(serializer* the_serializer, transcoder& the_transcoder);
+    emitter(
+        serializer* the_serializer, 
+        transcoder& the_transcoder,
+        bool aEmitAttributes = false);
 
     /**
      * Outputs the start of the serialized document, which, depending on
@@ -324,6 +334,7 @@
     store::AttributesIterator           * theAttrIter;
 
     bool                                  isFirstElementNode;
+    bool                                  theEmitAttributes;
   };
 
 
@@ -336,12 +347,19 @@
   class xml_emitter : public emitter
   {
   public:
-    xml_emitter(serializer* the_serializer, transcoder& the_transcoder);
+    xml_emitter(
+        serializer* the_serializer, 
+        transcoder& the_transcoder, 
+        bool aEmitAttributes = false
+    );
 
     virtual void emit_declaration();
 
   protected:
     virtual void emit_doctype(const zstring& elementName);
+
+  protected:
+    bool theEmitAttributes;
   };
 
   ///////////////////////////////////////////////////////////

=== modified file 'src/runtime/errors_and_diagnostics/errors_and_diagnostics_impl.cpp'
--- src/runtime/errors_and_diagnostics/errors_and_diagnostics_impl.cpp	2012-07-12 17:29:55 +0000
+++ src/runtime/errors_and_diagnostics/errors_and_diagnostics_impl.cpp	2012-07-13 08:35:21 +0000
@@ -27,10 +27,15 @@
 
 #include "store/api/item.h"
 #include "store/api/item_factory.h"
+#include "store/api/store.h"
+#include "store/api/temp_seq.h"
 
 #include "system/globalenv.h"
 #include "zorbatypes/zstring.h"
 
+#include "api/serialization/serializer.h"
+#include "api/serializerimpl.h"
+
 namespace zorba 
 {
 
@@ -97,14 +102,34 @@
     );
   }
 
+  if (state->theSerializer.get() == 0) {
+    state->theSerializer.reset(new serializer(0));
+    Zorba_SerializerOptions options;
+    options.omit_xml_declaration = ZORBA_OMIT_XML_DECLARATION_YES;
+    SerializerImpl::setSerializationParameters(
+                        *(state->theSerializer), 
+                        options);
+  }
   state->theOS = theSctx->get_trace_stream();
 
   while (consumeNext(result, theChildren[0], planState)) 
   {
     (*state->theOS) << state->theTagItem->getStringValue() 
-      << " [" << state->theIndex << "]: "
-      << result->show()
-      << std::endl;
+      << " [" << state->theIndex << "]: ";
+
+    {
+      store::Item_t lTmp = result;
+      store::TempSeq_t lSequence = GENV_STORE.createTempSeq(lTmp);
+      store::Iterator_t seq_iter = lSequence->getIterator();
+      seq_iter->open();
+      state->theSerializer->serialize(
+          seq_iter,
+          (*state->theOS),
+          true);
+      seq_iter->close(); 
+    }
+
+    (*state->theOS) << std::endl;
     ++state->theIndex;
 
     STACK_PUSH(true, state);

=== modified file 'src/runtime/errors_and_diagnostics/pregenerated/errors_and_diagnostics.cpp'
--- src/runtime/errors_and_diagnostics/pregenerated/errors_and_diagnostics.cpp	2012-07-12 17:29:55 +0000
+++ src/runtime/errors_and_diagnostics/pregenerated/errors_and_diagnostics.cpp	2012-07-13 08:35:21 +0000
@@ -95,6 +95,7 @@
   theTagItem = NULL;
   theIndex = 0;
   theOS = 0;
+  theSerializer = std::auto_ptr<serializer>(0);
 }
 
 void TraceIteratorState::reset(PlanState& planState) {
@@ -102,6 +103,7 @@
   theTagItem = NULL;
   theIndex = 0;
   theOS = 0;
+  theSerializer = std::auto_ptr<serializer>(0);
 }
 // </TraceIterator>
 

=== modified file 'src/runtime/errors_and_diagnostics/pregenerated/errors_and_diagnostics.h'
--- src/runtime/errors_and_diagnostics/pregenerated/errors_and_diagnostics.h	2012-07-12 17:29:55 +0000
+++ src/runtime/errors_and_diagnostics/pregenerated/errors_and_diagnostics.h	2012-07-13 08:35:21 +0000
@@ -73,6 +73,7 @@
   store::Item_t theTagItem; //
   uint32_t theIndex; //
   std::ostream* theOS; //
+  std::auto_ptr<serializer> theSerializer; //
 
   TraceIteratorState();
 

=== modified file 'src/runtime/spec/errors_and_diagnostics/errors_and_diagnostics.xml'
--- src/runtime/spec/errors_and_diagnostics/errors_and_diagnostics.xml	2012-07-12 17:29:55 +0000
+++ src/runtime/spec/errors_and_diagnostics/errors_and_diagnostics.xml	2012-07-13 08:35:21 +0000
@@ -88,6 +88,10 @@
       <zorba:member type="store::Item_t" name="theTagItem" defaultValue="NULL" brief=""/>
       <zorba:member type="uint32_t" name="theIndex" defaultValue="0" brief=""/>
       <zorba:member type="std::ostream*" name="theOS" defaultValue="0" brief=""/>
+      <zorba:member type="std::auto_ptr&lt;serializer&gt;"
+                    name="theSerializer"
+                    defaultValue="std::auto_ptr&lt;serializer&gt;(0)"
+                    brief=""/>
     </zorba:state>
     
   </zorba:iterator>