Merge lp:~zorba-coders/zorba/couchbase-module-new-options into lp:zorba/couchbase-module

[Juan Zacarias]

*Added new options to the cb:view Function:
-"limit" option, allows the user to limit the numbers of results shown by the view
-"stale" option, allows the user to specify when to update the view: update after the view is returned(default by couchbase), update before the view is returned, and don't update if the view function is called.

*Added a new option to the cb:put-text and cb:put-binary Functions:
-"wait" option, allows the user to wait for confirmation from couchbase that the key/value pair was stored in disk.

Modified test view.xq using the new options so it doesn't fail when ran for the first time.

[Chris Hillery (ceejatec)]

I was a bit confused by the choices for parameter values for "stale" and "wait", but it looks like those come directly from Couchbase itself, so OK then.

One change: This failed on the RQ the first time the test was run, but not on later RQ runs. That implies to me that later RQ runs are probably seeing leftover data from the earlier runs. The view.xq test case should clean up after itself by deleting the value that it inserts, at least IMHO. Possibly other test cases have a similar issue; please check.

review: Needs Fixing

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/couchbase-module-new-options-2013-01-14T22-34-42.475Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job couchbase-module-new-options-2013-01-14T22-34-42.475Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 1 Needs Fixing, 1 Pending.

[Juan Zacarias (juan457)]

It didn't fail because the created views remain on the database. They are not deleted since they shouldn't be cleaned with the module, but the views are database management work. I will add a delete view just like the create view which are management functions.

[lp:~zorba-coders/zorba/couchbase-module-new-options]

37. By Juan Zacarias on 2013-01-17

Added a delete view function added it to view.xq test

[Juan Zacarias (juan457)]

Fixed

[Chris Hillery (ceejatec)]

review: Approve

[Sorin Marian Nasoi (sorin.marian.nasoi)]

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/couchbase-module-new-options-2013-01-18T06-35-46.187Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job couchbase-module-new-options-2013-01-18T06-35-46.187Z is finished. The final status was:

All tests succeeded!

=== modified file 'examples/example.xq'
--- examples/example.xq	2013-01-10 07:59:35 +0000
+++ examples/example.xq	2013-01-17 22:52:18 +0000
@@ -7,7 +7,7 @@
   "password" : null,
   "bucket" : "default"});
 variable $view-name := cb:create-view($instance, "zip", "zip", {"key" : "doc.state", "values" : "doc.pop"});
-variable $data := cb:view($instance, $view-name);
+variable $data := cb:view($instance, $view-name, {"stale" : "false"});
 for $d in jn:members($data("rows"))
 let $state := $d("key")
 group by $state

=== modified file 'src/couchbase.xq'
--- src/couchbase.xq	2013-01-10 08:07:25 +0000
+++ src/couchbase.xq	2013-01-17 22:52:18 +0000
@@ -232,7 +232,10 @@
  : @option "operation" type of operation, possible values are 
  :         "add", "replace", "set", "append" and "prepend".
  : @option "encoding" the encoding that should be used for the
- :   value (default is UTF-8).
+ :         value (default is UTF-8).
+ : @option "wait" variable for setting if a wait for persistancy in 
+ :         the storing key is needed, possible values are "persist" 
+ :         and "false".
  : 
  : @error cb:LCB0002 if any error occurs in the communication with
  :   the server.
@@ -241,6 +244,7 @@
  : @error cb:CB0006 if the given encoding is not supported.
  : @error cb:CB0007 if any of the options is not supported.
  : @error cb:CB0009 if the given expiration time is not an xs:integer. 
+ : @error cb:CB0011 if the stored Variable was not stored
  :
  : @return a empty sequence.
  :)  
@@ -286,6 +290,9 @@
  :         expiration time in seconds.
  : @option "operation" type of operation, possible values are 
  :         "add", "replace", "set", "append" and "prepend".
+ : @option "wait" variable for setting if a wait for persistancy in 
+ :         the storing key is needed, possible values are "persist" 
+ :         and "false".
  :
  : @error cb:LCB0002 if any error occurs in the communication with
  :   the server.
@@ -293,6 +300,7 @@
  :   of values.
  : @error cb:CB0007 if any of the options is not supported.
  : @error cb:CB0009 if the given expiration time is not an xs:integer.
+ : @error cb:CB0011 if the stored Variable was not stored
  :
  : @return a empty sequence.
  :)  
@@ -365,8 +373,18 @@
  :        (e.g. "_design/test/_view/vies").
  : @param $options JSONiq object with additional options
  :
- : @option "encoding" string with the name of the encoding of the returned
- :   strings (if not UTF-8).
+ : @option Json object whith options for the querying the view. available options:
+ :         "encoding" string with the name of the encoding of the returned
+ :         strings (if not UTF-8).
+ :         "stale" option's value is a string with the type of stale to be used. 
+ :         Valid values:
+ :           "ok" : the view is not updated
+ :           "false" : the view is updated before the function view is executed, 
+ :            this options needs the key to be on disk before the call of the 
+ :            view function.
+ :           "update_after" : the view is updated after the call of view
+ :         "limit" option's value is an integer which sets a number of how many 
+ :         rows the view will show.  
  :
  : @error cb:LCB0002 if any error occurs in the communication with
  :   the server.
@@ -451,3 +469,23 @@
   $view-names as xs:string*,
   $options as object()*)
 as xs:string* external;
+
+
+(:~
+ : Delete a document/view.
+ 
+ : If the document doesn't exists, function does nothing. All the views hold in the
+ : Document are deleted, this function can't delete single views. 
+ :
+ : @param $db connection reference
+ : @param $doc-name name of the document to create.
+ :
+ : @error cb:LCB0002 if any error occurs in the communication with
+ :   the server.
+ :
+ : @return empty sequence.
+ :)
+declare %an:sequential function cb:delete-view(
+  $db as xs:anyURI,
+  $doc as xs:string*)
+as xs:string* external;

=== modified file 'src/couchbase.xq.src/couchbase.cpp'
--- src/couchbase.xq.src/couchbase.cpp	2013-01-10 07:59:35 +0000
+++ src/couchbase.xq.src/couchbase.cpp	2013-01-17 22:52:18 +0000
@@ -89,6 +89,10 @@
     {
       lFunc = new CreateViewFunction(this);
     }
+    else if (localname == "delete-view")
+    {
+      lFunc = new DeleteViewFunction(this);
+    }
   }
 
   return lFunc;
@@ -190,6 +194,30 @@
   return NULL;
 }
 
+String
+  CouchbaseFunction::ViewOptions::getPathOptions()
+{
+  String lPathOptions("?");
+  bool lAmp = false;
+  if (theStaleOption != "")
+  {
+    lPathOptions.append(theStaleOption);
+    lAmp = true;
+  }
+  if (theLimitOption != "")
+  {
+    if(lAmp)
+      lPathOptions.append("&");
+    lPathOptions.append(theLimitOption);
+    lAmp = true;
+  }
+
+  if (lPathOptions == "?")
+    lPathOptions = "";
+
+  return lPathOptions;
+}
+
 void
   CouchbaseFunction::ViewOptions::setOptions(Item& aOptions)
 {
@@ -216,6 +244,42 @@
         throwError("CB0006", lMsg.str().c_str());
       }
     }
+    else if (lStrKey == "stale")
+    {
+      Item lValue = aOptions.getObjectValue(lStrKey);
+      String lString = lValue.getStringValue();
+      if (lString == "false")
+      {
+        theStaleOption ="stale=false";
+      }
+      else if (lString == "ok")
+      {
+        theStaleOption ="stale=ok";
+      }
+      else if (lString == "update_after")
+      {
+        theStaleOption ="stale=update_after";
+      }
+      else
+      {
+        std::ostringstream lMsg;
+        lMsg << lStrKey << "=" << lString << ": option not supported";
+        throwError("CB0007", lMsg.str().c_str());
+      }
+    }
+    else if (lStrKey == "limit")
+    {
+      Item lValue = aOptions.getObjectValue(lStrKey);
+      try
+      {
+        int lLimit = lValue.getIntValue();
+        theLimitOption = "limit=" + lLimit;  
+      }
+      catch (ZorbaException& e)
+      {
+        throwError("CB0009", " limit option must be an integer value");
+      } 
+    }
     else
     {
       std::ostringstream lMsg;
@@ -371,6 +435,28 @@
             throwError("CB0006", lMsg.str().c_str());
       }
     }
+    else if (lStrKey == "wait")
+    {
+      Item lValue = aOptions.getObjectValue(lStrKey);
+      String lStrValue = lValue.getStringValue();
+      std::transform(
+        lStrValue.begin(), lStrValue.end(),
+        lStrValue.begin(), tolower);
+      if (lStrValue == "persist")
+      {
+        theWaitType = CB_WAIT_PERSIST;
+      }
+      else if (lStrValue == "false")
+      {
+        theWaitType = CB_WAIT_FALSE;
+      }
+      else
+      {
+        std::ostringstream lMsg;
+        lMsg << lStrKey << "=" << lStrValue << " : option not supported";
+        throwError("CB0007", lMsg.str().c_str());
+      }
+    }
     else
     {
       std::ostringstream lMsg;
@@ -731,6 +817,26 @@
 /*******************************************************************************
  ******************************************************************************/
 
+void CouchbaseFunction::observe_callback(lcb_t instance, const void *cookie, lcb_error_t error, const lcb_observe_resp_t *resp)
+{
+  if (error != LCB_SUCCESS)
+  {
+    libCouchbaseError (instance, error);
+  }
+  
+  PutOptions* lWait = (PutOptions*) cookie;
+  //verify is coming from the master
+  if (resp->v.v0.from_master > 0)
+  {
+    lcb_observe_t lStatus = resp->v.v0.status;if (lStatus == LCB_OBSERVE_NOT_FOUND)
+    {
+      throwError("CB0011", "Variable stored not found.");
+    }
+    //check for flag of persisntancy
+    lWait->setWaiting(lStatus & LCB_OBSERVE_PERSISTED?false:true);
+  }
+}
+
 void CouchbaseFunction::put (lcb_t aInstance, Iterator_t aKeys, Iterator_t aValues, PutOptions aOptions)
 {
   lcb_error_t lError;
@@ -800,9 +906,23 @@
     {
       libCouchbaseError (aInstance, lError);
     } 
-
-
+    //Wait for store
     lcb_wait(aInstance);
+    //Check if wait for disk
+    if (aOptions.getWaitType() != CB_WAIT_FALSE)
+    {     
+      lcb_set_observe_callback(aInstance, observe_callback);
+      PutOptions* lOptions = &aOptions;
+      do {
+        lcb_observe_cmd_t lObserve;
+        lObserve.version = 0;
+        lObserve.v.v0.key = lStrKey.c_str();
+        lObserve.v.v0.nkey = lStrKey.size();
+        lcb_observe_cmd_t* lCommands[1] = { &lObserve };
+        lcb_observe(aInstance, lOptions, 1, lCommands);
+        lcb_wait(aInstance);
+      }while(lOptions->isWaiting());
+    }
   }
 
   if (aValues->next(lValue))
@@ -1005,19 +1125,23 @@
   if (thePaths->next(lPath))
   {
     ViewOptions* lOptions = &theOptions;
-    
+    String lPathOptions = lOptions->getPathOptions();
+    String lPathString = lPath.getStringValue();    
+    if (lPathOptions != "")
+    {
+      lPathString.append(lPathOptions);
+    }    
     lcb_http_request_t lReq;
     lcb_http_cmd_t lCmd;
     lCmd.version = 0;
-    lCmd.v.v0.path = lPath.getStringValue().c_str();
-    lCmd.v.v0.npath = lPath.getStringValue().size();
+    lCmd.v.v0.path = lPathString.c_str();
+    lCmd.v.v0.npath = lPathString.size();
     lCmd.v.v0.body = NULL;
     lCmd.v.v0.nbody = 0;
     lCmd.v.v0.method = LCB_HTTP_METHOD_GET;
     lCmd.v.v0.chunked = 1;
     lCmd.v.v0.content_type = "application/json";
     lcb_error_t err = lcb_make_http_request(theInstance, lOptions, LCB_HTTP_TYPE_VIEW, &lCmd, &lReq);
-  
     if (err != LCB_SUCCESS)
     {
       libCouchbaseError (theInstance, err);
@@ -1065,8 +1189,46 @@
 
 /*******************************************************************************
  ******************************************************************************/
-
-
+zorba::ItemSequence_t
+DeleteViewFunction::evaluate(
+  const Arguments_t& aArgs,
+  const zorba::StaticContext* aSctx,
+  const zorba::DynamicContext* aDctx) const
+{
+  String lInstanceID = getOneStringArgument(aArgs, 0);
+  lcb_t lInstance = getInstance(aDctx, lInstanceID);
+
+  Iterator_t lDocNames = getIterArgument(aArgs, 1);
+  Item lDoc;  
+  lDocNames->open();
+  while(lDocNames->next(lDoc))
+  {
+    String lPath = "_design/" + lDoc.getStringValue();
+    lcb_http_request_t req;
+    lcb_http_cmd_t cmd;
+    cmd.version = 0;
+    cmd.v.v0.path = lPath.c_str();
+    cmd.v.v0.npath = lPath.size();
+    cmd.v.v0.body = NULL;
+    cmd.v.v0.nbody = 0;
+    cmd.v.v0.method = LCB_HTTP_METHOD_DELETE;
+    cmd.v.v0.chunked = false;
+    cmd.v.v0.content_type = "application/json";
+    lcb_error_t err = lcb_make_http_request(lInstance, NULL,
+                           LCB_HTTP_TYPE_VIEW, &cmd, &req);
+
+    if (err != LCB_SUCCESS)
+    libCouchbaseError (lInstance, err);
+
+    lcb_wait(lInstance);
+  }
+  lDocNames->close();
+
+  return ItemSequence_t(new EmptySequence());  
+}
+
+/*******************************************************************************
+ ******************************************************************************/
 zorba::ItemSequence_t
 CreateViewFunction::evaluate(
   const Arguments_t& aArgs,

=== modified file 'src/couchbase.xq.src/couchbase.h'
--- src/couchbase.xq.src/couchbase.h	2013-01-10 06:51:21 +0000
+++ src/couchbase.xq.src/couchbase.h	2013-01-17 22:52:18 +0000
@@ -86,16 +86,25 @@
 
     } lcb_storage_type_t;
 
+    typedef enum
+    {
+      CB_WAIT_FALSE = 0x00,
+      CB_WAIT_PERSIST = 0x01,
+      CB_WAIT_REPLICATE = 0x02
+    } cb_wait_type_t;
+
     class ViewOptions
     {
       protected:
         String theEncoding;
         String thePath;
+        String theStaleOption;
+        String theLimitOption;
 
       public:
         std::unique_ptr<std::stringstream>* theStream;
 
-        ViewOptions() : theEncoding("UTF-8"), thePath("") { theStream = NULL; }
+        ViewOptions() : theEncoding("UTF-8"), thePath(""), theStaleOption(""), theLimitOption("") { theStream = NULL; }
 
         ViewOptions(String& aPath) : theEncoding("UTF-8"), thePath(aPath) {}
 
@@ -106,6 +115,8 @@
         String getEncoding() { return theEncoding; }
 
         String getPath() { return thePath; }
+
+        String getPathOptions();
     };
 
     class GetOptions
@@ -137,17 +148,18 @@
     class PutOptions
     {
       protected:
-        
         lcb_storage_t theOperation;
         lcb_storage_type_t theType;
         unsigned int theExpTime;
         String theEncoding;
+        cb_wait_type_t theWaitType;
+        bool theIsWaiting;
 
       public:
 
-        PutOptions() : theOperation(LCB_ADD), theType(LCB_JSON), theExpTime(0), theEncoding("") { }
+        PutOptions() : theOperation(LCB_ADD), theType(LCB_JSON), theExpTime(0), theEncoding(""), theWaitType(CB_WAIT_FALSE), theIsWaiting(false){ }
 
-        PutOptions(lcb_storage_type_t aType) : theOperation(LCB_SET), theType(aType), theExpTime(0) { }
+        PutOptions(lcb_storage_type_t aType) : theOperation(LCB_SET), theType(aType), theExpTime(0), theWaitType(CB_WAIT_FALSE), theIsWaiting(false) { }
 
         void setOptions(Item& aOptions);
 
@@ -160,6 +172,12 @@
         unsigned int getExmpTime() { return theExpTime; }
 
         String getEncoding() { return theEncoding; }
+
+        cb_wait_type_t getWaitType() { return theWaitType; }
+
+        bool isWaiting() { return theIsWaiting; }
+
+        void setWaiting(bool isWaiting) { theIsWaiting = isWaiting; }
     };
 
     class ViewItemSequence : public ItemSequence
@@ -299,6 +317,14 @@
     static void
       put (lcb_t aInstance, Iterator_t aKeys, Iterator_t aValues, PutOptions aOptions);
 
+    
+    static void
+      observe_callback(
+        lcb_t instance, 
+        const void *cookie,
+        lcb_error_t error,
+        const lcb_observe_resp_t *resp);
+
   public:
     
     CouchbaseFunction(const CouchbaseModule* module);
@@ -560,6 +586,25 @@
                 const zorba::DynamicContext*) const;
 };
 
+/*******************************************************************************
+ ******************************************************************************/
+
+class DeleteViewFunction : public CouchbaseFunction
+{
+  public:
+    DeleteViewFunction(const CouchbaseModule* aModule)
+      : CouchbaseFunction(aModule) {}
+
+    virtual ~DeleteViewFunction(){}
+
+    virtual zorba::String
+      getLocalName() const { return "delete-view"; }
+
+    virtual zorba::ItemSequence_t
+      evaluate( const Arguments_t&,
+                const zorba::StaticContext*,
+                const zorba::DynamicContext*) const;
+};
 } /*namespace couchbase*/ } /*namespace zorba*/
 
 

=== modified file 'test/Queries/couchbase_module/view.xq'
--- test/Queries/couchbase_module/view.xq	2013-01-10 07:59:35 +0000
+++ test/Queries/couchbase_module/view.xq	2013-01-17 22:52:18 +0000
@@ -6,12 +6,16 @@
   "password" : null,
   "bucket" : "default"});
 
-cb:put-text($instance, "view", '{ "view" : 1 }');
-
-variable $view-name := cb:create-view($instance, "test-view", "test", {"key":"doc.view"});
-
-variable $data := cb:view($instance, $view-name);
+cb:remove($instance, "view");
+cb:put-text($instance, "view", '{ "view" : 1 }', { "wait" : "persist" });
+
+variable $cb-document := "test-view";
+variable $cb-view := "view";
+variable $view-name := cb:create-view($instance, $cb-document, $cb-view, {"key":"doc.view"});
+
+variable $data := cb:view($instance, $view-name, {"stale" : "false"});
+cb:delete-view($instance, $cb-document);
 for $d in jn:members($data("rows"))
-where $d("key") >0
+where $d("key") > 0
 return $d