Merge lp:~thisfred/u1db/fix-local-gen-before-sync into lp:u1db

[Eric Casteleijn]

Fixed local_gen to be an OUT parameter only, and return it from sync()

Fixed local_gen to be an OUT parameter only, and return it from sync()

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 8/15/2012 2:42 PM, Eric Casteleijn wrote:
> Eric Casteleijn has proposed merging
> lp:~thisfred/u1db/fix-local-gen-before-sync into lp:u1db.
>
> Requested reviews: Ubuntu One hackers (ubuntuone-hackers)
>
> For more details, see:
> https://code.launchpad.net/~thisfred/u1db/fix-local-gen-before-sync/+merge/119705
>
> Fixed local_gen to be an OUT parameter only, and return it from
> sync()
>

     // fprintf(stderr, "Starting\n");
- - if (db == NULL || target == NULL || local_gen_before_sync == NULL) {
+ if (db == NULL || target == NULL) {
         // fprintf(stderr, "DB, target, or local are NULL\n");

You should probably still check that "local_gen == NULL" here, so that
we generate an INVALID_PARAMETER error if it is not supplied. Or are
you intentionally making it optional? If so, you should update the
docstring to indicate that.

The rest looks good.

 merge: approve

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlArrJsACgkQJdeBCYSNAAPoygCgnAVKFl2EoKukAwlohz/DRhsd
5bsAn3vlT/AqJR4S5yfIIlgC9tQDedtt
=dSwQ
-----END PGP SIGNATURE-----

review: Approve

[Ubuntu One Auto Pilot (otto-pilot)]

The attempt to merge lp:~thisfred/u1db/fix-local-gen-before-sync into lp:u1db failed. Below is the output from the failed tests.

-- The C compiler identification is GNU 4.7.1
-- The CXX compiler identification is GNU 4.7.1
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Found Cython: /usr/bin/cython
-- Found CURL: /usr/lib/x86_64-linux-gnu/libcurl.so (found version "7.26.0")
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.26")
-- checking for module 'oauth'
-- found oauth, version 0.9.7
-- Found OAuth: /usr/lib/x86_64-linux-gnu/liboauth.so
-- checking for module 'json'
-- found json, version 0.9
-- Found JSON: /usr/lib/x86_64-linux-gnu/libjson.so
-- checking for module 'sqlite3'
-- found sqlite3, version 3.7.13
-- Found Sqlite3: /usr/lib/x86_64-linux-gnu/libsqlite3.so
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/tarmac/cache/u1db/trunk
[ 11%] Generating u1db_schema.c
Scanning dependencies of target u1db
[ 22%] Building C object src/CMakeFiles/u1db.dir/mkstemp_compat.c.o
[ 33%] Building C object src/CMakeFiles/u1db.dir/u1db.c.o
[ 44%] Building C object src/CMakeFiles/u1db.dir/u1db_http_sync_target.c.o
[ 55%] Building C object src/CMakeFiles/u1db.dir/u1db_query.c.o
[ 66%] Building C object src/CMakeFiles/u1db.dir/u1db_sync_target.c.o
[ 77%] Building C object src/CMakeFiles/u1db.dir/u1db_uuid.c.o
[ 88%] Building C object src/CMakeFiles/u1db.dir/u1db_vectorclock.c.o
[100%] Building C object src/CMakeFiles/u1db.dir/u1db_schema.c.o
Linking C static library libu1db.a
[100%] Built target u1db
Scanning dependencies of target ReplicatePythonSourceTree
[100%] Built target ReplicatePythonSourceTree
Scanning dependencies of target build-debug
running build_ext
cythoning u1db/tests/c_backend_wrapper.pyx to u1db/tests/c_backend_wrapper.c
building 'u1db.tests.c_backend_wrapper' extension
creating build
creating build/temp.linux-x86_64-2.7-pydebug
creating build/temp.linux-x86_64-2.7-pydebug/u1db
creating build/temp.linux-x86_64-2.7-pydebug/u1db/tests
gcc -pthread -fno-strict-aliasing -g -O0 -Wall -Wstrict-prototypes -fPIC -Iinclude -I/mnt/tarmac/cache/u1db/trunk/include -I/usr/include/python2.7_d -c u1db/tests/c_backend_wrapper.c -o build/temp.linux-x86_64-2.7-pydebug/u1db/tests/c_backend_wrapper.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro build/temp.linux-x86_64-2.7-pydebug/u1db/tests/c_backend_wrapper.o -Lsrc -lu1db -lsqlite3 -loauth -lcurl -ljson -o /mnt/tarmac/cache/u1db/trunk/u1db/tests/c_backend_wrapper_d.so
[100%] Built target build-debug
Scanning dependencies of target check-valgrind
Tests running...
======================================================================
FAIL: u1db.tests.test_sync.DatabaseSyncTests.test_sync_tracks_db_generation_of_other(c,csync)
----------------------------------------------------------------------
Tr...

[Ubuntu One Auto Pilot (otto-pilot)]

The attempt to merge lp:~thisfred/u1db/fix-local-gen-before-sync into lp:u1db failed. Below is the output from the failed tests.

-- The C compiler identification is GNU 4.7.1
-- The CXX compiler identification is GNU 4.7.1
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Found Cython: /usr/bin/cython
-- Found CURL: /usr/lib/x86_64-linux-gnu/libcurl.so (found version "7.26.0")
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.26")
-- checking for module 'oauth'
-- found oauth, version 0.9.7
-- Found OAuth: /usr/lib/x86_64-linux-gnu/liboauth.so
-- checking for module 'json'
-- found json, version 0.9
-- Found JSON: /usr/lib/x86_64-linux-gnu/libjson.so
-- checking for module 'sqlite3'
-- found sqlite3, version 3.7.13
-- Found Sqlite3: /usr/lib/x86_64-linux-gnu/libsqlite3.so
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/tarmac/cache/u1db/trunk
Scanning dependencies of target ReplicatePythonSourceTree
[ 0%] Built target ReplicatePythonSourceTree
[ 11%] Generating u1db_schema.c
Scanning dependencies of target u1db
[ 22%] Building C object src/CMakeFiles/u1db.dir/mkstemp_compat.c.o
[ 33%] Building C object src/CMakeFiles/u1db.dir/u1db.c.o
[ 44%] Building C object src/CMakeFiles/u1db.dir/u1db_http_sync_target.c.o
[ 55%] Building C object src/CMakeFiles/u1db.dir/u1db_query.c.o
[ 66%] Building C object src/CMakeFiles/u1db.dir/u1db_sync_target.c.o
[ 77%] Building C object src/CMakeFiles/u1db.dir/u1db_uuid.c.o
[ 88%] Building C object src/CMakeFiles/u1db.dir/u1db_vectorclock.c.o
[100%] Building C object src/CMakeFiles/u1db.dir/u1db_schema.c.o
Linking C static library libu1db.a
[100%] Built target u1db
Scanning dependencies of target build-debug
running build_ext
cythoning u1db/tests/c_backend_wrapper.pyx to u1db/tests/c_backend_wrapper.c
building 'u1db.tests.c_backend_wrapper' extension
creating build
creating build/temp.linux-x86_64-2.7-pydebug
creating build/temp.linux-x86_64-2.7-pydebug/u1db
creating build/temp.linux-x86_64-2.7-pydebug/u1db/tests
gcc -pthread -fno-strict-aliasing -g -O0 -Wall -Wstrict-prototypes -fPIC -Iinclude -I/mnt/tarmac/cache/u1db/trunk/include -I/usr/include/python2.7_d -c u1db/tests/c_backend_wrapper.c -o build/temp.linux-x86_64-2.7-pydebug/u1db/tests/c_backend_wrapper.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro build/temp.linux-x86_64-2.7-pydebug/u1db/tests/c_backend_wrapper.o -Lsrc -lu1db -lsqlite3 -loauth -lcurl -ljson -o /mnt/tarmac/cache/u1db/trunk/u1db/tests/c_backend_wrapper_d.so
[100%] Built target build-debug
Scanning dependencies of target check-valgrind
Tests running...
======================================================================
FAIL: u1db.tests.test_sync.DatabaseSyncTests.test_sync_tracks_db_generation_of_other(c,csync)
----------------------------------------------------------------------
Tr...

[lp:~thisfred/u1db/fix-local-gen-before-sync]

384. By Eric Casteleijn on 2012-08-15

init another int

=== modified file 'include/u1db/u1db.h'
--- include/u1db/u1db.h	2012-08-14 21:22:44 +0000
+++ include/u1db/u1db.h	2012-08-15 15:38:20 +0000
@@ -272,9 +272,13 @@
 /**
  * Synchronize db with the database at url.
  *
- * @param url    The URL of a remote database to synchronize with.
+ * @param url              The URL of a remote database to synchronize with.
+ * @param local_gen (OUT)  The local generation at the start of the sync. This
+ *                         is useful for applications to call whats_changed
+ *                         with to find out which documents were affected by a
+ *                         sync.   
  */
-int u1db_sync(u1database *db, const char *url);
+int u1db_sync(u1database *db, const char *url, int *local_gen);
 
 /**
  * Create an index that you can query for matching documents.

=== modified file 'include/u1db/u1db_internal.h'
--- include/u1db/u1db_internal.h	2012-07-26 18:07:54 +0000
+++ include/u1db/u1db_internal.h	2012-08-15 15:38:20 +0000
@@ -540,6 +540,11 @@
 
 /**
  * Sync a database with a sync target.
+ *
+ * @param local_gen_before_sync (OUT) Will return the local generation before
+ *                                    before the sync started, which clients
+ *                                    can use to know what was changed during
+ *                                    the sync.
  */
 int u1db__sync_db_to_target(u1database *db, u1db_sync_target *target,
                             int *local_gen_before_sync);

=== modified file 'src/u1db.c'
--- src/u1db.c	2012-08-14 23:05:22 +0000
+++ src/u1db.c	2012-08-15 15:38:20 +0000
@@ -2077,19 +2077,15 @@
 }
 
 int
-u1db_sync(u1database *db, const char *url)
+u1db_sync(u1database *db, const char *url, int *local_gen)
 {
     int status = U1DB_OK;
-    int local_gen;
     u1db_sync_target *target = NULL;
     status = u1db__create_http_sync_target(url, &target);
     if (status != U1DB_OK) {
         goto finish;
     }
-    status = u1db__get_generation(db, &local_gen);
-    if (status != U1DB_OK)
-        return status;
-    status = u1db__sync_db_to_target(db, target, &local_gen);
+    status = u1db__sync_db_to_target(db, target, local_gen);
 finish:
     if (target != NULL) {
         u1db__free_sync_target(&target);

=== modified file 'u1db/tests/c_backend_wrapper.pyx'
--- u1db/tests/c_backend_wrapper.pyx	2012-08-14 23:05:22 +0000
+++ u1db/tests/c_backend_wrapper.pyx	2012-08-15 15:38:20 +0000
@@ -98,7 +98,7 @@
                                   u1db_trans_info_callback cb)
     int u1db_get_doc_conflicts(u1database *db, char *doc_id, void *context,
                                u1db_doc_callback cb)
-    int u1db_sync(u1database *db, const_char_ptr url) nogil
+    int u1db_sync(u1database *db, const_char_ptr url, int *local_gen) nogil
     int u1db_create_index_list(u1database *db, char *index_name,
                                int n_expressions, const_char_ptr *expressions)
     int u1db_create_index(u1database *db, char *index_name, int n_expressions,
@@ -1232,10 +1232,12 @@
 
     def sync(self, url):
         cdef const_char_ptr c_url
+        cdef int local_gen = 0
         c_url = url
         with nogil:
-            status = u1db_sync(self._db, c_url)
+            status = u1db_sync(self._db, c_url, &local_gen)
         handle_status("sync", status)
+        return local_gen
 
     def list_indexes(self):
         a_list = []

=== modified file 'u1db/tests/test_c_backend.py'
--- u1db/tests/test_c_backend.py	2012-08-14 21:03:06 +0000
+++ u1db/tests/test_c_backend.py	2012-08-15 15:38:20 +0000
@@ -484,7 +484,12 @@
         url = self.getURL('test.db')
         db = c_backend_wrapper.CDatabase(':memory:')
         doc = db.create_doc_from_json(tests.simple_doc)
-        db.sync(url)
+        local_gen_before_sync = db.sync(url)
+        gen, _, changes = db.whats_changed(local_gen_before_sync)
+        self.assertEqual(1, len(changes))
+        self.assertEqual(mem_doc.doc_id, changes[0][0])
+        self.assertEqual(1, gen - local_gen_before_sync)
+        self.assertEqual(1, local_gen_before_sync)
         self.assertGetDoc(mem_db, doc.doc_id, doc.rev, doc.get_json(), False)
         self.assertGetDoc(db, mem_doc.doc_id, mem_doc.rev, mem_doc.get_json(),
                           False)

=== modified file 'u1db/tests/test_sync.py'
--- u1db/tests/test_sync.py	2012-08-09 20:46:13 +0000
+++ u1db/tests/test_sync.py	2012-08-15 15:38:20 +0000
@@ -1053,7 +1053,11 @@
         doc1 = self.db.create_doc_from_json(tests.simple_doc)
         doc2 = self.db2.create_doc_from_json(tests.nested_doc)
         db2_url = self.getURL('test2.db')
-        self.db.sync(db2_url)
+        local_gen_before_sync = self.db.sync(db2_url)
+        gen, _, changes = self.db.whats_changed(local_gen_before_sync)
+        self.assertEqual(1, len(changes))
+        self.assertEqual(doc2.doc_id, changes[0][0])
+        self.assertEqual(1, gen - local_gen_before_sync)
         self.assertGetDoc(self.db2, doc1.doc_id, doc1.rev, tests.simple_doc,
                           False)
         self.assertGetDoc(self.db, doc2.doc_id, doc2.rev, tests.nested_doc,