Merge lp:~kevin-wright-1/u1db-qt/synchronizer-07-juni-2013 into lp:u1db-qt

Proposed by Cris Dywan
Status: Merged
Merged at revision: 100
Proposed branch: lp:~kevin-wright-1/u1db-qt/synchronizer-07-juni-2013
Merge into: lp:u1db-qt
Diff against target: 1293 lines (+1197/-3) (has conflicts)
9 files modified
CMakeLists.txt (+0/-1)
examples/u1db-qt-example-6/u1db-qt-example-6.qdoc (+387/-0)
examples/u1db-qt-example-6/u1db-qt-example-6.qml (+174/-0)
modules/U1db/plugin.cpp (+2/-0)
src/CMakeLists.txt (+3/-1)
src/database.cpp (+4/-0)
src/index.cpp (+13/-1)
src/synchronizer.cpp (+526/-0)
src/synchronizer.h (+88/-0)
Text conflict in src/database.cpp
Text conflict in src/index.cpp
To merge this branch: bzr merge lp:~kevin-wright-1/u1db-qt/synchronizer-07-juni-2013
Reviewer Review Type Date Requested Status
U1DB Qt developers Pending
Review via email: mp+168058@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Cris Dywan (kalikiana) wrote :

syncLocalToLocal should use U1db::Database to re-use the initialization, which would also give it the existing sanity checks

the logic of onSyncChanged is probably best moved into a separate callback. Changing resolve_to_source and targets would trigger it as well. Thus it becomes possible to delay/ thread the real sync and avoid any delay at startup.

Even with keeping in mind it's incomplete //uncommented code and conflicts make reviewing a little hard.

I love the description of the sync via HTTP/ JSON, this would be nice to get into one qdoc-ified comment and make it accessible as documentation.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'CMakeLists.txt'
2--- CMakeLists.txt 2013-05-01 22:49:50 +0000
3+++ CMakeLists.txt 2013-06-07 13:40:37 +0000
4@@ -39,4 +39,3 @@
5 install(FILES ${CMAKE_CURRENT_BINARY_DIR}/${QT_U1DB_PKGCONFIG_FILE}
6 DESTINATION ${CMAKE_INSTALL_PREFIX}/lib/pkgconfig
7 )
8-
9
10=== added directory 'examples/u1db-qt-example-6'
11=== added file 'examples/u1db-qt-example-6/u1db-qt-example-6.qdoc'
12--- examples/u1db-qt-example-6/u1db-qt-example-6.qdoc 1970-01-01 00:00:00 +0000
13+++ examples/u1db-qt-example-6/u1db-qt-example-6.qdoc 2013-06-07 13:40:37 +0000
14@@ -0,0 +1,387 @@
15+/*!
16+
17+\page u1db-qt-tutorial-6.html
18+
19+\title U1Db-Qt Synchronizing Tutorial
20+
21+This tutorial is designed to demonstrate a variety of essential U1Db-Qt functionality and usage, including:
22+
23+\list 1
24+ \li Synchronizing two databases
25+ \li Utilizing the U1db-Qt Index element
26+ \li Various approaches to define U1db-Qt Document elements when using the Index element
27+ \li Partnering the U1db-Qt Index element and a QML ListView element
28+\endlist
29+
30+\section1 Storing Data
31+
32+\section2 The Database Element
33+
34+\section3 Creating a Database
35+
36+A Database is very simple to create. It only needs an id and a path where the file will be created. A Database is a model, which can be used by elements, such as the ListView further in this example.
37+
38+\code
39+U1db.Database {
40+ id: aDatabase
41+ path: "aDatabase4"
42+}
43+\endcode
44+
45+\section1 The Document Element
46+
47+\section2 Declaring Documents (at Runtime)
48+
49+A Document can be instantiated at runtime, or generated dynamically. The examples below demonstrate the former.
50+
51+A very basic Document could include its unique 'id' and 'docId' properties. While it is not mandatory to define these properties, in some cases they can be convenient references. More advanced applications would likely find these very useful, and in some cases may be an absolute necessity to achieve the objectives of the program.
52+
53+This example of a very simple Document will not initially do anything, until more properties are added and defined:
54+
55+\code
56+U1db.Document {
57+ id: aDocument1
58+ docId: 'helloworld1'
59+}
60+\endcode
61+
62+A basic but still practical Document definition contains several essential properties. In addition to 'id' and 'docId' (discussed above), the 'database', 'create', and 'defaults' properties are also very important, and are introduced below.
63+
64+The 'database' property ensures that the Document is attached to an already defined (or possibly soon to be defined one) identified by its id (in this case 'aDatabase'). For example:
65+
66+\code
67+U1db.Document {
68+ id: aDocument1
69+ database: aDatabase
70+ docId: 'helloworld1'
71+}
72+\endcode
73+
74+Should the Database not already contain a Document with the same docId ('hellowworld1' in this example) when a 'create' property is present and set to true it will be generated. For example:
75+
76+\code
77+U1db.Document {
78+ id: aDocument1
79+ database: aDatabase
80+ docId: 'helloworld1'
81+ create: true
82+}
83+\endcode
84+
85+However, the Document still requires some data to be useful, which is what the 'defaults' property provides. The value of 'defaults' is a map of data that will be stored in the database (again when the create property is et to true). It contain key:value pairs, where the value can be a string, number, or nested object (e.g. additional fields, lists). For example:
86+
87+\code
88+U1db.Document {
89+ id: aDocument1
90+ database: aDatabase
91+ docId: 'helloworld1'
92+ create: true
93+ defaults:{"hello": { "world": { "message":"Hello World", "id": 1 } } }
94+}
95+\endcode
96+
97+As mentioned above, lists can also be nested in Document data. Lists provide a convenient method for producing multiple instances of the same key (AKA 'field' or 'sub-field'). The example code below shows valid use of the 'message' and 'id' sub-fields multiple times within the same object.
98+
99+\code
100+U1db.Document {
101+ id: aDocument2
102+ database: aDatabase
103+ docId: 'helloworld2'
104+ create: true
105+ defaults:{"hello": { "world": [
106+ { "message":"Hello World", "id": 2 },
107+ { "message":"Hello World", "id": 2.5 }
108+ ] } }
109+}
110+\endcode
111+
112+When the default Javascript Object Notation itself is formatted with appropriate line breaks and indentation, it becomes easier to visualize an embedded list, containing sub-fields 'message' and 'id' (and their respective values):
113+
114+\code
115+{"hello":
116+ { "world":
117+ [
118+ { "message":"Hello World", "id": 2 },
119+ { "message":"Hello World", "id": 2.5 }
120+ ]
121+ }
122+}
123+\endcode
124+
125+In dot notation these sub-fields are represented by 'hello.world.message' and 'hello.world.id' respectively. Later in this tutorial these will be utilized within the 'expression' property of U1Db-Qt's Index element, in close collaboration with a QML ListView's delegates.
126+
127+
128+Normally when a docId already exists in a database, and when the set flag is set to true, the value in 'defaults' will be ignored (and the existing data in the database will remain untouched). Sometimes a developer needs to easily overwrite the data in an existing document. The 'contents' property can be used for just that purpose. When 'contents' is defined, its value will replace existing data in the database, for the document identified by the docId. In addition, 'contents' can be used to add new documents, in the same way as the 'create: true' + 'defaults' combination does; in other words, if the document defined by 'docId' does not exist it will be created.
129+
130+\code
131+U1db.Document {
132+ id: aDocument3
133+ database: aDatabase
134+ docId: 'helloworld3'
135+ contents:{"hello": { "world": [
136+ { "message":"Hello World", "id": 3 },
137+ { "message":"Hello World", "id": 3.33 },
138+ { "message":"Hello World", "id": 3.66 }
139+ ] } }
140+}
141+\endcode
142+
143+If 'defaults' exists, 'create' is set to 'true' (or 'false' for that matter) and 'contents' is also defined, it is the latter that takes precidence. In other words, 'create' and 'defaults' will be ignored. The following example demonstrates this scenario:
144+
145+\code
146+U1db.Document {
147+ id: aDocument3
148+ database: aDatabase
149+ docId: 'helloworld3'
150+ create: true
151+ default:{"hello": { "world": [{ "message":"Hello World", "id": 3 }] } }
152+ contents:{"hello": { "world": [
153+ { "message":"Hello World", "id": 3 },
154+ { "message":"Hello World", "id": 3.33 },
155+ { "message":"Hello World", "id": 3.66 }
156+ ] } }
157+}
158+\endcode
159+
160+This snippet simply represents the absence of the 'create' property, which is synonymous with 'create: false'. The Document can still be recognized within the application, but until applicable properties (such as those outlined above) are added and/or modified then nothing will be added or modified in the database, and this instance may have very little practical value.
161+
162+\code
163+U1db.Document {
164+ id: aDocument4
165+ database: aDatabase
166+ docId: 'helloworld4'
167+ defaults:{"hello": { "world": { "message":"Hello World", "id": 4 } } }
168+}
169+\endcode
170+
171+\section3 Samples of Stored Documents
172+
173+The data stored in the database after defining the above Document elements (and then running the application, will consist of the following:
174+
175+\table
176+\header
177+ \li docId
178+ \li content
179+\row
180+
181+ \li 'helloworld1'
182+ \li
183+\code
184+{
185+ "hello": {
186+ "world": {
187+ "id": 1,
188+ "message": "Hello World"
189+ }
190+ }
191+}
192+
193+\endcode
194+
195+
196+\row
197+
198+ \li 'helloworld2'
199+ \li
200+\code
201+{
202+ "hello": {
203+ "world": [
204+ {
205+ "id": 2,
206+ "message": "Hello World"
207+ },
208+ {
209+ "id": 2.5,
210+ "message": "Hello World"
211+ }
212+ ]
213+ }
214+}
215+\endcode
216+
217+\row
218+
219+ \li 'helloworld3'
220+ \li
221+\code
222+{
223+ "hello": {
224+ "world": [
225+ {
226+ "id": 3,
227+ "message": "Hello World"
228+ },
229+ {
230+ "id": 3.33,
231+ "message": "Hello World"
232+ },
233+ {
234+ "id": 3.66,
235+ "message": "Hello World"
236+ }
237+ ]
238+ }
239+}
240+\endcode
241+
242+
243+\endtable
244+
245+\section1 Retrieving Data
246+
247+To retrieve the Documents that were declared earlier requires two additional elements: Index and Query.
248+
249+\section2 The Index Element
250+
251+\section3 Creating and Index Element
252+
253+The Index element requires both a unique 'id' and a pointer to a 'database' in order to begin becoming useful, as demonstrated here:
254+
255+\code
256+U1db.Index{
257+ database: aDatabase
258+ id: by_helloworld
259+}
260+\endcode
261+
262+In the future, the Index element will support on disk storage of appropriate results / data. At the present time only in memory indexing is done, but once the storing capability is implemented, defining and identifying it is as simple as using the 'name' property (which will be stored in the database along with the relvent data that goes with it). The snippet below shows the use of the 'name' property:
263+
264+\code
265+U1db.Index{
266+ database: aDatabase
267+ id: by_helloworld
268+ //name: "by-helloworld"
269+}
270+\endcode
271+
272+The Index element describes, using dot notation, the fields and sub-fields where the developer expects to find information. That information is defined in a list, and added as the value for the 'expression' property. The list can contain one or more entries, as exemplified here (the property is commented out due to its current status):
273+
274+\code
275+U1db.Index{
276+ database: aDatabase
277+ id: by_helloworld
278+ //name: "by-helloworld"
279+ expression: ["hello.world.id","hello.world.message"]
280+}
281+\endcode
282+
283+\section2 The QueryElement
284+
285+\section3 Creating a Query Element
286+
287+The Query element has two responsibilities: a bridge from Database+Index to other parts of the application, as well as further filtering of data in the database (in addition to what Index provides).
288+
289+In order to fulfil its duties as a bridge to an Index (and Database), the 'index' property must point to an Index element, identified by its 'id'. For example:
290+
291+\code
292+U1db.Query{
293+ id: aQuery
294+ index: by_helloworld
295+}
296+\endcode
297+
298+While Index helps to filter data based on 'where' it is located (e.g. field.sub-field), Query helps determine the additional set of criteria for 'what' is being searched for. The intent of the 'query' property is to provide the mechanism for defnining the search criteria, but at the time of writing that functionality is not yet available. However, once the implementation is in place, using it is only requires defining the property's value (e.g. "Hello World"). Wild card searches using '*' are supported, which is the default query (i.e. if 'query' is not set it is assumed to be '*'). For example (the property is commented out due to its current status):
299+
300+\code
301+U1db.Query{
302+ id: aQuery
303+ index: by_helloworld
304+ //query: "*"
305+}
306+\endcode
307+
308+When the 'query' property becomes available, only wildcard search definitions for "starts with" will be suppoprted. Thus the following would be supported:
309+
310+\code
311+U1db.Query{
312+ id: aQuery
313+ index: by_helloworld
314+ //query: "Hello*"
315+}
316+\endcode
317+
318+
319+But this would not:
320+
321+\code
322+U1db.Query{
323+ id: aQuery
324+ index: by_helloworld
325+ //query: "*World"
326+}
327+\endcode
328+
329+Note: again, the 'query' property is commented out in the above two snippets due to its current status
330+
331+\section1 Using Data
332+
333+\section2 Data and the Application UI
334+
335+\section3 Using Data With Models and Views
336+
337+This simple snippet represents how to attach a ListModel to a ListView. In this instance the model 'aQuery' is representative of the Query + Index combination defined earlier:
338+
339+\code
340+ListView {
341+ width: units.gu(45)
342+ height: units.gu(80)
343+ model: aQuery
344+}
345+\endcode
346+
347+\section4 Data and Delegates
348+
349+How a model and ListView + delegates work together is a common QML concept, and not specific to U1Db-Qt. However, the asynchronous nature of this relationship is important to understand. When using QML ListView, delegates will be created based on particular properties such as the size of the application window, ListView, and delegate itself (amongst other factors). Each delegate can then represent a Document retrieved from the Database based on the record's index. This example demonstrates some of the property definitions that contribute to determining the number of delegates a ListView will contain:
350+
351+\code
352+ListView {
353+ width: units.gu(45)
354+ height: units.gu(80)
355+ model: aQuery
356+
357+ delegate: Text {
358+ x: 66; y: 77
359+ }
360+
361+}
362+\endcode
363+
364+
365+When the number of Documents is less than or equal to the number of delegates then there is a one to one mapping of index to delegate (e.g. the first delegate will represent the Document with an index = 0; the second, index = 1; and so on).
366+
367+When there are more Documents than delegates the ListView will request a new index depending on the situation (e.g. a user scrolls up or down). For example, if a ListView has 10 delegates, but 32 Documents to handle, when a user initially scrolls the first delegate will change from representing the Document with index = 0 to the Document that might have index = 8; the second, from index = 1 to index = 9; ...; the 10th delegate from index = 9 to index = 17. A second scrolling gesture the first index may change to 15, and the final index 24. And so on. Scrolling in the opposite direction will have a similar effect, but the Document index numbers for each delegate will obviously start to decline (towards their original values).
368+
369+The following snippet, which modifies the above delegate definition, could demonstrate this effect if there were enough Documents to do so (i.e. some number greater than the number of delegates):
370+
371+\code
372+ListView {
373+ width: units.gu(45)
374+ height: units.gu(80)
375+ model: aQuery
376+
377+ delegate: Text {
378+ x: 66; y: 77
379+ text: index
380+ }
381+
382+}
383+\endcode
384+
385+The object called 'contents' contains one or more properties. This example demonstrates the retrieval of data based on the U1db.Index defined earlier (id: by-helloworld). In this instance the Index contained two expressions simultaniously, "hello.world.id" and "hello.world.message"
386+
387+\code
388+ListView {
389+ width: units.gu(45)
390+ height: units.gu(80)
391+ model: aQuery
392+
393+ delegate: Text {
394+ x: 66; y: 77
395+ text: "(" + index + ") '" + contents.message + " " + contents.id + "'"
396+ }
397+
398+}
399+\endcode
400+
401+*/
402
403=== added file 'examples/u1db-qt-example-6/u1db-qt-example-6.qml'
404--- examples/u1db-qt-example-6/u1db-qt-example-6.qml 1970-01-01 00:00:00 +0000
405+++ examples/u1db-qt-example-6/u1db-qt-example-6.qml 2013-06-07 13:40:37 +0000
406@@ -0,0 +1,174 @@
407+/*
408+ * Copyright (C) 2013 Canonical, Ltd.
409+ *
410+ * Authors:
411+ * Kevin Wright <kevin.wright@canonical.com>
412+ *
413+ * This program is free software; you can redistribute it and/or modify
414+ * it under the terms of the GNU Lesser General Public License as published by
415+ * the Free Software Foundation; version 3.
416+ *
417+ * This program is distributed in the hope that it will be useful,
418+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
419+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
420+ * GNU Lesser General Public License for more details.
421+ *
422+ * You should have received a copy of the GNU Lesser General Public License
423+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
424+ */
425+
426+import QtQuick 2.0
427+import U1db 1.0 as U1db
428+import Ubuntu.Components 0.1
429+
430+
431+Item {
432+
433+ width: units.gu(45)
434+ height: units.gu(80)
435+
436+ U1db.Database {
437+ id: aDatabase
438+ path: "aDatabase6"
439+ }
440+
441+ U1db.Database {
442+ id: aTargetDatabase
443+ path: "aTargetDatabase6"
444+ }
445+
446+ U1db.Document {
447+ id: aDocument1
448+ database: aDatabase
449+ docId: 'helloworld'
450+ create: true
451+ defaults:{"hello": { "world": { "message":"Hello World", "id": 1 } } }
452+
453+ }
454+
455+ U1db.Document {
456+ id: aDocument2
457+ database: aTargetDatabase
458+ docId: 'helloworld'
459+ create: true
460+ defaults:{"hello": { "world": [
461+ { "message":"Hello World", "id": 2 },
462+ { "message":"Hello World", "id": 2.5 }
463+ ] } }
464+ }
465+
466+ U1db.Index{
467+ database: aDatabase
468+ id: by_helloworld
469+ expression: ["hello.world.id","hello.world.message"]
470+ }
471+
472+ U1db.Query{
473+ id: aQuery
474+ index: by_helloworld
475+ query: [{"id":"*"},{"message":"Hel*"}]
476+ }
477+
478+ U1db.Index{
479+ database: aTargetDatabase
480+ id: by_target_helloworld
481+ expression: ["hello.world.id","hello.world.message"]
482+ }
483+
484+ U1db.Query{
485+ id: aTargetQuery
486+ index: by_target_helloworld
487+ query: [{"id":"*"},{"message":"Hel*"}]
488+ }
489+
490+ U1db.Synchronizer{
491+ id: aSynchronizer
492+ source: aDatabase
493+ //local_targets: [aTargetDatabase]
494+ //remote_targets: ["http://somewhere"]
495+ targets: [{remote:true},
496+ {remote:false,location:"database7",resolve_to_source:true},
497+ {remote:"OK"}]
498+ resolve_to_source: true
499+ synchronize: false
500+ }
501+
502+ MainView {
503+
504+ id: u1dbView
505+ width: units.gu(45)
506+ height: units.gu(80)
507+ anchors.top: parent.top;
508+
509+ Tabs {
510+ id: tabs
511+ anchors.fill: parent
512+
513+ Tab {
514+ objectName: "Tab1"
515+
516+ title: i18n.tr("Hello U1Db!")
517+
518+ page: Page {
519+ id: helloPage
520+
521+ Rectangle {
522+ width: units.gu(45)
523+ height: units.gu(70)
524+ anchors.top: parent.top;
525+
526+ ListView {
527+ width: units.gu(45)
528+ height: units.gu(35)
529+ anchors.top: parent.top;
530+ model: aQuery
531+
532+ delegate: Text {
533+ x: 66; y: 77
534+ text: {
535+ text: "(" + index + ") '" + contents.message + " " + contents.id + "'"
536+ }
537+ }
538+ }
539+
540+ ListView {
541+ width: units.gu(45)
542+ height: units.gu(35)
543+ anchors.bottom: parent.bottom;
544+ model: aTargetQuery
545+
546+ delegate: Text {
547+ x: 66; y: 77
548+ text: {
549+ text: "(" + index + ") '" + contents.message + " " + contents.id + "'"
550+ }
551+ }
552+ }
553+ }
554+
555+ Rectangle {
556+ width: units.gu(45)
557+ height: units.gu(5)
558+ anchors.bottom: parent.bottom;
559+ color: "white"
560+
561+ Button{
562+ text: "Sync"
563+ onClicked: aSynchronizer.synchronize = true
564+ anchors.left: parent.left
565+ anchors.right: parent.right
566+
567+ }
568+
569+ }
570+ }
571+
572+ }
573+
574+ }
575+
576+ }
577+
578+}
579+
580+
581
582=== modified file 'modules/U1db/plugin.cpp'
583--- modules/U1db/plugin.cpp 2013-02-15 11:43:45 +0000
584+++ modules/U1db/plugin.cpp 2013-06-07 13:40:37 +0000
585@@ -21,6 +21,7 @@
586 #include "document.h"
587 #include "index.h"
588 #include "query.h"
589+#include "synchronizer.h"
590 #include "plugin.h"
591 #include <qqml.h>
592
593@@ -32,5 +33,6 @@
594 qmlRegisterType<Document>(uri, 1, 0, "Document");
595 qmlRegisterType<Index>(uri, 1, 0, "Index");
596 qmlRegisterType<Query>(uri, 1, 0, "Query");
597+ qmlRegisterType<Synchronizer>(uri, 1, 0, "Synchronizer");
598 }
599
600
601=== modified file 'src/CMakeLists.txt'
602--- src/CMakeLists.txt 2013-04-23 10:39:54 +0000
603+++ src/CMakeLists.txt 2013-06-07 13:40:37 +0000
604@@ -6,6 +6,7 @@
605 document.cpp
606 index.cpp
607 query.cpp
608+ synchronizer.cpp
609 )
610
611 # Generated files
612@@ -14,6 +15,7 @@
613 moc_document.cpp
614 moc_index.cpp
615 moc_query.cpp
616+ moc_synchronizer.cpp
617 )
618 set_directory_properties(PROPERTIES ADDITIONAL_MAKE_CLEAN_FILES "${U1DB_QT_GENERATED}")
619
620@@ -50,7 +52,7 @@
621 LIBRARY DESTINATION lib${LIB_SUFFIX}
622 )
623
624-install(FILES global.h database.h document.h index.h query.h
625+install(FILES global.h database.h document.h index.h query.h synchronizer.h
626 DESTINATION ${INCLUDE_INSTALL_DIR}
627 )
628
629
630=== modified file 'src/database.cpp'
631--- src/database.cpp 2013-04-29 23:54:19 +0000
632+++ src/database.cpp 2013-06-07 13:40:37 +0000
633@@ -106,7 +106,11 @@
634 /* A unique ID is used for the connection name to ensure that we aren't
635 re-using or replacing other opend databases. */
636 if (!m_db.isValid())
637+<<<<<<< TREE
638 m_db = QSqlDatabase::addDatabase("QSQLITE", QUuid::createUuid().toString());
639+=======
640+ m_db = QSqlDatabase::addDatabase("QSQLITE",QUuid::createUuid().toString());
641+>>>>>>> MERGE-SOURCE
642 if (!m_db.isValid())
643 return setError("QSqlDatabase error");
644 m_db.setDatabaseName(path);
645
646=== modified file 'src/index.cpp'
647--- src/index.cpp 2013-05-10 13:22:44 +0000
648+++ src/index.cpp 2013-06-07 13:40:37 +0000
649@@ -146,7 +146,19 @@
650 if (m_expression == expression)
651 return;
652
653- m_expression = expression;
654+<<<<<<< TREE
655+ m_expression = expression;
656+=======
657+ m_expression = expression;
658+
659+ if (m_database)
660+ {
661+ m_database->putIndex(m_name, expression);
662+ Q_EMIT dataInvalidated();
663+ }
664+
665+
666+>>>>>>> MERGE-SOURCE
667
668 if (m_database)
669 {
670
671=== added file 'src/synchronizer.cpp'
672--- src/synchronizer.cpp 1970-01-01 00:00:00 +0000
673+++ src/synchronizer.cpp 2013-06-07 13:40:37 +0000
674@@ -0,0 +1,526 @@
675+/*
676+ * Copyright (C) 2013 Canonical, Ltd.
677+ *
678+ * Authors:
679+ * Kevin Wright <kevin.wright@canonical.com>
680+ *
681+ * This program is free software; you can redistribute it and/or modify
682+ * it under the terms of the GNU Lesser General Public License as published by
683+ * the Free Software Foundation; version 3.
684+ *
685+ * This program is distributed in the hope that it will be useful,
686+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
687+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
688+ * GNU Lesser General Public License for more details.
689+ *
690+ * You should have received a copy of the GNU Lesser General Public License
691+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
692+ */
693+
694+#include <QDebug>
695+#include <QSqlQuery>
696+#include <QFile>
697+#include <QSqlError>
698+#include <QUuid>
699+#include <QStringList>
700+#include <QJsonDocument>
701+
702+#include "synchronizer.h"
703+#include "private.h"
704+
705+QT_BEGIN_NAMESPACE_U1DB
706+
707+/*!
708+ \class Synchronizer
709+ \inmodule U1db
710+ \ingroup modules
711+
712+ \brief The Synchronizer class handles synchronizing between two databases.
713+
714+*/
715+
716+/*
717+
718+ Below this line are methods that are standards in declarative API design. They include the element instantiation, default settings for properties and get/set methods for properties.
719+
720+*/
721+
722+/*!
723+ Instantiate a new Synchronizer with an optional \a parent, usually by declaring it as a QML item.
724+
725+ Synchronizer elements sync two databases together, a 'source' database and a remote or local 'target' database.
726+
727+ */
728+
729+Synchronizer::Synchronizer(QObject *parent) :
730+ QObject(parent), m_synchronize(false)
731+{
732+ qDebug()<<"Synchronizer::Synchronizer";
733+
734+ QObject::connect(this, &Synchronizer::syncChanged, this, &Synchronizer::onSyncChanged);
735+}
736+
737+/*!
738+ \property Synchronizer::source
739+ */
740+void Synchronizer::setSource(Database* source)
741+{
742+ qDebug()<<"Synchronizer::setSource";
743+
744+ if (m_source == source)
745+ return;
746+
747+ if (m_source)
748+ QObject::disconnect(m_source, 0, this, 0);
749+
750+ m_source = source;
751+
752+ Q_EMIT sourceChanged(source);
753+}
754+
755+/*!
756+ \property Synchronizer::local_targets
757+ */
758+void Synchronizer::setLocalTargets(QList<QObject*> local_targets)
759+{
760+ qDebug()<<"Synchronizer::setLocalTargets";
761+
762+ if (m_local_targets == local_targets)
763+ return;
764+
765+ //if (m_local_targets)
766+ // QObject::disconnect(m_local_targets, 0, this, 0);
767+
768+ m_local_targets = local_targets;
769+ Q_EMIT localTargetsChanged(local_targets);
770+}
771+
772+/*!
773+ \property Synchronizer::remote_targets
774+ */
775+void Synchronizer::setRemoteTargets(QList<QString> remote_targets)
776+{
777+ qDebug()<<"Synchronizer::setRemoteTargets";
778+
779+ if (m_remote_targets == remote_targets)
780+ return;
781+
782+ //if (m_local_targets)
783+ // QObject::disconnect(m_local_targets, 0, this, 0);
784+
785+ m_remote_targets = remote_targets;
786+ Q_EMIT remoteTargetsChanged(remote_targets);
787+}
788+
789+/*!
790+ \property Synchronizer::targets
791+ */
792+void Synchronizer::setTargets(QVariant targets)
793+{
794+ qDebug()<<"Synchronizer::setTargets";
795+
796+ if (m_targets == targets)
797+ return;
798+
799+ //if (m_targets)
800+ // QObject::disconnect(m_targets, 0, this, 0);
801+
802+ m_targets = targets;
803+ Q_EMIT targetsChanged(targets);
804+}
805+
806+/*!
807+ \property Synchronizer::synchronize
808+ */
809+void Synchronizer::setSync(bool synchronize)
810+{
811+ qDebug()<<"Synchronizer::setSync";
812+
813+ if (m_synchronize == synchronize)
814+ return;
815+
816+ m_synchronize = synchronize;
817+ Q_EMIT syncChanged(synchronize);
818+}
819+
820+/*!
821+ \property Synchronizer::resolve_to_source
822+ */
823+void Synchronizer::setResolveToSource(bool resolve_to_source)
824+{
825+ qDebug()<<"Synchronizer::setResolveToSource";
826+
827+ if (m_resolve_to_source == resolve_to_source)
828+ return;
829+
830+ m_resolve_to_source = resolve_to_source;
831+ Q_EMIT resolveToSourceChanged(resolve_to_source);
832+}
833+
834+/*!
835+
836+ */
837+Database* Synchronizer::getSource()
838+{
839+ qDebug()<<"Synchronizer::getSource";
840+
841+ return m_source;
842+}
843+
844+/*!
845+
846+ */
847+QList<QObject*> Synchronizer::getLocalTargets()
848+{
849+ qDebug()<<"Synchronizer::getLocalTargets";
850+
851+ return m_local_targets;
852+}
853+
854+/*!
855+
856+ */
857+QList<QString> Synchronizer::getRemoteTargets()
858+{
859+ qDebug()<<"Synchronizer::getRemoteTargets";
860+
861+ return m_remote_targets;
862+}
863+
864+/*!
865+
866+ */
867+QVariant Synchronizer::getTargets()
868+{
869+ qDebug()<<"Synchronizer::getTargets";
870+
871+ return m_targets;
872+}
873+
874+
875+/*!
876+
877+ */
878+bool Synchronizer::getSync()
879+{
880+ qDebug()<<"Synchronizer::getSync";
881+
882+ return m_synchronize;
883+}
884+
885+/*!
886+
887+ */
888+bool Synchronizer::getResolveToSource(){
889+
890+ qDebug()<<"Synchronizer::getResolveToSource";
891+
892+ return m_resolve_to_source;
893+}
894+
895+/*
896+
897+ Below this line represents the unique API methods that are not part of standard declarative API design.
898+
899+*/
900+
901+void Synchronizer::onSyncChanged(bool synchronize){
902+
903+ qDebug() << "Synchronizer::onSyncChanged = " << synchronize;
904+
905+ /*
906+ * `validator` is used to ensure that the QVariant types, for a given key,
907+ * within each QMap that represents a target db defintion, are correct.
908+ *
909+ * Similarily, `mandatory` checks to ensure the minimum necessary properties
910+ * have been set with some value.
911+ *
912+ * There may be a more elegant way to approach this, but for now it will do.
913+ *
914+ */
915+
916+ QList<QList<QString>> errors;
917+
918+ QMap<QString,QString>validator;
919+
920+ validator.insert("remote","bool");
921+ validator.insert("location","QString");
922+ validator.insert("resolve_to_source","bool");
923+ validator.insert("errors", "QStringList");
924+
925+ QList<QString>mandatory;
926+
927+ mandatory.append("remote");
928+ mandatory.append("location");
929+ mandatory.append("resolve_to_source");
930+
931+ if(synchronize == true){
932+
933+ int index = 0;
934+
935+
936+ Database* source = getSource();
937+
938+ QList<QVariant> targets = getTargets().toList();
939+ QList<QVariant> sync_targets;
940+
941+ Q_FOREACH (QVariant target_variant, targets)
942+ {
943+ index++;
944+ QString index_number = QString::number(index);
945+
946+ QMap<QString, QVariant> target = target_variant.toMap();
947+
948+ QList<QString> error;
949+
950+ bool valid = true;
951+ bool complete = true;
952+
953+ QMapIterator<QString, QVariant> i(target);
954+ while (i.hasNext()) {
955+
956+ i.next();
957+
958+ if(validator.contains(i.key())&&validator[i.key()]!=i.value().typeName()){
959+ valid = false;
960+
961+ error.append(index_number + ": For Key: " + i.key() + " Expecting Type: " + validator[i.key()] + " Received Type: " + i.value().typeName());
962+ target.insert("sync",false);
963+
964+ break;
965+ }
966+
967+ if(valid==false){
968+ targets.removeOne(target);
969+ break;
970+ }
971+ else{
972+ QListIterator<QString> j(mandatory);
973+
974+ while(j.hasNext()){
975+ QString value = j.next();
976+ if(!target.contains(value)){
977+ error.append(index_number + ": Expected Key: `" + value + "` but it is not present.");
978+ target.insert("sync",false);
979+ targets.removeOne(target);
980+ complete = false;
981+ break;
982+ }
983+ }
984+ if(complete==false){
985+ break;
986+ }
987+
988+ }
989+ }
990+
991+ if(target.contains("sync")&&target["sync"]==false){
992+ errors.append(error);
993+ }
994+ else
995+ {
996+ target.insert("sync",true);
997+ sync_targets.append(target);
998+ }
999+
1000+ }
1001+
1002+
1003+
1004+ Q_FOREACH (QStringList err, errors){
1005+ Q_FOREACH (QString error, err){
1006+ qDebug()<<error;
1007+ }
1008+ }
1009+
1010+ synchronizeTargets(source, sync_targets);
1011+
1012+/* The source replica asks the target replica for the information it has stored about the last time these two replicas were synchronised (if ever).*/
1013+
1014+ //QList<QObject*> local_targets = this->getLocalTargets();
1015+
1016+ /*Q_FOREACH (QObject* local_target, local_targets)
1017+ {
1018+
1019+ Database *target = (Database*)local_target;
1020+ qDebug() << "Synchronizer::onSyncChanged Q_FOREACH (Database* local_target, local_targets) = " << target->getPath();
1021+
1022+ //syncWithLocalTarget(Database *source, Database *target, bool resolve_to_source)
1023+
1024+ }*/
1025+
1026+ //QList<QString> remote_targets = this->getRemoteTargets();
1027+
1028+ /*Q_FOREACH (QString remote_target, remote_targets)
1029+ {
1030+
1031+ qDebug() << "Synchronizer::onSyncChanged Q_FOREACH (QObject* remote_target, remote_targets) = " << remote_target;
1032+
1033+ //syncWithRemoteTarget(Database *source, QString target_url, bool resolve_to_source)
1034+
1035+ }*/
1036+
1037+ QMap<QString, QVariant> sync_from_information;
1038+ //sync_from_information.insert("source_replica_uid",source_replica_uid);
1039+ sync_from_information.insert("source_replica_generation","");
1040+ sync_from_information.insert("source_transaction_id","");
1041+ //sync_from_information.insert("target_replica_uid",target_replica_uid);
1042+ sync_from_information.insert("target_replica_generation","");
1043+ sync_from_information.insert("target_replica_transaction_id","");
1044+
1045+/*
1046+
1047+The application wishing to synchronise sends the following GET request to the server:
1048+
1049+GET /thedb/sync-from/my_replica_uid
1050+
1051+Where thedb is the name of the database to be synchronised, and my_replica_uid is the replica id of the application’s (i.e. the local, or synchronisation source) database
1052+
1053+*/
1054+
1055+/*
1056+
1057+The target responds with a JSON document that looks like this:
1058+
1059+{
1060+ "target_replica_uid": "other_replica_uid",
1061+ "target_replica_generation": 12,
1062+ "target_replica_transaction_id": "T-sdkfj92292j",
1063+ "source_replica_uid": "my_replica_uid",
1064+ "source_replica_generation": 23,
1065+ "source_transaction_id": "T-39299sdsfla8"
1066+}
1067+
1068+With all the information it has stored for the most recent synchronisation between itself and this particular source replica. In this case it tells us that the synchronisation target believes that when it and the source were last synchronised, the target was at generation 12 and the source at generation 23.
1069+
1070+*/
1071+
1072+
1073+
1074+/* The source replica validates that its information regarding the last synchronisation is consistent with the target’s information, and raises an error if not. (This could happen for instance if one of the replicas was lost and restored from backup, or if a user inadvertently tries to synchronise a copied database.) */
1075+
1076+
1077+
1078+/* The source replica generates a list of changes since the last change the target replica knows of. */
1079+
1080+
1081+
1082+/* The source replica checks what the last change is it knows about on the target replica. */
1083+
1084+
1085+
1086+/* If there have been no changes on either replica that the other side has not seen, the synchronisation stops here. */
1087+
1088+
1089+
1090+/* The source replica sends the changed documents to the target, along with what the latest change is that it knows about on the target replica. */
1091+
1092+
1093+
1094+/* The target processes the changed documents, and records the source replica’s latest change. */
1095+
1096+
1097+
1098+/* The target responds with the documents that have changes that the source does not yet know about. */
1099+
1100+
1101+
1102+/* The source processes the changed documents, and records the target replica’s latest change. */
1103+
1104+
1105+
1106+/* If the source has seen no changes unrelated to the synchronisation during this whole process, it now sends the target what its latest change is, so that the next synchronisation does not have to consider changes that were the result of this one.*/
1107+
1108+ setSync(false);
1109+
1110+
1111+ }
1112+ else{
1113+
1114+ }
1115+}
1116+
1117+void Synchronizer::synchronizeTargets(Database *source, QVariant targets){
1118+
1119+ qDebug() << "Synchronizer::synchronizeTargets";
1120+
1121+ if(targets.typeName()== QStringLiteral("QVariantList")){
1122+
1123+ QList<QVariant> target_list = targets.toList();
1124+
1125+ QListIterator<QVariant> i(target_list);
1126+
1127+ while(i.hasNext()){
1128+
1129+ QVariant target = i.next();
1130+
1131+ if(target.typeName()== QStringLiteral("QVariantMap")){
1132+ QMap<QString,QVariant> target_map = target.toMap();
1133+
1134+ if(target_map.contains("remote")&&target_map["remote"]==false){
1135+ if(target_map.contains("sync")&&target_map["sync"]==true){
1136+ syncLocalToLocal(source, target_map);
1137+ }
1138+ }
1139+ else if(target_map.contains("remote")&&target_map["remote"]==true){
1140+ if(target_map.contains("sync")&&target_map["sync"]==true){
1141+ qDebug() << "Remote database sync is under construction. Try again later.";
1142+ }
1143+ }
1144+ else{
1145+
1146+ }
1147+ }
1148+
1149+
1150+ }
1151+
1152+ }
1153+
1154+
1155+}
1156+
1157+void Synchronizer::syncLocalToLocal(Database *source, QMap<QString,QVariant> target)
1158+{
1159+ qDebug() << "Synchronizer::syncLocalToLocal";
1160+
1161+ QSqlDatabase db;
1162+
1163+ db = QSqlDatabase::addDatabase("QSQLITE",QUuid::createUuid().toString());
1164+
1165+ QString target_db_name = target["location"].toString();
1166+
1167+ QFile target_db_file(target_db_name);
1168+
1169+ if(!target_db_file.exists())
1170+ {
1171+ qDebug()<< "Local database " << target_db_name << " does not exist";
1172+ }
1173+ else
1174+ {
1175+ db.setDatabaseName(target_db_name);
1176+ if (!db.open()){
1177+ qDebug() << db.lastError().text();
1178+ }
1179+ else{
1180+ QString source_uid;
1181+
1182+ QSqlQuery query (db.exec("SELECT value FROM u1db_config WHERE name = 'replica_uid'"));
1183+
1184+ if(!query.lastError().isValid() && query.next()){
1185+ source_uid = query.value(0).toString();
1186+ }
1187+ else{
1188+ qDebug() << query.lastError().text();
1189+ }
1190+ }
1191+
1192+ }
1193+
1194+}
1195+
1196+
1197+QT_END_NAMESPACE_U1DB
1198+
1199+#include "moc_synchronizer.cpp"
1200+
1201
1202=== added file 'src/synchronizer.h'
1203--- src/synchronizer.h 1970-01-01 00:00:00 +0000
1204+++ src/synchronizer.h 2013-06-07 13:40:37 +0000
1205@@ -0,0 +1,88 @@
1206+/*
1207+ * Copyright (C) 2013 Canonical, Ltd.
1208+ *
1209+ * Authors:
1210+ * Kevin Wright <kevin.wright@canonical.com>
1211+ *
1212+ * This program is free software; you can redistribute it and/or modify
1213+ * it under the terms of the GNU Lesser General Public License as published by
1214+ * the Free Software Foundation; version 3.
1215+ *
1216+ * This program is distributed in the hope that it will be useful,
1217+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
1218+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1219+ * GNU Lesser General Public License for more details.
1220+ *
1221+ * You should have received a copy of the GNU Lesser General Public License
1222+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
1223+ */
1224+
1225+#ifndef U1DB_SYNCHRONIZER_H
1226+#define U1DB_SYNCHRONIZER_H
1227+
1228+#include <QtCore/QObject>
1229+#include <QSqlDatabase>
1230+#include <QVariant>
1231+
1232+#include "database.h"
1233+
1234+QT_BEGIN_NAMESPACE_U1DB
1235+
1236+class Q_DECL_EXPORT Synchronizer : public QObject {
1237+ Q_OBJECT
1238+#ifdef Q_QDOC
1239+ Q_PROPERTY(Database* source READ getSource WRITE setSource NOTIFY sourceChanged)
1240+#else
1241+ Q_PROPERTY(QT_PREPEND_NAMESPACE_U1DB(Database*) source READ getSource WRITE setSource NOTIFY sourceChanged)
1242+#endif
1243+ Q_PROPERTY(bool synchronize READ getSync WRITE setSync NOTIFY syncChanged)
1244+ Q_PROPERTY(bool resolve_to_source READ getResolveToSource WRITE setResolveToSource NOTIFY resolveToSourceChanged)
1245+ //Q_PROPERTY(QList <QObject*> local_targets READ getLocalTargets WRITE setLocalTargets NOTIFY localTargetsChanged)
1246+ //Q_PROPERTY(QList <QString> remote_targets READ getRemoteTargets WRITE setRemoteTargets NOTIFY remoteTargetsChanged)
1247+ Q_PROPERTY(QVariant targets READ getTargets WRITE setTargets NOTIFY targetsChanged)
1248+
1249+public:
1250+ Synchronizer(QObject* parent = 0);
1251+ Database* getSource();
1252+ QList<QObject*> getLocalTargets();
1253+ QList<QString> getRemoteTargets();
1254+ QVariant getTargets();
1255+ bool getSync();
1256+ bool getResolveToSource();
1257+ void setSource(Database* source);
1258+ void setLocalTargets(QList<QObject*> local_targets);
1259+ void setRemoteTargets(QList<QString> remote_targets);
1260+ void setTargets(QVariant targets);
1261+ void setSync(bool synchronize);
1262+ void setResolveToSource(bool resolve_to_source);
1263+
1264+ void syncWithLocalTarget(Database *source, Database *target, bool resolve_to_source);
1265+ void syncWithRemoteTarget(Database *source, QString target_url, bool resolve_to_source);
1266+ void syncLocalToLocal(Database *source, QMap<QString,QVariant> target);
1267+ void synchronizeTargets(Database *source, QVariant targets);
1268+
1269+Q_SIGNALS:
1270+ void sourceChanged(Database* source);
1271+ void localTargetsChanged(QList<QObject*> local_targets);
1272+ void remoteTargetsChanged(QList<QString> remote_targets);
1273+ void targetsChanged(QVariant targets);
1274+ void syncChanged(bool synchronize);
1275+ void resolveToSourceChanged(bool resolve_to_source);
1276+private:
1277+ Q_DISABLE_COPY(Synchronizer)
1278+ Database* m_source;
1279+ bool m_synchronize;
1280+ bool m_resolve_to_source;
1281+ QList <QObject*> m_local_targets;
1282+ QList <QString> m_remote_targets;
1283+ QVariant m_targets;
1284+ QList <QStringList> m_errors;
1285+
1286+ void onSyncChanged(bool synchronize);
1287+
1288+};
1289+
1290+QT_END_NAMESPACE_U1DB
1291+
1292+#endif // U1DB_SYNCHRONIZER_H
1293+

Subscribers

People subscribed via source and target branches

to all changes: