Merge lp:~kevin-wright-1/u1db-qt/synchronizer-merged-with-trunk-8-aug into lp:u1db-qt

Proposed by Kevin Wright
Status: Merged
Approved by: Cris Dywan
Approved revision: 126
Merged at revision: 100
Proposed branch: lp:~kevin-wright-1/u1db-qt/synchronizer-merged-with-trunk-8-aug
Merge into: lp:u1db-qt
Diff against target: 2776 lines (+2415/-23)
15 files modified
CMakeLists.txt (+2/-2)
documentation/u1db.qdocconf (+3/-1)
examples/u1db-qt-example-6/u1db-qt-example-6.qdoc (+387/-0)
examples/u1db-qt-example-6/u1db-qt-example-6.qml (+173/-0)
modules/U1db/CMakeLists.txt (+4/-0)
modules/U1db/plugin.cpp (+2/-0)
src/CMakeLists.txt (+5/-1)
src/database.cpp (+395/-14)
src/database.h (+19/-2)
src/query.cpp (+16/-0)
src/query.h (+2/-0)
src/synchronizer.cpp (+1257/-0)
src/synchronizer.h (+146/-0)
tests/tst_database.qml (+3/-3)
tests/tst_query.qml (+1/-0)
To merge this branch: bzr merge lp:~kevin-wright-1/u1db-qt/synchronizer-merged-with-trunk-8-aug
Reviewer Review Type Date Requested Status
PS Jenkins bot continuous-integration Approve
Cris Dywan Approve
Review via email: mp+179173@code.launchpad.net

Description of the change

This branch merges the latest trunk with the code for synchronization of databases.

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Cris Dywan (kalikiana) wrote :

The code deals with id's being wrapped in {} in places. That's based on behavior of the core code; I think we need a newid() function and solve this in one place.

There's a few instances of the pattern "if (!query.exec()) qDebug" which should probably be using setError.

resetModel can be used in more places currently still doing its equivalent calls.

getValidTargets() and synchronizeTargets() hard-code HTML formatting - that's not good if we plan to see errors in console or XML and will make those cases harder to deal with. The messages should be predictable without using markup - if needed, it's still an option to use regex.
m_errors.append("<b><font color=\"red\">Database "+index_number+"</font></b>: F

FIXMEs that ought to be sorted out:
##KW## The two lists below are currently not used for anything
##KW## maybe this can be removed as it is replaced by revision_number

Unused code:
//targetDb->updateSyncLog(true, QString uid, QString generation, QString transaction_id);
//double target_replica_generation = replyMap["target_replica_generation"].toDouble();

This should probably say the real app name + " (U1Db-Qt)"
request.setRawHeader("User-Agent", "My app name v0.1");
request.setRawHeader("X-Custom-User-Agent", "My app name v0.1");

For good measure processDataFromRemoteServer might want to qDebug if an unexpected i.key() is seen?

This should not be writable, it's only set internally
Q_PROPERTY(QList<QString> errors READ getErrors WRITE setErrors NOTIFY errorsChanged)

"where u1db has been downloaded/branched"
This is very ambiguous and doesn't say where to obtain it. It should say
"where the u1db Python reference implemented has been downloaded from lp:u1db"

review: Needs Fixing
114. By Kevin Wright

Removed an unecessary function call and code comment from src/database.cpp.

115. By Kevin Wright

Removed an unecessary code from src/synchronizer.cpp, and updated putDoc in src/database.cpp to return QString instead of int.

116. By Kevin Wright

Updated code comments in src/synchronizer.cpp to be more descriptive with respect to the instructions for how to use the plugin with the python implementation (the server part to be specific).

117. By Kevin Wright

Added 'U1Db-Qt v1.0' to the raw header information sent to the server.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 08/08/2013 16:24, skrev Christian Dywan:
> Review: Needs Fixing
>
> The code deals with id's being wrapped in {} in places. That's based on behavior of the core code; I think we need a newid() function and solve this in one place.

Agreed. Will add.

> There's a few instances of the pattern "if (!query.exec()) qDebug" which should probably be using setError.

Agreed. Will modify.
>
> resetModel can be used in more places currently still doing its equivalent calls.
Not sure what this comment means.
> getValidTargets() and synchronizeTargets() hard-code HTML formatting - that's not good if we plan to see errors in console or XML and will make those cases harder to deal with. The messages should be predictable without using markup - if needed, it's still an option to use regex.
> m_errors.append("<b><font color=\"red\">Database "+index_number+"</font></b>: F

The type of errors here are for the benefit of app devs and app users at
the UI level, not console etc. The messages are for use in a QML
application as output from a sync transation to notify re: errors,
warnings and things that went according to plan.

An alternative approach, with broader and more flexible possibilities,
is to have the error information available in a model (QVariantMap) that
includes useful meta data and values (e.g. type of message, message
text, other?). In that way it can be available to app developers as well
as easily adapted for other purposes (e.g. log files, console output).

>
> FIXMEs that ought to be sorted out:
> ##KW## The two lists below are currently not used for anything
> ##KW## maybe this can be removed as it is replaced by revision_number

Both removed.

>
> Unused code:
> //targetDb->updateSyncLog(true, QString uid, QString generation, QString transaction_id);
Could not find this one. Perhaps it was deleted in another modification.
> //double target_replica_generation = replyMap["target_replica_generation"].toDouble();

This variable is probably needed in upcoming modifications, so it is
there but commented out so it does not produce an annoying warning at
compile time.

>
> This should probably say the real app name + " (U1Db-Qt)"
> request.setRawHeader("User-Agent", "My app name v0.1");
> request.setRawHeader("X-Custom-User-Agent", "My app name v0.1");

Changed to 'U1Db-Qt v1.0' on both lines. Those two lines may not even be
necessary, as the server may ignore them, but are kept there and posted
anyway for good measure.

>
> For good measure processDataFromRemoteServer might want to qDebug if an unexpected i.key() is seen?

Not sure it would be needed. If an unexpected key showed up it would be
ignored (as it should be).

>
> This should not be writable, it's only set internally
> Q_PROPERTY(QList<QString> errors READ getErrors WRITE setErrors NOTIFY errorsChanged)

Agreed. Will modify accordingly.

> "where u1db has been downloaded/branched"
> This is very ambiguous and doesn't say where to obtain it. It should say
> "where the u1db Python reference implemented has been downloaded from lp:u1db"
Changed.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Cris Dywan (kalikiana) wrote :

> > getValidTargets() and synchronizeTargets() hard-code HTML formatting -
> that's not good if we plan to see errors in console or XML and will make those
> cases harder to deal with. The messages should be predictable without using
> markup - if needed, it's still an option to use regex.
> > m_errors.append("<b><font color=\"red\">Database
> "+index_number+"</font></b>: F
>
> The type of errors here are for the benefit of app devs and app users at
> the UI level, not console etc. The messages are for use in a QML
> application as output from a sync transation to notify re: errors,
> warnings and things that went according to plan.
>
> An alternative approach, with broader and more flexible possibilities,
> is to have the error information available in a model (QVariantMap) that
> includes useful meta data and values (e.g. type of message, message
> text, other?). In that way it can be available to app developers as well
> as easily adapted for other purposes (e.g. log files, console output).

Okay, I better see the reasoning now. And I think you're mixing two concepts into here, UI error messages and error handling API for letting developers distinguish errors. The user must not see verbatim key names. And the app developer needs a way to distinguish a failing sync from a bug in his code.

I had a thought in that direction in respect to Database::setError originally. SQL errors aren't interesting to anyone but U1Db-Qt developers - they need to be simple and pastable. The same is often true for U1Db-Qt errors. Errors about invalid id's or json are indeed relevant for developers - but only in the console. There's no need to "handle" them, wrong use of the API is to be fixed. One kind of error that we should have API for is: no space left. I apparently never filed a bug for that.

Now for sync, I'd still want the same console output for obvious mistakes like invalid URLs and wrong key names. That is what I was having in mind when I suggested to not have HTML. It's a one-time see the mistake and fix it and not for UI. And it is useful to always log if a connection failed in any way. For the developer's benefit we should have "signal syncFailed". That should suffice to prompt the user appropriately - the developer knows their target audience, what language to use etc.

> > For good measure processDataFromRemoteServer might want to qDebug if an
> unexpected i.key() is seen?
>
> Not sure it would be needed. If an unexpected key showed up it would be
> ignored (as it should be).

I should clarify: if an unexpected key turns up that's a bug in the code. Q_ASSERT is better I guess.

review: Needs Fixing
Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 08/08/2013 17:32, skrev Christian Dywan:
> Review: Needs Fixing
>
>
>>> For good measure processDataFromRemoteServer might want to qDebug if an
>> unexpected i.key() is seen?
>>
>> Not sure it would be needed. If an unexpected key showed up it would be
>> ignored (as it should be).
> I should clarify: if an unexpected key turns up that's a bug in the code. Q_ASSERT is better I guess.

In fact there are other key and value pairs that the server does return
that are not yet being used by the Synchronizer class.

Of course it is harmless to notify that there is more information than
expected (or in this case information that is expected but not being
used), and discounting the possibility that at some point in time there
might be a benefit to doing so would not be wise.

For the record though, in this particular case, if there were an
unexpected key (and associated value) that turns up it is because it was
information delivered directly from the server.

Kevin

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :
Download full text (3.9 KiB)

Den 08/08/2013 17:32, skrev Christian Dywan:
> Review: Needs Fixing
>
>>> getValidTargets() and synchronizeTargets() hard-code HTML formatting -
>> that's not good if we plan to see errors in console or XML and will make those
>> cases harder to deal with. The messages should be predictable without using
>> markup - if needed, it's still an option to use regex.
>>> m_errors.append("<b><font color=\"red\">Database
>> "+index_number+"</font></b>: F
>>
>> The type of errors here are for the benefit of app devs and app users at
>> the UI level, not console etc. The messages are for use in a QML
>> application as output from a sync transation to notify re: errors,
>> warnings and things that went according to plan.
>>
>> An alternative approach, with broader and more flexible possibilities,
>> is to have the error information available in a model (QVariantMap) that
>> includes useful meta data and values (e.g. type of message, message
>> text, other?). In that way it can be available to app developers as well
>> as easily adapted for other purposes (e.g. log files, console output).
> Okay, I better see the reasoning now. And I think you're mixing two concepts into here, UI error messages and error handling API for letting developers distinguish errors. The user must not see verbatim key names. And the app developer needs a way to distinguish a failing sync from a bug in his code.

The messages can and should be modified. But the app developer does not
need to necessarily use the log/error/warning output from sync in its
raw form either. It should be up to the developer to determine how they
want to work with the data.

>
> I had a thought in that direction in respect to Database::setError originally. SQL errors aren't interesting to anyone but U1Db-Qt developers - they need to be simple and pastable. The same is often true for U1Db-Qt errors. Errors about invalid id's or json are indeed relevant for developers - but only in the console. There's no need to "handle" them, wrong use of the API is to be fixed. One kind of error that we should have API for is: no space left. I apparently never filed a bug for that.
>
> Now for sync, I'd still want the same console output for obvious mistakes like invalid URLs and wrong key names. That is what I was having in mind when I suggested to not have HTML. It's a one-time see the mistake and fix it and not for UI. And it is useful to always log if a connection failed in any way. For the developer's benefit we should have "signal syncFailed". That should suffice to prompt the user appropriately - the developer knows their target audience, what language to use etc.

Assuming too much about what the app developer wants, the endless
possible types of errors that might be encountered (e.g. faulty app
logic, user input variations, API problems), or who their audience is,
along rigid lines (and then use that logic for designing the API), may
be asking for trouble. What one developer to the next might want (or
need) could be different enough that "signal syncFailed" isn't enough.
This is why I suggested having a much more flexible and less
discriminating approach of making the data vailable using QVarientMap,...

Read more...

Revision history for this message
Cris Dywan (kalikiana) wrote :

> Assuming too much about what the app developer wants, the endless
> possible types of errors that might be encountered (e.g. faulty app
> logic, user input variations, API problems), or who their audience is,
> along rigid lines (and then use that logic for designing the API), may
> be asking for trouble. What one developer to the next might want (or
> need) could be different enough that "signal syncFailed" isn't enough.
> This is why I suggested having a much more flexible and less
> discriminating approach of making the data vailable using QVarientMap,
> with meta data included so u1db dev and app dev can use the information
> in ways they choose, for the ultimate benefit of the end user.
>
> The existing approach is indeed immature in that respect, but the
> suggestion of using a model (QVarientMap) moves beyond current
> limitations and is flexible enough for a broad set of use cases, some of
> which we cannot anticipate on our own, but without creating huge overhead.
>
> It is intended to provide a rich mechanism for developers to create
> robust applications for users that do more than simply give feedback
> that only says "fail". Additionally it can hopefully provide useful
> information for u1db-qt devs as well (who also might happen to build
> apps with the plugin...including examples).

syncFailed was an example only. And I stress free-form strings are not a basis for proper error handling API.

At this point I'd prefer to only use simple qDebug() calls and leave proper error API for a separate task. I created bugs for this: bug 1210448 and bug 1210450.

review: Needs Fixing
Revision history for this message
Cris Dywan (kalikiana) wrote :

These are the doc errors I'm currently seeing with the latest changes. They need to be resolve before the branch can be merged:

./src/database.cpp:427: warning: Unknown command '\param'
 [Maybe you meant '\part'?]
./src/database.cpp:428: warning: Unknown command '\param'
 [Maybe you meant '\part'?]
./src/database.cpp:429: warning: Unknown command '\param'
 [Maybe you meant '\part'?]
./src/synchronizer.cpp:33: warning: Cannot find Synchronizer specified with \class in any header file
./src/synchronizer.cpp:156: warning: Cannot find Synchronizer::source specified with \property in any header file
./src/synchronizer.cpp:173: warning: Cannot find Synchronizer::targets specified with \property in any header file
./src/synchronizer.cpp:189: warning: Cannot find Synchronizer::synchronize specified with \property in any header file
./src/synchronizer.cpp:202: warning: Cannot find Synchronizer::errors specified with \property in any header file
./src/synchronizer.cpp:214: warning: Cannot find Synchronizer::errors specified with \property in any header file
./src/synchronizer.cpp:358: warning: Unknown command '\param'
 [Maybe you meant '\part'?]
./src/synchronizer.cpp:359: warning: Unknown command '\param'
 [Maybe you meant '\part'?]
./src/synchronizer.cpp:360: warning: Unknown command '\return'
./src/synchronizer.cpp:783: warning: Unknown command '\param'
 [Maybe you meant '\part'?]
./src/synchronizer.cpp:817: warning: Unknown command '\param'
 [Maybe you meant '\part'?]
./src/synchronizer.cpp:818: warning: Unknown command '\param'
 [Maybe you meant '\part'?]
./src/synchronizer.cpp:889: warning: Unknown command '\param'
 [Maybe you meant '\part'?]
./src/database.cpp:398: warning: Undocumented parameter doc_id in Database::getCurrentDocRevisionNumber()
./src/database.cpp:255: warning: No documentation for 'Database::getDocumentContents()'
./src/database.cpp:324: warning: Undocumented parameter doc_id in Database::getNextDocRevisionNumber()
./src/database.cpp:818: warning: No documentation for 'Database::getSyncLogInfo()'
./src/database.cpp:792: warning: No documentation for 'Database::listTransactionsSince()'
./src/database.cpp:619: warning: No documentation for 'Database::resetModel()'
./src/database.cpp:463: warning: No documentation for 'Database::updateDocRevisionNumber()'
./src/database.cpp:425: warning: Undocumented parameter transaction_id in Database::updateSyncLog()
./src/database.cpp:425: warning: Undocumented parameter generation in Database::updateSyncLog()
./src/database.cpp:425: warning: Undocumented parameter insert in Database::updateSyncLog()
./src/database.cpp:425: warning: Undocumented parameter uid in Database::updateSyncLog()
./src/query.cpp:161: warning: No documentation for 'Query::resetModel()'

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

[snip]
>
> The existing approach is indeed immature in that respect, but the
> suggestion of using a model (QVarientMap) moves beyond current
> limitations and is flexible enough for a broad set of use cases, some of
> which we cannot anticipate on our own, but without creating huge overhead.
>
> It is intended to provide a rich mechanism for developers to create
> robust applications for users that do more than simply give feedback
> that only says "fail". Additionally it can hopefully provide useful
> information for u1db-qt devs as well (who also might happen to build
> apps with the plugin...including examples).
> syncFailed was an example only. And I stress free-form strings are not a basis for proper error handling API.
>
> At this point I'd prefer to only use simple qDebug() calls and leave proper error API for a separate task. I created bugs for this: bug 1210448 and bug 1210450.

I modified things already.

I renamed it sync_output instead of errors (since not everything was an
error), and changed it so that a QVariantMap is returned (rather than
QString) with various meta data, depending on the sync output. There is
a descriptive message for each sync_output record, but it comes along
with hard data, such as what property is related to the message, any
particulars about the database (e.g. name, its index within the targets
definition). Additionally the messages themselves have been modified,
with some former details that were included in the description pushed to
the various meta-data instead. A set of error IDs could be created as well.

The information can be used at the application level, or used by methods
within the plugin itself (e.g. console output, log file).

I'll push those changes since they over write the original error
handling that needed removal/modification.

Kevin

118. By Kevin Wright

Modified 'errors'. Changed to 'sync_output' (since not everything was an error); changed from QString to QVariantMap that includes more meta-data; and created more generic output descriptions (i.e. removed most hard data and placed into meta-data instead). This approach is still primarily intented as a mechanism for application developers to troubleshoot (or simply keep themselves/users informed of background activity), but can be utilized within the plugin itself as well (e.g. log files, console output). It does not presently include error codes (where appropriate) but should, and additionally it needs to be documented so people can take advantage of it and use it properly. The example u1db-qt-example-6.qml uses it as of this commit, but is only applying the output message and not any of the meta-data.

Revision history for this message
Cris Dywan (kalikiana) wrote :

> The information can be used at the application level, or used by methods
> within the plugin itself (e.g. console output, log file).
>
> I'll push those changes since they over write the original error
> handling that needed removal/modification.

Nice stuff actually. Much cleaner than I expected! It should lend itself extremely nicely for test cases as well. I wouldn't have expected this in this MR but I'm not going to complain now ;-)

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
119. By Kevin Wright

Modified several source files to fix warnings from qdoc.

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :
Download full text (3.2 KiB)

Den 09/08/2013 11:20, skrev Christian Dywan:
> These are the doc errors I'm currently seeing with the latest changes. They need to be resolve before the branch can be merged:
>
> ./src/database.cpp:427: warning: Unknown command '\param'
> [Maybe you meant '\part'?]
> ./src/database.cpp:428: warning: Unknown command '\param'
> [Maybe you meant '\part'?]
> ./src/database.cpp:429: warning: Unknown command '\param'
> [Maybe you meant '\part'?]
> ./src/synchronizer.cpp:33: warning: Cannot find Synchronizer specified with \class in any header file
> ./src/synchronizer.cpp:156: warning: Cannot find Synchronizer::source specified with \property in any header file
> ./src/synchronizer.cpp:173: warning: Cannot find Synchronizer::targets specified with \property in any header file
> ./src/synchronizer.cpp:189: warning: Cannot find Synchronizer::synchronize specified with \property in any header file
> ./src/synchronizer.cpp:202: warning: Cannot find Synchronizer::errors specified with \property in any header file
> ./src/synchronizer.cpp:214: warning: Cannot find Synchronizer::errors specified with \property in any header file
> ./src/synchronizer.cpp:358: warning: Unknown command '\param'
> [Maybe you meant '\part'?]
> ./src/synchronizer.cpp:359: warning: Unknown command '\param'
> [Maybe you meant '\part'?]
> ./src/synchronizer.cpp:360: warning: Unknown command '\return'
> ./src/synchronizer.cpp:783: warning: Unknown command '\param'
> [Maybe you meant '\part'?]
> ./src/synchronizer.cpp:817: warning: Unknown command '\param'
> [Maybe you meant '\part'?]
> ./src/synchronizer.cpp:818: warning: Unknown command '\param'
> [Maybe you meant '\part'?]
> ./src/synchronizer.cpp:889: warning: Unknown command '\param'
> [Maybe you meant '\part'?]
> ./src/database.cpp:398: warning: Undocumented parameter doc_id in Database::getCurrentDocRevisionNumber()
> ./src/database.cpp:255: warning: No documentation for 'Database::getDocumentContents()'
> ./src/database.cpp:324: warning: Undocumented parameter doc_id in Database::getNextDocRevisionNumber()
> ./src/database.cpp:818: warning: No documentation for 'Database::getSyncLogInfo()'
> ./src/database.cpp:792: warning: No documentation for 'Database::listTransactionsSince()'
> ./src/database.cpp:619: warning: No documentation for 'Database::resetModel()'
> ./src/database.cpp:463: warning: No documentation for 'Database::updateDocRevisionNumber()'
> ./src/database.cpp:425: warning: Undocumented parameter transaction_id in Database::updateSyncLog()
> ./src/database.cpp:425: warning: Undocumented parameter generation in Database::updateSyncLog()
> ./src/database.cpp:425: warning: Undocumented parameter insert in Database::updateSyncLog()
> ./src/database.cpp:425: warning: Undocumented parameter uid in Database::updateSyncLog()
> ./src/query.cpp:161: warning: No documentation for 'Query::resetModel()'

Hopefully all fixed now. Could not find anything specific to resolve the
warnings about \property, but maybe this is resolved by the change to
fix the warning about \class.

Also \param is used elsewhere in source files, but perhaps was not used
properly in these instances. The references have been remov...

Read more...

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 08/08/2013 16:24, skrev Christian Dywan:
> Review: Needs Fixing
>
> The code deals with id's being wrapped in {} in places. That's based on behavior of the core code; I think we need a newid() function and solve this in one place.
>
Have not done anything about this. I'm not sure it needs attending to
ahead of higher priority items.
> There's a few instances of the pattern "if (!query.exec()) qDebug" which should probably be using setError.

Have not added any new qDebug messages yet.
>
> resetModel can be used in more places currently still doing its equivalent calls.
Still unsure about this comment -- maybe it was mentioned in another thread.

>
> getValidTargets() and synchronizeTargets() hard-code HTML formatting - that's not good if we plan to see errors in console or XML and will make those cases harder to deal with. The messages should be predictable without using markup - if needed, it's still an option to use regex.
> m_errors.append("<b><font color=\"red\">Database "+index_number+"</font></b>: F

Discussed in another thread. Has been changed completely. No more markup
is included in output messages.

>
> FIXMEs that ought to be sorted out:
> ##KW## The two lists below are currently not used for anything
> ##KW## maybe this can be removed as it is replaced by revision_number
Already removed.
>
> Unused code:
> //targetDb->updateSyncLog(true, QString uid, QString generation, QString transaction_id);
Already removed?
> //double target_replica_generation = replyMap["target_replica_generation"].toDouble();
Already commented on this -- variable might be used in further
modifications. Commented out to avoid seeing warning during builds.

> This should probably say the real app name + " (U1Db-Qt)"
> request.setRawHeader("User-Agent", "My app name v0.1");
> request.setRawHeader("X-Custom-User-Agent", "My app name v0.1");
Already fixed.
>
> For good measure processDataFromRemoteServer might want to qDebug if an unexpected i.key() is seen?

Have not touched this -- will add an else clause.

>
> This should not be writable, it's only set internally
> Q_PROPERTY(QList<QString> errors READ getErrors WRITE setErrors NOTIFY errorsChanged)

Modified to sync_output, QList<QVariant> and read only (commit message
has more details).

>
> "where u1db has been downloaded/branched"
> This is very ambiguous and doesn't say where to obtain it. It should say
> "where the u1db Python reference implemented has been downloaded from lp:u1db"
Already fixed.

Kevin

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 09/08/2013 12:50, skrev Christian Dywan:
>> The information can be used at the application level, or used by methods
>> within the plugin itself (e.g. console output, log file).
>>
>> I'll push those changes since they over write the original error
>> handling that needed removal/modification.
> Nice stuff actually. Much cleaner than I expected! It should lend itself extremely nicely for test cases as well. I wouldn't have expected this in this MR but I'm not going to complain now ;-)

It was actually quite easy to implement. Took less than 30 minutes to
put together.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
120. By Kevin Wright

Modified database.cpp so that all query.exec() statements have either qDebug output of the error message, or are covered by setError. Eventually error handling should be more consistant, and perhaps utilize the same mechanism as sync_output in the Synchronizer class. Some functions in Database are used side by side with other functions in Synchronizer, so perhaps this is a wise idea where applicable.

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

>> There's a few instances of the pattern "if (!query.exec()) qDebug"
>> which should probably be using setError.
>
> Have not added any new qDebug messages yet.

Whoops. Misread this one. Added qDebug statements where there was
nothing. Will fix to use setError.

Kevin

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 09/08/2013 13:48, skrev Kevin Wright:
>>> There's a few instances of the pattern "if (!query.exec()) qDebug"
>>> which should probably be using setError.
>> Have not added any new qDebug messages yet.
> Whoops. Misread this one. Added qDebug statements where there was
> nothing. Will fix to use setError.
>
> Kevin
>

Hopefully all done now.

Kevin

121. By Kevin Wright

Modified database.cpp so that all query.exec() statements are covered by setError. Unfortunately setSerror cannot be the return value in some places, because the method is void or something other than a string.

122. By Kevin Wright

Modified database.cpp to fix a small bug re: setError.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Cris Dywan (kalikiana) wrote :

> Den 09/08/2013 11:20, skrev Christian Dywan:
> > These are the doc errors I'm currently seeing with the latest changes. They
> need to be resolve before the branch can be merged:
> Hopefully all fixed now. Could not find anything specific to resolve the
> warnings about \property, but maybe this is resolved by the change to
> fix the warning about \class.
>
> Also \param is used elsewhere in source files, but perhaps was not used
> properly in these instances. The references have been removed in the
> latest commit, but perhaps this needs to be looked at again.

There's still 17 qdoc warnings. See https://jenkins.qa.ubuntu.com/job/u1db-qt-quantal-amd64-ci/7/console I locally see the same.

review: Needs Fixing
123. By Kevin Wright

Changed qdoc markup class to qmlclass in synchronizer.cpp.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
124. By Kevin Wright

Fixed qdoc warnings.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Cris Dywan (kalikiana) wrote :

I get a unit test failure with the latest update:

  <testcase result="fail" name="U1dbDatabase::test_1_databasePopulated">
    <failure message="Compared values are not the same
   Actual (): false
   Expected (): true" result="fail"/>
  </testcase>

Running "qmltestrunner -import ./_build/modules -input ./tests" gives me the line number:

tests/tst_database.qml(75)

review: Needs Fixing
Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 12/08/2013 11:26, skrev Christian Dywan:
> Review: Needs Fixing
>
> I get a unit test failure with the latest update:
>
> <testcase result="fail" name="U1dbDatabase::test_1_databasePopulated">
> <failure message="Compared values are not the same
> Actual (): false
> Expected (): true" result="fail"/>
> </testcase>
>
> Running "qmltestrunner -import ./_build/modules -input ./tests" gives me the line number:
>
> tests/tst_database.qml(75)

The test message seems ambiguous.

Compared values of what?

Kevin

Revision history for this message
Cris Dywan (kalikiana) wrote :

There's also Jenkins brokenness with precise:
  Err http://ppa.launchpad.net precise/main i386 Packages
  404 Not Found

Not at all related to u1db-qt - I'm investigating what we can do there.

Revision history for this message
Cris Dywan (kalikiana) wrote :

> Den 12/08/2013 11:26, skrev Christian Dywan:
> > Review: Needs Fixing
> >
> > I get a unit test failure with the latest update:
> >
> > <testcase result="fail" name="U1dbDatabase::test_1_databasePopulated">
> > <failure message="Compared values are not the same
> > Actual (): false
> > Expected (): true" result="fail"/>
> > </testcase>
> >
> > Running "qmltestrunner -import ./_build/modules -input ./tests" gives me the
> line number:
> >
> > tests/tst_database.qml(75)
>
> The test message seems ambiguous.
>
> Compared values of what?

Tell me about it, hte default messages of qmltestrunner suck :-(
But the line is:

compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]}) > -1, true)

So that translates to "Expected -1 but got something else from putDoc()"

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 12/08/2013 11:35, skrev Christian Dywan:
>> Den 12/08/2013 11:26, skrev Christian Dywan:
>>> Review: Needs Fixing
>>>
>>> I get a unit test failure with the latest update:
>>>
>>> <testcase result="fail" name="U1dbDatabase::test_1_databasePopulated">
>>> <failure message="Compared values are not the same
>>> Actual (): false
>>> Expected (): true" result="fail"/>
>>> </testcase>
>>>
>>> Running "qmltestrunner -import ./_build/modules -input ./tests" gives me the
>> line number:
>>> tests/tst_database.qml(75)
>> The test message seems ambiguous.
>>
>> Compared values of what?
> Tell me about it, hte default messages of qmltestrunner suck :-(
> But the line is:
>
> compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]}) > -1, true)
>
> So that translates to "Expected -1 but got something else from putDoc()"
Yep, found it. putDoc returns a QString, but the test expects bool.

Not sure how to modify the test properly because the return value is
based on the UID of the database, which will be different with each new
database created -- in addition the function that retrieves the database
UID is private, so unless that is made public the only thing that could
be tested for is whether the return string is empty or not. Well, if the
return string contains ":1" that could also be tested for -- not quite a
good test but perhaps better than nothing.

Kevin

Revision history for this message
Cris Dywan (kalikiana) wrote :

> > compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]}) > -1,
> true)
> >
> > So that translates to "Expected -1 but got something else from putDoc()"
> Yep, found it. putDoc returns a QString, but the test expects bool.
>
> Not sure how to modify the test properly because the return value is
> based on the UID of the database, which will be different with each new
> database created -- in addition the function that retrieves the database
> UID is private, so unless that is made public the only thing that could
> be tested for is whether the return string is empty or not. Well, if the
> return string contains ":1" that could also be tested for -- not quite a
> good test but perhaps better than nothing.

It sounds sensible to check the empty string - the test only checks if it succeeds or not at the moment. I would leave it to a separate task to extend it to see if that id is valid, works, etc.

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 12/08/2013 11:55, skrev Christian Dywan:
>>> compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]}) > -1,
>> true)
>>> So that translates to "Expected -1 but got something else from putDoc()"
>> Yep, found it. putDoc returns a QString, but the test expects bool.
>>
>> Not sure how to modify the test properly because the return value is
>> based on the UID of the database, which will be different with each new
>> database created -- in addition the function that retrieves the database
>> UID is private, so unless that is made public the only thing that could
>> be tested for is whether the return string is empty or not. Well, if the
>> return string contains ":1" that could also be tested for -- not quite a
>> good test but perhaps better than nothing.
> It sounds sensible to check the empty string - the test only checks if it succeeds or not at the moment. I would leave it to a separate task to extend it to see if that id is valid, works, etc.
I misunderstood slightly what the original test was looking for...it was
checking for an int actually, and whether it was > -1. In any case I
have changed it to check the return string and whether != empty string.

The qmltestrunner program seems to hang sometimes. It is difficult to
know if it is running or not. Is that normal?

Kevin

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 12/08/2013 12:02, skrev <email address hidden>:
> Den 12/08/2013 11:55, skrev Christian Dywan:
>>>> compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]}) >
>>>> -1,
>>> true)
>>>> So that translates to "Expected -1 but got something else from
>>>> putDoc()"
>>> Yep, found it. putDoc returns a QString, but the test expects bool.
>>>
>>> Not sure how to modify the test properly because the return value is
>>> based on the UID of the database, which will be different with each new
>>> database created -- in addition the function that retrieves the
>>> database
>>> UID is private, so unless that is made public the only thing that could
>>> be tested for is whether the return string is empty or not. Well, if
>>> the
>>> return string contains ":1" that could also be tested for -- not
>>> quite a
>>> good test but perhaps better than nothing.
>> It sounds sensible to check the empty string - the test only checks
>> if it succeeds or not at the moment. I would leave it to a separate
>> task to extend it to see if that id is valid, works, etc.
> I misunderstood slightly what the original test was looking for...it
> was checking for an int actually, and whether it was > -1. In any case
> I have changed it to check the return string and whether != empty string.
>
> The qmltestrunner program seems to hang sometimes. It is difficult to
> know if it is running or not. Is that normal?
>
> Kevin
>
>
Checking for an empty string does not appear to be working no matter
which way I try. Will keep at it.

However, it also appears that there is a check for the file name of the
database. The test is only the name of the file itself, but the value
recieved appears to be the absolute path.

Kevin

Revision history for this message
Kevin Wright (kevin-wright-1) wrote :

Den 12/08/2013 12:22, skrev <email address hidden>:
> Den 12/08/2013 12:02, skrev <email address hidden>:
>> Den 12/08/2013 11:55, skrev Christian Dywan:
>>>>> compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]})
>>>>> > -1,
>>>> true)
>>>>> So that translates to "Expected -1 but got something else from
>>>>> putDoc()"
>>>> Yep, found it. putDoc returns a QString, but the test expects bool.
>>>>
>>>> Not sure how to modify the test properly because the return value is
>>>> based on the UID of the database, which will be different with each
>>>> new
>>>> database created -- in addition the function that retrieves the
>>>> database
>>>> UID is private, so unless that is made public the only thing that
>>>> could
>>>> be tested for is whether the return string is empty or not. Well,
>>>> if the
>>>> return string contains ":1" that could also be tested for -- not
>>>> quite a
>>>> good test but perhaps better than nothing.
>>> It sounds sensible to check the empty string - the test only checks
>>> if it succeeds or not at the moment. I would leave it to a separate
>>> task to extend it to see if that id is valid, works, etc.
>> I misunderstood slightly what the original test was looking for...it
>> was checking for an int actually, and whether it was > -1. In any
>> case I have changed it to check the return string and whether !=
>> empty string.
>>
>> The qmltestrunner program seems to hang sometimes. It is difficult to
>> know if it is running or not. Is that normal?
>>
>> Kevin
>>
>>
> Checking for an empty string does not appear to be working no matter
> which way I try. Will keep at it.
>
> However, it also appears that there is a check for the file name of
> the database. The test is only the name of the file itself, but the
> value recieved appears to be the absolute path.

Scratch this last one. Seems to be a misunderstanding.

>
> Kevin

Revision history for this message
Cris Dywan (kalikiana) wrote :

> There's also Jenkins brokenness with precise:
> Err http://ppa.launchpad.net precise/main i386 Packages
> 404 Not Found
>
> Not at all related to u1db-qt - I'm investigating what we can do there.

FYI A work-around for the issue is in place (real fix will take longer). The next automated build should run fine.

125. By Kevin Wright

Modified tests. In one instance a query property was missing from a Query element (in tst_query.qml), and in another occurance the test needed to be updated to reflect a different return value (in test_database.qml).

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
126. By Kevin Wright

Fixed one test to remove a warning.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Cris Dywan (kalikiana) wrote :

ERROR: No artifacts found that match the file pattern "**/*test*.xml". Configuration error?
ERROR: '**/*test*.xml' doesn't match anything: '**' exists but not '**/*test*.xml'

I'm seeing to get this sorted. As far as I'm concerned, the build, tests and docs pass and this is down to Jenkins.

review: Approve
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'CMakeLists.txt'
2--- CMakeLists.txt 2013-07-31 12:22:38 +0000
3+++ CMakeLists.txt 2013-08-12 16:20:45 +0000
4@@ -8,11 +8,12 @@
5 include(FindPkgConfig)
6 include(GNUInstallDirs)
7 find_package(Qt5Core REQUIRED)
8+find_package(Qt5Network REQUIRED)
9 find_package(Qt5Sql REQUIRED)
10 add_definitions(-DWITHQT5=1)
11
12 set(U1DB_QT_LIBNAME u1db-qt5)
13-set(QT_PKGCONFIG_DEPENDENCIES "Qt5Core Qt5Quick Qt5Sql")
14+set(QT_PKGCONFIG_DEPENDENCIES "Qt5Core Qt5Network Qt5Quick Qt5Sql")
15 set(QT_U1DB_PKGCONFIG_FILE lib${U1DB_QT_LIBNAME}.pc)
16
17 # Build flags
18@@ -42,4 +43,3 @@
19 install(FILES ${CMAKE_CURRENT_BINARY_DIR}/${QT_U1DB_PKGCONFIG_FILE}
20 DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig
21 )
22-
23
24=== modified file 'documentation/u1db.qdocconf'
25--- documentation/u1db.qdocconf 2013-05-13 12:24:31 +0000
26+++ documentation/u1db.qdocconf 2013-08-12 16:20:45 +0000
27@@ -9,6 +9,7 @@
28 ../src/document.h \
29 ../src/index.h \
30 ../src/query.h \
31+ ../src/synchronizer.h \
32 ../src/global.h
33 Cpp.ignoretokens = Q_DECL_EXPORT \
34 Q_PROPERTY \
35@@ -25,7 +26,8 @@
36 ../examples/u1db-qt-example-2b \
37 ../examples/u1db-qt-example-3 \
38 ../examples/u1db-qt-example-4 \
39-../examples/u1db-qt-example-5
40+../examples/u1db-qt-example-5 \
41+../examples/u1db-qt-example-6
42
43 imagedirs = images
44
45
46=== added directory 'examples/u1db-qt-example-6'
47=== added file 'examples/u1db-qt-example-6/u1db-qt-example-6.qdoc'
48--- examples/u1db-qt-example-6/u1db-qt-example-6.qdoc 1970-01-01 00:00:00 +0000
49+++ examples/u1db-qt-example-6/u1db-qt-example-6.qdoc 2013-08-12 16:20:45 +0000
50@@ -0,0 +1,387 @@
51+/*!
52+
53+\page u1db-qt-tutorial-6.html
54+
55+\title U1Db-Qt Synchronizing Tutorial
56+
57+This tutorial is designed to demonstrate a variety of essential U1Db-Qt functionality and usage, including:
58+
59+\list 1
60+ \li Synchronizing two databases
61+ \li Utilizing the U1db-Qt Index element
62+ \li Various approaches to define U1db-Qt Document elements when using the Index element
63+ \li Partnering the U1db-Qt Index element and a QML ListView element
64+\endlist
65+
66+\section1 Storing Data
67+
68+\section2 The Database Element
69+
70+\section3 Creating a Database
71+
72+A Database is very simple to create. It only needs an id and a path where the file will be created. A Database is a model, which can be used by elements, such as the ListView further in this example.
73+
74+\code
75+U1db.Database {
76+ id: aDatabase
77+ path: "aDatabase4"
78+}
79+\endcode
80+
81+\section1 The Document Element
82+
83+\section2 Declaring Documents (at Runtime)
84+
85+A Document can be instantiated at runtime, or generated dynamically. The examples below demonstrate the former.
86+
87+A very basic Document could include its unique 'id' and 'docId' properties. While it is not mandatory to define these properties, in some cases they can be convenient references. More advanced applications would likely find these very useful, and in some cases may be an absolute necessity to achieve the objectives of the program.
88+
89+This example of a very simple Document will not initially do anything, until more properties are added and defined:
90+
91+\code
92+U1db.Document {
93+ id: aDocument1
94+ docId: 'helloworld1'
95+}
96+\endcode
97+
98+A basic but still practical Document definition contains several essential properties. In addition to 'id' and 'docId' (discussed above), the 'database', 'create', and 'defaults' properties are also very important, and are introduced below.
99+
100+The 'database' property ensures that the Document is attached to an already defined (or possibly soon to be defined one) identified by its id (in this case 'aDatabase'). For example:
101+
102+\code
103+U1db.Document {
104+ id: aDocument1
105+ database: aDatabase
106+ docId: 'helloworld1'
107+}
108+\endcode
109+
110+Should the Database not already contain a Document with the same docId ('hellowworld1' in this example) when a 'create' property is present and set to true it will be generated. For example:
111+
112+\code
113+U1db.Document {
114+ id: aDocument1
115+ database: aDatabase
116+ docId: 'helloworld1'
117+ create: true
118+}
119+\endcode
120+
121+However, the Document still requires some data to be useful, which is what the 'defaults' property provides. The value of 'defaults' is a map of data that will be stored in the database (again when the create property is et to true). It contain key:value pairs, where the value can be a string, number, or nested object (e.g. additional fields, lists). For example:
122+
123+\code
124+U1db.Document {
125+ id: aDocument1
126+ database: aDatabase
127+ docId: 'helloworld1'
128+ create: true
129+ defaults:{"hello": { "world": { "message":"Hello World", "id": 1 } } }
130+}
131+\endcode
132+
133+As mentioned above, lists can also be nested in Document data. Lists provide a convenient method for producing multiple instances of the same key (AKA 'field' or 'sub-field'). The example code below shows valid use of the 'message' and 'id' sub-fields multiple times within the same object.
134+
135+\code
136+U1db.Document {
137+ id: aDocument2
138+ database: aDatabase
139+ docId: 'helloworld2'
140+ create: true
141+ defaults:{"hello": { "world": [
142+ { "message":"Hello World", "id": 2 },
143+ { "message":"Hello World", "id": 2.5 }
144+ ] } }
145+}
146+\endcode
147+
148+When the default Javascript Object Notation itself is formatted with appropriate line breaks and indentation, it becomes easier to visualize an embedded list, containing sub-fields 'message' and 'id' (and their respective values):
149+
150+\code
151+{"hello":
152+ { "world":
153+ [
154+ { "message":"Hello World", "id": 2 },
155+ { "message":"Hello World", "id": 2.5 }
156+ ]
157+ }
158+}
159+\endcode
160+
161+In dot notation these sub-fields are represented by 'hello.world.message' and 'hello.world.id' respectively. Later in this tutorial these will be utilized within the 'expression' property of U1Db-Qt's Index element, in close collaboration with a QML ListView's delegates.
162+
163+
164+Normally when a docId already exists in a database, and when the set flag is set to true, the value in 'defaults' will be ignored (and the existing data in the database will remain untouched). Sometimes a developer needs to easily overwrite the data in an existing document. The 'contents' property can be used for just that purpose. When 'contents' is defined, its value will replace existing data in the database, for the document identified by the docId. In addition, 'contents' can be used to add new documents, in the same way as the 'create: true' + 'defaults' combination does; in other words, if the document defined by 'docId' does not exist it will be created.
165+
166+\code
167+U1db.Document {
168+ id: aDocument3
169+ database: aDatabase
170+ docId: 'helloworld3'
171+ contents:{"hello": { "world": [
172+ { "message":"Hello World", "id": 3 },
173+ { "message":"Hello World", "id": 3.33 },
174+ { "message":"Hello World", "id": 3.66 }
175+ ] } }
176+}
177+\endcode
178+
179+If 'defaults' exists, 'create' is set to 'true' (or 'false' for that matter) and 'contents' is also defined, it is the latter that takes precidence. In other words, 'create' and 'defaults' will be ignored. The following example demonstrates this scenario:
180+
181+\code
182+U1db.Document {
183+ id: aDocument3
184+ database: aDatabase
185+ docId: 'helloworld3'
186+ create: true
187+ default:{"hello": { "world": [{ "message":"Hello World", "id": 3 }] } }
188+ contents:{"hello": { "world": [
189+ { "message":"Hello World", "id": 3 },
190+ { "message":"Hello World", "id": 3.33 },
191+ { "message":"Hello World", "id": 3.66 }
192+ ] } }
193+}
194+\endcode
195+
196+This snippet simply represents the absence of the 'create' property, which is synonymous with 'create: false'. The Document can still be recognized within the application, but until applicable properties (such as those outlined above) are added and/or modified then nothing will be added or modified in the database, and this instance may have very little practical value.
197+
198+\code
199+U1db.Document {
200+ id: aDocument4
201+ database: aDatabase
202+ docId: 'helloworld4'
203+ defaults:{"hello": { "world": { "message":"Hello World", "id": 4 } } }
204+}
205+\endcode
206+
207+\section3 Samples of Stored Documents
208+
209+The data stored in the database after defining the above Document elements (and then running the application, will consist of the following:
210+
211+\table
212+\header
213+ \li docId
214+ \li content
215+\row
216+
217+ \li 'helloworld1'
218+ \li
219+\code
220+{
221+ "hello": {
222+ "world": {
223+ "id": 1,
224+ "message": "Hello World"
225+ }
226+ }
227+}
228+
229+\endcode
230+
231+
232+\row
233+
234+ \li 'helloworld2'
235+ \li
236+\code
237+{
238+ "hello": {
239+ "world": [
240+ {
241+ "id": 2,
242+ "message": "Hello World"
243+ },
244+ {
245+ "id": 2.5,
246+ "message": "Hello World"
247+ }
248+ ]
249+ }
250+}
251+\endcode
252+
253+\row
254+
255+ \li 'helloworld3'
256+ \li
257+\code
258+{
259+ "hello": {
260+ "world": [
261+ {
262+ "id": 3,
263+ "message": "Hello World"
264+ },
265+ {
266+ "id": 3.33,
267+ "message": "Hello World"
268+ },
269+ {
270+ "id": 3.66,
271+ "message": "Hello World"
272+ }
273+ ]
274+ }
275+}
276+\endcode
277+
278+
279+\endtable
280+
281+\section1 Retrieving Data
282+
283+To retrieve the Documents that were declared earlier requires two additional elements: Index and Query.
284+
285+\section2 The Index Element
286+
287+\section3 Creating and Index Element
288+
289+The Index element requires both a unique 'id' and a pointer to a 'database' in order to begin becoming useful, as demonstrated here:
290+
291+\code
292+U1db.Index{
293+ database: aDatabase
294+ id: by_helloworld
295+}
296+\endcode
297+
298+In the future, the Index element will support on disk storage of appropriate results / data. At the present time only in memory indexing is done, but once the storing capability is implemented, defining and identifying it is as simple as using the 'name' property (which will be stored in the database along with the relvent data that goes with it). The snippet below shows the use of the 'name' property:
299+
300+\code
301+U1db.Index{
302+ database: aDatabase
303+ id: by_helloworld
304+ //name: "by-helloworld"
305+}
306+\endcode
307+
308+The Index element describes, using dot notation, the fields and sub-fields where the developer expects to find information. That information is defined in a list, and added as the value for the 'expression' property. The list can contain one or more entries, as exemplified here (the property is commented out due to its current status):
309+
310+\code
311+U1db.Index{
312+ database: aDatabase
313+ id: by_helloworld
314+ //name: "by-helloworld"
315+ expression: ["hello.world.id","hello.world.message"]
316+}
317+\endcode
318+
319+\section2 The QueryElement
320+
321+\section3 Creating a Query Element
322+
323+The Query element has two responsibilities: a bridge from Database+Index to other parts of the application, as well as further filtering of data in the database (in addition to what Index provides).
324+
325+In order to fulfil its duties as a bridge to an Index (and Database), the 'index' property must point to an Index element, identified by its 'id'. For example:
326+
327+\code
328+U1db.Query{
329+ id: aQuery
330+ index: by_helloworld
331+}
332+\endcode
333+
334+While Index helps to filter data based on 'where' it is located (e.g. field.sub-field), Query helps determine the additional set of criteria for 'what' is being searched for. The intent of the 'query' property is to provide the mechanism for defnining the search criteria, but at the time of writing that functionality is not yet available. However, once the implementation is in place, using it is only requires defining the property's value (e.g. "Hello World"). Wild card searches using '*' are supported, which is the default query (i.e. if 'query' is not set it is assumed to be '*'). For example (the property is commented out due to its current status):
335+
336+\code
337+U1db.Query{
338+ id: aQuery
339+ index: by_helloworld
340+ //query: "*"
341+}
342+\endcode
343+
344+When the 'query' property becomes available, only wildcard search definitions for "starts with" will be suppoprted. Thus the following would be supported:
345+
346+\code
347+U1db.Query{
348+ id: aQuery
349+ index: by_helloworld
350+ //query: "Hello*"
351+}
352+\endcode
353+
354+
355+But this would not:
356+
357+\code
358+U1db.Query{
359+ id: aQuery
360+ index: by_helloworld
361+ //query: "*World"
362+}
363+\endcode
364+
365+Note: again, the 'query' property is commented out in the above two snippets due to its current status
366+
367+\section1 Using Data
368+
369+\section2 Data and the Application UI
370+
371+\section3 Using Data With Models and Views
372+
373+This simple snippet represents how to attach a ListModel to a ListView. In this instance the model 'aQuery' is representative of the Query + Index combination defined earlier:
374+
375+\code
376+ListView {
377+ width: units.gu(45)
378+ height: units.gu(80)
379+ model: aQuery
380+}
381+\endcode
382+
383+\section4 Data and Delegates
384+
385+How a model and ListView + delegates work together is a common QML concept, and not specific to U1Db-Qt. However, the asynchronous nature of this relationship is important to understand. When using QML ListView, delegates will be created based on particular properties such as the size of the application window, ListView, and delegate itself (amongst other factors). Each delegate can then represent a Document retrieved from the Database based on the record's index. This example demonstrates some of the property definitions that contribute to determining the number of delegates a ListView will contain:
386+
387+\code
388+ListView {
389+ width: units.gu(45)
390+ height: units.gu(80)
391+ model: aQuery
392+
393+ delegate: Text {
394+ x: 66; y: 77
395+ }
396+
397+}
398+\endcode
399+
400+
401+When the number of Documents is less than or equal to the number of delegates then there is a one to one mapping of index to delegate (e.g. the first delegate will represent the Document with an index = 0; the second, index = 1; and so on).
402+
403+When there are more Documents than delegates the ListView will request a new index depending on the situation (e.g. a user scrolls up or down). For example, if a ListView has 10 delegates, but 32 Documents to handle, when a user initially scrolls the first delegate will change from representing the Document with index = 0 to the Document that might have index = 8; the second, from index = 1 to index = 9; ...; the 10th delegate from index = 9 to index = 17. A second scrolling gesture the first index may change to 15, and the final index 24. And so on. Scrolling in the opposite direction will have a similar effect, but the Document index numbers for each delegate will obviously start to decline (towards their original values).
404+
405+The following snippet, which modifies the above delegate definition, could demonstrate this effect if there were enough Documents to do so (i.e. some number greater than the number of delegates):
406+
407+\code
408+ListView {
409+ width: units.gu(45)
410+ height: units.gu(80)
411+ model: aQuery
412+
413+ delegate: Text {
414+ x: 66; y: 77
415+ text: index
416+ }
417+
418+}
419+\endcode
420+
421+The object called 'contents' contains one or more properties. This example demonstrates the retrieval of data based on the U1db.Index defined earlier (id: by-helloworld). In this instance the Index contained two expressions simultaniously, "hello.world.id" and "hello.world.message"
422+
423+\code
424+ListView {
425+ width: units.gu(45)
426+ height: units.gu(80)
427+ model: aQuery
428+
429+ delegate: Text {
430+ x: 66; y: 77
431+ text: "(" + index + ") '" + contents.message + " " + contents.id + "'"
432+ }
433+
434+}
435+\endcode
436+
437+*/
438
439=== added file 'examples/u1db-qt-example-6/u1db-qt-example-6.qml'
440--- examples/u1db-qt-example-6/u1db-qt-example-6.qml 1970-01-01 00:00:00 +0000
441+++ examples/u1db-qt-example-6/u1db-qt-example-6.qml 2013-08-12 16:20:45 +0000
442@@ -0,0 +1,173 @@
443+/*
444+ * Copyright (C) 2013 Canonical, Ltd.
445+ *
446+ * Authors:
447+ * Kevin Wright <kevin.wright@canonical.com>
448+ *
449+ * This program is free software; you can redistribute it and/or modify
450+ * it under the terms of the GNU Lesser General Public License as published by
451+ * the Free Software Foundation; version 3.
452+ *
453+ * This program is distributed in the hope that it will be useful,
454+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
455+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
456+ * GNU Lesser General Public License for more details.
457+ *
458+ * You should have received a copy of the GNU Lesser General Public License
459+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
460+ */
461+
462+import QtQuick 2.0
463+import U1db 1.0 as U1db
464+import Ubuntu.Components 0.1
465+
466+
467+Item {
468+
469+ width: units.gu(45)
470+ height: units.gu(80)
471+
472+ U1db.Database {
473+ id: aDatabase
474+ path: "aDatabase6"
475+ }
476+
477+ U1db.Document {
478+ id: aDocument1
479+ database: aDatabase
480+ docId: 'helloworld'
481+ create: true
482+ contents:{"hello": { "world": [ { "message": "Hello World" } ] } }
483+
484+ }
485+
486+ U1db.Index{
487+ database: aDatabase
488+ id: by_helloworld
489+ expression: ["hello.world.message"]
490+ }
491+
492+ U1db.Query{
493+ id: aQuery
494+ index: by_helloworld
495+ query: [{"message":"Hel*"}]
496+ }
497+
498+ U1db.Synchronizer{
499+ id: aSynchronizer
500+ source: aDatabase
501+ targets: [{remote:true},
502+ {remote:true,
503+ ip:"127.0.0.1",
504+ port: 7777,
505+ name:"example1.u1db",
506+ resolve_to_source:true},
507+ {remote:"OK"}]
508+ synchronize: false
509+ }
510+
511+ MainView {
512+
513+ id: u1dbView
514+ width: units.gu(45)
515+ height: units.gu(80)
516+ anchors.top: parent.top;
517+
518+ Tabs {
519+ id: tabs
520+ anchors.fill: parent
521+
522+ Tab {
523+ objectName: "Tab1"
524+
525+ title: i18n.tr("Hello U1Db!")
526+
527+ page: Page {
528+ id: helloPage
529+
530+ Rectangle {
531+ width: units.gu(45)
532+ height: units.gu(40)
533+ anchors.top: parent.top;
534+ border.width: 1
535+
536+ Text {
537+ id: sourceLabel
538+ anchors.top: parent.top;
539+ font.bold: true
540+ text: "aDatabase6 Contents"
541+ }
542+
543+ ListView {
544+ id: sourceListView
545+ width: units.gu(45)
546+ height: units.gu(35)
547+ anchors.top: sourceLabel.bottom;
548+ model: aQuery
549+
550+ delegate: Text {
551+ wrapMode: Text.WordWrap
552+ x: 6; y: 77
553+ text: {
554+ text: "(" + index + ") '" + contents.message + "'"
555+ }
556+ }
557+ }
558+
559+
560+ }
561+
562+ Rectangle {
563+ id: lowerRectangle
564+ width: units.gu(45)
565+ height: units.gu(35)
566+ anchors.bottom: parent.bottom;
567+ border.width: 1
568+
569+ Text {
570+ id: errorsLabel
571+ anchors.top: parent.top;
572+ font.bold: true
573+ text: "Log:"
574+ }
575+
576+ ListView {
577+
578+ parent: lowerRectangle
579+ width: units.gu(45)
580+ height: units.gu(30)
581+ anchors.top: errorsLabel.bottom;
582+ model: aSynchronizer
583+ delegate:Text {
584+ width: units.gu(40)
585+ anchors.left: parent.left
586+ anchors.right: parent.right
587+ wrapMode: Text.WordWrap
588+ text: {
589+ text: sync_output
590+ }
591+ }
592+ }
593+
594+ Button{
595+ parent: lowerRectangle
596+ anchors.bottom: parent.bottom;
597+ text: "Sync"
598+ onClicked: aSynchronizer.synchronize = true
599+ anchors.left: parent.left
600+ anchors.right: parent.right
601+
602+ }
603+
604+ }
605+ }
606+
607+ }
608+
609+ }
610+
611+ }
612+
613+}
614+
615+
616
617=== modified file 'modules/U1db/CMakeLists.txt'
618--- modules/U1db/CMakeLists.txt 2013-05-01 22:49:50 +0000
619+++ modules/U1db/CMakeLists.txt 2013-08-12 16:20:45 +0000
620@@ -1,4 +1,5 @@
621 find_package(Qt5Quick REQUIRED)
622+find_package(Qt5Network REQUIRED)
623
624 get_target_property(QMAKE_EXECUTABLE Qt5::qmake LOCATION)
625 # See http://doc-snapshot.qt-project.org/5.0/qtcore/qlibraryinfo.html#LibraryLocation-enum
626@@ -13,6 +14,7 @@
627 include_directories(
628 ${CMAKE_CURRENT_SOURCE_DIR}
629 ${Qt5Sql_INCLUDE_DIRS}
630+ ${Qt5Network_INCLUDE_DIRS}
631 )
632
633 add_library(U1DBPlugin SHARED ${U1DBPlugin_SRCS})
634@@ -21,6 +23,7 @@
635 target_link_libraries(U1DBPlugin
636 ${U1DB_QT_LIBNAME}
637 ${Qt5Quick_LIBRARIES}
638+ ${Qt5Network_LIBRARIES}
639 )
640
641 include_directories(
642@@ -28,6 +31,7 @@
643 ${U1DB_INCLUDE_DIRS}
644 ${CMAKE_CURRENT_BINARY_DIR}
645 ${Qt5Quick_INCLUDE_DIRS}
646+ ${Qt5Network_INCLUDE_DIRS}
647 )
648
649 # copy qmldir file into build directory for shadow builds
650
651=== modified file 'modules/U1db/plugin.cpp'
652--- modules/U1db/plugin.cpp 2013-02-15 11:43:45 +0000
653+++ modules/U1db/plugin.cpp 2013-08-12 16:20:45 +0000
654@@ -21,6 +21,7 @@
655 #include "document.h"
656 #include "index.h"
657 #include "query.h"
658+#include "synchronizer.h"
659 #include "plugin.h"
660 #include <qqml.h>
661
662@@ -32,5 +33,6 @@
663 qmlRegisterType<Document>(uri, 1, 0, "Document");
664 qmlRegisterType<Index>(uri, 1, 0, "Index");
665 qmlRegisterType<Query>(uri, 1, 0, "Query");
666+ qmlRegisterType<Synchronizer>(uri, 1, 0, "Synchronizer");
667 }
668
669
670=== modified file 'src/CMakeLists.txt'
671--- src/CMakeLists.txt 2013-07-31 12:22:38 +0000
672+++ src/CMakeLists.txt 2013-08-12 16:20:45 +0000
673@@ -6,6 +6,7 @@
674 document.cpp
675 index.cpp
676 query.cpp
677+ synchronizer.cpp
678 )
679
680 # Generated files
681@@ -14,6 +15,7 @@
682 moc_document.cpp
683 moc_index.cpp
684 moc_query.cpp
685+ moc_synchronizer.cpp
686 )
687 set_directory_properties(PROPERTIES ADDITIONAL_MAKE_CLEAN_FILES "${U1DB_QT_GENERATED}")
688
689@@ -27,6 +29,7 @@
690 ${CMAKE_CURRENT_SOURCE_DIR}
691 ${CMAKE_CURRENT_BINARY_DIR}
692 ${Qt5Core_INCLUDE_DIRS}
693+ ${Qt5Network_INCLUDE_DIRS}
694 ${Qt5Sql_INCLUDE_DIRS}
695 ${U1DB_INCLUDE_DIRS}
696 )
697@@ -35,6 +38,7 @@
698 target_link_libraries(${U1DB_QT_LIBNAME}
699 ${Qt5Core_LIBRARIES}
700 ${Qt5Sql_LIBRARIES}
701+ ${Qt5Network_LIBRARIES}
702 ${U1DB_LDFLAGS}
703 )
704
705@@ -50,7 +54,7 @@
706 LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
707 )
708
709-install(FILES global.h database.h document.h index.h query.h
710+install(FILES global.h database.h document.h index.h query.h synchronizer.h
711 DESTINATION ${INCLUDE_INSTALL_DIR}
712 )
713
714
715=== modified file 'src/database.cpp'
716--- src/database.cpp 2013-04-29 23:54:19 +0000
717+++ src/database.cpp 2013-08-12 16:20:45 +0000
718@@ -106,7 +106,8 @@
719 /* A unique ID is used for the connection name to ensure that we aren't
720 re-using or replacing other opend databases. */
721 if (!m_db.isValid())
722- m_db = QSqlDatabase::addDatabase("QSQLITE", QUuid::createUuid().toString());
723+ m_db = QSqlDatabase::addDatabase("QSQLITE",QUuid::createUuid().toString());
724+
725 if (!m_db.isValid())
726 return setError("QSqlDatabase error");
727 m_db.setDatabaseName(path);
728@@ -251,6 +252,42 @@
729 }
730
731 /*!
732+ * \internal
733+ * \brief Database::getDocumentContents
734+ *
735+ * Returns the string representation of a document that has
736+ * been selected from the database using a document id.
737+ *
738+ */
739+
740+QString
741+Database::getDocumentContents(const QString& docId)
742+{
743+ if (!initializeIfNeeded())
744+ return QString();
745+
746+ QSqlQuery query(m_db.exec());
747+ query.prepare("SELECT document.doc_rev, document.content, "
748+ "count(conflicts.doc_rev) AS conflicts FROM document LEFT OUTER JOIN "
749+ "conflicts ON conflicts.doc_id = document.doc_id WHERE "
750+ "document.doc_id = :docId GROUP BY document.doc_id, "
751+ "document.doc_rev, document.content");
752+ query.bindValue(":docId", docId);
753+ if (query.exec())
754+ {
755+ if (query.next())
756+ {
757+ if (query.value("conflicts").toInt() > 0)
758+ setError(QString("Conflicts in %1").arg(docId));
759+ return query.value("content").toString();
760+ }
761+ return setError(QString("Failed to get document %1: No document").arg(docId)) ? QString() : QString();
762+ }
763+ return setError(QString("Failed to get document %1: %2\n%3").arg(docId).arg(query.lastError().text()).arg(query.lastQuery())) ? QString() : QString();
764+}
765+
766+
767+/*!
768 Returns the contents of a document by \a docId in a form that QML recognizes
769 as a Variant object, it's identical to Document::getContents() with the
770 same \a docId.
771@@ -284,6 +321,9 @@
772 return setError(QString("Failed to get document %1: %2\n%3").arg(docId).arg(query.lastError().text()).arg(query.lastQuery())) ? QVariant() : QVariant();
773 }
774
775+/*!
776+ The increaseVectorClockRev(int oldRev) function is deprecated.
777+ */
778 static int
779 increaseVectorClockRev(int oldRev)
780 {
781@@ -291,54 +331,304 @@
782 }
783
784 /*!
785+ * \internal
786+ This function creates a new revision number.
787+
788+ It returns a string for use in the document table's 'doc_rev' field.
789+ */
790+
791+QString Database::getNextDocRevisionNumber(QString doc_id)
792+{
793+
794+ QString revision_number = getReplicaUid()+":1";
795+
796+ QString current_revision_number = getCurrentDocRevisionNumber(doc_id);
797+
798+ /*!
799+ Some revisions contain information from previous
800+conflicts/syncs. Revisions are delimited by '|'.
801+
802+ */
803+
804+ QStringList current_revision_list = current_revision_number.split("|");
805+
806+ Q_FOREACH (QString current_revision, current_revision_list) {
807+
808+ /*!
809+ Each revision contains two pieces of information,
810+the uid of the database that made the revsion, and a counter
811+for the revsion. This information is delimited by ':'.
812+
813+ */
814+
815+ QStringList current_revision_number_list = current_revision.split(":");
816+
817+ if(current_revision_number_list[0]==getReplicaUid()) {
818+
819+ /*!
820+ If the current revision uid is the same as this Database's uid the counter portion is increased by one.
821+
822+ */
823+
824+ int revision_generation_number = current_revision_number_list[1].toInt()+1;
825+
826+ revision_number = getReplicaUid()+":"+QString::number(revision_generation_number);
827+
828+ }
829+ else {
830+
831+ /*!
832+ If the current revision uid is not the same as this Database's uid then the revision represents a change that originated in another database.
833+
834+ */
835+ //revision_number+="|"+current_revision;
836+
837+ /* Not sure if the above is necessary,
838+ *and did not appear to be working as intended either.
839+ *
840+ * Commented out, but maybe OK to delete.
841+ */
842+ }
843+
844+ }
845+
846+ /*!
847+ The Database UID has curly brackets, but they are not required for the revision number and need to be removed.
848+
849+ */
850+
851+ revision_number = revision_number.replace("{","");
852+
853+ revision_number = revision_number.replace("}","");
854+
855+ return revision_number;
856+
857+}
858+
859+/*!
860+ * \internal
861+ The getCurrentDocRevisionNumber(QString doc_id) function
862+returns the current string value from the document table's
863+doc_rev field.
864+
865+ */
866+
867+
868+QString Database::getCurrentDocRevisionNumber(QString doc_id){
869+ if (!initializeIfNeeded())
870+ return QString();
871+
872+ QSqlQuery query(m_db.exec());
873+
874+ query.prepare("SELECT doc_rev from document WHERE doc_id = :docId");
875+ query.bindValue(":docId", doc_id);
876+
877+ if (query.exec())
878+ {
879+ while (query.next())
880+ {
881+ return query.value("doc_rev").toString();
882+ }
883+
884+ }
885+ else{
886+ return setError(query.lastError().text()) ? QString() : QString();
887+ }
888+ return QString();
889+}
890+
891+/*!
892+ * \internal
893+ * \brief Database::updateSyncLog
894+ *
895+ * This method is used at the end of a synchronization session,
896+ * to update the database with the latest information known about the peer
897+ * database that was synced against.
898+ */
899+void Database::updateSyncLog(bool insert, QString uid, QString generation, QString transaction_id)
900+{
901+
902+ if (!initializeIfNeeded())
903+ return;
904+
905+ QSqlQuery query(m_db.exec());
906+
907+ if(insert==true){
908+ query.prepare("INSERT INTO sync_log(known_generation,known_transation_id,known_transation_id) VALUES(:knownGeneration, :knownTransactionId, :replicaUid)");
909+
910+ }
911+ else{
912+ query.prepare("UPDATE sync_log SET known_generation = :knownGeneration, known_transation_id = :knownTransactionId WHERE replica_uid = :replicaUid");
913+ }
914+
915+
916+ query.bindValue(":replicaUid", uid);
917+ query.bindValue(":knownGeneration", generation);
918+ query.bindValue(":knownTransactionId", transaction_id);
919+ if (!query.exec())
920+ {
921+ setError(query.lastError().text());
922+
923+ }
924+
925+}
926+
927+/*!
928+ * \internal
929+ * \brief Database::updateDocRevisionNumber
930+ *
931+ * Whenever a document as added or modified it needs a new revision number.
932+ *
933+ * The revision number contains information about revisions made at the source,
934+ * but also revisions to the document by target databases (and then synced with the source).
935+ *
936+ */
937+
938+void Database::updateDocRevisionNumber(QString doc_id,QString revision){
939+ if (!initializeIfNeeded())
940+ return;
941+
942+ QSqlQuery query(m_db.exec());
943+
944+ query.prepare("UPDATE document SET doc_rev = :revisionId WHERE doc_id = :docId");
945+ query.bindValue(":docId", doc_id);
946+ query.bindValue(":revisionId", revision);
947+ if (!query.exec())
948+ {
949+ setError(query.lastError().text());
950+
951+ }
952+
953+}
954+
955+/*!
956+ The getCurrentGenerationNumber() function searches for the
957+current generation number from the sqlite_sequence table.
958+The return value can then be used during a synchronization session,
959+amongst other things.
960+
961+ */
962+
963+int Database::getCurrentGenerationNumber(){
964+
965+ int sequence_number = -1;
966+
967+ QSqlQuery query(m_db.exec());
968+
969+ query.prepare("SELECT seq FROM sqlite_sequence WHERE name = 'transaction_log'");
970+
971+ if (query.exec())
972+ {
973+ while (query.next())
974+ {
975+ sequence_number = (query.value("seq").toInt());
976+
977+ }
978+
979+ }
980+ else{
981+ setError(query.lastError().text());
982+ }
983+
984+ return sequence_number;
985+
986+}
987+
988+/*!
989+ The generateNewTransactionId() function generates a random
990+transaction id string, for use when creating new transations.
991+
992+ */
993+
994+QString Database::generateNewTransactionId(){
995+
996+ QString uid = "T-"+QUuid::createUuid().toString();
997+ uid = uid.replace("}","");
998+ uid = uid.replace("{","");
999+ return uid;
1000+
1001+}
1002+
1003+/*!
1004+ Each time a document in the Database is created or updated a
1005+new transaction is performed, and information about it inserted into the
1006+transation_log table using the createNewTransaction(QString doc_id)
1007+function.
1008+
1009+ */
1010+
1011+int Database::createNewTransaction(QString doc_id){
1012+
1013+ QString transaction_id = generateNewTransactionId();
1014+
1015+ QSqlQuery query(m_db.exec());
1016+
1017+ QString queryString = "INSERT INTO transaction_log(doc_id, transaction_id) VALUES('"+doc_id+"', '"+transaction_id+"')";
1018+
1019+ if (!query.exec(queryString)){
1020+ return -1;
1021+ }
1022+ else{
1023+ return 0;
1024+ }
1025+
1026+ return -1;
1027+}
1028+
1029+/*!
1030 Updates the existing \a contents of the document identified by \a docId if
1031 there's no error.
1032 If no \a docId is given or \a docId is an empty string the \a contents will be
1033 stored under an autogenerated name.
1034 Returns the new revision of the document, or -1 on failure.
1035 */
1036-int
1037+QString
1038 Database::putDoc(QVariant contents, const QString& docId)
1039 {
1040 if (!initializeIfNeeded())
1041- return -1;
1042+ return "";
1043
1044 QString newOrEmptyDocId(docId);
1045 QVariant oldDoc = newOrEmptyDocId.isEmpty() ? QVariant() : getDocUnchecked(newOrEmptyDocId);
1046- /* FIXME: Conflicts */
1047-
1048- int newRev = increaseVectorClockRev(7/*contents.rev*/);
1049+
1050+ QString revision_number = getNextDocRevisionNumber(newOrEmptyDocId);
1051+
1052 QSqlQuery query(m_db.exec());
1053 if (oldDoc.isValid())
1054 {
1055 query.prepare("UPDATE document SET doc_rev=:docRev, content=:docJson WHERE doc_id = :docId");
1056 query.bindValue(":docId", newOrEmptyDocId);
1057- query.bindValue(":docRev", newRev);
1058+ query.bindValue(":docRev", revision_number);
1059 // Parse Variant from QML as JsonDocument, fallback to string
1060 QString json(QJsonDocument::fromVariant(contents).toJson());
1061 query.bindValue(":docJson", json.isEmpty() ? contents : json);
1062 if (!query.exec())
1063- return setError(QString("Failed to put/ update document %1: %2\n%3").arg(newOrEmptyDocId).arg(query.lastError().text()).arg(query.lastQuery())) ? -1 : -1;
1064+ return setError(QString("Failed to put/ update document %1: %2\n%3").arg(newOrEmptyDocId).arg(query.lastError().text()).arg(query.lastQuery())) ? "" : "";
1065 query.prepare("DELETE FROM document_fields WHERE doc_id = :docId");
1066 query.bindValue(":docId", newOrEmptyDocId);
1067 if (!query.exec())
1068- return setError(QString("Failed to delete document field %1: %2\n%3").arg(newOrEmptyDocId).arg(query.lastError().text()).arg(query.lastQuery())) ? -1 : -1;
1069+ return setError(QString("Failed to delete document field %1: %2\n%3").arg(newOrEmptyDocId).arg(query.lastError().text()).arg(query.lastQuery())) ? "" : "";
1070+
1071+ createNewTransaction(newOrEmptyDocId);
1072+
1073 }
1074 else
1075 {
1076 if (newOrEmptyDocId.isEmpty())
1077 newOrEmptyDocId = QString("D-%1").arg(QUuid::createUuid().toString().mid(1).replace("}",""));
1078 if (!QRegExp("^[a-zA-Z0-9.%_-]+$").exactMatch(newOrEmptyDocId))
1079- return setError(QString("Invalid docID %1").arg(newOrEmptyDocId)) ? -1 : -1;
1080+ return setError(QString("Invalid docID %1").arg(newOrEmptyDocId)) ? "" : "";
1081
1082 query.prepare("INSERT INTO document (doc_id, doc_rev, content) VALUES (:docId, :docRev, :docJson)");
1083 query.bindValue(":docId", newOrEmptyDocId);
1084- query.bindValue(":docRev", newRev);
1085+ query.bindValue(":docRev", revision_number);
1086 // Parse Variant from QML as JsonDocument, fallback to string
1087 QJsonDocument json(QJsonDocument::fromVariant(contents));
1088 query.bindValue(":docJson", json.isEmpty() ? contents : json.toJson());
1089 if (!query.exec())
1090- return setError(QString("Failed to put document %1: %2\n%3").arg(docId).arg(query.lastError().text()).arg(query.lastQuery())) ? -1 : -1;
1091+ return setError(QString("Failed to put document %1: %2\n%3").arg(docId).arg(query.lastError().text()).arg(query.lastQuery())) ? "" : "";
1092+
1093+ createNewTransaction(newOrEmptyDocId);
1094 }
1095
1096 beginResetModel();
1097@@ -350,8 +640,22 @@
1098
1099 Q_EMIT docChanged(newOrEmptyDocId, contents);
1100
1101- return newRev;
1102-}
1103+ return revision_number;
1104+}
1105+
1106+/*!
1107+ * \brief Database::resetModel
1108+ *
1109+ * Resets the Database model.
1110+ */
1111+
1112+void Database::resetModel(){
1113+
1114+ beginResetModel();
1115+ endResetModel();
1116+
1117+}
1118+
1119
1120 /*!
1121 Returns a list of all stored documents by their docId.
1122@@ -376,6 +680,7 @@
1123 }
1124 return list;
1125 }
1126+
1127 return setError(QString("Failed to list documents: %1\n%2").arg(query.lastError().text()).arg(query.lastQuery())) ? list : list;
1128 }
1129
1130@@ -400,6 +705,12 @@
1131 Q_EMIT pathChanged(path);
1132 }
1133
1134+/*!
1135+ * \brief Database::getPath
1136+ *
1137+ * Simply returns the path of the database.
1138+ *
1139+ */
1140 QString
1141 Database::getPath()
1142 {
1143@@ -516,6 +827,76 @@
1144 return list;
1145 }
1146
1147+/* Handy functions for synchronization. */
1148+
1149+/*!
1150+ * \internal
1151+ * \brief Database::listTransactionsSince
1152+ *
1153+ * This lists transactions for the database since a particular generation number.
1154+ *
1155+ */
1156+
1157+QList<QString> Database::listTransactionsSince(int generation){
1158+
1159+ QList<QString> list;
1160+
1161+ if (!initializeIfNeeded())
1162+ return list;
1163+
1164+ QSqlQuery query(m_db.exec());
1165+
1166+ QString queryStmt = "SELECT generation, doc_id, transaction_id FROM transaction_log where generation > "+QString::number(generation);
1167+
1168+ if (query.exec(queryStmt))
1169+ {
1170+ while (query.next())
1171+ {
1172+ list.append(query.value("generation").toString()+"|"+query.value("doc_id").toString()+"|"+query.value("transaction_id").toString());
1173+ }
1174+
1175+ return list;
1176+
1177+ }
1178+
1179+ return list;
1180+
1181+}
1182+
1183+/*!
1184+ * \internal
1185+ * \brief Database::getSyncLogInfo
1186+ *
1187+ * Provides the information about previous synchronizations between the database and another (if any).
1188+ *
1189+ */
1190+
1191+QMap<QString,QVariant> Database::getSyncLogInfo(QMap<QString,QVariant> lastSyncInformation, QString uid, QString prefix){
1192+
1193+ if (!initializeIfNeeded())
1194+ return lastSyncInformation;
1195+
1196+ QString queryStmt = "SELECT known_transaction_id, known_generation FROM sync_log WHERE replica_uid = '"+uid +"'";
1197+
1198+ QSqlQuery query(m_db.exec());
1199+
1200+ if (query.exec(queryStmt))
1201+ {
1202+ while (query.next())
1203+ {
1204+ lastSyncInformation.insert(prefix + "_replica_generation", query.value(1).toInt());
1205+ lastSyncInformation.insert(prefix + "_replica_transaction_id",query.value(0).toString());
1206+ return lastSyncInformation;
1207+ }
1208+
1209+ }
1210+ else{
1211+ setError(query.lastError().text());
1212+ }
1213+
1214+ return lastSyncInformation;
1215+}
1216+
1217 QT_END_NAMESPACE_U1DB
1218
1219 #include "moc_database.cpp"
1220
1221=== modified file 'src/database.h'
1222--- src/database.h 2013-04-23 15:17:24 +0000
1223+++ src/database.h 2013-08-12 16:20:45 +0000
1224@@ -36,21 +36,33 @@
1225 public:
1226 Database(QObject* parent = 0);
1227
1228+
1229 // QAbstractListModel
1230 QVariant data(const QModelIndex & index, int role = Qt::DisplayRole) const;
1231 QHash<int, QByteArray>roleNames() const;
1232 int rowCount(const QModelIndex & parent = QModelIndex()) const;
1233+ void resetModel();
1234
1235 QString getPath();
1236 void setPath(const QString& path);
1237 Q_INVOKABLE QVariant getDoc(const QString& docId);
1238+ QString getDocumentContents(const QString& docId);
1239 QVariant getDocUnchecked(const QString& docId) const;
1240- Q_INVOKABLE int putDoc(QVariant newDoc, const QString& docID=QString());
1241+ Q_INVOKABLE QString putDoc(QVariant newDoc, const QString& docID=QString());
1242 Q_INVOKABLE QList<QString> listDocs();
1243 Q_INVOKABLE QString lastError();
1244 Q_INVOKABLE QString putIndex(const QString& index_name, QStringList expressions);
1245 Q_INVOKABLE QStringList getIndexExpressions(const QString& indexName);
1246 Q_INVOKABLE QStringList getIndexKeys(const QString& indexName);
1247+
1248+ /* Functions handy for Synchronization */
1249+ QString getNextDocRevisionNumber(QString doc_id);
1250+ QString getCurrentDocRevisionNumber(QString doc_id);
1251+ void updateDocRevisionNumber(QString doc_id,QString revision);
1252+ void updateSyncLog(bool insert, QString uid, QString generation, QString transaction_id);
1253+ QList<QString> listTransactionsSince(int generation);
1254+ QMap<QString,QVariant> getSyncLogInfo(QMap<QString,QVariant> lastSyncInformation, QString uid, QString prefix);
1255+
1256 Q_SIGNALS:
1257 void pathChanged(const QString& path);
1258 void errorChanged(const QString& error);
1259@@ -63,7 +75,7 @@
1260 */
1261 void docLoaded(const QString& docId, QVariant content) const;
1262 private:
1263- Q_DISABLE_COPY(Database)
1264+ //Q_DISABLE_COPY(Database)
1265 QString m_path;
1266 QSqlDatabase m_db;
1267 QString m_error;
1268@@ -73,6 +85,11 @@
1269 bool initializeIfNeeded(const QString& path=":memory:");
1270 bool setError(const QString& error);
1271 QString getDocIdByRow(int row) const;
1272+
1273+ int createNewTransaction(QString doc_id);
1274+ QString generateNewTransactionId();
1275+ int getCurrentGenerationNumber();
1276+
1277 };
1278
1279 QT_END_NAMESPACE_U1DB
1280
1281=== modified file 'src/query.cpp'
1282--- src/query.cpp 2013-05-01 23:06:01 +0000
1283+++ src/query.cpp 2013-08-12 16:20:45 +0000
1284@@ -59,6 +59,7 @@
1285 QVariant
1286 Query::data(const QModelIndex & index, int role) const
1287 {
1288+
1289 if (role == 0) // contents
1290 return m_results.at(index.row());
1291 if (role == 1) // docId
1292@@ -150,8 +151,23 @@
1293
1294 }
1295
1296+ resetModel();
1297+
1298 Q_EMIT documentsChanged(m_documents);
1299 Q_EMIT resultsChanged(m_results);
1300+
1301+}
1302+
1303+/*!
1304+ * \brief Query::resetModel
1305+ *
1306+ * Resets the model of the Query
1307+ *
1308+ */
1309+
1310+void Query::resetModel(){
1311+ beginResetModel();
1312+ endResetModel();
1313 }
1314
1315 /*!
1316
1317=== modified file 'src/query.h'
1318--- src/query.h 2013-04-25 13:38:33 +0000
1319+++ src/query.h 2013-08-12 16:20:45 +0000
1320@@ -52,6 +52,8 @@
1321 QStringList getDocuments();
1322 QList<QVariant> getResults();
1323
1324+ void resetModel();
1325+
1326 Q_SIGNALS:
1327 void indexChanged(Index* index);
1328 void queryChanged(QVariant query);
1329
1330=== added file 'src/synchronizer.cpp'
1331--- src/synchronizer.cpp 1970-01-01 00:00:00 +0000
1332+++ src/synchronizer.cpp 2013-08-12 16:20:45 +0000
1333@@ -0,0 +1,1257 @@
1334+/*
1335+ * Copyright (C) 2013 Canonical, Ltd.
1336+ *
1337+ * Authors:
1338+ * Kevin Wright <kevin.wright@canonical.com>
1339+ *
1340+ * This program is free software; you can redistribute it and/or modify
1341+ * it under the terms of the GNU Lesser General Public License as published by
1342+ * the Free Software Foundation; version 3.
1343+ *
1344+ * This program is distributed in the hope that it will be useful,
1345+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
1346+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1347+ * GNU Lesser General Public License for more details.
1348+ *
1349+ * You should have received a copy of the GNU Lesser General Public License
1350+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
1351+ */
1352+
1353+#include <QDebug>
1354+#include <QSqlQuery>
1355+#include <QFile>
1356+#include <QSqlError>
1357+#include <QUuid>
1358+#include <QStringList>
1359+#include <QJsonDocument>
1360+
1361+#include "synchronizer.h"
1362+#include "private.h"
1363+
1364+QT_BEGIN_NAMESPACE_U1DB
1365+
1366+/*!
1367+ \class Synchronizer
1368+ \inmodule U1Db
1369+ \ingroup modules
1370+
1371+ \brief The Synchronizer class handles synchronizing between two databases.
1372+
1373+*/
1374+
1375+/*
1376+
1377+ Below this line are general methods for this class, such as setting/getting values for various properties.
1378+
1379+*/
1380+
1381+/*!
1382+ Create a new Synchronizer element, with an optional \a parent, usually by declaring it as a QML item.
1383+
1384+ Synchronizer elements sync two databases together, a 'source' database and a remote or local 'target' database.
1385+
1386+ Example use in a QML application:
1387+
1388+ U1db.Synchronizer{
1389+ id: aSynchronizer
1390+ synchronize: false
1391+ source: aDatabase
1392+ targets: [{remote:true,
1393+ ip:"127.0.0.1",
1394+ port: 7777,
1395+ name:"example1.u1db",
1396+ resolve_to_source:true}]
1397+
1398+ }
1399+
1400+ Short description of properties:
1401+
1402+ id: The element's identification.
1403+
1404+ bool synchronize: Is the element actively synching or not. Should be set to false.
1405+
1406+ U1DB::Database source: The id of a local database that will be used for synchronization.
1407+
1408+ QVariant targets: One or more target databases that will be synched with the local database.
1409+
1410+ bool targets.remote: Is the target database a remote or local database.
1411+
1412+ QString targets.ip: The ip address of a remote database (if applicable).
1413+
1414+ int targets.port: Port number of the remote server.
1415+
1416+ QString targets.name: The name of the database.
1417+
1418+ bool targets.resolve_to_source: In case of conflict should the sync resolve to the source's data (if true).
1419+
1420+ Example use with u1db-serve:
1421+
1422+ 1. In a terminal cd into a directory where the u1db Python reference implemented has been downloaded from lp:u1db.
1423+ 2. Using Python create a database called 'example1.u1db' using u1db, and a document 'helloworld':
1424+
1425+ # python
1426+
1427+ >>> import u1db
1428+ >>> db = u1db.open("example1.u1db",create=True)
1429+ >>> content = {"hello": { "world": { "message":"Hello World Updated" } } }
1430+ >>> db.create_doc(content, doc_id="helloworld")
1431+
1432+ ctrl+d
1433+
1434+ 3. From the u1db directory above type './u1db-serve --port=7777' and hit enter.
1435+ 4. Open another terminal tab.
1436+ 5. Change into a directory containing u1db-qt (assuming this class is included in that directory and the installed version on the host computer).
1437+ 6. Change into the directory where u1db-qt-example-6.qml is located.
1438+ 7. Type 'qmlscene u1db-qt-example-6.qml' and hit enter.
1439+ 8. Click the button labelled 'Sync'.
1440+ 9. Check the terminal windows for output from either the client or server.
1441+
1442+ */
1443+
1444+Synchronizer::Synchronizer(QObject *parent) :
1445+ QAbstractListModel(parent), m_synchronize(false), m_source(NULL)
1446+{
1447+ QObject::connect(this, &Synchronizer::syncChanged, this, &Synchronizer::onSyncChanged);
1448+}
1449+
1450+/*!
1451+ \internal
1452+ *Used to implement QAbstractListModel
1453+ *Implements the variables exposed to the Delegate in a model
1454+ */
1455+QVariant
1456+Synchronizer::data(const QModelIndex & index, int role) const
1457+{
1458+ if (role == 0)
1459+ return m_sync_output.at(index.row());
1460+ return QVariant();
1461+}
1462+
1463+/*!
1464+ \internal
1465+ Used to implement QAbstractListModel
1466+
1467+ The number of rows: the number of documents given by the query.
1468+ */
1469+int
1470+Synchronizer::rowCount(const QModelIndex & parent) const
1471+{
1472+ return m_sync_output.count();
1473+}
1474+
1475+/*!
1476+ \internal
1477+ Used to implement QAbstractListModel
1478+
1479+ */
1480+QHash<int, QByteArray>
1481+Synchronizer::roleNames() const
1482+{
1483+ QHash<int, QByteArray> roles;
1484+ roles.insert(0, "sync_output");
1485+ return roles;
1486+}
1487+
1488+
1489+/*!
1490+
1491+
1492+ Sets the source database.
1493+
1494+ */
1495+void Synchronizer::setSource(Database* source)
1496+{
1497+
1498+ if (m_source == source)
1499+ return;
1500+
1501+ if (m_source)
1502+ QObject::disconnect(m_source, 0, this, 0);
1503+
1504+ m_source = source;
1505+
1506+ Q_EMIT sourceChanged(source);
1507+}
1508+
1509+
1510+/*!
1511+ * \property Synchronizer::targets
1512+ *
1513+ * Sets meta-data for databases to be used during a synchronization session.
1514+ *
1515+ * The QVariant is a list that can contain definitions for more than one database
1516+ * to be used as a target. For example:
1517+ *
1518+ * targets: [{remote:true},
1519+ * {remote:true,
1520+ * ip:"127.0.0.1",
1521+ * port: 7777,
1522+ * name:"example1.u1db",
1523+ * resolve_to_source:true},
1524+ * {remote:"OK"}]
1525+ *
1526+ * The above example defines three databases. Two of the three definitions in the
1527+ * example are invalid, the first ({remote:true}) and the third ({remote:"OK"}),
1528+ * because they are incomplete.
1529+ *
1530+ * The second definition is a fully defined and valid definition for a local to
1531+ * remote synchronization of two databases:
1532+ *
1533+ * {remote:true,
1534+ * ip:"127.0.0.1",
1535+ * port: 7777,
1536+ * name:"example1.u1db",
1537+ * resolve_to_source:true}
1538+ *
1539+ * 'remote' determines whether the database is on disk or located on a server.
1540+ * 'ip' and 'port' for a server are used only when 'remote' is set to true
1541+ * 'name' is the name of the local (on disk) or remote database.
1542+ * Note: If 'remote' is false this is the relative/absolute file location.
1543+ * 'resolve_to_source' determines whether to resolve conflicts automatically
1544+ * in favor of the source (aka local) database's values or the target's.
1545+ *
1546+ */
1547+
1548+void Synchronizer::setTargets(QVariant targets)
1549+{
1550+
1551+ if (m_targets == targets)
1552+ return;
1553+
1554+ //if (m_targets)
1555+ // QObject::disconnect(m_targets, 0, this, 0);
1556+
1557+ m_targets = targets;
1558+ Q_EMIT targetsChanged(targets);
1559+}
1560+
1561+/*!
1562+ * \property Synchronizer::synchronize
1563+ */
1564+
1565+void Synchronizer::setSync(bool synchronize)
1566+{
1567+
1568+ if (m_synchronize == synchronize)
1569+ return;
1570+
1571+ m_synchronize = synchronize;
1572+ Q_EMIT syncChanged(synchronize);
1573+}
1574+
1575+
1576+/*!
1577+ * \property Synchronizer::resolve_to_source
1578+ */
1579+
1580+void Synchronizer::setResolveToSource(bool resolve_to_source)
1581+{
1582+ if (m_resolve_to_source == resolve_to_source)
1583+ return;
1584+
1585+ m_resolve_to_source = resolve_to_source;
1586+ Q_EMIT resolveToSourceChanged(resolve_to_source);
1587+}
1588+
1589+
1590+/*!
1591+ * \fn void Synchronizer::setSyncOutput(QList<QVariant> sync_output)
1592+ *
1593+ * Sets the current value for the active session's \a sync_output.
1594+ *
1595+ */
1596+
1597+void Synchronizer::setSyncOutput(QList<QVariant> sync_output)
1598+{
1599+ if (m_sync_output == sync_output)
1600+ return;
1601+
1602+ m_sync_output = sync_output;
1603+ Q_EMIT syncOutputChanged(sync_output);
1604+}
1605+
1606+/*!
1607+ * \property Synchronizer::source
1608+ *
1609+ *
1610+ * Returns a source Database.
1611+ *
1612+ */
1613+Database* Synchronizer::getSource()
1614+{
1615+ return m_source;
1616+}
1617+
1618+/*!
1619+ * \brief Synchronizer::getTargets
1620+ *
1621+ *
1622+ * Returns meta-data for all target databases.
1623+ *
1624+ */
1625+
1626+QVariant Synchronizer::getTargets()
1627+{
1628+ return m_targets;
1629+}
1630+
1631+/*!
1632+ * \brief Synchronizer::getSync
1633+ *
1634+ *
1635+ * Returns the current value of synchronize. If true then the synchronize
1636+ * session is initiated.
1637+ *
1638+ * This should probaby always be set to false on application start up.
1639+ * The application developer should use some trigger to switch it to true
1640+ * when needed (e.g. button click).
1641+ *
1642+ */
1643+
1644+bool Synchronizer::getSync()
1645+{
1646+ return m_synchronize;
1647+}
1648+
1649+/*!
1650+ * \brief Synchronizer::getResolveToSource
1651+ *
1652+ *
1653+ * If set to true, any document conflicts created during a sync session
1654+ * will be resolved in favor of the content from the source database. If false
1655+ * the content from the target database will replace the document content in
1656+ * the source database.
1657+ *
1658+ */
1659+
1660+bool Synchronizer::getResolveToSource(){
1661+ return m_resolve_to_source;
1662+}
1663+
1664+/*!
1665+ * \property Synchronizer::sync_output
1666+ * \brief Synchronizer::getSyncOutput
1667+ *
1668+ * Returns the output from a sync session. The list should contain numerous
1669+ * QVariantMaps, each of which will have various meta-data with informative
1670+ * information about what happened in the background of the session.
1671+ *
1672+ * In some cases the information will be about errors or warnings, and in
1673+ * other cases simple log messages. Also included would noramlly be associated
1674+ * properties, elements and other data.
1675+ *
1676+ * The information can be used in any number of ways, such as on screen within an app,
1677+ * testing, console output, logs and more. This is designed to be flexible enough that
1678+ * the app developer can decide themselves how to best use the data.
1679+ *
1680+ */
1681+
1682+QList<QVariant> Synchronizer::getSyncOutput(){
1683+ return m_sync_output;
1684+}
1685+
1686+/*
1687+
1688+ Below this line represents the class' more unique functionality.
1689+ In other words, methods that do more than simply modify/retrieve
1690+ an element's property values.
1691+
1692+*/
1693+
1694+
1695+/*!
1696+ * \brief Synchronizer::onSyncChanged
1697+ *
1698+ * The synchroization process begins here.
1699+ *
1700+ */
1701+
1702+void Synchronizer::onSyncChanged(bool synchronize){
1703+
1704+ Database* source = getSource();
1705+
1706+ QList<QVariant> sync_targets;
1707+
1708+ /*!
1709+ * The validator map contains key and value pair definitions,
1710+ *that are used to confirm that the values provided for each
1711+ *database target are of the expected type for a particular key.
1712+ */
1713+
1714+ QMap<QString,QString>validator;
1715+
1716+ validator.insert("remote","bool");
1717+ validator.insert("location","QString");
1718+ validator.insert("resolve_to_source","bool");
1719+
1720+
1721+ /*!
1722+ * The mandatory map contains the keys that are used to confirm
1723+ *that a database target definition contains all the mandatory keys
1724+ *necessary for synchronizing.
1725+ */
1726+
1727+ QList<QString>mandatory;
1728+
1729+ mandatory.append("remote");
1730+ mandatory.append("resolve_to_source");
1731+
1732+ if(synchronize == true){
1733+
1734+ /*!
1735+ A list of valid sync target databases is generated by calling the getValidTargets(validator, mandatory) method, and adding the return value to a QList (sync_targets).
1736+ */
1737+
1738+ sync_targets=getValidTargets(validator, mandatory);
1739+
1740+ /*!
1741+ * Once the list of sync targets has been generated the sync activity
1742+ *can be initiated via the synchronizeTargets function.
1743+ */
1744+
1745+ synchronizeTargets(source, sync_targets);
1746+
1747+ /*!
1748+ * After the synchronization is complete the model is reset so that
1749+ *log and error messages are available at the application level.
1750+ *
1751+ */
1752+
1753+ beginResetModel();
1754+ endResetModel();
1755+
1756+ /*!
1757+ The convenience signals syncOutputChanged and syncCompleted are
1758+emitted after the model has been reset.
1759+ */
1760+
1761+ Q_EMIT syncOutputChanged(m_sync_output);
1762+ Q_EMIT syncCompleted();
1763+
1764+ /*!
1765+ * The sync boolean value is reset to its default value (false)
1766+ *once all sync activity is complete.
1767+ */
1768+
1769+ setSync(false);
1770+
1771+ }
1772+ else{
1773+
1774+ }
1775+}
1776+
1777+
1778+/*!
1779+ * \internal
1780+ * \brief Synchronizer::getValidTargets
1781+ *
1782+ *
1783+ * This method confirms that each sync target definition is valid, based
1784+ * on predefined criteria contained in the validator and mandatory lists.
1785+ *
1786+ */
1787+
1788+QList<QVariant> Synchronizer::getValidTargets(QMap<QString,QString>validator, QList<QString>mandatory){
1789+
1790+ QList<QVariant> sync_targets;
1791+
1792+ int index = 0;
1793+
1794+ QList<QVariant> targets = getTargets().toList();
1795+
1796+ Q_FOREACH (QVariant target_variant, targets)
1797+ {
1798+ index++;
1799+ QString index_number = QString::number(index);
1800+
1801+ QMap<QString, QVariant> target = target_variant.toMap();
1802+
1803+ bool valid = true;
1804+ bool complete = true;
1805+
1806+ QMapIterator<QString, QVariant> i(target);
1807+ while (i.hasNext()) {
1808+
1809+ i.next();
1810+
1811+ if(validator.contains(i.key())&&validator[i.key()]!=i.value().typeName()){
1812+ valid = false;
1813+
1814+ QString message_value = "For property `" + i.key() + "` Expecting type `" + validator[i.key()] + "`, but received type `" + i.value().typeName()+"`";
1815+
1816+ QVariantMap output_map;
1817+ output_map.insert("concerning_property","targets");
1818+ output_map.insert("concerning_index",index_number);
1819+ output_map.insert("message_type","error");
1820+ output_map.insert("message_value",message_value);
1821+ m_sync_output.append(output_map);
1822+
1823+ target.insert("sync",false);
1824+
1825+ break;
1826+ }
1827+
1828+ if(valid==false){
1829+ targets.removeOne(target);
1830+ break;
1831+ }
1832+ else{
1833+ QListIterator<QString> j(mandatory);
1834+
1835+ while(j.hasNext()){
1836+ QString value = j.next();
1837+ if(!target.contains(value)){
1838+
1839+ QString message_value = "Expected property `" + value + "`, but it is not present.";
1840+
1841+ QVariantMap output_map;
1842+ output_map.insert("concerning_property","targets");
1843+ output_map.insert("concerning_index",index_number);
1844+ output_map.insert("message_type","error");
1845+ output_map.insert("message_value",message_value);
1846+ m_sync_output.append(output_map);
1847+
1848+ target.insert("sync",false);
1849+ targets.removeOne(target);
1850+ complete = false;
1851+ break;
1852+ }
1853+ }
1854+ if(complete==false){
1855+ break;
1856+ }
1857+
1858+ }
1859+ }
1860+
1861+ if(target.contains("sync")&&target["sync"]==false){
1862+
1863+ QString message_value = "Not synced due to errors with properties.";
1864+
1865+ QVariantMap output_map;
1866+ output_map.insert("concerning_property","targets");
1867+ output_map.insert("concerning_index",index_number);
1868+ output_map.insert("message_type","error");
1869+ output_map.insert("message_value",message_value);
1870+ m_sync_output.append(output_map);
1871+
1872+ }
1873+ else
1874+ {
1875+ target.insert("sync",true);
1876+ sync_targets.append(target);
1877+
1878+ QString message_value = "Mandatory properties were included and their values are valid.";
1879+
1880+ QVariantMap output_map;
1881+ output_map.insert("concerning_property","targets");
1882+ output_map.insert("concerning_index",index_number);
1883+ output_map.insert("message_type","no-errors");
1884+ output_map.insert("message_value",message_value);
1885+ m_sync_output.append(output_map);
1886+
1887+ }
1888+
1889+ }
1890+
1891+ return sync_targets;
1892+
1893+}
1894+
1895+/*!
1896+ * \internal
1897+ * \brief Synchronizer::synchronizeTargets
1898+ *
1899+ * The source database is synchronized with the target databases contained
1900+ * in the 'targets' list. That list should only contain valid targets, as
1901+ * determined by Synchronizer::getValidTargets.
1902+ *
1903+ */
1904+
1905+void Synchronizer::synchronizeTargets(Database *source, QVariant targets){
1906+
1907+ if(targets.typeName()== QStringLiteral("QVariantList")){
1908+
1909+ QList<QVariant> target_list = targets.toList();
1910+
1911+ QListIterator<QVariant> i(target_list);
1912+
1913+ int target_index = -1;
1914+
1915+ while(i.hasNext()){
1916+
1917+ target_index++;
1918+
1919+ QVariant target = i.next();
1920+
1921+ if(target.typeName()== QStringLiteral("QVariantMap")){
1922+ QMap<QString,QVariant> target_map = target.toMap();
1923+
1924+ if(target_map.contains("remote")&&target_map["remote"]==false){
1925+ if(target_map.contains("sync")&&target_map["sync"]==true){
1926+
1927+ QString message_value = "Valid local target.";
1928+
1929+ QVariantMap output_map;
1930+ output_map.insert("concerning_property","targets");
1931+ output_map.insert("concerning_index",target_index);
1932+ output_map.insert("message_type","no-errors");
1933+ output_map.insert("message_value",message_value);
1934+ m_sync_output.append(output_map);
1935+
1936+ syncLocalToLocal(source, target_map);
1937+ }
1938+ }
1939+ else if(target_map.contains("remote")&&target_map["remote"]==true){
1940+ if(target_map.contains("sync")&&target_map["sync"]==true){
1941+
1942+ //ip
1943+ //port
1944+ //name
1945+ //GET /thedb/sync-from/my_replica_uid
1946+
1947+ QString source_uid = getUidFromLocalDb(source->getPath());
1948+ QString get_string = target_map["name"].toString()+"/sync-from/"+source_uid;
1949+ QString url_string = "http://"+target_map["ip"].toString();
1950+ QString full_get_request = url_string+"/"+get_string;
1951+ int port_number = target_map["port"].toInt();
1952+
1953+ QNetworkAccessManager *manager = new QNetworkAccessManager(source);
1954+
1955+ QUrl url(full_get_request);
1956+ url.setPort(port_number);
1957+
1958+ QNetworkRequest request(url);
1959+
1960+ connect(manager, &QNetworkAccessManager::finished, this, &Synchronizer::remoteGetSyncInfoFinished);
1961+
1962+ QString message_value = "Valid remote target.";
1963+
1964+ QVariantMap output_map;
1965+ output_map.insert("concerning_property","targets");
1966+ output_map.insert("concerning_index",target_index);
1967+ output_map.insert("message_type","no-errors");
1968+ output_map.insert("message_value",message_value);
1969+ m_sync_output.append(output_map);
1970+
1971+ manager->get(QNetworkRequest(request));
1972+
1973+ }
1974+ }
1975+ else{
1976+
1977+ QString message_value = "Unknown error. Please check properties";
1978+
1979+ QVariantMap output_map;
1980+ output_map.insert("concerning_property","targets");
1981+ output_map.insert("concerning_index",target_index);
1982+ output_map.insert("message_type","error");
1983+ output_map.insert("message_value",message_value);
1984+ m_sync_output.append(output_map);
1985+
1986+ }
1987+
1988+ }
1989+
1990+ }
1991+
1992+ }
1993+
1994+}
1995+
1996+/*!
1997+ * \internal
1998+ * \brief Synchronizer::syncLocalToLocal
1999+ *
2000+ * This function synchronizes two local databases, a source database and a target database.
2001+ *
2002+ */
2003+
2004+void Synchronizer::syncLocalToLocal(Database *sourceDb, QMap<QString,QVariant> target)
2005+{
2006+
2007+ QString target_db_name = target["location"].toString();
2008+
2009+ Database *targetDb;
2010+ Index *targetIndex;
2011+ Query *targetQuery;
2012+
2013+ Index *sourceIndex;
2014+ Query *sourceQuery;
2015+
2016+ if(target.contains("id")){
2017+ targetDb = (Database*)target["id"].value<QObject*>();
2018+ }
2019+
2020+ if(target.contains("target_query")){
2021+ targetQuery = (Query*)target["target_query"].value<QObject*>();
2022+ targetIndex = targetQuery->getIndex();
2023+ targetDb = targetIndex->getDatabase();
2024+ }
2025+
2026+ if(target.contains("source_query")){
2027+ sourceQuery = (Query*)target["source_query"].value<QObject*>();
2028+ sourceIndex = sourceQuery->getIndex();
2029+ sourceDb = sourceIndex->getDatabase();
2030+ }
2031+
2032+ if(sourceDb == NULL || targetDb == NULL){
2033+
2034+ QString message_value = "Either source or target does not exist or is not active.";
2035+
2036+ QVariantMap output_map;
2037+ output_map.insert("concerning_property","source|targets");
2038+ //output_map.insert("concerning_index",index_number); // no access to targets index?
2039+ output_map.insert("message_type","error");
2040+ output_map.insert("message_value",message_value);
2041+ m_sync_output.append(output_map);
2042+
2043+ return;
2044+ }
2045+
2046+ QMap<QString,QVariant> lastSyncInformation;
2047+
2048+ lastSyncInformation.insert("target_replica_uid",getUidFromLocalDb(target_db_name));
2049+ lastSyncInformation.insert("target_replica_generation","");
2050+ lastSyncInformation.insert("target_replica_transaction_id",-1);
2051+ lastSyncInformation.insert("source_replica_uid",getUidFromLocalDb(sourceDb->getPath()));
2052+ lastSyncInformation.insert("source_replica_generation","");
2053+ lastSyncInformation.insert("source_replica_transaction_id",-1);
2054+
2055+ lastSyncInformation = getLastSyncInformation(sourceDb, targetDb, false, lastSyncInformation);
2056+
2057+ QList<QString> transactionsFromSource;
2058+ QList<QString> transactionsFromTarget;
2059+
2060+ // Check if target and source have ever been synced before
2061+
2062+ if(lastSyncInformation["target_replica_uid"].toString() != "" && lastSyncInformation["target_replica_generation"].toString() != "" && lastSyncInformation["target_replica_transaction_id"].toInt() != -1 && lastSyncInformation["source_replica_uid"].toString() != "" && lastSyncInformation["source_replica_generation"].toString() != "" && lastSyncInformation["source_replica_transaction_id"].toInt() != -1)
2063+ {
2064+
2065+ QString message_value = "Source and local database have previously synced.";
2066+
2067+ QVariantMap output_map;
2068+ output_map.insert("concerning_property","source|targets");
2069+ output_map.insert("concerning_source",sourceDb->getPath());
2070+ output_map.insert("concerning_target",target_db_name);
2071+ output_map.insert("message_type","no-errors");
2072+ output_map.insert("message_value",message_value);
2073+ m_sync_output.append(output_map);
2074+
2075+ //Do some syncing
2076+
2077+ transactionsFromSource = sourceDb->listTransactionsSince(lastSyncInformation["source_replica_generation"].toInt());
2078+
2079+ transactionsFromTarget = targetDb->listTransactionsSince(lastSyncInformation["target_replica_generation"].toInt());
2080+
2081+
2082+ }
2083+ else{
2084+
2085+ QString message_value = "Source and local database have not previously synced.";
2086+
2087+ QVariantMap output_map;
2088+ output_map.insert("concerning_property","source|targets");
2089+ output_map.insert("concerning_source",sourceDb->getPath());
2090+ output_map.insert("concerning_target",target_db_name);
2091+ output_map.insert("message_type","no-errors");
2092+ output_map.insert("message_value",message_value);
2093+ m_sync_output.append(output_map);
2094+
2095+ //There is a first time for everything, let's sync!
2096+
2097+ transactionsFromSource = sourceDb->listTransactionsSince(0);
2098+
2099+ transactionsFromTarget = targetDb->listTransactionsSince(0);
2100+ }
2101+
2102+ /*!
2103+ * With two distinct lists present, it is now possible to check what
2104+ * updates should be made, or new documents created in one or the other
2105+ * database, depending on conditions.
2106+ *
2107+ * However, two additional lists containing transactions IDs are required
2108+ * because the information is contained within delimited strings (see
2109+ * below for details).
2110+ *
2111+ */
2112+
2113+ QList<QString> transactionIdsFromSource;
2114+ QList<QString> transactionIdsFromTarget;
2115+
2116+ Q_FOREACH(QString sourceTransaction, transactionsFromSource){
2117+
2118+ /*!
2119+ * Each sourceTransaction is a pipe delimited string containing
2120+ * generation number, document ID, and transaction ID details
2121+ * in that order.
2122+ *
2123+ * Splitting the string into its component pieces provides a
2124+ * document ID (the second key in the list).
2125+ */
2126+
2127+ QStringList transactionDetails = sourceTransaction.split("|");
2128+
2129+ /*!
2130+ * It is only necessary to have unique instances of the
2131+ * document ID.
2132+ */
2133+
2134+ if(!transactionIdsFromSource.contains(transactionDetails[1]))
2135+ transactionIdsFromSource.append(transactionDetails[1]);
2136+
2137+ }
2138+
2139+ Q_FOREACH(QString targetTransaction, transactionsFromTarget){
2140+
2141+ /*!
2142+ * Each targetTransaction is a pipe delimited string containing
2143+ * generation number, document ID, and transaction ID details
2144+ * in that order.
2145+ *
2146+ * Splitting the string into its component pieces provides a
2147+ * document ID (the second key in the list).
2148+ */
2149+
2150+ QStringList transactionDetails = targetTransaction.split("|");
2151+
2152+ /*!
2153+ * It is only necessary to have unique instances of the
2154+ * document ID.
2155+ */
2156+
2157+ if(!transactionIdsFromTarget.contains(transactionDetails[1]))
2158+ transactionIdsFromTarget.append(transactionDetails[1]);
2159+
2160+ }
2161+
2162+
2163+ /* The source replica asks the target replica for the information it has stored about the last time these two replicas were synchronised (if ever).*/
2164+
2165+
2166+ /*
2167+
2168+ The application wishing to synchronise sends the following GET request to the server:
2169+
2170+ GET /thedb/sync-from/my_replica_uid
2171+
2172+ Where thedb is the name of the database to be synchronised, and my_replica_uid is the replica id of the application’s (i.e. the local, or synchronisation source) database
2173+
2174+ */
2175+
2176+ /*
2177+
2178+ The target responds with a JSON document that looks like this:
2179+
2180+ {
2181+ "target_replica_uid": "other_replica_uid",
2182+ "target_replica_generation": 12,
2183+ "target_replica_transaction_id": "T-sdkfj92292j",
2184+ "source_replica_uid": "my_replica_uid",
2185+ "source_replica_generation": 23,
2186+ "source_transaction_id": "T-39299sdsfla8"
2187+ }
2188+
2189+ With all the information it has stored for the most recent synchronisation between itself and this particular source replica. In this case it tells us that the synchronisation target believes that when it and the source were last synchronised, the target was at generation 12 and the source at generation 23.
2190+
2191+ */
2192+
2193+
2194+ /* The source replica validates that its information regarding the last synchronisation is consistent with the target’s information, and raises an error if not. (This could happen for instance if one of the replicas was lost and restored from backup, or if a user inadvertently tries to synchronise a copied database.) */
2195+
2196+
2197+
2198+ /* The source replica generates a list of changes since the last change the target replica knows of. */
2199+
2200+
2201+
2202+ /* The source replica checks what the last change is it knows about on the target replica. */
2203+
2204+
2205+
2206+ /* If there have been no changes on either replica that the other side has not seen, the synchronisation stops here. */
2207+
2208+
2209+
2210+ /* The source replica sends the changed documents to the target, along with what the latest change is that it knows about on the target replica. */
2211+
2212+
2213+
2214+ /* The target processes the changed documents, and records the source replica’s latest change. */
2215+
2216+
2217+
2218+ /* The target responds with the documents that have changes that the source does not yet know about. */
2219+
2220+ /* The source processes the changed documents, and records the target replica’s latest change. */
2221+
2222+
2223+
2224+ /* If the source has seen no changes unrelated to the synchronisation during this whole process, it now sends the target what its latest change is, so that the next synchronisation does not have to consider changes that were the result of this one.*/
2225+
2226+
2227+}
2228+
2229+/*!
2230+ * \internal
2231+ * \brief Synchronizer::syncDocument
2232+ *
2233+ *
2234+ * This method is used to synchronize documents from one local database
2235+ * to another local database.
2236+ *
2237+ */
2238+
2239+
2240+QVariant Synchronizer::syncDocument(Database *from, Database *to, QString docId)
2241+{
2242+ QVariant document = from->getDoc(docId);
2243+ to->putDoc(document, docId);
2244+ QString revision = from->getCurrentDocRevisionNumber(docId);
2245+ to->updateDocRevisionNumber(docId,revision);
2246+
2247+ return document;
2248+}
2249+
2250+/*!
2251+ * \internal
2252+ * \brief Synchronizer::getLastSyncInformation
2253+ *
2254+ *
2255+ * If the source and target database have ever been synced before the information
2256+ * from that previous session is returned. This is only used for local to local
2257+ * databases. The local to remote procedure is handled elsewhere in a different manner.
2258+ *
2259+ */
2260+
2261+QMap<QString,QVariant> Synchronizer::getLastSyncInformation(Database *sourceDb, Database *targetDb, bool remote, QMap<QString,QVariant> lastSyncInformation){
2262+
2263+ if(remote == true){
2264+
2265+ QString message_value = "Sync information from remote target not available at this time.";
2266+
2267+ QVariantMap output_map;
2268+ output_map.insert("concerning_property","source|targets");
2269+ output_map.insert("concerning_source",sourceDb->getPath());
2270+ output_map.insert("message_type","warning");
2271+ output_map.insert("message_value",message_value);
2272+ m_sync_output.append(output_map);
2273+
2274+ return lastSyncInformation;
2275+
2276+ }
2277+ else{
2278+
2279+ lastSyncInformation["source_replica_uid"].toString();
2280+
2281+ lastSyncInformation = targetDb->getSyncLogInfo(lastSyncInformation, lastSyncInformation["source_replica_uid"].toString(),"target");
2282+
2283+ lastSyncInformation = sourceDb->getSyncLogInfo(lastSyncInformation, lastSyncInformation["target_replica_uid"].toString(),"source");
2284+
2285+ }
2286+
2287+ return lastSyncInformation;
2288+
2289+}
2290+
2291+/*!
2292+ * \internal
2293+ * \brief Synchronizer::getUidFromLocalDb
2294+ *
2295+ *
2296+ * The unique id of a database is needed in certain situations of a
2297+ * synchronize session. This method retrieves that id from a local database.
2298+ *
2299+ */
2300+
2301+QString Synchronizer::getUidFromLocalDb(QString dbFileName)
2302+{
2303+
2304+ QString dbUid;
2305+
2306+ QSqlDatabase db;
2307+
2308+ db = QSqlDatabase::addDatabase("QSQLITE",QUuid::createUuid().toString());
2309+
2310+ QFile db_file(dbFileName);
2311+
2312+ if(!db_file.exists())
2313+ {
2314+
2315+ QString message_value = "Database does not exist.";
2316+
2317+ QVariantMap output_map;
2318+ output_map.insert("concerning_property","source|targets");
2319+ output_map.insert("concerning_database",dbFileName);
2320+ output_map.insert("message_type","error");
2321+ output_map.insert("message_value",message_value);
2322+ m_sync_output.append(output_map);
2323+
2324+ return dbUid;
2325+ }
2326+ else
2327+ {
2328+ db.setDatabaseName(dbFileName);
2329+ if (!db.open()){
2330+
2331+ QString message_value = db.lastError().text();
2332+
2333+ QVariantMap output_map;
2334+ output_map.insert("concerning_property","source|targets");
2335+ output_map.insert("concerning_database",dbFileName);
2336+ output_map.insert("message_type","error");
2337+ output_map.insert("message_value",message_value);
2338+ m_sync_output.append(output_map);
2339+
2340+ }
2341+ else{
2342+ QSqlQuery query (db.exec("SELECT value FROM u1db_config WHERE name = 'replica_uid'"));
2343+
2344+ if(!query.lastError().isValid() && query.next()){
2345+ dbUid = query.value(0).toString();
2346+ db.close();
2347+
2348+ dbUid = dbUid.replace("{","");
2349+
2350+ dbUid = dbUid.replace("}","");
2351+
2352+ return dbUid;
2353+ }
2354+ else{
2355+ qDebug() << query.lastError().text();
2356+ db.close();
2357+ return dbUid;
2358+ }
2359+ }
2360+
2361+ }
2362+
2363+ return dbUid;
2364+}
2365+
2366+/*!
2367+ * \internal
2368+ * \brief Synchronizer::remoteGetSyncInfoFinished
2369+ *
2370+ * Once the initial exchange between the client application
2371+ * and the remote server is complete, this method retrieves
2372+ * necessary information from the reply that came from the
2373+ * server, and is required for posting data from the client
2374+ * in the steps that will follow.
2375+ *
2376+ * After the data is saved to a string and the network reply
2377+ * is closed, the appropriate method for posting
2378+ * (postDataFromClientToRemoteServer) is then called.
2379+ *
2380+ */
2381+
2382+void Synchronizer::remoteGetSyncInfoFinished(QNetworkReply* reply)
2383+{
2384+
2385+ QNetworkAccessManager *manager = reply->manager();
2386+
2387+ Database *source = qobject_cast<Database *>(manager->parent());
2388+
2389+ QUrl postUrl = reply->request().url();
2390+
2391+ QByteArray data = reply->readAll();
2392+
2393+ QString replyData = QString(data);
2394+
2395+ reply->close();
2396+
2397+ postDataFromClientToRemoteServer(source, postUrl, replyData);
2398+}
2399+
2400+
2401+/*!
2402+ * \internal
2403+ * \brief Synchronizer::postDataFromClientToRemoteServer
2404+ *
2405+ * This method builds a string for posting from the client
2406+ * application to the remote server, using information previously
2407+ * gathered from the source and target databases,
2408+ * and then initiates the post.
2409+ *
2410+ */
2411+
2412+void Synchronizer::postDataFromClientToRemoteServer(Database *source, QUrl postUrl, QString replyData)
2413+{
2414+
2415+ QVariantMap replyMap;
2416+
2417+ QJsonDocument replyJson = QJsonDocument::fromJson(replyData.toUtf8());
2418+
2419+ QVariant replyVariant = replyJson.toVariant();
2420+
2421+ replyMap = replyVariant.toMap();
2422+
2423+ double source_replica_generation = replyMap["source_replica_generation"].toDouble();
2424+ QString source_replica_uid = replyMap["source_replica_uid"].toString();
2425+ QString source_replica_transaction_id = replyMap["source_transaction_id"].toString();
2426+ //double target_replica_generation = replyMap["target_replica_generation"].toDouble();
2427+ QString target_replica_transaction_id = replyMap["target_replica_transaction_id"].toString();
2428+ QString target_replica_uid = replyMap["target_replica_uid"].toString();
2429+
2430+ QNetworkAccessManager *manager = new QNetworkAccessManager(source);
2431+
2432+ connect(manager, &QNetworkAccessManager::finished, this, &Synchronizer::remotePostSyncInfoFinished);
2433+
2434+ QByteArray postString;
2435+
2436+ postString = "[\r\n";
2437+ postString.append("{\"last_known_generation\": ");
2438+ postString.append(QByteArray::number(source_replica_generation));
2439+ postString.append(", \"last_known_trans_id\": \"");
2440+ postString.append(source_replica_transaction_id);
2441+ postString.append("\"}");
2442+
2443+ QList<QString> transactions = m_source->listTransactionsSince(source_replica_generation);
2444+
2445+ Q_FOREACH(QString transaction,transactions){
2446+ QStringList transactionData = transaction.split("|");
2447+
2448+ QString content = source->getDocumentContents(transactionData[1]);
2449+ content = content.replace("\r\n","");
2450+ content = content.replace("\r","");
2451+ content = content.replace("\n","");
2452+ content = content.replace("\"","\\\"");
2453+
2454+ postString.append(",\r\n{\"content\": \""+content+"\",\"rev\": \""+m_source->getCurrentDocRevisionNumber(transactionData[1])+"\", \"id\": \""+transactionData[1]+"\",\"trans_id\": \""+transactionData[2]+"\",\"gen\": "+transactionData[0]+"}");
2455+
2456+ }
2457+
2458+ postString.append("\r\n]");
2459+
2460+ QByteArray postDataSize = QByteArray::number(postString.size());
2461+
2462+ QNetworkRequest request(postUrl);
2463+ request.setRawHeader("User-Agent", "U1Db-Qt v1.0");
2464+ request.setRawHeader("X-Custom-User-Agent", "U1Db-Qt v1.0");
2465+ request.setRawHeader("Content-Type", "application/x-u1db-sync-stream");
2466+ request.setRawHeader("Content-Length", postDataSize);
2467+
2468+ manager->post(QNetworkRequest(request),postString);
2469+
2470+}
2471+
2472+/*!
2473+ * \internal
2474+ * \brief Synchronizer::remotePostSyncInfoFinished
2475+ *
2476+ * This method is a slot, which is called once the data
2477+ * from the client application has been posted to the remote
2478+ * server.
2479+ *
2480+ * This is where any new data from the
2481+ * remote server is gathered, and then the further steps
2482+ * for processing on the client side are begun.
2483+ */
2484+
2485+void Synchronizer::remotePostSyncInfoFinished(QNetworkReply* reply)
2486+{
2487+
2488+ QNetworkAccessManager *manager = reply->manager();
2489+
2490+ Database *source = qobject_cast<Database *>(manager->parent());
2491+
2492+ QByteArray data = reply->readAll();
2493+
2494+ QString replyData = QString(data);
2495+
2496+ reply->close();
2497+
2498+ processDataFromRemoteServer(source, replyData);
2499+
2500+}
2501+
2502+/*!
2503+ * \internal
2504+ * \brief Synchronizer::processDataFromRemoteServer
2505+ *
2506+ * After the remote target database has replied back, the data it has sent
2507+ * is processed to determine what action to take on any relevant documents
2508+ * that may have been included in the data.
2509+ *
2510+ */
2511+
2512+void Synchronizer::processDataFromRemoteServer(Database *source, QString replyData)
2513+{
2514+
2515+ replyData = replyData.replace("\r\n","");
2516+
2517+ QJsonDocument replyJson = QJsonDocument::fromJson(replyData.toUtf8());
2518+
2519+ QVariant replyVariant = replyJson.toVariant();
2520+
2521+ QVariantList replyList = replyVariant.toList();
2522+
2523+ QListIterator<QVariant> i(replyList);
2524+
2525+ int index = -1;
2526+
2527+ while(i.hasNext()){
2528+
2529+ index++;
2530+
2531+ QVariant current = i.next();
2532+
2533+ QString type_name = QString::fromUtf8(current.typeName());
2534+
2535+ if(type_name == "QVariantMap")
2536+ {
2537+
2538+ QVariantMap map = current.toMap();
2539+
2540+ if(index == 0)
2541+ {
2542+ // Meta data
2543+ }
2544+ else
2545+ {
2546+ // Document to update
2547+
2548+ QString id("");
2549+ QVariant content("");
2550+ QString rev("");
2551+
2552+ QMapIterator<QString, QVariant> i(map);
2553+
2554+ while (i.hasNext()) {
2555+
2556+ i.next();
2557+
2558+ if(i.key()=="content")
2559+ {
2560+ content = i.value();
2561+ }
2562+ else if(i.key()=="id")
2563+ {
2564+ id = i.value().toString();
2565+ }
2566+ else if(i.key()=="rev")
2567+ {
2568+ rev = i.value().toString();
2569+ }
2570+
2571+ }
2572+
2573+ if(content!=""&&id!=""&&rev!="")
2574+ {
2575+ source->putDoc(content,id);
2576+ source->updateDocRevisionNumber(id,rev);
2577+ }
2578+
2579+ }
2580+
2581+ }
2582+
2583+ }
2584+
2585+}
2586+
2587+QT_END_NAMESPACE_U1DB
2588+
2589+#include "moc_synchronizer.cpp"
2590+
2591
2592=== added file 'src/synchronizer.h'
2593--- src/synchronizer.h 1970-01-01 00:00:00 +0000
2594+++ src/synchronizer.h 2013-08-12 16:20:45 +0000
2595@@ -0,0 +1,146 @@
2596+/*
2597+ * Copyright (C) 2013 Canonical, Ltd.
2598+ *
2599+ * Authors:
2600+ * Kevin Wright <kevin.wright@canonical.com>
2601+ *
2602+ * This program is free software; you can redistribute it and/or modify
2603+ * it under the terms of the GNU Lesser General Public License as published by
2604+ * the Free Software Foundation; version 3.
2605+ *
2606+ * This program is distributed in the hope that it will be useful,
2607+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
2608+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2609+ * GNU Lesser General Public License for more details.
2610+ *
2611+ * You should have received a copy of the GNU Lesser General Public License
2612+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
2613+ */
2614+
2615+#ifndef U1DB_SYNCHRONIZER_H
2616+#define U1DB_SYNCHRONIZER_H
2617+
2618+#include <QtCore/QObject>
2619+#include <QSqlDatabase>
2620+#include <QVariant>
2621+#include <QMetaType>
2622+#include <QtNetwork>
2623+#include <QNetworkAccessManager>
2624+
2625+
2626+#include "database.h"
2627+#include "index.h"
2628+#include "query.h"
2629+
2630+QT_BEGIN_NAMESPACE_U1DB
2631+
2632+class Q_DECL_EXPORT Synchronizer : public QAbstractListModel {
2633+ Q_OBJECT
2634+#ifdef Q_QDOC
2635+ Q_PROPERTY(Database* source READ getSource WRITE setSource NOTIFY sourceChanged)
2636+#else
2637+ Q_PROPERTY(QT_PREPEND_NAMESPACE_U1DB(Database*) source READ getSource WRITE setSource NOTIFY sourceChanged)
2638+#endif
2639+ Q_PROPERTY(bool synchronize READ getSync WRITE setSync NOTIFY syncChanged)
2640+ Q_PROPERTY(bool resolve_to_source READ getResolveToSource WRITE setResolveToSource NOTIFY resolveToSourceChanged)
2641+ Q_PROPERTY(QVariant targets READ getTargets WRITE setTargets NOTIFY targetsChanged)
2642+ Q_PROPERTY(QList<QVariant> sync_output READ getSyncOutput NOTIFY syncOutputChanged)
2643+
2644+public:
2645+ Synchronizer(QObject* parent = 0);
2646+
2647+ // QAbstractListModel
2648+ QVariant data(const QModelIndex & index, int role = Qt::DisplayRole) const;
2649+ QHash<int, QByteArray>roleNames() const;
2650+ int rowCount(const QModelIndex & parent = QModelIndex()) const;
2651+
2652+ QList<QVariant> getValidTargets(QMap<QString,QString>validator, QList<QString>mandatory);
2653+ QMap<QString,QVariant> getLastSyncInformation(Database *sourceDb, Database *targetDb, bool remote, QMap<QString,QVariant> lastSyncInformation);
2654+
2655+
2656+ Database* getSource();
2657+ QVariant getTargets();
2658+ bool getSync();
2659+ bool getResolveToSource();
2660+ QList<QVariant> getSyncOutput();
2661+
2662+ void setSource(Database* source);
2663+ void setTargets(QVariant targets);
2664+ void setSync(bool synchronize);
2665+ void setResolveToSource(bool resolve_to_source);
2666+ void setSyncOutput(QList<QVariant> sync_output);
2667+
2668+ void syncLocalToLocal(Database *sourceDb, QMap<QString,QVariant> target);
2669+ void synchronizeTargets(Database *source, QVariant targets);
2670+
2671+ QVariant syncDocument(Database *from, Database *to, QString docId);
2672+ QString getUidFromLocalDb(QString dbFileName);
2673+
2674+ void postDataFromClientToRemoteServer(Database *source, QUrl postUrl, QString replyData);
2675+ void processDataFromRemoteServer(Database *source, QString replyData);
2676+
2677+
2678+Q_SIGNALS:
2679+
2680+ /*!
2681+ * \brief sourceChanged
2682+ * \param source
2683+ */
2684+
2685+ void sourceChanged(Database* source);
2686+
2687+ /*!
2688+ * \brief targetsChanged
2689+ * \param targets
2690+ */
2691+
2692+ void targetsChanged(QVariant targets);
2693+
2694+ /*!
2695+ * \brief syncChanged
2696+ * \param synchronize
2697+ */
2698+
2699+ void syncChanged(bool synchronize);
2700+
2701+ /*!
2702+ * \brief resolveToSourceChanged
2703+ * \param resolve_to_source
2704+ */
2705+
2706+ void resolveToSourceChanged(bool resolve_to_source);
2707+
2708+ /*!
2709+ * \brief syncOutputChanged
2710+ * \param sync_output
2711+ */
2712+
2713+ void syncOutputChanged(QList<QVariant> sync_output);
2714+
2715+ /*!
2716+ * \brief syncCompleted
2717+ */
2718+
2719+ void syncCompleted();
2720+
2721+private:
2722+ //Q_DISABLE_COPY(Synchronizer)
2723+ Database* m_source;
2724+ bool m_synchronize;
2725+ bool m_resolve_to_source;
2726+ QVariant m_targets;
2727+ QList<QVariant> m_sync_output;
2728+
2729+ void onSyncChanged(bool synchronize);
2730+ void remoteGetSyncInfoFinished(QNetworkReply* reply);
2731+ void remotePostSyncInfoFinished(QNetworkReply* reply);
2732+
2733+};
2734+
2735+QT_END_NAMESPACE_U1DB
2736+
2737+
2738+
2739+#endif // U1DB_SYNCHRONIZER_H
2740+
2741+
2742
2743=== modified file 'tests/tst_database.qml'
2744--- tests/tst_database.qml 2013-07-16 17:17:53 +0000
2745+++ tests/tst_database.qml 2013-08-12 16:20:45 +0000
2746@@ -72,15 +72,15 @@
2747
2748 function test_1_databasePopulated () {
2749 spyListCompleted.wait()
2750- compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]}) > -1, true)
2751+ compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]}) == '', false)
2752
2753 var myPath = "aDatabaseA"
2754 myDatabase.path = myPath
2755 spyPathChanged.wait()
2756 compare(myDatabase.path, myPath)
2757- compare(myDatabase.putDoc({"spam": "eggs"}) > -1, true)
2758+ compare(myDatabase.putDoc({"spam": "eggs"}) == '', false)
2759 var json = {"foo": "bar"}
2760- compare(myDatabase.putDoc(json, "hijklmn") > -1, true)
2761+ compare(myDatabase.putDoc(json, "hijklmn") == '', false)
2762 compare(myDatabase.getDoc("hijklmn"), json)
2763 compare(myDatabase.getDoc("hijklmn"), json)
2764 }
2765
2766=== modified file 'tests/tst_query.qml'
2767--- tests/tst_query.qml 2013-05-10 16:04:04 +0000
2768+++ tests/tst_query.qml 2013-08-12 16:20:45 +0000
2769@@ -117,6 +117,7 @@
2770 U1db.Query {
2771 id: wrongQuery
2772 index: byNamePhone
2773+ query: [{ 'name': 'Ivanka', 'phone': '*' }]
2774 }
2775
2776 SignalSpy {

Subscribers

People subscribed via source and target branches

to all changes: