Merge lp:~kevin-wright-1/u1db-qt/synchronizer-merged-with-trunk-8-aug into lp:u1db-qt
- synchronizer-merged-with-trunk-8-aug
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Cris Dywan | ||||
Approved revision: | 126 | ||||
Merged at revision: | 100 | ||||
Proposed branch: | lp:~kevin-wright-1/u1db-qt/synchronizer-merged-with-trunk-8-aug | ||||
Merge into: | lp:u1db-qt | ||||
Diff against target: |
2776 lines (+2415/-23) 15 files modified
CMakeLists.txt (+2/-2) documentation/u1db.qdocconf (+3/-1) examples/u1db-qt-example-6/u1db-qt-example-6.qdoc (+387/-0) examples/u1db-qt-example-6/u1db-qt-example-6.qml (+173/-0) modules/U1db/CMakeLists.txt (+4/-0) modules/U1db/plugin.cpp (+2/-0) src/CMakeLists.txt (+5/-1) src/database.cpp (+395/-14) src/database.h (+19/-2) src/query.cpp (+16/-0) src/query.h (+2/-0) src/synchronizer.cpp (+1257/-0) src/synchronizer.h (+146/-0) tests/tst_database.qml (+3/-3) tests/tst_query.qml (+1/-0) |
||||
To merge this branch: | bzr merge lp:~kevin-wright-1/u1db-qt/synchronizer-merged-with-trunk-8-aug | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
PS Jenkins bot | continuous-integration | Approve | |
Cris Dywan | Approve | ||
Review via email: mp+179173@code.launchpad.net |
Commit message
Description of the change
This branch merges the latest trunk with the code for synchronization of databases.
PS Jenkins bot (ps-jenkins) wrote : | # |
Cris Dywan (kalikiana) wrote : | # |
The code deals with id's being wrapped in {} in places. That's based on behavior of the core code; I think we need a newid() function and solve this in one place.
There's a few instances of the pattern "if (!query.exec()) qDebug" which should probably be using setError.
resetModel can be used in more places currently still doing its equivalent calls.
getValidTargets() and synchronizeTarg
m_errors.
FIXMEs that ought to be sorted out:
##KW## The two lists below are currently not used for anything
##KW## maybe this can be removed as it is replaced by revision_number
Unused code:
//targetDb-
//double target_
This should probably say the real app name + " (U1Db-Qt)"
request.
request.
For good measure processDataFrom
This should not be writable, it's only set internally
Q_PROPERTY(
"where u1db has been downloaded/
This is very ambiguous and doesn't say where to obtain it. It should say
"where the u1db Python reference implemented has been downloaded from lp:u1db"
- 114. By Kevin Wright
-
Removed an unecessary function call and code comment from src/database.cpp.
- 115. By Kevin Wright
-
Removed an unecessary code from src/synchronize
r.cpp, and updated putDoc in src/database.cpp to return QString instead of int. - 116. By Kevin Wright
-
Updated code comments in src/synchronize
r.cpp to be more descriptive with respect to the instructions for how to use the plugin with the python implementation (the server part to be specific). - 117. By Kevin Wright
-
Added 'U1Db-Qt v1.0' to the raw header information sent to the server.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:116
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
Kevin Wright (kevin-wright-1) wrote : | # |
Den 08/08/2013 16:24, skrev Christian Dywan:
> Review: Needs Fixing
>
> The code deals with id's being wrapped in {} in places. That's based on behavior of the core code; I think we need a newid() function and solve this in one place.
Agreed. Will add.
> There's a few instances of the pattern "if (!query.exec()) qDebug" which should probably be using setError.
Agreed. Will modify.
>
> resetModel can be used in more places currently still doing its equivalent calls.
Not sure what this comment means.
> getValidTargets() and synchronizeTarg
> m_errors.
The type of errors here are for the benefit of app devs and app users at
the UI level, not console etc. The messages are for use in a QML
application as output from a sync transation to notify re: errors,
warnings and things that went according to plan.
An alternative approach, with broader and more flexible possibilities,
is to have the error information available in a model (QVariantMap) that
includes useful meta data and values (e.g. type of message, message
text, other?). In that way it can be available to app developers as well
as easily adapted for other purposes (e.g. log files, console output).
>
> FIXMEs that ought to be sorted out:
> ##KW## The two lists below are currently not used for anything
> ##KW## maybe this can be removed as it is replaced by revision_number
Both removed.
>
> Unused code:
> //targetDb-
Could not find this one. Perhaps it was deleted in another modification.
> //double target_
This variable is probably needed in upcoming modifications, so it is
there but commented out so it does not produce an annoying warning at
compile time.
>
> This should probably say the real app name + " (U1Db-Qt)"
> request.
> request.
Changed to 'U1Db-Qt v1.0' on both lines. Those two lines may not even be
necessary, as the server may ignore them, but are kept there and posted
anyway for good measure.
>
> For good measure processDataFrom
Not sure it would be needed. If an unexpected key showed up it would be
ignored (as it should be).
>
> This should not be writable, it's only set internally
> Q_PROPERTY(
Agreed. Will modify accordingly.
> "where u1db has been downloaded/
> This is very ambiguous and doesn't say where to obtain it. It should say
> "where the u1db Python reference implemented has been downloaded from lp:u1db"
Changed.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:117
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
Cris Dywan (kalikiana) wrote : | # |
> > getValidTargets() and synchronizeTarg
> that's not good if we plan to see errors in console or XML and will make those
> cases harder to deal with. The messages should be predictable without using
> markup - if needed, it's still an option to use regex.
> > m_errors.
> "+index_
>
> The type of errors here are for the benefit of app devs and app users at
> the UI level, not console etc. The messages are for use in a QML
> application as output from a sync transation to notify re: errors,
> warnings and things that went according to plan.
>
> An alternative approach, with broader and more flexible possibilities,
> is to have the error information available in a model (QVariantMap) that
> includes useful meta data and values (e.g. type of message, message
> text, other?). In that way it can be available to app developers as well
> as easily adapted for other purposes (e.g. log files, console output).
Okay, I better see the reasoning now. And I think you're mixing two concepts into here, UI error messages and error handling API for letting developers distinguish errors. The user must not see verbatim key names. And the app developer needs a way to distinguish a failing sync from a bug in his code.
I had a thought in that direction in respect to Database::setError originally. SQL errors aren't interesting to anyone but U1Db-Qt developers - they need to be simple and pastable. The same is often true for U1Db-Qt errors. Errors about invalid id's or json are indeed relevant for developers - but only in the console. There's no need to "handle" them, wrong use of the API is to be fixed. One kind of error that we should have API for is: no space left. I apparently never filed a bug for that.
Now for sync, I'd still want the same console output for obvious mistakes like invalid URLs and wrong key names. That is what I was having in mind when I suggested to not have HTML. It's a one-time see the mistake and fix it and not for UI. And it is useful to always log if a connection failed in any way. For the developer's benefit we should have "signal syncFailed". That should suffice to prompt the user appropriately - the developer knows their target audience, what language to use etc.
> > For good measure processDataFrom
> unexpected i.key() is seen?
>
> Not sure it would be needed. If an unexpected key showed up it would be
> ignored (as it should be).
I should clarify: if an unexpected key turns up that's a bug in the code. Q_ASSERT is better I guess.
Kevin Wright (kevin-wright-1) wrote : | # |
Den 08/08/2013 17:32, skrev Christian Dywan:
> Review: Needs Fixing
>
>
>>> For good measure processDataFrom
>> unexpected i.key() is seen?
>>
>> Not sure it would be needed. If an unexpected key showed up it would be
>> ignored (as it should be).
> I should clarify: if an unexpected key turns up that's a bug in the code. Q_ASSERT is better I guess.
In fact there are other key and value pairs that the server does return
that are not yet being used by the Synchronizer class.
Of course it is harmless to notify that there is more information than
expected (or in this case information that is expected but not being
used), and discounting the possibility that at some point in time there
might be a benefit to doing so would not be wise.
For the record though, in this particular case, if there were an
unexpected key (and associated value) that turns up it is because it was
information delivered directly from the server.
Kevin
Kevin Wright (kevin-wright-1) wrote : | # |
Den 08/08/2013 17:32, skrev Christian Dywan:
> Review: Needs Fixing
>
>>> getValidTargets() and synchronizeTarg
>> that's not good if we plan to see errors in console or XML and will make those
>> cases harder to deal with. The messages should be predictable without using
>> markup - if needed, it's still an option to use regex.
>>> m_errors.
>> "+index_
>>
>> The type of errors here are for the benefit of app devs and app users at
>> the UI level, not console etc. The messages are for use in a QML
>> application as output from a sync transation to notify re: errors,
>> warnings and things that went according to plan.
>>
>> An alternative approach, with broader and more flexible possibilities,
>> is to have the error information available in a model (QVariantMap) that
>> includes useful meta data and values (e.g. type of message, message
>> text, other?). In that way it can be available to app developers as well
>> as easily adapted for other purposes (e.g. log files, console output).
> Okay, I better see the reasoning now. And I think you're mixing two concepts into here, UI error messages and error handling API for letting developers distinguish errors. The user must not see verbatim key names. And the app developer needs a way to distinguish a failing sync from a bug in his code.
The messages can and should be modified. But the app developer does not
need to necessarily use the log/error/warning output from sync in its
raw form either. It should be up to the developer to determine how they
want to work with the data.
>
> I had a thought in that direction in respect to Database::setError originally. SQL errors aren't interesting to anyone but U1Db-Qt developers - they need to be simple and pastable. The same is often true for U1Db-Qt errors. Errors about invalid id's or json are indeed relevant for developers - but only in the console. There's no need to "handle" them, wrong use of the API is to be fixed. One kind of error that we should have API for is: no space left. I apparently never filed a bug for that.
>
> Now for sync, I'd still want the same console output for obvious mistakes like invalid URLs and wrong key names. That is what I was having in mind when I suggested to not have HTML. It's a one-time see the mistake and fix it and not for UI. And it is useful to always log if a connection failed in any way. For the developer's benefit we should have "signal syncFailed". That should suffice to prompt the user appropriately - the developer knows their target audience, what language to use etc.
Assuming too much about what the app developer wants, the endless
possible types of errors that might be encountered (e.g. faulty app
logic, user input variations, API problems), or who their audience is,
along rigid lines (and then use that logic for designing the API), may
be asking for trouble. What one developer to the next might want (or
need) could be different enough that "signal syncFailed" isn't enough.
This is why I suggested having a much more flexible and less
discriminating approach of making the data vailable using QVarientMap,...
Cris Dywan (kalikiana) wrote : | # |
> Assuming too much about what the app developer wants, the endless
> possible types of errors that might be encountered (e.g. faulty app
> logic, user input variations, API problems), or who their audience is,
> along rigid lines (and then use that logic for designing the API), may
> be asking for trouble. What one developer to the next might want (or
> need) could be different enough that "signal syncFailed" isn't enough.
> This is why I suggested having a much more flexible and less
> discriminating approach of making the data vailable using QVarientMap,
> with meta data included so u1db dev and app dev can use the information
> in ways they choose, for the ultimate benefit of the end user.
>
> The existing approach is indeed immature in that respect, but the
> suggestion of using a model (QVarientMap) moves beyond current
> limitations and is flexible enough for a broad set of use cases, some of
> which we cannot anticipate on our own, but without creating huge overhead.
>
> It is intended to provide a rich mechanism for developers to create
> robust applications for users that do more than simply give feedback
> that only says "fail". Additionally it can hopefully provide useful
> information for u1db-qt devs as well (who also might happen to build
> apps with the plugin...including examples).
syncFailed was an example only. And I stress free-form strings are not a basis for proper error handling API.
At this point I'd prefer to only use simple qDebug() calls and leave proper error API for a separate task. I created bugs for this: bug 1210448 and bug 1210450.
Cris Dywan (kalikiana) wrote : | # |
These are the doc errors I'm currently seeing with the latest changes. They need to be resolve before the branch can be merged:
./src/database.
[Maybe you meant '\part'?]
./src/database.
[Maybe you meant '\part'?]
./src/database.
[Maybe you meant '\part'?]
./src/synchroni
./src/synchroni
./src/synchroni
./src/synchroni
./src/synchroni
./src/synchroni
./src/synchroni
[Maybe you meant '\part'?]
./src/synchroni
[Maybe you meant '\part'?]
./src/synchroni
./src/synchroni
[Maybe you meant '\part'?]
./src/synchroni
[Maybe you meant '\part'?]
./src/synchroni
[Maybe you meant '\part'?]
./src/synchroni
[Maybe you meant '\part'?]
./src/database.
./src/database.
./src/database.
./src/database.
./src/database.
./src/database.
./src/database.
./src/database.
./src/database.
./src/database.
./src/database.
./src/query.
Kevin Wright (kevin-wright-1) wrote : | # |
[snip]
>
> The existing approach is indeed immature in that respect, but the
> suggestion of using a model (QVarientMap) moves beyond current
> limitations and is flexible enough for a broad set of use cases, some of
> which we cannot anticipate on our own, but without creating huge overhead.
>
> It is intended to provide a rich mechanism for developers to create
> robust applications for users that do more than simply give feedback
> that only says "fail". Additionally it can hopefully provide useful
> information for u1db-qt devs as well (who also might happen to build
> apps with the plugin...including examples).
> syncFailed was an example only. And I stress free-form strings are not a basis for proper error handling API.
>
> At this point I'd prefer to only use simple qDebug() calls and leave proper error API for a separate task. I created bugs for this: bug 1210448 and bug 1210450.
I modified things already.
I renamed it sync_output instead of errors (since not everything was an
error), and changed it so that a QVariantMap is returned (rather than
QString) with various meta data, depending on the sync output. There is
a descriptive message for each sync_output record, but it comes along
with hard data, such as what property is related to the message, any
particulars about the database (e.g. name, its index within the targets
definition). Additionally the messages themselves have been modified,
with some former details that were included in the description pushed to
the various meta-data instead. A set of error IDs could be created as well.
The information can be used at the application level, or used by methods
within the plugin itself (e.g. console output, log file).
I'll push those changes since they over write the original error
handling that needed removal/
Kevin
- 118. By Kevin Wright
-
Modified 'errors'. Changed to 'sync_output' (since not everything was an error); changed from QString to QVariantMap that includes more meta-data; and created more generic output descriptions (i.e. removed most hard data and placed into meta-data instead). This approach is still primarily intented as a mechanism for application developers to troubleshoot (or simply keep themselves/users informed of background activity), but can be utilized within the plugin itself as well (e.g. log files, console output). It does not presently include error codes (where appropriate) but should, and additionally it needs to be documented so people can take advantage of it and use it properly. The example u1db-qt-
example- 6.qml uses it as of this commit, but is only applying the output message and not any of the meta-data.
Cris Dywan (kalikiana) wrote : | # |
> The information can be used at the application level, or used by methods
> within the plugin itself (e.g. console output, log file).
>
> I'll push those changes since they over write the original error
> handling that needed removal/
Nice stuff actually. Much cleaner than I expected! It should lend itself extremely nicely for test cases as well. I wouldn't have expected this in this MR but I'm not going to complain now ;-)
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:118
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 119. By Kevin Wright
-
Modified several source files to fix warnings from qdoc.
Kevin Wright (kevin-wright-1) wrote : | # |
Den 09/08/2013 11:20, skrev Christian Dywan:
> These are the doc errors I'm currently seeing with the latest changes. They need to be resolve before the branch can be merged:
>
> ./src/database.
> [Maybe you meant '\part'?]
> ./src/database.
> [Maybe you meant '\part'?]
> ./src/database.
> [Maybe you meant '\part'?]
> ./src/synchroni
> ./src/synchroni
> ./src/synchroni
> ./src/synchroni
> ./src/synchroni
> ./src/synchroni
> ./src/synchroni
> [Maybe you meant '\part'?]
> ./src/synchroni
> [Maybe you meant '\part'?]
> ./src/synchroni
> ./src/synchroni
> [Maybe you meant '\part'?]
> ./src/synchroni
> [Maybe you meant '\part'?]
> ./src/synchroni
> [Maybe you meant '\part'?]
> ./src/synchroni
> [Maybe you meant '\part'?]
> ./src/database.
> ./src/database.
> ./src/database.
> ./src/database.
> ./src/database.
> ./src/database.
> ./src/database.
> ./src/database.
> ./src/database.
> ./src/database.
> ./src/database.
> ./src/query.
Hopefully all fixed now. Could not find anything specific to resolve the
warnings about \property, but maybe this is resolved by the change to
fix the warning about \class.
Also \param is used elsewhere in source files, but perhaps was not used
properly in these instances. The references have been remov...
Kevin Wright (kevin-wright-1) wrote : | # |
Den 08/08/2013 16:24, skrev Christian Dywan:
> Review: Needs Fixing
>
> The code deals with id's being wrapped in {} in places. That's based on behavior of the core code; I think we need a newid() function and solve this in one place.
>
Have not done anything about this. I'm not sure it needs attending to
ahead of higher priority items.
> There's a few instances of the pattern "if (!query.exec()) qDebug" which should probably be using setError.
Have not added any new qDebug messages yet.
>
> resetModel can be used in more places currently still doing its equivalent calls.
Still unsure about this comment -- maybe it was mentioned in another thread.
>
> getValidTargets() and synchronizeTarg
> m_errors.
Discussed in another thread. Has been changed completely. No more markup
is included in output messages.
>
> FIXMEs that ought to be sorted out:
> ##KW## The two lists below are currently not used for anything
> ##KW## maybe this can be removed as it is replaced by revision_number
Already removed.
>
> Unused code:
> //targetDb-
Already removed?
> //double target_
Already commented on this -- variable might be used in further
modifications. Commented out to avoid seeing warning during builds.
> This should probably say the real app name + " (U1Db-Qt)"
> request.
> request.
Already fixed.
>
> For good measure processDataFrom
Have not touched this -- will add an else clause.
>
> This should not be writable, it's only set internally
> Q_PROPERTY(
Modified to sync_output, QList<QVariant> and read only (commit message
has more details).
>
> "where u1db has been downloaded/
> This is very ambiguous and doesn't say where to obtain it. It should say
> "where the u1db Python reference implemented has been downloaded from lp:u1db"
Already fixed.
Kevin
Kevin Wright (kevin-wright-1) wrote : | # |
Den 09/08/2013 12:50, skrev Christian Dywan:
>> The information can be used at the application level, or used by methods
>> within the plugin itself (e.g. console output, log file).
>>
>> I'll push those changes since they over write the original error
>> handling that needed removal/
> Nice stuff actually. Much cleaner than I expected! It should lend itself extremely nicely for test cases as well. I wouldn't have expected this in this MR but I'm not going to complain now ;-)
It was actually quite easy to implement. Took less than 30 minutes to
put together.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:119
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 120. By Kevin Wright
-
Modified database.cpp so that all query.exec() statements have either qDebug output of the error message, or are covered by setError. Eventually error handling should be more consistant, and perhaps utilize the same mechanism as sync_output in the Synchronizer class. Some functions in Database are used side by side with other functions in Synchronizer, so perhaps this is a wise idea where applicable.
Kevin Wright (kevin-wright-1) wrote : | # |
>> There's a few instances of the pattern "if (!query.exec()) qDebug"
>> which should probably be using setError.
>
> Have not added any new qDebug messages yet.
Whoops. Misread this one. Added qDebug statements where there was
nothing. Will fix to use setError.
Kevin
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:120
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
Kevin Wright (kevin-wright-1) wrote : | # |
Den 09/08/2013 13:48, skrev Kevin Wright:
>>> There's a few instances of the pattern "if (!query.exec()) qDebug"
>>> which should probably be using setError.
>> Have not added any new qDebug messages yet.
> Whoops. Misread this one. Added qDebug statements where there was
> nothing. Will fix to use setError.
>
> Kevin
>
Hopefully all done now.
Kevin
- 121. By Kevin Wright
-
Modified database.cpp so that all query.exec() statements are covered by setError. Unfortunately setSerror cannot be the return value in some places, because the method is void or something other than a string.
- 122. By Kevin Wright
-
Modified database.cpp to fix a small bug re: setError.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:122
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
Cris Dywan (kalikiana) wrote : | # |
> Den 09/08/2013 11:20, skrev Christian Dywan:
> > These are the doc errors I'm currently seeing with the latest changes. They
> need to be resolve before the branch can be merged:
> Hopefully all fixed now. Could not find anything specific to resolve the
> warnings about \property, but maybe this is resolved by the change to
> fix the warning about \class.
>
> Also \param is used elsewhere in source files, but perhaps was not used
> properly in these instances. The references have been removed in the
> latest commit, but perhaps this needs to be looked at again.
There's still 17 qdoc warnings. See https:/
- 123. By Kevin Wright
-
Changed qdoc markup class to qmlclass in synchronizer.cpp.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:123
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 124. By Kevin Wright
-
Fixed qdoc warnings.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:124
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
Cris Dywan (kalikiana) wrote : | # |
I get a unit test failure with the latest update:
<testcase result="fail" name="U1dbDatab
<failure message="Compared values are not the same
Actual (): false
Expected (): true" result="fail"/>
</testcase>
Running "qmltestrunner -import ./_build/modules -input ./tests" gives me the line number:
tests/tst_
Kevin Wright (kevin-wright-1) wrote : | # |
Den 12/08/2013 11:26, skrev Christian Dywan:
> Review: Needs Fixing
>
> I get a unit test failure with the latest update:
>
> <testcase result="fail" name="U1dbDatab
> <failure message="Compared values are not the same
> Actual (): false
> Expected (): true" result="fail"/>
> </testcase>
>
> Running "qmltestrunner -import ./_build/modules -input ./tests" gives me the line number:
>
> tests/tst_
The test message seems ambiguous.
Compared values of what?
Kevin
Cris Dywan (kalikiana) wrote : | # |
There's also Jenkins brokenness with precise:
Err http://
404 Not Found
Not at all related to u1db-qt - I'm investigating what we can do there.
Cris Dywan (kalikiana) wrote : | # |
> Den 12/08/2013 11:26, skrev Christian Dywan:
> > Review: Needs Fixing
> >
> > I get a unit test failure with the latest update:
> >
> > <testcase result="fail" name="U1dbDatab
> > <failure message="Compared values are not the same
> > Actual (): false
> > Expected (): true" result="fail"/>
> > </testcase>
> >
> > Running "qmltestrunner -import ./_build/modules -input ./tests" gives me the
> line number:
> >
> > tests/tst_
>
> The test message seems ambiguous.
>
> Compared values of what?
Tell me about it, hte default messages of qmltestrunner suck :-(
But the line is:
compare(
So that translates to "Expected -1 but got something else from putDoc()"
Kevin Wright (kevin-wright-1) wrote : | # |
Den 12/08/2013 11:35, skrev Christian Dywan:
>> Den 12/08/2013 11:26, skrev Christian Dywan:
>>> Review: Needs Fixing
>>>
>>> I get a unit test failure with the latest update:
>>>
>>> <testcase result="fail" name="U1dbDatab
>>> <failure message="Compared values are not the same
>>> Actual (): false
>>> Expected (): true" result="fail"/>
>>> </testcase>
>>>
>>> Running "qmltestrunner -import ./_build/modules -input ./tests" gives me the
>> line number:
>>> tests/tst_
>> The test message seems ambiguous.
>>
>> Compared values of what?
> Tell me about it, hte default messages of qmltestrunner suck :-(
> But the line is:
>
> compare(
>
> So that translates to "Expected -1 but got something else from putDoc()"
Yep, found it. putDoc returns a QString, but the test expects bool.
Not sure how to modify the test properly because the return value is
based on the UID of the database, which will be different with each new
database created -- in addition the function that retrieves the database
UID is private, so unless that is made public the only thing that could
be tested for is whether the return string is empty or not. Well, if the
return string contains ":1" that could also be tested for -- not quite a
good test but perhaps better than nothing.
Kevin
Cris Dywan (kalikiana) wrote : | # |
> > compare(
> true)
> >
> > So that translates to "Expected -1 but got something else from putDoc()"
> Yep, found it. putDoc returns a QString, but the test expects bool.
>
> Not sure how to modify the test properly because the return value is
> based on the UID of the database, which will be different with each new
> database created -- in addition the function that retrieves the database
> UID is private, so unless that is made public the only thing that could
> be tested for is whether the return string is empty or not. Well, if the
> return string contains ":1" that could also be tested for -- not quite a
> good test but perhaps better than nothing.
It sounds sensible to check the empty string - the test only checks if it succeeds or not at the moment. I would leave it to a separate task to extend it to see if that id is valid, works, etc.
Kevin Wright (kevin-wright-1) wrote : | # |
Den 12/08/2013 11:55, skrev Christian Dywan:
>>> compare(
>> true)
>>> So that translates to "Expected -1 but got something else from putDoc()"
>> Yep, found it. putDoc returns a QString, but the test expects bool.
>>
>> Not sure how to modify the test properly because the return value is
>> based on the UID of the database, which will be different with each new
>> database created -- in addition the function that retrieves the database
>> UID is private, so unless that is made public the only thing that could
>> be tested for is whether the return string is empty or not. Well, if the
>> return string contains ":1" that could also be tested for -- not quite a
>> good test but perhaps better than nothing.
> It sounds sensible to check the empty string - the test only checks if it succeeds or not at the moment. I would leave it to a separate task to extend it to see if that id is valid, works, etc.
I misunderstood slightly what the original test was looking for...it was
checking for an int actually, and whether it was > -1. In any case I
have changed it to check the return string and whether != empty string.
The qmltestrunner program seems to hang sometimes. It is difficult to
know if it is running or not. Is that normal?
Kevin
Kevin Wright (kevin-wright-1) wrote : | # |
Den 12/08/2013 12:02, skrev <email address hidden>:
> Den 12/08/2013 11:55, skrev Christian Dywan:
>>>> compare(
>>>> -1,
>>> true)
>>>> So that translates to "Expected -1 but got something else from
>>>> putDoc()"
>>> Yep, found it. putDoc returns a QString, but the test expects bool.
>>>
>>> Not sure how to modify the test properly because the return value is
>>> based on the UID of the database, which will be different with each new
>>> database created -- in addition the function that retrieves the
>>> database
>>> UID is private, so unless that is made public the only thing that could
>>> be tested for is whether the return string is empty or not. Well, if
>>> the
>>> return string contains ":1" that could also be tested for -- not
>>> quite a
>>> good test but perhaps better than nothing.
>> It sounds sensible to check the empty string - the test only checks
>> if it succeeds or not at the moment. I would leave it to a separate
>> task to extend it to see if that id is valid, works, etc.
> I misunderstood slightly what the original test was looking for...it
> was checking for an int actually, and whether it was > -1. In any case
> I have changed it to check the return string and whether != empty string.
>
> The qmltestrunner program seems to hang sometimes. It is difficult to
> know if it is running or not. Is that normal?
>
> Kevin
>
>
Checking for an empty string does not appear to be working no matter
which way I try. Will keep at it.
However, it also appears that there is a check for the file name of the
database. The test is only the name of the file itself, but the value
recieved appears to be the absolute path.
Kevin
Kevin Wright (kevin-wright-1) wrote : | # |
Den 12/08/2013 12:22, skrev <email address hidden>:
> Den 12/08/2013 12:02, skrev <email address hidden>:
>> Den 12/08/2013 11:55, skrev Christian Dywan:
>>>>> compare(
>>>>> > -1,
>>>> true)
>>>>> So that translates to "Expected -1 but got something else from
>>>>> putDoc()"
>>>> Yep, found it. putDoc returns a QString, but the test expects bool.
>>>>
>>>> Not sure how to modify the test properly because the return value is
>>>> based on the UID of the database, which will be different with each
>>>> new
>>>> database created -- in addition the function that retrieves the
>>>> database
>>>> UID is private, so unless that is made public the only thing that
>>>> could
>>>> be tested for is whether the return string is empty or not. Well,
>>>> if the
>>>> return string contains ":1" that could also be tested for -- not
>>>> quite a
>>>> good test but perhaps better than nothing.
>>> It sounds sensible to check the empty string - the test only checks
>>> if it succeeds or not at the moment. I would leave it to a separate
>>> task to extend it to see if that id is valid, works, etc.
>> I misunderstood slightly what the original test was looking for...it
>> was checking for an int actually, and whether it was > -1. In any
>> case I have changed it to check the return string and whether !=
>> empty string.
>>
>> The qmltestrunner program seems to hang sometimes. It is difficult to
>> know if it is running or not. Is that normal?
>>
>> Kevin
>>
>>
> Checking for an empty string does not appear to be working no matter
> which way I try. Will keep at it.
>
> However, it also appears that there is a check for the file name of
> the database. The test is only the name of the file itself, but the
> value recieved appears to be the absolute path.
Scratch this last one. Seems to be a misunderstanding.
>
> Kevin
Cris Dywan (kalikiana) wrote : | # |
> There's also Jenkins brokenness with precise:
> Err http://
> 404 Not Found
>
> Not at all related to u1db-qt - I'm investigating what we can do there.
FYI A work-around for the issue is in place (real fix will take longer). The next automated build should run fine.
- 125. By Kevin Wright
-
Modified tests. In one instance a query property was missing from a Query element (in tst_query.qml), and in another occurance the test needed to be updated to reflect a different return value (in test_database.qml).
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:125
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 126. By Kevin Wright
-
Fixed one test to remove a warning.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:126
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
Cris Dywan (kalikiana) wrote : | # |
ERROR: No artifacts found that match the file pattern "**/*test*.xml". Configuration error?
ERROR: '**/*test*.xml' doesn't match anything: '**' exists but not '**/*test*.xml'
I'm seeing to get this sorted. As far as I'm concerned, the build, tests and docs pass and this is down to Jenkins.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:126
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:126
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:126
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:126
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
Preview Diff
1 | === modified file 'CMakeLists.txt' |
2 | --- CMakeLists.txt 2013-07-31 12:22:38 +0000 |
3 | +++ CMakeLists.txt 2013-08-12 16:20:45 +0000 |
4 | @@ -8,11 +8,12 @@ |
5 | include(FindPkgConfig) |
6 | include(GNUInstallDirs) |
7 | find_package(Qt5Core REQUIRED) |
8 | +find_package(Qt5Network REQUIRED) |
9 | find_package(Qt5Sql REQUIRED) |
10 | add_definitions(-DWITHQT5=1) |
11 | |
12 | set(U1DB_QT_LIBNAME u1db-qt5) |
13 | -set(QT_PKGCONFIG_DEPENDENCIES "Qt5Core Qt5Quick Qt5Sql") |
14 | +set(QT_PKGCONFIG_DEPENDENCIES "Qt5Core Qt5Network Qt5Quick Qt5Sql") |
15 | set(QT_U1DB_PKGCONFIG_FILE lib${U1DB_QT_LIBNAME}.pc) |
16 | |
17 | # Build flags |
18 | @@ -42,4 +43,3 @@ |
19 | install(FILES ${CMAKE_CURRENT_BINARY_DIR}/${QT_U1DB_PKGCONFIG_FILE} |
20 | DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig |
21 | ) |
22 | - |
23 | |
24 | === modified file 'documentation/u1db.qdocconf' |
25 | --- documentation/u1db.qdocconf 2013-05-13 12:24:31 +0000 |
26 | +++ documentation/u1db.qdocconf 2013-08-12 16:20:45 +0000 |
27 | @@ -9,6 +9,7 @@ |
28 | ../src/document.h \ |
29 | ../src/index.h \ |
30 | ../src/query.h \ |
31 | + ../src/synchronizer.h \ |
32 | ../src/global.h |
33 | Cpp.ignoretokens = Q_DECL_EXPORT \ |
34 | Q_PROPERTY \ |
35 | @@ -25,7 +26,8 @@ |
36 | ../examples/u1db-qt-example-2b \ |
37 | ../examples/u1db-qt-example-3 \ |
38 | ../examples/u1db-qt-example-4 \ |
39 | -../examples/u1db-qt-example-5 |
40 | +../examples/u1db-qt-example-5 \ |
41 | +../examples/u1db-qt-example-6 |
42 | |
43 | imagedirs = images |
44 | |
45 | |
46 | === added directory 'examples/u1db-qt-example-6' |
47 | === added file 'examples/u1db-qt-example-6/u1db-qt-example-6.qdoc' |
48 | --- examples/u1db-qt-example-6/u1db-qt-example-6.qdoc 1970-01-01 00:00:00 +0000 |
49 | +++ examples/u1db-qt-example-6/u1db-qt-example-6.qdoc 2013-08-12 16:20:45 +0000 |
50 | @@ -0,0 +1,387 @@ |
51 | +/*! |
52 | + |
53 | +\page u1db-qt-tutorial-6.html |
54 | + |
55 | +\title U1Db-Qt Synchronizing Tutorial |
56 | + |
57 | +This tutorial is designed to demonstrate a variety of essential U1Db-Qt functionality and usage, including: |
58 | + |
59 | +\list 1 |
60 | + \li Synchronizing two databases |
61 | + \li Utilizing the U1db-Qt Index element |
62 | + \li Various approaches to define U1db-Qt Document elements when using the Index element |
63 | + \li Partnering the U1db-Qt Index element and a QML ListView element |
64 | +\endlist |
65 | + |
66 | +\section1 Storing Data |
67 | + |
68 | +\section2 The Database Element |
69 | + |
70 | +\section3 Creating a Database |
71 | + |
72 | +A Database is very simple to create. It only needs an id and a path where the file will be created. A Database is a model, which can be used by elements, such as the ListView further in this example. |
73 | + |
74 | +\code |
75 | +U1db.Database { |
76 | + id: aDatabase |
77 | + path: "aDatabase4" |
78 | +} |
79 | +\endcode |
80 | + |
81 | +\section1 The Document Element |
82 | + |
83 | +\section2 Declaring Documents (at Runtime) |
84 | + |
85 | +A Document can be instantiated at runtime, or generated dynamically. The examples below demonstrate the former. |
86 | + |
87 | +A very basic Document could include its unique 'id' and 'docId' properties. While it is not mandatory to define these properties, in some cases they can be convenient references. More advanced applications would likely find these very useful, and in some cases may be an absolute necessity to achieve the objectives of the program. |
88 | + |
89 | +This example of a very simple Document will not initially do anything, until more properties are added and defined: |
90 | + |
91 | +\code |
92 | +U1db.Document { |
93 | + id: aDocument1 |
94 | + docId: 'helloworld1' |
95 | +} |
96 | +\endcode |
97 | + |
98 | +A basic but still practical Document definition contains several essential properties. In addition to 'id' and 'docId' (discussed above), the 'database', 'create', and 'defaults' properties are also very important, and are introduced below. |
99 | + |
100 | +The 'database' property ensures that the Document is attached to an already defined (or possibly soon to be defined one) identified by its id (in this case 'aDatabase'). For example: |
101 | + |
102 | +\code |
103 | +U1db.Document { |
104 | + id: aDocument1 |
105 | + database: aDatabase |
106 | + docId: 'helloworld1' |
107 | +} |
108 | +\endcode |
109 | + |
110 | +Should the Database not already contain a Document with the same docId ('hellowworld1' in this example) when a 'create' property is present and set to true it will be generated. For example: |
111 | + |
112 | +\code |
113 | +U1db.Document { |
114 | + id: aDocument1 |
115 | + database: aDatabase |
116 | + docId: 'helloworld1' |
117 | + create: true |
118 | +} |
119 | +\endcode |
120 | + |
121 | +However, the Document still requires some data to be useful, which is what the 'defaults' property provides. The value of 'defaults' is a map of data that will be stored in the database (again when the create property is et to true). It contain key:value pairs, where the value can be a string, number, or nested object (e.g. additional fields, lists). For example: |
122 | + |
123 | +\code |
124 | +U1db.Document { |
125 | + id: aDocument1 |
126 | + database: aDatabase |
127 | + docId: 'helloworld1' |
128 | + create: true |
129 | + defaults:{"hello": { "world": { "message":"Hello World", "id": 1 } } } |
130 | +} |
131 | +\endcode |
132 | + |
133 | +As mentioned above, lists can also be nested in Document data. Lists provide a convenient method for producing multiple instances of the same key (AKA 'field' or 'sub-field'). The example code below shows valid use of the 'message' and 'id' sub-fields multiple times within the same object. |
134 | + |
135 | +\code |
136 | +U1db.Document { |
137 | + id: aDocument2 |
138 | + database: aDatabase |
139 | + docId: 'helloworld2' |
140 | + create: true |
141 | + defaults:{"hello": { "world": [ |
142 | + { "message":"Hello World", "id": 2 }, |
143 | + { "message":"Hello World", "id": 2.5 } |
144 | + ] } } |
145 | +} |
146 | +\endcode |
147 | + |
148 | +When the default Javascript Object Notation itself is formatted with appropriate line breaks and indentation, it becomes easier to visualize an embedded list, containing sub-fields 'message' and 'id' (and their respective values): |
149 | + |
150 | +\code |
151 | +{"hello": |
152 | + { "world": |
153 | + [ |
154 | + { "message":"Hello World", "id": 2 }, |
155 | + { "message":"Hello World", "id": 2.5 } |
156 | + ] |
157 | + } |
158 | +} |
159 | +\endcode |
160 | + |
161 | +In dot notation these sub-fields are represented by 'hello.world.message' and 'hello.world.id' respectively. Later in this tutorial these will be utilized within the 'expression' property of U1Db-Qt's Index element, in close collaboration with a QML ListView's delegates. |
162 | + |
163 | + |
164 | +Normally when a docId already exists in a database, and when the set flag is set to true, the value in 'defaults' will be ignored (and the existing data in the database will remain untouched). Sometimes a developer needs to easily overwrite the data in an existing document. The 'contents' property can be used for just that purpose. When 'contents' is defined, its value will replace existing data in the database, for the document identified by the docId. In addition, 'contents' can be used to add new documents, in the same way as the 'create: true' + 'defaults' combination does; in other words, if the document defined by 'docId' does not exist it will be created. |
165 | + |
166 | +\code |
167 | +U1db.Document { |
168 | + id: aDocument3 |
169 | + database: aDatabase |
170 | + docId: 'helloworld3' |
171 | + contents:{"hello": { "world": [ |
172 | + { "message":"Hello World", "id": 3 }, |
173 | + { "message":"Hello World", "id": 3.33 }, |
174 | + { "message":"Hello World", "id": 3.66 } |
175 | + ] } } |
176 | +} |
177 | +\endcode |
178 | + |
179 | +If 'defaults' exists, 'create' is set to 'true' (or 'false' for that matter) and 'contents' is also defined, it is the latter that takes precidence. In other words, 'create' and 'defaults' will be ignored. The following example demonstrates this scenario: |
180 | + |
181 | +\code |
182 | +U1db.Document { |
183 | + id: aDocument3 |
184 | + database: aDatabase |
185 | + docId: 'helloworld3' |
186 | + create: true |
187 | + default:{"hello": { "world": [{ "message":"Hello World", "id": 3 }] } } |
188 | + contents:{"hello": { "world": [ |
189 | + { "message":"Hello World", "id": 3 }, |
190 | + { "message":"Hello World", "id": 3.33 }, |
191 | + { "message":"Hello World", "id": 3.66 } |
192 | + ] } } |
193 | +} |
194 | +\endcode |
195 | + |
196 | +This snippet simply represents the absence of the 'create' property, which is synonymous with 'create: false'. The Document can still be recognized within the application, but until applicable properties (such as those outlined above) are added and/or modified then nothing will be added or modified in the database, and this instance may have very little practical value. |
197 | + |
198 | +\code |
199 | +U1db.Document { |
200 | + id: aDocument4 |
201 | + database: aDatabase |
202 | + docId: 'helloworld4' |
203 | + defaults:{"hello": { "world": { "message":"Hello World", "id": 4 } } } |
204 | +} |
205 | +\endcode |
206 | + |
207 | +\section3 Samples of Stored Documents |
208 | + |
209 | +The data stored in the database after defining the above Document elements (and then running the application, will consist of the following: |
210 | + |
211 | +\table |
212 | +\header |
213 | + \li docId |
214 | + \li content |
215 | +\row |
216 | + |
217 | + \li 'helloworld1' |
218 | + \li |
219 | +\code |
220 | +{ |
221 | + "hello": { |
222 | + "world": { |
223 | + "id": 1, |
224 | + "message": "Hello World" |
225 | + } |
226 | + } |
227 | +} |
228 | + |
229 | +\endcode |
230 | + |
231 | + |
232 | +\row |
233 | + |
234 | + \li 'helloworld2' |
235 | + \li |
236 | +\code |
237 | +{ |
238 | + "hello": { |
239 | + "world": [ |
240 | + { |
241 | + "id": 2, |
242 | + "message": "Hello World" |
243 | + }, |
244 | + { |
245 | + "id": 2.5, |
246 | + "message": "Hello World" |
247 | + } |
248 | + ] |
249 | + } |
250 | +} |
251 | +\endcode |
252 | + |
253 | +\row |
254 | + |
255 | + \li 'helloworld3' |
256 | + \li |
257 | +\code |
258 | +{ |
259 | + "hello": { |
260 | + "world": [ |
261 | + { |
262 | + "id": 3, |
263 | + "message": "Hello World" |
264 | + }, |
265 | + { |
266 | + "id": 3.33, |
267 | + "message": "Hello World" |
268 | + }, |
269 | + { |
270 | + "id": 3.66, |
271 | + "message": "Hello World" |
272 | + } |
273 | + ] |
274 | + } |
275 | +} |
276 | +\endcode |
277 | + |
278 | + |
279 | +\endtable |
280 | + |
281 | +\section1 Retrieving Data |
282 | + |
283 | +To retrieve the Documents that were declared earlier requires two additional elements: Index and Query. |
284 | + |
285 | +\section2 The Index Element |
286 | + |
287 | +\section3 Creating and Index Element |
288 | + |
289 | +The Index element requires both a unique 'id' and a pointer to a 'database' in order to begin becoming useful, as demonstrated here: |
290 | + |
291 | +\code |
292 | +U1db.Index{ |
293 | + database: aDatabase |
294 | + id: by_helloworld |
295 | +} |
296 | +\endcode |
297 | + |
298 | +In the future, the Index element will support on disk storage of appropriate results / data. At the present time only in memory indexing is done, but once the storing capability is implemented, defining and identifying it is as simple as using the 'name' property (which will be stored in the database along with the relvent data that goes with it). The snippet below shows the use of the 'name' property: |
299 | + |
300 | +\code |
301 | +U1db.Index{ |
302 | + database: aDatabase |
303 | + id: by_helloworld |
304 | + //name: "by-helloworld" |
305 | +} |
306 | +\endcode |
307 | + |
308 | +The Index element describes, using dot notation, the fields and sub-fields where the developer expects to find information. That information is defined in a list, and added as the value for the 'expression' property. The list can contain one or more entries, as exemplified here (the property is commented out due to its current status): |
309 | + |
310 | +\code |
311 | +U1db.Index{ |
312 | + database: aDatabase |
313 | + id: by_helloworld |
314 | + //name: "by-helloworld" |
315 | + expression: ["hello.world.id","hello.world.message"] |
316 | +} |
317 | +\endcode |
318 | + |
319 | +\section2 The QueryElement |
320 | + |
321 | +\section3 Creating a Query Element |
322 | + |
323 | +The Query element has two responsibilities: a bridge from Database+Index to other parts of the application, as well as further filtering of data in the database (in addition to what Index provides). |
324 | + |
325 | +In order to fulfil its duties as a bridge to an Index (and Database), the 'index' property must point to an Index element, identified by its 'id'. For example: |
326 | + |
327 | +\code |
328 | +U1db.Query{ |
329 | + id: aQuery |
330 | + index: by_helloworld |
331 | +} |
332 | +\endcode |
333 | + |
334 | +While Index helps to filter data based on 'where' it is located (e.g. field.sub-field), Query helps determine the additional set of criteria for 'what' is being searched for. The intent of the 'query' property is to provide the mechanism for defnining the search criteria, but at the time of writing that functionality is not yet available. However, once the implementation is in place, using it is only requires defining the property's value (e.g. "Hello World"). Wild card searches using '*' are supported, which is the default query (i.e. if 'query' is not set it is assumed to be '*'). For example (the property is commented out due to its current status): |
335 | + |
336 | +\code |
337 | +U1db.Query{ |
338 | + id: aQuery |
339 | + index: by_helloworld |
340 | + //query: "*" |
341 | +} |
342 | +\endcode |
343 | + |
344 | +When the 'query' property becomes available, only wildcard search definitions for "starts with" will be suppoprted. Thus the following would be supported: |
345 | + |
346 | +\code |
347 | +U1db.Query{ |
348 | + id: aQuery |
349 | + index: by_helloworld |
350 | + //query: "Hello*" |
351 | +} |
352 | +\endcode |
353 | + |
354 | + |
355 | +But this would not: |
356 | + |
357 | +\code |
358 | +U1db.Query{ |
359 | + id: aQuery |
360 | + index: by_helloworld |
361 | + //query: "*World" |
362 | +} |
363 | +\endcode |
364 | + |
365 | +Note: again, the 'query' property is commented out in the above two snippets due to its current status |
366 | + |
367 | +\section1 Using Data |
368 | + |
369 | +\section2 Data and the Application UI |
370 | + |
371 | +\section3 Using Data With Models and Views |
372 | + |
373 | +This simple snippet represents how to attach a ListModel to a ListView. In this instance the model 'aQuery' is representative of the Query + Index combination defined earlier: |
374 | + |
375 | +\code |
376 | +ListView { |
377 | + width: units.gu(45) |
378 | + height: units.gu(80) |
379 | + model: aQuery |
380 | +} |
381 | +\endcode |
382 | + |
383 | +\section4 Data and Delegates |
384 | + |
385 | +How a model and ListView + delegates work together is a common QML concept, and not specific to U1Db-Qt. However, the asynchronous nature of this relationship is important to understand. When using QML ListView, delegates will be created based on particular properties such as the size of the application window, ListView, and delegate itself (amongst other factors). Each delegate can then represent a Document retrieved from the Database based on the record's index. This example demonstrates some of the property definitions that contribute to determining the number of delegates a ListView will contain: |
386 | + |
387 | +\code |
388 | +ListView { |
389 | + width: units.gu(45) |
390 | + height: units.gu(80) |
391 | + model: aQuery |
392 | + |
393 | + delegate: Text { |
394 | + x: 66; y: 77 |
395 | + } |
396 | + |
397 | +} |
398 | +\endcode |
399 | + |
400 | + |
401 | +When the number of Documents is less than or equal to the number of delegates then there is a one to one mapping of index to delegate (e.g. the first delegate will represent the Document with an index = 0; the second, index = 1; and so on). |
402 | + |
403 | +When there are more Documents than delegates the ListView will request a new index depending on the situation (e.g. a user scrolls up or down). For example, if a ListView has 10 delegates, but 32 Documents to handle, when a user initially scrolls the first delegate will change from representing the Document with index = 0 to the Document that might have index = 8; the second, from index = 1 to index = 9; ...; the 10th delegate from index = 9 to index = 17. A second scrolling gesture the first index may change to 15, and the final index 24. And so on. Scrolling in the opposite direction will have a similar effect, but the Document index numbers for each delegate will obviously start to decline (towards their original values). |
404 | + |
405 | +The following snippet, which modifies the above delegate definition, could demonstrate this effect if there were enough Documents to do so (i.e. some number greater than the number of delegates): |
406 | + |
407 | +\code |
408 | +ListView { |
409 | + width: units.gu(45) |
410 | + height: units.gu(80) |
411 | + model: aQuery |
412 | + |
413 | + delegate: Text { |
414 | + x: 66; y: 77 |
415 | + text: index |
416 | + } |
417 | + |
418 | +} |
419 | +\endcode |
420 | + |
421 | +The object called 'contents' contains one or more properties. This example demonstrates the retrieval of data based on the U1db.Index defined earlier (id: by-helloworld). In this instance the Index contained two expressions simultaniously, "hello.world.id" and "hello.world.message" |
422 | + |
423 | +\code |
424 | +ListView { |
425 | + width: units.gu(45) |
426 | + height: units.gu(80) |
427 | + model: aQuery |
428 | + |
429 | + delegate: Text { |
430 | + x: 66; y: 77 |
431 | + text: "(" + index + ") '" + contents.message + " " + contents.id + "'" |
432 | + } |
433 | + |
434 | +} |
435 | +\endcode |
436 | + |
437 | +*/ |
438 | |
439 | === added file 'examples/u1db-qt-example-6/u1db-qt-example-6.qml' |
440 | --- examples/u1db-qt-example-6/u1db-qt-example-6.qml 1970-01-01 00:00:00 +0000 |
441 | +++ examples/u1db-qt-example-6/u1db-qt-example-6.qml 2013-08-12 16:20:45 +0000 |
442 | @@ -0,0 +1,173 @@ |
443 | +/* |
444 | + * Copyright (C) 2013 Canonical, Ltd. |
445 | + * |
446 | + * Authors: |
447 | + * Kevin Wright <kevin.wright@canonical.com> |
448 | + * |
449 | + * This program is free software; you can redistribute it and/or modify |
450 | + * it under the terms of the GNU Lesser General Public License as published by |
451 | + * the Free Software Foundation; version 3. |
452 | + * |
453 | + * This program is distributed in the hope that it will be useful, |
454 | + * but WITHOUT ANY WARRANTY; without even the implied warranty of |
455 | + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
456 | + * GNU Lesser General Public License for more details. |
457 | + * |
458 | + * You should have received a copy of the GNU Lesser General Public License |
459 | + * along with this program. If not, see <http://www.gnu.org/licenses/>. |
460 | + */ |
461 | + |
462 | +import QtQuick 2.0 |
463 | +import U1db 1.0 as U1db |
464 | +import Ubuntu.Components 0.1 |
465 | + |
466 | + |
467 | +Item { |
468 | + |
469 | + width: units.gu(45) |
470 | + height: units.gu(80) |
471 | + |
472 | + U1db.Database { |
473 | + id: aDatabase |
474 | + path: "aDatabase6" |
475 | + } |
476 | + |
477 | + U1db.Document { |
478 | + id: aDocument1 |
479 | + database: aDatabase |
480 | + docId: 'helloworld' |
481 | + create: true |
482 | + contents:{"hello": { "world": [ { "message": "Hello World" } ] } } |
483 | + |
484 | + } |
485 | + |
486 | + U1db.Index{ |
487 | + database: aDatabase |
488 | + id: by_helloworld |
489 | + expression: ["hello.world.message"] |
490 | + } |
491 | + |
492 | + U1db.Query{ |
493 | + id: aQuery |
494 | + index: by_helloworld |
495 | + query: [{"message":"Hel*"}] |
496 | + } |
497 | + |
498 | + U1db.Synchronizer{ |
499 | + id: aSynchronizer |
500 | + source: aDatabase |
501 | + targets: [{remote:true}, |
502 | + {remote:true, |
503 | + ip:"127.0.0.1", |
504 | + port: 7777, |
505 | + name:"example1.u1db", |
506 | + resolve_to_source:true}, |
507 | + {remote:"OK"}] |
508 | + synchronize: false |
509 | + } |
510 | + |
511 | + MainView { |
512 | + |
513 | + id: u1dbView |
514 | + width: units.gu(45) |
515 | + height: units.gu(80) |
516 | + anchors.top: parent.top; |
517 | + |
518 | + Tabs { |
519 | + id: tabs |
520 | + anchors.fill: parent |
521 | + |
522 | + Tab { |
523 | + objectName: "Tab1" |
524 | + |
525 | + title: i18n.tr("Hello U1Db!") |
526 | + |
527 | + page: Page { |
528 | + id: helloPage |
529 | + |
530 | + Rectangle { |
531 | + width: units.gu(45) |
532 | + height: units.gu(40) |
533 | + anchors.top: parent.top; |
534 | + border.width: 1 |
535 | + |
536 | + Text { |
537 | + id: sourceLabel |
538 | + anchors.top: parent.top; |
539 | + font.bold: true |
540 | + text: "aDatabase6 Contents" |
541 | + } |
542 | + |
543 | + ListView { |
544 | + id: sourceListView |
545 | + width: units.gu(45) |
546 | + height: units.gu(35) |
547 | + anchors.top: sourceLabel.bottom; |
548 | + model: aQuery |
549 | + |
550 | + delegate: Text { |
551 | + wrapMode: Text.WordWrap |
552 | + x: 6; y: 77 |
553 | + text: { |
554 | + text: "(" + index + ") '" + contents.message + "'" |
555 | + } |
556 | + } |
557 | + } |
558 | + |
559 | + |
560 | + } |
561 | + |
562 | + Rectangle { |
563 | + id: lowerRectangle |
564 | + width: units.gu(45) |
565 | + height: units.gu(35) |
566 | + anchors.bottom: parent.bottom; |
567 | + border.width: 1 |
568 | + |
569 | + Text { |
570 | + id: errorsLabel |
571 | + anchors.top: parent.top; |
572 | + font.bold: true |
573 | + text: "Log:" |
574 | + } |
575 | + |
576 | + ListView { |
577 | + |
578 | + parent: lowerRectangle |
579 | + width: units.gu(45) |
580 | + height: units.gu(30) |
581 | + anchors.top: errorsLabel.bottom; |
582 | + model: aSynchronizer |
583 | + delegate:Text { |
584 | + width: units.gu(40) |
585 | + anchors.left: parent.left |
586 | + anchors.right: parent.right |
587 | + wrapMode: Text.WordWrap |
588 | + text: { |
589 | + text: sync_output |
590 | + } |
591 | + } |
592 | + } |
593 | + |
594 | + Button{ |
595 | + parent: lowerRectangle |
596 | + anchors.bottom: parent.bottom; |
597 | + text: "Sync" |
598 | + onClicked: aSynchronizer.synchronize = true |
599 | + anchors.left: parent.left |
600 | + anchors.right: parent.right |
601 | + |
602 | + } |
603 | + |
604 | + } |
605 | + } |
606 | + |
607 | + } |
608 | + |
609 | + } |
610 | + |
611 | + } |
612 | + |
613 | +} |
614 | + |
615 | + |
616 | |
617 | === modified file 'modules/U1db/CMakeLists.txt' |
618 | --- modules/U1db/CMakeLists.txt 2013-05-01 22:49:50 +0000 |
619 | +++ modules/U1db/CMakeLists.txt 2013-08-12 16:20:45 +0000 |
620 | @@ -1,4 +1,5 @@ |
621 | find_package(Qt5Quick REQUIRED) |
622 | +find_package(Qt5Network REQUIRED) |
623 | |
624 | get_target_property(QMAKE_EXECUTABLE Qt5::qmake LOCATION) |
625 | # See http://doc-snapshot.qt-project.org/5.0/qtcore/qlibraryinfo.html#LibraryLocation-enum |
626 | @@ -13,6 +14,7 @@ |
627 | include_directories( |
628 | ${CMAKE_CURRENT_SOURCE_DIR} |
629 | ${Qt5Sql_INCLUDE_DIRS} |
630 | + ${Qt5Network_INCLUDE_DIRS} |
631 | ) |
632 | |
633 | add_library(U1DBPlugin SHARED ${U1DBPlugin_SRCS}) |
634 | @@ -21,6 +23,7 @@ |
635 | target_link_libraries(U1DBPlugin |
636 | ${U1DB_QT_LIBNAME} |
637 | ${Qt5Quick_LIBRARIES} |
638 | + ${Qt5Network_LIBRARIES} |
639 | ) |
640 | |
641 | include_directories( |
642 | @@ -28,6 +31,7 @@ |
643 | ${U1DB_INCLUDE_DIRS} |
644 | ${CMAKE_CURRENT_BINARY_DIR} |
645 | ${Qt5Quick_INCLUDE_DIRS} |
646 | + ${Qt5Network_INCLUDE_DIRS} |
647 | ) |
648 | |
649 | # copy qmldir file into build directory for shadow builds |
650 | |
651 | === modified file 'modules/U1db/plugin.cpp' |
652 | --- modules/U1db/plugin.cpp 2013-02-15 11:43:45 +0000 |
653 | +++ modules/U1db/plugin.cpp 2013-08-12 16:20:45 +0000 |
654 | @@ -21,6 +21,7 @@ |
655 | #include "document.h" |
656 | #include "index.h" |
657 | #include "query.h" |
658 | +#include "synchronizer.h" |
659 | #include "plugin.h" |
660 | #include <qqml.h> |
661 | |
662 | @@ -32,5 +33,6 @@ |
663 | qmlRegisterType<Document>(uri, 1, 0, "Document"); |
664 | qmlRegisterType<Index>(uri, 1, 0, "Index"); |
665 | qmlRegisterType<Query>(uri, 1, 0, "Query"); |
666 | + qmlRegisterType<Synchronizer>(uri, 1, 0, "Synchronizer"); |
667 | } |
668 | |
669 | |
670 | === modified file 'src/CMakeLists.txt' |
671 | --- src/CMakeLists.txt 2013-07-31 12:22:38 +0000 |
672 | +++ src/CMakeLists.txt 2013-08-12 16:20:45 +0000 |
673 | @@ -6,6 +6,7 @@ |
674 | document.cpp |
675 | index.cpp |
676 | query.cpp |
677 | + synchronizer.cpp |
678 | ) |
679 | |
680 | # Generated files |
681 | @@ -14,6 +15,7 @@ |
682 | moc_document.cpp |
683 | moc_index.cpp |
684 | moc_query.cpp |
685 | + moc_synchronizer.cpp |
686 | ) |
687 | set_directory_properties(PROPERTIES ADDITIONAL_MAKE_CLEAN_FILES "${U1DB_QT_GENERATED}") |
688 | |
689 | @@ -27,6 +29,7 @@ |
690 | ${CMAKE_CURRENT_SOURCE_DIR} |
691 | ${CMAKE_CURRENT_BINARY_DIR} |
692 | ${Qt5Core_INCLUDE_DIRS} |
693 | + ${Qt5Network_INCLUDE_DIRS} |
694 | ${Qt5Sql_INCLUDE_DIRS} |
695 | ${U1DB_INCLUDE_DIRS} |
696 | ) |
697 | @@ -35,6 +38,7 @@ |
698 | target_link_libraries(${U1DB_QT_LIBNAME} |
699 | ${Qt5Core_LIBRARIES} |
700 | ${Qt5Sql_LIBRARIES} |
701 | + ${Qt5Network_LIBRARIES} |
702 | ${U1DB_LDFLAGS} |
703 | ) |
704 | |
705 | @@ -50,7 +54,7 @@ |
706 | LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} |
707 | ) |
708 | |
709 | -install(FILES global.h database.h document.h index.h query.h |
710 | +install(FILES global.h database.h document.h index.h query.h synchronizer.h |
711 | DESTINATION ${INCLUDE_INSTALL_DIR} |
712 | ) |
713 | |
714 | |
715 | === modified file 'src/database.cpp' |
716 | --- src/database.cpp 2013-04-29 23:54:19 +0000 |
717 | +++ src/database.cpp 2013-08-12 16:20:45 +0000 |
718 | @@ -106,7 +106,8 @@ |
719 | /* A unique ID is used for the connection name to ensure that we aren't |
720 | re-using or replacing other opend databases. */ |
721 | if (!m_db.isValid()) |
722 | - m_db = QSqlDatabase::addDatabase("QSQLITE", QUuid::createUuid().toString()); |
723 | + m_db = QSqlDatabase::addDatabase("QSQLITE",QUuid::createUuid().toString()); |
724 | + |
725 | if (!m_db.isValid()) |
726 | return setError("QSqlDatabase error"); |
727 | m_db.setDatabaseName(path); |
728 | @@ -251,6 +252,42 @@ |
729 | } |
730 | |
731 | /*! |
732 | + * \internal |
733 | + * \brief Database::getDocumentContents |
734 | + * |
735 | + * Returns the string representation of a document that has |
736 | + * been selected from the database using a document id. |
737 | + * |
738 | + */ |
739 | + |
740 | +QString |
741 | +Database::getDocumentContents(const QString& docId) |
742 | +{ |
743 | + if (!initializeIfNeeded()) |
744 | + return QString(); |
745 | + |
746 | + QSqlQuery query(m_db.exec()); |
747 | + query.prepare("SELECT document.doc_rev, document.content, " |
748 | + "count(conflicts.doc_rev) AS conflicts FROM document LEFT OUTER JOIN " |
749 | + "conflicts ON conflicts.doc_id = document.doc_id WHERE " |
750 | + "document.doc_id = :docId GROUP BY document.doc_id, " |
751 | + "document.doc_rev, document.content"); |
752 | + query.bindValue(":docId", docId); |
753 | + if (query.exec()) |
754 | + { |
755 | + if (query.next()) |
756 | + { |
757 | + if (query.value("conflicts").toInt() > 0) |
758 | + setError(QString("Conflicts in %1").arg(docId)); |
759 | + return query.value("content").toString(); |
760 | + } |
761 | + return setError(QString("Failed to get document %1: No document").arg(docId)) ? QString() : QString(); |
762 | + } |
763 | + return setError(QString("Failed to get document %1: %2\n%3").arg(docId).arg(query.lastError().text()).arg(query.lastQuery())) ? QString() : QString(); |
764 | +} |
765 | + |
766 | + |
767 | +/*! |
768 | Returns the contents of a document by \a docId in a form that QML recognizes |
769 | as a Variant object, it's identical to Document::getContents() with the |
770 | same \a docId. |
771 | @@ -284,6 +321,9 @@ |
772 | return setError(QString("Failed to get document %1: %2\n%3").arg(docId).arg(query.lastError().text()).arg(query.lastQuery())) ? QVariant() : QVariant(); |
773 | } |
774 | |
775 | +/*! |
776 | + The increaseVectorClockRev(int oldRev) function is deprecated. |
777 | + */ |
778 | static int |
779 | increaseVectorClockRev(int oldRev) |
780 | { |
781 | @@ -291,54 +331,304 @@ |
782 | } |
783 | |
784 | /*! |
785 | + * \internal |
786 | + This function creates a new revision number. |
787 | + |
788 | + It returns a string for use in the document table's 'doc_rev' field. |
789 | + */ |
790 | + |
791 | +QString Database::getNextDocRevisionNumber(QString doc_id) |
792 | +{ |
793 | + |
794 | + QString revision_number = getReplicaUid()+":1"; |
795 | + |
796 | + QString current_revision_number = getCurrentDocRevisionNumber(doc_id); |
797 | + |
798 | + /*! |
799 | + Some revisions contain information from previous |
800 | +conflicts/syncs. Revisions are delimited by '|'. |
801 | + |
802 | + */ |
803 | + |
804 | + QStringList current_revision_list = current_revision_number.split("|"); |
805 | + |
806 | + Q_FOREACH (QString current_revision, current_revision_list) { |
807 | + |
808 | + /*! |
809 | + Each revision contains two pieces of information, |
810 | +the uid of the database that made the revsion, and a counter |
811 | +for the revsion. This information is delimited by ':'. |
812 | + |
813 | + */ |
814 | + |
815 | + QStringList current_revision_number_list = current_revision.split(":"); |
816 | + |
817 | + if(current_revision_number_list[0]==getReplicaUid()) { |
818 | + |
819 | + /*! |
820 | + If the current revision uid is the same as this Database's uid the counter portion is increased by one. |
821 | + |
822 | + */ |
823 | + |
824 | + int revision_generation_number = current_revision_number_list[1].toInt()+1; |
825 | + |
826 | + revision_number = getReplicaUid()+":"+QString::number(revision_generation_number); |
827 | + |
828 | + } |
829 | + else { |
830 | + |
831 | + /*! |
832 | + If the current revision uid is not the same as this Database's uid then the revision represents a change that originated in another database. |
833 | + |
834 | + */ |
835 | + //revision_number+="|"+current_revision; |
836 | + |
837 | + /* Not sure if the above is necessary, |
838 | + *and did not appear to be working as intended either. |
839 | + * |
840 | + * Commented out, but maybe OK to delete. |
841 | + */ |
842 | + } |
843 | + |
844 | + } |
845 | + |
846 | + /*! |
847 | + The Database UID has curly brackets, but they are not required for the revision number and need to be removed. |
848 | + |
849 | + */ |
850 | + |
851 | + revision_number = revision_number.replace("{",""); |
852 | + |
853 | + revision_number = revision_number.replace("}",""); |
854 | + |
855 | + return revision_number; |
856 | + |
857 | +} |
858 | + |
859 | +/*! |
860 | + * \internal |
861 | + The getCurrentDocRevisionNumber(QString doc_id) function |
862 | +returns the current string value from the document table's |
863 | +doc_rev field. |
864 | + |
865 | + */ |
866 | + |
867 | + |
868 | +QString Database::getCurrentDocRevisionNumber(QString doc_id){ |
869 | + if (!initializeIfNeeded()) |
870 | + return QString(); |
871 | + |
872 | + QSqlQuery query(m_db.exec()); |
873 | + |
874 | + query.prepare("SELECT doc_rev from document WHERE doc_id = :docId"); |
875 | + query.bindValue(":docId", doc_id); |
876 | + |
877 | + if (query.exec()) |
878 | + { |
879 | + while (query.next()) |
880 | + { |
881 | + return query.value("doc_rev").toString(); |
882 | + } |
883 | + |
884 | + } |
885 | + else{ |
886 | + return setError(query.lastError().text()) ? QString() : QString(); |
887 | + } |
888 | + return QString(); |
889 | +} |
890 | + |
891 | +/*! |
892 | + * \internal |
893 | + * \brief Database::updateSyncLog |
894 | + * |
895 | + * This method is used at the end of a synchronization session, |
896 | + * to update the database with the latest information known about the peer |
897 | + * database that was synced against. |
898 | + */ |
899 | +void Database::updateSyncLog(bool insert, QString uid, QString generation, QString transaction_id) |
900 | +{ |
901 | + |
902 | + if (!initializeIfNeeded()) |
903 | + return; |
904 | + |
905 | + QSqlQuery query(m_db.exec()); |
906 | + |
907 | + if(insert==true){ |
908 | + query.prepare("INSERT INTO sync_log(known_generation,known_transation_id,known_transation_id) VALUES(:knownGeneration, :knownTransactionId, :replicaUid)"); |
909 | + |
910 | + } |
911 | + else{ |
912 | + query.prepare("UPDATE sync_log SET known_generation = :knownGeneration, known_transation_id = :knownTransactionId WHERE replica_uid = :replicaUid"); |
913 | + } |
914 | + |
915 | + |
916 | + query.bindValue(":replicaUid", uid); |
917 | + query.bindValue(":knownGeneration", generation); |
918 | + query.bindValue(":knownTransactionId", transaction_id); |
919 | + if (!query.exec()) |
920 | + { |
921 | + setError(query.lastError().text()); |
922 | + |
923 | + } |
924 | + |
925 | +} |
926 | + |
927 | +/*! |
928 | + * \internal |
929 | + * \brief Database::updateDocRevisionNumber |
930 | + * |
931 | + * Whenever a document as added or modified it needs a new revision number. |
932 | + * |
933 | + * The revision number contains information about revisions made at the source, |
934 | + * but also revisions to the document by target databases (and then synced with the source). |
935 | + * |
936 | + */ |
937 | + |
938 | +void Database::updateDocRevisionNumber(QString doc_id,QString revision){ |
939 | + if (!initializeIfNeeded()) |
940 | + return; |
941 | + |
942 | + QSqlQuery query(m_db.exec()); |
943 | + |
944 | + query.prepare("UPDATE document SET doc_rev = :revisionId WHERE doc_id = :docId"); |
945 | + query.bindValue(":docId", doc_id); |
946 | + query.bindValue(":revisionId", revision); |
947 | + if (!query.exec()) |
948 | + { |
949 | + setError(query.lastError().text()); |
950 | + |
951 | + } |
952 | + |
953 | +} |
954 | + |
955 | +/*! |
956 | + The getCurrentGenerationNumber() function searches for the |
957 | +current generation number from the sqlite_sequence table. |
958 | +The return value can then be used during a synchronization session, |
959 | +amongst other things. |
960 | + |
961 | + */ |
962 | + |
963 | +int Database::getCurrentGenerationNumber(){ |
964 | + |
965 | + int sequence_number = -1; |
966 | + |
967 | + QSqlQuery query(m_db.exec()); |
968 | + |
969 | + query.prepare("SELECT seq FROM sqlite_sequence WHERE name = 'transaction_log'"); |
970 | + |
971 | + if (query.exec()) |
972 | + { |
973 | + while (query.next()) |
974 | + { |
975 | + sequence_number = (query.value("seq").toInt()); |
976 | + |
977 | + } |
978 | + |
979 | + } |
980 | + else{ |
981 | + setError(query.lastError().text()); |
982 | + } |
983 | + |
984 | + return sequence_number; |
985 | + |
986 | +} |
987 | + |
988 | +/*! |
989 | + The generateNewTransactionId() function generates a random |
990 | +transaction id string, for use when creating new transations. |
991 | + |
992 | + */ |
993 | + |
994 | +QString Database::generateNewTransactionId(){ |
995 | + |
996 | + QString uid = "T-"+QUuid::createUuid().toString(); |
997 | + uid = uid.replace("}",""); |
998 | + uid = uid.replace("{",""); |
999 | + return uid; |
1000 | + |
1001 | +} |
1002 | + |
1003 | +/*! |
1004 | + Each time a document in the Database is created or updated a |
1005 | +new transaction is performed, and information about it inserted into the |
1006 | +transation_log table using the createNewTransaction(QString doc_id) |
1007 | +function. |
1008 | + |
1009 | + */ |
1010 | + |
1011 | +int Database::createNewTransaction(QString doc_id){ |
1012 | + |
1013 | + QString transaction_id = generateNewTransactionId(); |
1014 | + |
1015 | + QSqlQuery query(m_db.exec()); |
1016 | + |
1017 | + QString queryString = "INSERT INTO transaction_log(doc_id, transaction_id) VALUES('"+doc_id+"', '"+transaction_id+"')"; |
1018 | + |
1019 | + if (!query.exec(queryString)){ |
1020 | + return -1; |
1021 | + } |
1022 | + else{ |
1023 | + return 0; |
1024 | + } |
1025 | + |
1026 | + return -1; |
1027 | +} |
1028 | + |
1029 | +/*! |
1030 | Updates the existing \a contents of the document identified by \a docId if |
1031 | there's no error. |
1032 | If no \a docId is given or \a docId is an empty string the \a contents will be |
1033 | stored under an autogenerated name. |
1034 | Returns the new revision of the document, or -1 on failure. |
1035 | */ |
1036 | -int |
1037 | +QString |
1038 | Database::putDoc(QVariant contents, const QString& docId) |
1039 | { |
1040 | if (!initializeIfNeeded()) |
1041 | - return -1; |
1042 | + return ""; |
1043 | |
1044 | QString newOrEmptyDocId(docId); |
1045 | QVariant oldDoc = newOrEmptyDocId.isEmpty() ? QVariant() : getDocUnchecked(newOrEmptyDocId); |
1046 | - /* FIXME: Conflicts */ |
1047 | - |
1048 | - int newRev = increaseVectorClockRev(7/*contents.rev*/); |
1049 | + |
1050 | + QString revision_number = getNextDocRevisionNumber(newOrEmptyDocId); |
1051 | + |
1052 | QSqlQuery query(m_db.exec()); |
1053 | if (oldDoc.isValid()) |
1054 | { |
1055 | query.prepare("UPDATE document SET doc_rev=:docRev, content=:docJson WHERE doc_id = :docId"); |
1056 | query.bindValue(":docId", newOrEmptyDocId); |
1057 | - query.bindValue(":docRev", newRev); |
1058 | + query.bindValue(":docRev", revision_number); |
1059 | // Parse Variant from QML as JsonDocument, fallback to string |
1060 | QString json(QJsonDocument::fromVariant(contents).toJson()); |
1061 | query.bindValue(":docJson", json.isEmpty() ? contents : json); |
1062 | if (!query.exec()) |
1063 | - return setError(QString("Failed to put/ update document %1: %2\n%3").arg(newOrEmptyDocId).arg(query.lastError().text()).arg(query.lastQuery())) ? -1 : -1; |
1064 | + return setError(QString("Failed to put/ update document %1: %2\n%3").arg(newOrEmptyDocId).arg(query.lastError().text()).arg(query.lastQuery())) ? "" : ""; |
1065 | query.prepare("DELETE FROM document_fields WHERE doc_id = :docId"); |
1066 | query.bindValue(":docId", newOrEmptyDocId); |
1067 | if (!query.exec()) |
1068 | - return setError(QString("Failed to delete document field %1: %2\n%3").arg(newOrEmptyDocId).arg(query.lastError().text()).arg(query.lastQuery())) ? -1 : -1; |
1069 | + return setError(QString("Failed to delete document field %1: %2\n%3").arg(newOrEmptyDocId).arg(query.lastError().text()).arg(query.lastQuery())) ? "" : ""; |
1070 | + |
1071 | + createNewTransaction(newOrEmptyDocId); |
1072 | + |
1073 | } |
1074 | else |
1075 | { |
1076 | if (newOrEmptyDocId.isEmpty()) |
1077 | newOrEmptyDocId = QString("D-%1").arg(QUuid::createUuid().toString().mid(1).replace("}","")); |
1078 | if (!QRegExp("^[a-zA-Z0-9.%_-]+$").exactMatch(newOrEmptyDocId)) |
1079 | - return setError(QString("Invalid docID %1").arg(newOrEmptyDocId)) ? -1 : -1; |
1080 | + return setError(QString("Invalid docID %1").arg(newOrEmptyDocId)) ? "" : ""; |
1081 | |
1082 | query.prepare("INSERT INTO document (doc_id, doc_rev, content) VALUES (:docId, :docRev, :docJson)"); |
1083 | query.bindValue(":docId", newOrEmptyDocId); |
1084 | - query.bindValue(":docRev", newRev); |
1085 | + query.bindValue(":docRev", revision_number); |
1086 | // Parse Variant from QML as JsonDocument, fallback to string |
1087 | QJsonDocument json(QJsonDocument::fromVariant(contents)); |
1088 | query.bindValue(":docJson", json.isEmpty() ? contents : json.toJson()); |
1089 | if (!query.exec()) |
1090 | - return setError(QString("Failed to put document %1: %2\n%3").arg(docId).arg(query.lastError().text()).arg(query.lastQuery())) ? -1 : -1; |
1091 | + return setError(QString("Failed to put document %1: %2\n%3").arg(docId).arg(query.lastError().text()).arg(query.lastQuery())) ? "" : ""; |
1092 | + |
1093 | + createNewTransaction(newOrEmptyDocId); |
1094 | } |
1095 | |
1096 | beginResetModel(); |
1097 | @@ -350,8 +640,22 @@ |
1098 | |
1099 | Q_EMIT docChanged(newOrEmptyDocId, contents); |
1100 | |
1101 | - return newRev; |
1102 | -} |
1103 | + return revision_number; |
1104 | +} |
1105 | + |
1106 | +/*! |
1107 | + * \brief Database::resetModel |
1108 | + * |
1109 | + * Resets the Database model. |
1110 | + */ |
1111 | + |
1112 | +void Database::resetModel(){ |
1113 | + |
1114 | + beginResetModel(); |
1115 | + endResetModel(); |
1116 | + |
1117 | +} |
1118 | + |
1119 | |
1120 | /*! |
1121 | Returns a list of all stored documents by their docId. |
1122 | @@ -376,6 +680,7 @@ |
1123 | } |
1124 | return list; |
1125 | } |
1126 | + |
1127 | return setError(QString("Failed to list documents: %1\n%2").arg(query.lastError().text()).arg(query.lastQuery())) ? list : list; |
1128 | } |
1129 | |
1130 | @@ -400,6 +705,12 @@ |
1131 | Q_EMIT pathChanged(path); |
1132 | } |
1133 | |
1134 | +/*! |
1135 | + * \brief Database::getPath |
1136 | + * |
1137 | + * Simply returns the path of the database. |
1138 | + * |
1139 | + */ |
1140 | QString |
1141 | Database::getPath() |
1142 | { |
1143 | @@ -516,6 +827,76 @@ |
1144 | return list; |
1145 | } |
1146 | |
1147 | +/* Handy functions for synchronization. */ |
1148 | + |
1149 | +/*! |
1150 | + * \internal |
1151 | + * \brief Database::listTransactionsSince |
1152 | + * |
1153 | + * This lists transactions for the database since a particular generation number. |
1154 | + * |
1155 | + */ |
1156 | + |
1157 | +QList<QString> Database::listTransactionsSince(int generation){ |
1158 | + |
1159 | + QList<QString> list; |
1160 | + |
1161 | + if (!initializeIfNeeded()) |
1162 | + return list; |
1163 | + |
1164 | + QSqlQuery query(m_db.exec()); |
1165 | + |
1166 | + QString queryStmt = "SELECT generation, doc_id, transaction_id FROM transaction_log where generation > "+QString::number(generation); |
1167 | + |
1168 | + if (query.exec(queryStmt)) |
1169 | + { |
1170 | + while (query.next()) |
1171 | + { |
1172 | + list.append(query.value("generation").toString()+"|"+query.value("doc_id").toString()+"|"+query.value("transaction_id").toString()); |
1173 | + } |
1174 | + |
1175 | + return list; |
1176 | + |
1177 | + } |
1178 | + |
1179 | + return list; |
1180 | + |
1181 | +} |
1182 | + |
1183 | +/*! |
1184 | + * \internal |
1185 | + * \brief Database::getSyncLogInfo |
1186 | + * |
1187 | + * Provides the information about previous synchronizations between the database and another (if any). |
1188 | + * |
1189 | + */ |
1190 | + |
1191 | +QMap<QString,QVariant> Database::getSyncLogInfo(QMap<QString,QVariant> lastSyncInformation, QString uid, QString prefix){ |
1192 | + |
1193 | + if (!initializeIfNeeded()) |
1194 | + return lastSyncInformation; |
1195 | + |
1196 | + QString queryStmt = "SELECT known_transaction_id, known_generation FROM sync_log WHERE replica_uid = '"+uid +"'"; |
1197 | + |
1198 | + QSqlQuery query(m_db.exec()); |
1199 | + |
1200 | + if (query.exec(queryStmt)) |
1201 | + { |
1202 | + while (query.next()) |
1203 | + { |
1204 | + lastSyncInformation.insert(prefix + "_replica_generation", query.value(1).toInt()); |
1205 | + lastSyncInformation.insert(prefix + "_replica_transaction_id",query.value(0).toString()); |
1206 | + return lastSyncInformation; |
1207 | + } |
1208 | + |
1209 | + } |
1210 | + else{ |
1211 | + setError(query.lastError().text()); |
1212 | + } |
1213 | + |
1214 | + return lastSyncInformation; |
1215 | +} |
1216 | + |
1217 | QT_END_NAMESPACE_U1DB |
1218 | |
1219 | #include "moc_database.cpp" |
1220 | |
1221 | === modified file 'src/database.h' |
1222 | --- src/database.h 2013-04-23 15:17:24 +0000 |
1223 | +++ src/database.h 2013-08-12 16:20:45 +0000 |
1224 | @@ -36,21 +36,33 @@ |
1225 | public: |
1226 | Database(QObject* parent = 0); |
1227 | |
1228 | + |
1229 | // QAbstractListModel |
1230 | QVariant data(const QModelIndex & index, int role = Qt::DisplayRole) const; |
1231 | QHash<int, QByteArray>roleNames() const; |
1232 | int rowCount(const QModelIndex & parent = QModelIndex()) const; |
1233 | + void resetModel(); |
1234 | |
1235 | QString getPath(); |
1236 | void setPath(const QString& path); |
1237 | Q_INVOKABLE QVariant getDoc(const QString& docId); |
1238 | + QString getDocumentContents(const QString& docId); |
1239 | QVariant getDocUnchecked(const QString& docId) const; |
1240 | - Q_INVOKABLE int putDoc(QVariant newDoc, const QString& docID=QString()); |
1241 | + Q_INVOKABLE QString putDoc(QVariant newDoc, const QString& docID=QString()); |
1242 | Q_INVOKABLE QList<QString> listDocs(); |
1243 | Q_INVOKABLE QString lastError(); |
1244 | Q_INVOKABLE QString putIndex(const QString& index_name, QStringList expressions); |
1245 | Q_INVOKABLE QStringList getIndexExpressions(const QString& indexName); |
1246 | Q_INVOKABLE QStringList getIndexKeys(const QString& indexName); |
1247 | + |
1248 | + /* Functions handy for Synchronization */ |
1249 | + QString getNextDocRevisionNumber(QString doc_id); |
1250 | + QString getCurrentDocRevisionNumber(QString doc_id); |
1251 | + void updateDocRevisionNumber(QString doc_id,QString revision); |
1252 | + void updateSyncLog(bool insert, QString uid, QString generation, QString transaction_id); |
1253 | + QList<QString> listTransactionsSince(int generation); |
1254 | + QMap<QString,QVariant> getSyncLogInfo(QMap<QString,QVariant> lastSyncInformation, QString uid, QString prefix); |
1255 | + |
1256 | Q_SIGNALS: |
1257 | void pathChanged(const QString& path); |
1258 | void errorChanged(const QString& error); |
1259 | @@ -63,7 +75,7 @@ |
1260 | */ |
1261 | void docLoaded(const QString& docId, QVariant content) const; |
1262 | private: |
1263 | - Q_DISABLE_COPY(Database) |
1264 | + //Q_DISABLE_COPY(Database) |
1265 | QString m_path; |
1266 | QSqlDatabase m_db; |
1267 | QString m_error; |
1268 | @@ -73,6 +85,11 @@ |
1269 | bool initializeIfNeeded(const QString& path=":memory:"); |
1270 | bool setError(const QString& error); |
1271 | QString getDocIdByRow(int row) const; |
1272 | + |
1273 | + int createNewTransaction(QString doc_id); |
1274 | + QString generateNewTransactionId(); |
1275 | + int getCurrentGenerationNumber(); |
1276 | + |
1277 | }; |
1278 | |
1279 | QT_END_NAMESPACE_U1DB |
1280 | |
1281 | === modified file 'src/query.cpp' |
1282 | --- src/query.cpp 2013-05-01 23:06:01 +0000 |
1283 | +++ src/query.cpp 2013-08-12 16:20:45 +0000 |
1284 | @@ -59,6 +59,7 @@ |
1285 | QVariant |
1286 | Query::data(const QModelIndex & index, int role) const |
1287 | { |
1288 | + |
1289 | if (role == 0) // contents |
1290 | return m_results.at(index.row()); |
1291 | if (role == 1) // docId |
1292 | @@ -150,8 +151,23 @@ |
1293 | |
1294 | } |
1295 | |
1296 | + resetModel(); |
1297 | + |
1298 | Q_EMIT documentsChanged(m_documents); |
1299 | Q_EMIT resultsChanged(m_results); |
1300 | + |
1301 | +} |
1302 | + |
1303 | +/*! |
1304 | + * \brief Query::resetModel |
1305 | + * |
1306 | + * Resets the model of the Query |
1307 | + * |
1308 | + */ |
1309 | + |
1310 | +void Query::resetModel(){ |
1311 | + beginResetModel(); |
1312 | + endResetModel(); |
1313 | } |
1314 | |
1315 | /*! |
1316 | |
1317 | === modified file 'src/query.h' |
1318 | --- src/query.h 2013-04-25 13:38:33 +0000 |
1319 | +++ src/query.h 2013-08-12 16:20:45 +0000 |
1320 | @@ -52,6 +52,8 @@ |
1321 | QStringList getDocuments(); |
1322 | QList<QVariant> getResults(); |
1323 | |
1324 | + void resetModel(); |
1325 | + |
1326 | Q_SIGNALS: |
1327 | void indexChanged(Index* index); |
1328 | void queryChanged(QVariant query); |
1329 | |
1330 | === added file 'src/synchronizer.cpp' |
1331 | --- src/synchronizer.cpp 1970-01-01 00:00:00 +0000 |
1332 | +++ src/synchronizer.cpp 2013-08-12 16:20:45 +0000 |
1333 | @@ -0,0 +1,1257 @@ |
1334 | +/* |
1335 | + * Copyright (C) 2013 Canonical, Ltd. |
1336 | + * |
1337 | + * Authors: |
1338 | + * Kevin Wright <kevin.wright@canonical.com> |
1339 | + * |
1340 | + * This program is free software; you can redistribute it and/or modify |
1341 | + * it under the terms of the GNU Lesser General Public License as published by |
1342 | + * the Free Software Foundation; version 3. |
1343 | + * |
1344 | + * This program is distributed in the hope that it will be useful, |
1345 | + * but WITHOUT ANY WARRANTY; without even the implied warranty of |
1346 | + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1347 | + * GNU Lesser General Public License for more details. |
1348 | + * |
1349 | + * You should have received a copy of the GNU Lesser General Public License |
1350 | + * along with this program. If not, see <http://www.gnu.org/licenses/>. |
1351 | + */ |
1352 | + |
1353 | +#include <QDebug> |
1354 | +#include <QSqlQuery> |
1355 | +#include <QFile> |
1356 | +#include <QSqlError> |
1357 | +#include <QUuid> |
1358 | +#include <QStringList> |
1359 | +#include <QJsonDocument> |
1360 | + |
1361 | +#include "synchronizer.h" |
1362 | +#include "private.h" |
1363 | + |
1364 | +QT_BEGIN_NAMESPACE_U1DB |
1365 | + |
1366 | +/*! |
1367 | + \class Synchronizer |
1368 | + \inmodule U1Db |
1369 | + \ingroup modules |
1370 | + |
1371 | + \brief The Synchronizer class handles synchronizing between two databases. |
1372 | + |
1373 | +*/ |
1374 | + |
1375 | +/* |
1376 | + |
1377 | + Below this line are general methods for this class, such as setting/getting values for various properties. |
1378 | + |
1379 | +*/ |
1380 | + |
1381 | +/*! |
1382 | + Create a new Synchronizer element, with an optional \a parent, usually by declaring it as a QML item. |
1383 | + |
1384 | + Synchronizer elements sync two databases together, a 'source' database and a remote or local 'target' database. |
1385 | + |
1386 | + Example use in a QML application: |
1387 | + |
1388 | + U1db.Synchronizer{ |
1389 | + id: aSynchronizer |
1390 | + synchronize: false |
1391 | + source: aDatabase |
1392 | + targets: [{remote:true, |
1393 | + ip:"127.0.0.1", |
1394 | + port: 7777, |
1395 | + name:"example1.u1db", |
1396 | + resolve_to_source:true}] |
1397 | + |
1398 | + } |
1399 | + |
1400 | + Short description of properties: |
1401 | + |
1402 | + id: The element's identification. |
1403 | + |
1404 | + bool synchronize: Is the element actively synching or not. Should be set to false. |
1405 | + |
1406 | + U1DB::Database source: The id of a local database that will be used for synchronization. |
1407 | + |
1408 | + QVariant targets: One or more target databases that will be synched with the local database. |
1409 | + |
1410 | + bool targets.remote: Is the target database a remote or local database. |
1411 | + |
1412 | + QString targets.ip: The ip address of a remote database (if applicable). |
1413 | + |
1414 | + int targets.port: Port number of the remote server. |
1415 | + |
1416 | + QString targets.name: The name of the database. |
1417 | + |
1418 | + bool targets.resolve_to_source: In case of conflict should the sync resolve to the source's data (if true). |
1419 | + |
1420 | + Example use with u1db-serve: |
1421 | + |
1422 | + 1. In a terminal cd into a directory where the u1db Python reference implemented has been downloaded from lp:u1db. |
1423 | + 2. Using Python create a database called 'example1.u1db' using u1db, and a document 'helloworld': |
1424 | + |
1425 | + # python |
1426 | + |
1427 | + >>> import u1db |
1428 | + >>> db = u1db.open("example1.u1db",create=True) |
1429 | + >>> content = {"hello": { "world": { "message":"Hello World Updated" } } } |
1430 | + >>> db.create_doc(content, doc_id="helloworld") |
1431 | + |
1432 | + ctrl+d |
1433 | + |
1434 | + 3. From the u1db directory above type './u1db-serve --port=7777' and hit enter. |
1435 | + 4. Open another terminal tab. |
1436 | + 5. Change into a directory containing u1db-qt (assuming this class is included in that directory and the installed version on the host computer). |
1437 | + 6. Change into the directory where u1db-qt-example-6.qml is located. |
1438 | + 7. Type 'qmlscene u1db-qt-example-6.qml' and hit enter. |
1439 | + 8. Click the button labelled 'Sync'. |
1440 | + 9. Check the terminal windows for output from either the client or server. |
1441 | + |
1442 | + */ |
1443 | + |
1444 | +Synchronizer::Synchronizer(QObject *parent) : |
1445 | + QAbstractListModel(parent), m_synchronize(false), m_source(NULL) |
1446 | +{ |
1447 | + QObject::connect(this, &Synchronizer::syncChanged, this, &Synchronizer::onSyncChanged); |
1448 | +} |
1449 | + |
1450 | +/*! |
1451 | + \internal |
1452 | + *Used to implement QAbstractListModel |
1453 | + *Implements the variables exposed to the Delegate in a model |
1454 | + */ |
1455 | +QVariant |
1456 | +Synchronizer::data(const QModelIndex & index, int role) const |
1457 | +{ |
1458 | + if (role == 0) |
1459 | + return m_sync_output.at(index.row()); |
1460 | + return QVariant(); |
1461 | +} |
1462 | + |
1463 | +/*! |
1464 | + \internal |
1465 | + Used to implement QAbstractListModel |
1466 | + |
1467 | + The number of rows: the number of documents given by the query. |
1468 | + */ |
1469 | +int |
1470 | +Synchronizer::rowCount(const QModelIndex & parent) const |
1471 | +{ |
1472 | + return m_sync_output.count(); |
1473 | +} |
1474 | + |
1475 | +/*! |
1476 | + \internal |
1477 | + Used to implement QAbstractListModel |
1478 | + |
1479 | + */ |
1480 | +QHash<int, QByteArray> |
1481 | +Synchronizer::roleNames() const |
1482 | +{ |
1483 | + QHash<int, QByteArray> roles; |
1484 | + roles.insert(0, "sync_output"); |
1485 | + return roles; |
1486 | +} |
1487 | + |
1488 | + |
1489 | +/*! |
1490 | + |
1491 | + |
1492 | + Sets the source database. |
1493 | + |
1494 | + */ |
1495 | +void Synchronizer::setSource(Database* source) |
1496 | +{ |
1497 | + |
1498 | + if (m_source == source) |
1499 | + return; |
1500 | + |
1501 | + if (m_source) |
1502 | + QObject::disconnect(m_source, 0, this, 0); |
1503 | + |
1504 | + m_source = source; |
1505 | + |
1506 | + Q_EMIT sourceChanged(source); |
1507 | +} |
1508 | + |
1509 | + |
1510 | +/*! |
1511 | + * \property Synchronizer::targets |
1512 | + * |
1513 | + * Sets meta-data for databases to be used during a synchronization session. |
1514 | + * |
1515 | + * The QVariant is a list that can contain definitions for more than one database |
1516 | + * to be used as a target. For example: |
1517 | + * |
1518 | + * targets: [{remote:true}, |
1519 | + * {remote:true, |
1520 | + * ip:"127.0.0.1", |
1521 | + * port: 7777, |
1522 | + * name:"example1.u1db", |
1523 | + * resolve_to_source:true}, |
1524 | + * {remote:"OK"}] |
1525 | + * |
1526 | + * The above example defines three databases. Two of the three definitions in the |
1527 | + * example are invalid, the first ({remote:true}) and the third ({remote:"OK"}), |
1528 | + * because they are incomplete. |
1529 | + * |
1530 | + * The second definition is a fully defined and valid definition for a local to |
1531 | + * remote synchronization of two databases: |
1532 | + * |
1533 | + * {remote:true, |
1534 | + * ip:"127.0.0.1", |
1535 | + * port: 7777, |
1536 | + * name:"example1.u1db", |
1537 | + * resolve_to_source:true} |
1538 | + * |
1539 | + * 'remote' determines whether the database is on disk or located on a server. |
1540 | + * 'ip' and 'port' for a server are used only when 'remote' is set to true |
1541 | + * 'name' is the name of the local (on disk) or remote database. |
1542 | + * Note: If 'remote' is false this is the relative/absolute file location. |
1543 | + * 'resolve_to_source' determines whether to resolve conflicts automatically |
1544 | + * in favor of the source (aka local) database's values or the target's. |
1545 | + * |
1546 | + */ |
1547 | + |
1548 | +void Synchronizer::setTargets(QVariant targets) |
1549 | +{ |
1550 | + |
1551 | + if (m_targets == targets) |
1552 | + return; |
1553 | + |
1554 | + //if (m_targets) |
1555 | + // QObject::disconnect(m_targets, 0, this, 0); |
1556 | + |
1557 | + m_targets = targets; |
1558 | + Q_EMIT targetsChanged(targets); |
1559 | +} |
1560 | + |
1561 | +/*! |
1562 | + * \property Synchronizer::synchronize |
1563 | + */ |
1564 | + |
1565 | +void Synchronizer::setSync(bool synchronize) |
1566 | +{ |
1567 | + |
1568 | + if (m_synchronize == synchronize) |
1569 | + return; |
1570 | + |
1571 | + m_synchronize = synchronize; |
1572 | + Q_EMIT syncChanged(synchronize); |
1573 | +} |
1574 | + |
1575 | + |
1576 | +/*! |
1577 | + * \property Synchronizer::resolve_to_source |
1578 | + */ |
1579 | + |
1580 | +void Synchronizer::setResolveToSource(bool resolve_to_source) |
1581 | +{ |
1582 | + if (m_resolve_to_source == resolve_to_source) |
1583 | + return; |
1584 | + |
1585 | + m_resolve_to_source = resolve_to_source; |
1586 | + Q_EMIT resolveToSourceChanged(resolve_to_source); |
1587 | +} |
1588 | + |
1589 | + |
1590 | +/*! |
1591 | + * \fn void Synchronizer::setSyncOutput(QList<QVariant> sync_output) |
1592 | + * |
1593 | + * Sets the current value for the active session's \a sync_output. |
1594 | + * |
1595 | + */ |
1596 | + |
1597 | +void Synchronizer::setSyncOutput(QList<QVariant> sync_output) |
1598 | +{ |
1599 | + if (m_sync_output == sync_output) |
1600 | + return; |
1601 | + |
1602 | + m_sync_output = sync_output; |
1603 | + Q_EMIT syncOutputChanged(sync_output); |
1604 | +} |
1605 | + |
1606 | +/*! |
1607 | + * \property Synchronizer::source |
1608 | + * |
1609 | + * |
1610 | + * Returns a source Database. |
1611 | + * |
1612 | + */ |
1613 | +Database* Synchronizer::getSource() |
1614 | +{ |
1615 | + return m_source; |
1616 | +} |
1617 | + |
1618 | +/*! |
1619 | + * \brief Synchronizer::getTargets |
1620 | + * |
1621 | + * |
1622 | + * Returns meta-data for all target databases. |
1623 | + * |
1624 | + */ |
1625 | + |
1626 | +QVariant Synchronizer::getTargets() |
1627 | +{ |
1628 | + return m_targets; |
1629 | +} |
1630 | + |
1631 | +/*! |
1632 | + * \brief Synchronizer::getSync |
1633 | + * |
1634 | + * |
1635 | + * Returns the current value of synchronize. If true then the synchronize |
1636 | + * session is initiated. |
1637 | + * |
1638 | + * This should probaby always be set to false on application start up. |
1639 | + * The application developer should use some trigger to switch it to true |
1640 | + * when needed (e.g. button click). |
1641 | + * |
1642 | + */ |
1643 | + |
1644 | +bool Synchronizer::getSync() |
1645 | +{ |
1646 | + return m_synchronize; |
1647 | +} |
1648 | + |
1649 | +/*! |
1650 | + * \brief Synchronizer::getResolveToSource |
1651 | + * |
1652 | + * |
1653 | + * If set to true, any document conflicts created during a sync session |
1654 | + * will be resolved in favor of the content from the source database. If false |
1655 | + * the content from the target database will replace the document content in |
1656 | + * the source database. |
1657 | + * |
1658 | + */ |
1659 | + |
1660 | +bool Synchronizer::getResolveToSource(){ |
1661 | + return m_resolve_to_source; |
1662 | +} |
1663 | + |
1664 | +/*! |
1665 | + * \property Synchronizer::sync_output |
1666 | + * \brief Synchronizer::getSyncOutput |
1667 | + * |
1668 | + * Returns the output from a sync session. The list should contain numerous |
1669 | + * QVariantMaps, each of which will have various meta-data with informative |
1670 | + * information about what happened in the background of the session. |
1671 | + * |
1672 | + * In some cases the information will be about errors or warnings, and in |
1673 | + * other cases simple log messages. Also included would noramlly be associated |
1674 | + * properties, elements and other data. |
1675 | + * |
1676 | + * The information can be used in any number of ways, such as on screen within an app, |
1677 | + * testing, console output, logs and more. This is designed to be flexible enough that |
1678 | + * the app developer can decide themselves how to best use the data. |
1679 | + * |
1680 | + */ |
1681 | + |
1682 | +QList<QVariant> Synchronizer::getSyncOutput(){ |
1683 | + return m_sync_output; |
1684 | +} |
1685 | + |
1686 | +/* |
1687 | + |
1688 | + Below this line represents the class' more unique functionality. |
1689 | + In other words, methods that do more than simply modify/retrieve |
1690 | + an element's property values. |
1691 | + |
1692 | +*/ |
1693 | + |
1694 | + |
1695 | +/*! |
1696 | + * \brief Synchronizer::onSyncChanged |
1697 | + * |
1698 | + * The synchroization process begins here. |
1699 | + * |
1700 | + */ |
1701 | + |
1702 | +void Synchronizer::onSyncChanged(bool synchronize){ |
1703 | + |
1704 | + Database* source = getSource(); |
1705 | + |
1706 | + QList<QVariant> sync_targets; |
1707 | + |
1708 | + /*! |
1709 | + * The validator map contains key and value pair definitions, |
1710 | + *that are used to confirm that the values provided for each |
1711 | + *database target are of the expected type for a particular key. |
1712 | + */ |
1713 | + |
1714 | + QMap<QString,QString>validator; |
1715 | + |
1716 | + validator.insert("remote","bool"); |
1717 | + validator.insert("location","QString"); |
1718 | + validator.insert("resolve_to_source","bool"); |
1719 | + |
1720 | + |
1721 | + /*! |
1722 | + * The mandatory map contains the keys that are used to confirm |
1723 | + *that a database target definition contains all the mandatory keys |
1724 | + *necessary for synchronizing. |
1725 | + */ |
1726 | + |
1727 | + QList<QString>mandatory; |
1728 | + |
1729 | + mandatory.append("remote"); |
1730 | + mandatory.append("resolve_to_source"); |
1731 | + |
1732 | + if(synchronize == true){ |
1733 | + |
1734 | + /*! |
1735 | + A list of valid sync target databases is generated by calling the getValidTargets(validator, mandatory) method, and adding the return value to a QList (sync_targets). |
1736 | + */ |
1737 | + |
1738 | + sync_targets=getValidTargets(validator, mandatory); |
1739 | + |
1740 | + /*! |
1741 | + * Once the list of sync targets has been generated the sync activity |
1742 | + *can be initiated via the synchronizeTargets function. |
1743 | + */ |
1744 | + |
1745 | + synchronizeTargets(source, sync_targets); |
1746 | + |
1747 | + /*! |
1748 | + * After the synchronization is complete the model is reset so that |
1749 | + *log and error messages are available at the application level. |
1750 | + * |
1751 | + */ |
1752 | + |
1753 | + beginResetModel(); |
1754 | + endResetModel(); |
1755 | + |
1756 | + /*! |
1757 | + The convenience signals syncOutputChanged and syncCompleted are |
1758 | +emitted after the model has been reset. |
1759 | + */ |
1760 | + |
1761 | + Q_EMIT syncOutputChanged(m_sync_output); |
1762 | + Q_EMIT syncCompleted(); |
1763 | + |
1764 | + /*! |
1765 | + * The sync boolean value is reset to its default value (false) |
1766 | + *once all sync activity is complete. |
1767 | + */ |
1768 | + |
1769 | + setSync(false); |
1770 | + |
1771 | + } |
1772 | + else{ |
1773 | + |
1774 | + } |
1775 | +} |
1776 | + |
1777 | + |
1778 | +/*! |
1779 | + * \internal |
1780 | + * \brief Synchronizer::getValidTargets |
1781 | + * |
1782 | + * |
1783 | + * This method confirms that each sync target definition is valid, based |
1784 | + * on predefined criteria contained in the validator and mandatory lists. |
1785 | + * |
1786 | + */ |
1787 | + |
1788 | +QList<QVariant> Synchronizer::getValidTargets(QMap<QString,QString>validator, QList<QString>mandatory){ |
1789 | + |
1790 | + QList<QVariant> sync_targets; |
1791 | + |
1792 | + int index = 0; |
1793 | + |
1794 | + QList<QVariant> targets = getTargets().toList(); |
1795 | + |
1796 | + Q_FOREACH (QVariant target_variant, targets) |
1797 | + { |
1798 | + index++; |
1799 | + QString index_number = QString::number(index); |
1800 | + |
1801 | + QMap<QString, QVariant> target = target_variant.toMap(); |
1802 | + |
1803 | + bool valid = true; |
1804 | + bool complete = true; |
1805 | + |
1806 | + QMapIterator<QString, QVariant> i(target); |
1807 | + while (i.hasNext()) { |
1808 | + |
1809 | + i.next(); |
1810 | + |
1811 | + if(validator.contains(i.key())&&validator[i.key()]!=i.value().typeName()){ |
1812 | + valid = false; |
1813 | + |
1814 | + QString message_value = "For property `" + i.key() + "` Expecting type `" + validator[i.key()] + "`, but received type `" + i.value().typeName()+"`"; |
1815 | + |
1816 | + QVariantMap output_map; |
1817 | + output_map.insert("concerning_property","targets"); |
1818 | + output_map.insert("concerning_index",index_number); |
1819 | + output_map.insert("message_type","error"); |
1820 | + output_map.insert("message_value",message_value); |
1821 | + m_sync_output.append(output_map); |
1822 | + |
1823 | + target.insert("sync",false); |
1824 | + |
1825 | + break; |
1826 | + } |
1827 | + |
1828 | + if(valid==false){ |
1829 | + targets.removeOne(target); |
1830 | + break; |
1831 | + } |
1832 | + else{ |
1833 | + QListIterator<QString> j(mandatory); |
1834 | + |
1835 | + while(j.hasNext()){ |
1836 | + QString value = j.next(); |
1837 | + if(!target.contains(value)){ |
1838 | + |
1839 | + QString message_value = "Expected property `" + value + "`, but it is not present."; |
1840 | + |
1841 | + QVariantMap output_map; |
1842 | + output_map.insert("concerning_property","targets"); |
1843 | + output_map.insert("concerning_index",index_number); |
1844 | + output_map.insert("message_type","error"); |
1845 | + output_map.insert("message_value",message_value); |
1846 | + m_sync_output.append(output_map); |
1847 | + |
1848 | + target.insert("sync",false); |
1849 | + targets.removeOne(target); |
1850 | + complete = false; |
1851 | + break; |
1852 | + } |
1853 | + } |
1854 | + if(complete==false){ |
1855 | + break; |
1856 | + } |
1857 | + |
1858 | + } |
1859 | + } |
1860 | + |
1861 | + if(target.contains("sync")&&target["sync"]==false){ |
1862 | + |
1863 | + QString message_value = "Not synced due to errors with properties."; |
1864 | + |
1865 | + QVariantMap output_map; |
1866 | + output_map.insert("concerning_property","targets"); |
1867 | + output_map.insert("concerning_index",index_number); |
1868 | + output_map.insert("message_type","error"); |
1869 | + output_map.insert("message_value",message_value); |
1870 | + m_sync_output.append(output_map); |
1871 | + |
1872 | + } |
1873 | + else |
1874 | + { |
1875 | + target.insert("sync",true); |
1876 | + sync_targets.append(target); |
1877 | + |
1878 | + QString message_value = "Mandatory properties were included and their values are valid."; |
1879 | + |
1880 | + QVariantMap output_map; |
1881 | + output_map.insert("concerning_property","targets"); |
1882 | + output_map.insert("concerning_index",index_number); |
1883 | + output_map.insert("message_type","no-errors"); |
1884 | + output_map.insert("message_value",message_value); |
1885 | + m_sync_output.append(output_map); |
1886 | + |
1887 | + } |
1888 | + |
1889 | + } |
1890 | + |
1891 | + return sync_targets; |
1892 | + |
1893 | +} |
1894 | + |
1895 | +/*! |
1896 | + * \internal |
1897 | + * \brief Synchronizer::synchronizeTargets |
1898 | + * |
1899 | + * The source database is synchronized with the target databases contained |
1900 | + * in the 'targets' list. That list should only contain valid targets, as |
1901 | + * determined by Synchronizer::getValidTargets. |
1902 | + * |
1903 | + */ |
1904 | + |
1905 | +void Synchronizer::synchronizeTargets(Database *source, QVariant targets){ |
1906 | + |
1907 | + if(targets.typeName()== QStringLiteral("QVariantList")){ |
1908 | + |
1909 | + QList<QVariant> target_list = targets.toList(); |
1910 | + |
1911 | + QListIterator<QVariant> i(target_list); |
1912 | + |
1913 | + int target_index = -1; |
1914 | + |
1915 | + while(i.hasNext()){ |
1916 | + |
1917 | + target_index++; |
1918 | + |
1919 | + QVariant target = i.next(); |
1920 | + |
1921 | + if(target.typeName()== QStringLiteral("QVariantMap")){ |
1922 | + QMap<QString,QVariant> target_map = target.toMap(); |
1923 | + |
1924 | + if(target_map.contains("remote")&&target_map["remote"]==false){ |
1925 | + if(target_map.contains("sync")&&target_map["sync"]==true){ |
1926 | + |
1927 | + QString message_value = "Valid local target."; |
1928 | + |
1929 | + QVariantMap output_map; |
1930 | + output_map.insert("concerning_property","targets"); |
1931 | + output_map.insert("concerning_index",target_index); |
1932 | + output_map.insert("message_type","no-errors"); |
1933 | + output_map.insert("message_value",message_value); |
1934 | + m_sync_output.append(output_map); |
1935 | + |
1936 | + syncLocalToLocal(source, target_map); |
1937 | + } |
1938 | + } |
1939 | + else if(target_map.contains("remote")&&target_map["remote"]==true){ |
1940 | + if(target_map.contains("sync")&&target_map["sync"]==true){ |
1941 | + |
1942 | + //ip |
1943 | + //port |
1944 | + //name |
1945 | + //GET /thedb/sync-from/my_replica_uid |
1946 | + |
1947 | + QString source_uid = getUidFromLocalDb(source->getPath()); |
1948 | + QString get_string = target_map["name"].toString()+"/sync-from/"+source_uid; |
1949 | + QString url_string = "http://"+target_map["ip"].toString(); |
1950 | + QString full_get_request = url_string+"/"+get_string; |
1951 | + int port_number = target_map["port"].toInt(); |
1952 | + |
1953 | + QNetworkAccessManager *manager = new QNetworkAccessManager(source); |
1954 | + |
1955 | + QUrl url(full_get_request); |
1956 | + url.setPort(port_number); |
1957 | + |
1958 | + QNetworkRequest request(url); |
1959 | + |
1960 | + connect(manager, &QNetworkAccessManager::finished, this, &Synchronizer::remoteGetSyncInfoFinished); |
1961 | + |
1962 | + QString message_value = "Valid remote target."; |
1963 | + |
1964 | + QVariantMap output_map; |
1965 | + output_map.insert("concerning_property","targets"); |
1966 | + output_map.insert("concerning_index",target_index); |
1967 | + output_map.insert("message_type","no-errors"); |
1968 | + output_map.insert("message_value",message_value); |
1969 | + m_sync_output.append(output_map); |
1970 | + |
1971 | + manager->get(QNetworkRequest(request)); |
1972 | + |
1973 | + } |
1974 | + } |
1975 | + else{ |
1976 | + |
1977 | + QString message_value = "Unknown error. Please check properties"; |
1978 | + |
1979 | + QVariantMap output_map; |
1980 | + output_map.insert("concerning_property","targets"); |
1981 | + output_map.insert("concerning_index",target_index); |
1982 | + output_map.insert("message_type","error"); |
1983 | + output_map.insert("message_value",message_value); |
1984 | + m_sync_output.append(output_map); |
1985 | + |
1986 | + } |
1987 | + |
1988 | + } |
1989 | + |
1990 | + } |
1991 | + |
1992 | + } |
1993 | + |
1994 | +} |
1995 | + |
1996 | +/*! |
1997 | + * \internal |
1998 | + * \brief Synchronizer::syncLocalToLocal |
1999 | + * |
2000 | + * This function synchronizes two local databases, a source database and a target database. |
2001 | + * |
2002 | + */ |
2003 | + |
2004 | +void Synchronizer::syncLocalToLocal(Database *sourceDb, QMap<QString,QVariant> target) |
2005 | +{ |
2006 | + |
2007 | + QString target_db_name = target["location"].toString(); |
2008 | + |
2009 | + Database *targetDb; |
2010 | + Index *targetIndex; |
2011 | + Query *targetQuery; |
2012 | + |
2013 | + Index *sourceIndex; |
2014 | + Query *sourceQuery; |
2015 | + |
2016 | + if(target.contains("id")){ |
2017 | + targetDb = (Database*)target["id"].value<QObject*>(); |
2018 | + } |
2019 | + |
2020 | + if(target.contains("target_query")){ |
2021 | + targetQuery = (Query*)target["target_query"].value<QObject*>(); |
2022 | + targetIndex = targetQuery->getIndex(); |
2023 | + targetDb = targetIndex->getDatabase(); |
2024 | + } |
2025 | + |
2026 | + if(target.contains("source_query")){ |
2027 | + sourceQuery = (Query*)target["source_query"].value<QObject*>(); |
2028 | + sourceIndex = sourceQuery->getIndex(); |
2029 | + sourceDb = sourceIndex->getDatabase(); |
2030 | + } |
2031 | + |
2032 | + if(sourceDb == NULL || targetDb == NULL){ |
2033 | + |
2034 | + QString message_value = "Either source or target does not exist or is not active."; |
2035 | + |
2036 | + QVariantMap output_map; |
2037 | + output_map.insert("concerning_property","source|targets"); |
2038 | + //output_map.insert("concerning_index",index_number); // no access to targets index? |
2039 | + output_map.insert("message_type","error"); |
2040 | + output_map.insert("message_value",message_value); |
2041 | + m_sync_output.append(output_map); |
2042 | + |
2043 | + return; |
2044 | + } |
2045 | + |
2046 | + QMap<QString,QVariant> lastSyncInformation; |
2047 | + |
2048 | + lastSyncInformation.insert("target_replica_uid",getUidFromLocalDb(target_db_name)); |
2049 | + lastSyncInformation.insert("target_replica_generation",""); |
2050 | + lastSyncInformation.insert("target_replica_transaction_id",-1); |
2051 | + lastSyncInformation.insert("source_replica_uid",getUidFromLocalDb(sourceDb->getPath())); |
2052 | + lastSyncInformation.insert("source_replica_generation",""); |
2053 | + lastSyncInformation.insert("source_replica_transaction_id",-1); |
2054 | + |
2055 | + lastSyncInformation = getLastSyncInformation(sourceDb, targetDb, false, lastSyncInformation); |
2056 | + |
2057 | + QList<QString> transactionsFromSource; |
2058 | + QList<QString> transactionsFromTarget; |
2059 | + |
2060 | + // Check if target and source have ever been synced before |
2061 | + |
2062 | + if(lastSyncInformation["target_replica_uid"].toString() != "" && lastSyncInformation["target_replica_generation"].toString() != "" && lastSyncInformation["target_replica_transaction_id"].toInt() != -1 && lastSyncInformation["source_replica_uid"].toString() != "" && lastSyncInformation["source_replica_generation"].toString() != "" && lastSyncInformation["source_replica_transaction_id"].toInt() != -1) |
2063 | + { |
2064 | + |
2065 | + QString message_value = "Source and local database have previously synced."; |
2066 | + |
2067 | + QVariantMap output_map; |
2068 | + output_map.insert("concerning_property","source|targets"); |
2069 | + output_map.insert("concerning_source",sourceDb->getPath()); |
2070 | + output_map.insert("concerning_target",target_db_name); |
2071 | + output_map.insert("message_type","no-errors"); |
2072 | + output_map.insert("message_value",message_value); |
2073 | + m_sync_output.append(output_map); |
2074 | + |
2075 | + //Do some syncing |
2076 | + |
2077 | + transactionsFromSource = sourceDb->listTransactionsSince(lastSyncInformation["source_replica_generation"].toInt()); |
2078 | + |
2079 | + transactionsFromTarget = targetDb->listTransactionsSince(lastSyncInformation["target_replica_generation"].toInt()); |
2080 | + |
2081 | + |
2082 | + } |
2083 | + else{ |
2084 | + |
2085 | + QString message_value = "Source and local database have not previously synced."; |
2086 | + |
2087 | + QVariantMap output_map; |
2088 | + output_map.insert("concerning_property","source|targets"); |
2089 | + output_map.insert("concerning_source",sourceDb->getPath()); |
2090 | + output_map.insert("concerning_target",target_db_name); |
2091 | + output_map.insert("message_type","no-errors"); |
2092 | + output_map.insert("message_value",message_value); |
2093 | + m_sync_output.append(output_map); |
2094 | + |
2095 | + //There is a first time for everything, let's sync! |
2096 | + |
2097 | + transactionsFromSource = sourceDb->listTransactionsSince(0); |
2098 | + |
2099 | + transactionsFromTarget = targetDb->listTransactionsSince(0); |
2100 | + } |
2101 | + |
2102 | + /*! |
2103 | + * With two distinct lists present, it is now possible to check what |
2104 | + * updates should be made, or new documents created in one or the other |
2105 | + * database, depending on conditions. |
2106 | + * |
2107 | + * However, two additional lists containing transactions IDs are required |
2108 | + * because the information is contained within delimited strings (see |
2109 | + * below for details). |
2110 | + * |
2111 | + */ |
2112 | + |
2113 | + QList<QString> transactionIdsFromSource; |
2114 | + QList<QString> transactionIdsFromTarget; |
2115 | + |
2116 | + Q_FOREACH(QString sourceTransaction, transactionsFromSource){ |
2117 | + |
2118 | + /*! |
2119 | + * Each sourceTransaction is a pipe delimited string containing |
2120 | + * generation number, document ID, and transaction ID details |
2121 | + * in that order. |
2122 | + * |
2123 | + * Splitting the string into its component pieces provides a |
2124 | + * document ID (the second key in the list). |
2125 | + */ |
2126 | + |
2127 | + QStringList transactionDetails = sourceTransaction.split("|"); |
2128 | + |
2129 | + /*! |
2130 | + * It is only necessary to have unique instances of the |
2131 | + * document ID. |
2132 | + */ |
2133 | + |
2134 | + if(!transactionIdsFromSource.contains(transactionDetails[1])) |
2135 | + transactionIdsFromSource.append(transactionDetails[1]); |
2136 | + |
2137 | + } |
2138 | + |
2139 | + Q_FOREACH(QString targetTransaction, transactionsFromTarget){ |
2140 | + |
2141 | + /*! |
2142 | + * Each targetTransaction is a pipe delimited string containing |
2143 | + * generation number, document ID, and transaction ID details |
2144 | + * in that order. |
2145 | + * |
2146 | + * Splitting the string into its component pieces provides a |
2147 | + * document ID (the second key in the list). |
2148 | + */ |
2149 | + |
2150 | + QStringList transactionDetails = targetTransaction.split("|"); |
2151 | + |
2152 | + /*! |
2153 | + * It is only necessary to have unique instances of the |
2154 | + * document ID. |
2155 | + */ |
2156 | + |
2157 | + if(!transactionIdsFromTarget.contains(transactionDetails[1])) |
2158 | + transactionIdsFromTarget.append(transactionDetails[1]); |
2159 | + |
2160 | + } |
2161 | + |
2162 | + |
2163 | + /* The source replica asks the target replica for the information it has stored about the last time these two replicas were synchronised (if ever).*/ |
2164 | + |
2165 | + |
2166 | + /* |
2167 | + |
2168 | + The application wishing to synchronise sends the following GET request to the server: |
2169 | + |
2170 | + GET /thedb/sync-from/my_replica_uid |
2171 | + |
2172 | + Where thedb is the name of the database to be synchronised, and my_replica_uid is the replica id of the application’s (i.e. the local, or synchronisation source) database |
2173 | + |
2174 | + */ |
2175 | + |
2176 | + /* |
2177 | + |
2178 | + The target responds with a JSON document that looks like this: |
2179 | + |
2180 | + { |
2181 | + "target_replica_uid": "other_replica_uid", |
2182 | + "target_replica_generation": 12, |
2183 | + "target_replica_transaction_id": "T-sdkfj92292j", |
2184 | + "source_replica_uid": "my_replica_uid", |
2185 | + "source_replica_generation": 23, |
2186 | + "source_transaction_id": "T-39299sdsfla8" |
2187 | + } |
2188 | + |
2189 | + With all the information it has stored for the most recent synchronisation between itself and this particular source replica. In this case it tells us that the synchronisation target believes that when it and the source were last synchronised, the target was at generation 12 and the source at generation 23. |
2190 | + |
2191 | + */ |
2192 | + |
2193 | + |
2194 | + /* The source replica validates that its information regarding the last synchronisation is consistent with the target’s information, and raises an error if not. (This could happen for instance if one of the replicas was lost and restored from backup, or if a user inadvertently tries to synchronise a copied database.) */ |
2195 | + |
2196 | + |
2197 | + |
2198 | + /* The source replica generates a list of changes since the last change the target replica knows of. */ |
2199 | + |
2200 | + |
2201 | + |
2202 | + /* The source replica checks what the last change is it knows about on the target replica. */ |
2203 | + |
2204 | + |
2205 | + |
2206 | + /* If there have been no changes on either replica that the other side has not seen, the synchronisation stops here. */ |
2207 | + |
2208 | + |
2209 | + |
2210 | + /* The source replica sends the changed documents to the target, along with what the latest change is that it knows about on the target replica. */ |
2211 | + |
2212 | + |
2213 | + |
2214 | + /* The target processes the changed documents, and records the source replica’s latest change. */ |
2215 | + |
2216 | + |
2217 | + |
2218 | + /* The target responds with the documents that have changes that the source does not yet know about. */ |
2219 | + |
2220 | + /* The source processes the changed documents, and records the target replica’s latest change. */ |
2221 | + |
2222 | + |
2223 | + |
2224 | + /* If the source has seen no changes unrelated to the synchronisation during this whole process, it now sends the target what its latest change is, so that the next synchronisation does not have to consider changes that were the result of this one.*/ |
2225 | + |
2226 | + |
2227 | +} |
2228 | + |
2229 | +/*! |
2230 | + * \internal |
2231 | + * \brief Synchronizer::syncDocument |
2232 | + * |
2233 | + * |
2234 | + * This method is used to synchronize documents from one local database |
2235 | + * to another local database. |
2236 | + * |
2237 | + */ |
2238 | + |
2239 | + |
2240 | +QVariant Synchronizer::syncDocument(Database *from, Database *to, QString docId) |
2241 | +{ |
2242 | + QVariant document = from->getDoc(docId); |
2243 | + to->putDoc(document, docId); |
2244 | + QString revision = from->getCurrentDocRevisionNumber(docId); |
2245 | + to->updateDocRevisionNumber(docId,revision); |
2246 | + |
2247 | + return document; |
2248 | +} |
2249 | + |
2250 | +/*! |
2251 | + * \internal |
2252 | + * \brief Synchronizer::getLastSyncInformation |
2253 | + * |
2254 | + * |
2255 | + * If the source and target database have ever been synced before the information |
2256 | + * from that previous session is returned. This is only used for local to local |
2257 | + * databases. The local to remote procedure is handled elsewhere in a different manner. |
2258 | + * |
2259 | + */ |
2260 | + |
2261 | +QMap<QString,QVariant> Synchronizer::getLastSyncInformation(Database *sourceDb, Database *targetDb, bool remote, QMap<QString,QVariant> lastSyncInformation){ |
2262 | + |
2263 | + if(remote == true){ |
2264 | + |
2265 | + QString message_value = "Sync information from remote target not available at this time."; |
2266 | + |
2267 | + QVariantMap output_map; |
2268 | + output_map.insert("concerning_property","source|targets"); |
2269 | + output_map.insert("concerning_source",sourceDb->getPath()); |
2270 | + output_map.insert("message_type","warning"); |
2271 | + output_map.insert("message_value",message_value); |
2272 | + m_sync_output.append(output_map); |
2273 | + |
2274 | + return lastSyncInformation; |
2275 | + |
2276 | + } |
2277 | + else{ |
2278 | + |
2279 | + lastSyncInformation["source_replica_uid"].toString(); |
2280 | + |
2281 | + lastSyncInformation = targetDb->getSyncLogInfo(lastSyncInformation, lastSyncInformation["source_replica_uid"].toString(),"target"); |
2282 | + |
2283 | + lastSyncInformation = sourceDb->getSyncLogInfo(lastSyncInformation, lastSyncInformation["target_replica_uid"].toString(),"source"); |
2284 | + |
2285 | + } |
2286 | + |
2287 | + return lastSyncInformation; |
2288 | + |
2289 | +} |
2290 | + |
2291 | +/*! |
2292 | + * \internal |
2293 | + * \brief Synchronizer::getUidFromLocalDb |
2294 | + * |
2295 | + * |
2296 | + * The unique id of a database is needed in certain situations of a |
2297 | + * synchronize session. This method retrieves that id from a local database. |
2298 | + * |
2299 | + */ |
2300 | + |
2301 | +QString Synchronizer::getUidFromLocalDb(QString dbFileName) |
2302 | +{ |
2303 | + |
2304 | + QString dbUid; |
2305 | + |
2306 | + QSqlDatabase db; |
2307 | + |
2308 | + db = QSqlDatabase::addDatabase("QSQLITE",QUuid::createUuid().toString()); |
2309 | + |
2310 | + QFile db_file(dbFileName); |
2311 | + |
2312 | + if(!db_file.exists()) |
2313 | + { |
2314 | + |
2315 | + QString message_value = "Database does not exist."; |
2316 | + |
2317 | + QVariantMap output_map; |
2318 | + output_map.insert("concerning_property","source|targets"); |
2319 | + output_map.insert("concerning_database",dbFileName); |
2320 | + output_map.insert("message_type","error"); |
2321 | + output_map.insert("message_value",message_value); |
2322 | + m_sync_output.append(output_map); |
2323 | + |
2324 | + return dbUid; |
2325 | + } |
2326 | + else |
2327 | + { |
2328 | + db.setDatabaseName(dbFileName); |
2329 | + if (!db.open()){ |
2330 | + |
2331 | + QString message_value = db.lastError().text(); |
2332 | + |
2333 | + QVariantMap output_map; |
2334 | + output_map.insert("concerning_property","source|targets"); |
2335 | + output_map.insert("concerning_database",dbFileName); |
2336 | + output_map.insert("message_type","error"); |
2337 | + output_map.insert("message_value",message_value); |
2338 | + m_sync_output.append(output_map); |
2339 | + |
2340 | + } |
2341 | + else{ |
2342 | + QSqlQuery query (db.exec("SELECT value FROM u1db_config WHERE name = 'replica_uid'")); |
2343 | + |
2344 | + if(!query.lastError().isValid() && query.next()){ |
2345 | + dbUid = query.value(0).toString(); |
2346 | + db.close(); |
2347 | + |
2348 | + dbUid = dbUid.replace("{",""); |
2349 | + |
2350 | + dbUid = dbUid.replace("}",""); |
2351 | + |
2352 | + return dbUid; |
2353 | + } |
2354 | + else{ |
2355 | + qDebug() << query.lastError().text(); |
2356 | + db.close(); |
2357 | + return dbUid; |
2358 | + } |
2359 | + } |
2360 | + |
2361 | + } |
2362 | + |
2363 | + return dbUid; |
2364 | +} |
2365 | + |
2366 | +/*! |
2367 | + * \internal |
2368 | + * \brief Synchronizer::remoteGetSyncInfoFinished |
2369 | + * |
2370 | + * Once the initial exchange between the client application |
2371 | + * and the remote server is complete, this method retrieves |
2372 | + * necessary information from the reply that came from the |
2373 | + * server, and is required for posting data from the client |
2374 | + * in the steps that will follow. |
2375 | + * |
2376 | + * After the data is saved to a string and the network reply |
2377 | + * is closed, the appropriate method for posting |
2378 | + * (postDataFromClientToRemoteServer) is then called. |
2379 | + * |
2380 | + */ |
2381 | + |
2382 | +void Synchronizer::remoteGetSyncInfoFinished(QNetworkReply* reply) |
2383 | +{ |
2384 | + |
2385 | + QNetworkAccessManager *manager = reply->manager(); |
2386 | + |
2387 | + Database *source = qobject_cast<Database *>(manager->parent()); |
2388 | + |
2389 | + QUrl postUrl = reply->request().url(); |
2390 | + |
2391 | + QByteArray data = reply->readAll(); |
2392 | + |
2393 | + QString replyData = QString(data); |
2394 | + |
2395 | + reply->close(); |
2396 | + |
2397 | + postDataFromClientToRemoteServer(source, postUrl, replyData); |
2398 | +} |
2399 | + |
2400 | + |
2401 | +/*! |
2402 | + * \internal |
2403 | + * \brief Synchronizer::postDataFromClientToRemoteServer |
2404 | + * |
2405 | + * This method builds a string for posting from the client |
2406 | + * application to the remote server, using information previously |
2407 | + * gathered from the source and target databases, |
2408 | + * and then initiates the post. |
2409 | + * |
2410 | + */ |
2411 | + |
2412 | +void Synchronizer::postDataFromClientToRemoteServer(Database *source, QUrl postUrl, QString replyData) |
2413 | +{ |
2414 | + |
2415 | + QVariantMap replyMap; |
2416 | + |
2417 | + QJsonDocument replyJson = QJsonDocument::fromJson(replyData.toUtf8()); |
2418 | + |
2419 | + QVariant replyVariant = replyJson.toVariant(); |
2420 | + |
2421 | + replyMap = replyVariant.toMap(); |
2422 | + |
2423 | + double source_replica_generation = replyMap["source_replica_generation"].toDouble(); |
2424 | + QString source_replica_uid = replyMap["source_replica_uid"].toString(); |
2425 | + QString source_replica_transaction_id = replyMap["source_transaction_id"].toString(); |
2426 | + //double target_replica_generation = replyMap["target_replica_generation"].toDouble(); |
2427 | + QString target_replica_transaction_id = replyMap["target_replica_transaction_id"].toString(); |
2428 | + QString target_replica_uid = replyMap["target_replica_uid"].toString(); |
2429 | + |
2430 | + QNetworkAccessManager *manager = new QNetworkAccessManager(source); |
2431 | + |
2432 | + connect(manager, &QNetworkAccessManager::finished, this, &Synchronizer::remotePostSyncInfoFinished); |
2433 | + |
2434 | + QByteArray postString; |
2435 | + |
2436 | + postString = "[\r\n"; |
2437 | + postString.append("{\"last_known_generation\": "); |
2438 | + postString.append(QByteArray::number(source_replica_generation)); |
2439 | + postString.append(", \"last_known_trans_id\": \""); |
2440 | + postString.append(source_replica_transaction_id); |
2441 | + postString.append("\"}"); |
2442 | + |
2443 | + QList<QString> transactions = m_source->listTransactionsSince(source_replica_generation); |
2444 | + |
2445 | + Q_FOREACH(QString transaction,transactions){ |
2446 | + QStringList transactionData = transaction.split("|"); |
2447 | + |
2448 | + QString content = source->getDocumentContents(transactionData[1]); |
2449 | + content = content.replace("\r\n",""); |
2450 | + content = content.replace("\r",""); |
2451 | + content = content.replace("\n",""); |
2452 | + content = content.replace("\"","\\\""); |
2453 | + |
2454 | + postString.append(",\r\n{\"content\": \""+content+"\",\"rev\": \""+m_source->getCurrentDocRevisionNumber(transactionData[1])+"\", \"id\": \""+transactionData[1]+"\",\"trans_id\": \""+transactionData[2]+"\",\"gen\": "+transactionData[0]+"}"); |
2455 | + |
2456 | + } |
2457 | + |
2458 | + postString.append("\r\n]"); |
2459 | + |
2460 | + QByteArray postDataSize = QByteArray::number(postString.size()); |
2461 | + |
2462 | + QNetworkRequest request(postUrl); |
2463 | + request.setRawHeader("User-Agent", "U1Db-Qt v1.0"); |
2464 | + request.setRawHeader("X-Custom-User-Agent", "U1Db-Qt v1.0"); |
2465 | + request.setRawHeader("Content-Type", "application/x-u1db-sync-stream"); |
2466 | + request.setRawHeader("Content-Length", postDataSize); |
2467 | + |
2468 | + manager->post(QNetworkRequest(request),postString); |
2469 | + |
2470 | +} |
2471 | + |
2472 | +/*! |
2473 | + * \internal |
2474 | + * \brief Synchronizer::remotePostSyncInfoFinished |
2475 | + * |
2476 | + * This method is a slot, which is called once the data |
2477 | + * from the client application has been posted to the remote |
2478 | + * server. |
2479 | + * |
2480 | + * This is where any new data from the |
2481 | + * remote server is gathered, and then the further steps |
2482 | + * for processing on the client side are begun. |
2483 | + */ |
2484 | + |
2485 | +void Synchronizer::remotePostSyncInfoFinished(QNetworkReply* reply) |
2486 | +{ |
2487 | + |
2488 | + QNetworkAccessManager *manager = reply->manager(); |
2489 | + |
2490 | + Database *source = qobject_cast<Database *>(manager->parent()); |
2491 | + |
2492 | + QByteArray data = reply->readAll(); |
2493 | + |
2494 | + QString replyData = QString(data); |
2495 | + |
2496 | + reply->close(); |
2497 | + |
2498 | + processDataFromRemoteServer(source, replyData); |
2499 | + |
2500 | +} |
2501 | + |
2502 | +/*! |
2503 | + * \internal |
2504 | + * \brief Synchronizer::processDataFromRemoteServer |
2505 | + * |
2506 | + * After the remote target database has replied back, the data it has sent |
2507 | + * is processed to determine what action to take on any relevant documents |
2508 | + * that may have been included in the data. |
2509 | + * |
2510 | + */ |
2511 | + |
2512 | +void Synchronizer::processDataFromRemoteServer(Database *source, QString replyData) |
2513 | +{ |
2514 | + |
2515 | + replyData = replyData.replace("\r\n",""); |
2516 | + |
2517 | + QJsonDocument replyJson = QJsonDocument::fromJson(replyData.toUtf8()); |
2518 | + |
2519 | + QVariant replyVariant = replyJson.toVariant(); |
2520 | + |
2521 | + QVariantList replyList = replyVariant.toList(); |
2522 | + |
2523 | + QListIterator<QVariant> i(replyList); |
2524 | + |
2525 | + int index = -1; |
2526 | + |
2527 | + while(i.hasNext()){ |
2528 | + |
2529 | + index++; |
2530 | + |
2531 | + QVariant current = i.next(); |
2532 | + |
2533 | + QString type_name = QString::fromUtf8(current.typeName()); |
2534 | + |
2535 | + if(type_name == "QVariantMap") |
2536 | + { |
2537 | + |
2538 | + QVariantMap map = current.toMap(); |
2539 | + |
2540 | + if(index == 0) |
2541 | + { |
2542 | + // Meta data |
2543 | + } |
2544 | + else |
2545 | + { |
2546 | + // Document to update |
2547 | + |
2548 | + QString id(""); |
2549 | + QVariant content(""); |
2550 | + QString rev(""); |
2551 | + |
2552 | + QMapIterator<QString, QVariant> i(map); |
2553 | + |
2554 | + while (i.hasNext()) { |
2555 | + |
2556 | + i.next(); |
2557 | + |
2558 | + if(i.key()=="content") |
2559 | + { |
2560 | + content = i.value(); |
2561 | + } |
2562 | + else if(i.key()=="id") |
2563 | + { |
2564 | + id = i.value().toString(); |
2565 | + } |
2566 | + else if(i.key()=="rev") |
2567 | + { |
2568 | + rev = i.value().toString(); |
2569 | + } |
2570 | + |
2571 | + } |
2572 | + |
2573 | + if(content!=""&&id!=""&&rev!="") |
2574 | + { |
2575 | + source->putDoc(content,id); |
2576 | + source->updateDocRevisionNumber(id,rev); |
2577 | + } |
2578 | + |
2579 | + } |
2580 | + |
2581 | + } |
2582 | + |
2583 | + } |
2584 | + |
2585 | +} |
2586 | + |
2587 | +QT_END_NAMESPACE_U1DB |
2588 | + |
2589 | +#include "moc_synchronizer.cpp" |
2590 | + |
2591 | |
2592 | === added file 'src/synchronizer.h' |
2593 | --- src/synchronizer.h 1970-01-01 00:00:00 +0000 |
2594 | +++ src/synchronizer.h 2013-08-12 16:20:45 +0000 |
2595 | @@ -0,0 +1,146 @@ |
2596 | +/* |
2597 | + * Copyright (C) 2013 Canonical, Ltd. |
2598 | + * |
2599 | + * Authors: |
2600 | + * Kevin Wright <kevin.wright@canonical.com> |
2601 | + * |
2602 | + * This program is free software; you can redistribute it and/or modify |
2603 | + * it under the terms of the GNU Lesser General Public License as published by |
2604 | + * the Free Software Foundation; version 3. |
2605 | + * |
2606 | + * This program is distributed in the hope that it will be useful, |
2607 | + * but WITHOUT ANY WARRANTY; without even the implied warranty of |
2608 | + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
2609 | + * GNU Lesser General Public License for more details. |
2610 | + * |
2611 | + * You should have received a copy of the GNU Lesser General Public License |
2612 | + * along with this program. If not, see <http://www.gnu.org/licenses/>. |
2613 | + */ |
2614 | + |
2615 | +#ifndef U1DB_SYNCHRONIZER_H |
2616 | +#define U1DB_SYNCHRONIZER_H |
2617 | + |
2618 | +#include <QtCore/QObject> |
2619 | +#include <QSqlDatabase> |
2620 | +#include <QVariant> |
2621 | +#include <QMetaType> |
2622 | +#include <QtNetwork> |
2623 | +#include <QNetworkAccessManager> |
2624 | + |
2625 | + |
2626 | +#include "database.h" |
2627 | +#include "index.h" |
2628 | +#include "query.h" |
2629 | + |
2630 | +QT_BEGIN_NAMESPACE_U1DB |
2631 | + |
2632 | +class Q_DECL_EXPORT Synchronizer : public QAbstractListModel { |
2633 | + Q_OBJECT |
2634 | +#ifdef Q_QDOC |
2635 | + Q_PROPERTY(Database* source READ getSource WRITE setSource NOTIFY sourceChanged) |
2636 | +#else |
2637 | + Q_PROPERTY(QT_PREPEND_NAMESPACE_U1DB(Database*) source READ getSource WRITE setSource NOTIFY sourceChanged) |
2638 | +#endif |
2639 | + Q_PROPERTY(bool synchronize READ getSync WRITE setSync NOTIFY syncChanged) |
2640 | + Q_PROPERTY(bool resolve_to_source READ getResolveToSource WRITE setResolveToSource NOTIFY resolveToSourceChanged) |
2641 | + Q_PROPERTY(QVariant targets READ getTargets WRITE setTargets NOTIFY targetsChanged) |
2642 | + Q_PROPERTY(QList<QVariant> sync_output READ getSyncOutput NOTIFY syncOutputChanged) |
2643 | + |
2644 | +public: |
2645 | + Synchronizer(QObject* parent = 0); |
2646 | + |
2647 | + // QAbstractListModel |
2648 | + QVariant data(const QModelIndex & index, int role = Qt::DisplayRole) const; |
2649 | + QHash<int, QByteArray>roleNames() const; |
2650 | + int rowCount(const QModelIndex & parent = QModelIndex()) const; |
2651 | + |
2652 | + QList<QVariant> getValidTargets(QMap<QString,QString>validator, QList<QString>mandatory); |
2653 | + QMap<QString,QVariant> getLastSyncInformation(Database *sourceDb, Database *targetDb, bool remote, QMap<QString,QVariant> lastSyncInformation); |
2654 | + |
2655 | + |
2656 | + Database* getSource(); |
2657 | + QVariant getTargets(); |
2658 | + bool getSync(); |
2659 | + bool getResolveToSource(); |
2660 | + QList<QVariant> getSyncOutput(); |
2661 | + |
2662 | + void setSource(Database* source); |
2663 | + void setTargets(QVariant targets); |
2664 | + void setSync(bool synchronize); |
2665 | + void setResolveToSource(bool resolve_to_source); |
2666 | + void setSyncOutput(QList<QVariant> sync_output); |
2667 | + |
2668 | + void syncLocalToLocal(Database *sourceDb, QMap<QString,QVariant> target); |
2669 | + void synchronizeTargets(Database *source, QVariant targets); |
2670 | + |
2671 | + QVariant syncDocument(Database *from, Database *to, QString docId); |
2672 | + QString getUidFromLocalDb(QString dbFileName); |
2673 | + |
2674 | + void postDataFromClientToRemoteServer(Database *source, QUrl postUrl, QString replyData); |
2675 | + void processDataFromRemoteServer(Database *source, QString replyData); |
2676 | + |
2677 | + |
2678 | +Q_SIGNALS: |
2679 | + |
2680 | + /*! |
2681 | + * \brief sourceChanged |
2682 | + * \param source |
2683 | + */ |
2684 | + |
2685 | + void sourceChanged(Database* source); |
2686 | + |
2687 | + /*! |
2688 | + * \brief targetsChanged |
2689 | + * \param targets |
2690 | + */ |
2691 | + |
2692 | + void targetsChanged(QVariant targets); |
2693 | + |
2694 | + /*! |
2695 | + * \brief syncChanged |
2696 | + * \param synchronize |
2697 | + */ |
2698 | + |
2699 | + void syncChanged(bool synchronize); |
2700 | + |
2701 | + /*! |
2702 | + * \brief resolveToSourceChanged |
2703 | + * \param resolve_to_source |
2704 | + */ |
2705 | + |
2706 | + void resolveToSourceChanged(bool resolve_to_source); |
2707 | + |
2708 | + /*! |
2709 | + * \brief syncOutputChanged |
2710 | + * \param sync_output |
2711 | + */ |
2712 | + |
2713 | + void syncOutputChanged(QList<QVariant> sync_output); |
2714 | + |
2715 | + /*! |
2716 | + * \brief syncCompleted |
2717 | + */ |
2718 | + |
2719 | + void syncCompleted(); |
2720 | + |
2721 | +private: |
2722 | + //Q_DISABLE_COPY(Synchronizer) |
2723 | + Database* m_source; |
2724 | + bool m_synchronize; |
2725 | + bool m_resolve_to_source; |
2726 | + QVariant m_targets; |
2727 | + QList<QVariant> m_sync_output; |
2728 | + |
2729 | + void onSyncChanged(bool synchronize); |
2730 | + void remoteGetSyncInfoFinished(QNetworkReply* reply); |
2731 | + void remotePostSyncInfoFinished(QNetworkReply* reply); |
2732 | + |
2733 | +}; |
2734 | + |
2735 | +QT_END_NAMESPACE_U1DB |
2736 | + |
2737 | + |
2738 | + |
2739 | +#endif // U1DB_SYNCHRONIZER_H |
2740 | + |
2741 | + |
2742 | |
2743 | === modified file 'tests/tst_database.qml' |
2744 | --- tests/tst_database.qml 2013-07-16 17:17:53 +0000 |
2745 | +++ tests/tst_database.qml 2013-08-12 16:20:45 +0000 |
2746 | @@ -72,15 +72,15 @@ |
2747 | |
2748 | function test_1_databasePopulated () { |
2749 | spyListCompleted.wait() |
2750 | - compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]}) > -1, true) |
2751 | + compare(myDatabase.putDoc({"animals": ["cat", "dog", "hamster"]}) == '', false) |
2752 | |
2753 | var myPath = "aDatabaseA" |
2754 | myDatabase.path = myPath |
2755 | spyPathChanged.wait() |
2756 | compare(myDatabase.path, myPath) |
2757 | - compare(myDatabase.putDoc({"spam": "eggs"}) > -1, true) |
2758 | + compare(myDatabase.putDoc({"spam": "eggs"}) == '', false) |
2759 | var json = {"foo": "bar"} |
2760 | - compare(myDatabase.putDoc(json, "hijklmn") > -1, true) |
2761 | + compare(myDatabase.putDoc(json, "hijklmn") == '', false) |
2762 | compare(myDatabase.getDoc("hijklmn"), json) |
2763 | compare(myDatabase.getDoc("hijklmn"), json) |
2764 | } |
2765 | |
2766 | === modified file 'tests/tst_query.qml' |
2767 | --- tests/tst_query.qml 2013-05-10 16:04:04 +0000 |
2768 | +++ tests/tst_query.qml 2013-08-12 16:20:45 +0000 |
2769 | @@ -117,6 +117,7 @@ |
2770 | U1db.Query { |
2771 | id: wrongQuery |
2772 | index: byNamePhone |
2773 | + query: [{ 'name': 'Ivanka', 'phone': '*' }] |
2774 | } |
2775 | |
2776 | SignalSpy { |
FAILED: Continuous integration, rev:113 jenkins. qa.ubuntu. com/job/ u1db-qt- ci/1/ jenkins. qa.ubuntu. com/job/ u1db-qt- precise- amd64-ci/ 1/console jenkins. qa.ubuntu. com/job/ u1db-qt- quantal- amd64-ci/ 1/console jenkins. qa.ubuntu. com/job/ u1db-qt- raring- amd64-ci/ 1/console jenkins. qa.ubuntu. com/job/ u1db-qt- saucy-amd64- ci/1/console jenkins. qa.ubuntu. com/job/ u1db-qt- saucy-armhf- ci/1/console
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild: s-jenkins: 8080/job/ u1db-qt- ci/1/rebuild
http://