Merge lp:~xavi-garcia-mena/keeper/restore into lp:keeper/devel

Proposed by Xavi Garcia
Status: Merged
Approved by: Charles Kerr
Approved revision: 142
Merged at revision: 121
Proposed branch: lp:~xavi-garcia-mena/keeper/restore
Merge into: lp:keeper/devel
Prerequisite: lp:~xavi-garcia-mena/keeper/manifest-after-last-task
Diff against target: 1993 lines (+1234/-88)
37 files modified
data/helper-registry.json.in (+6/-1)
include/helper/data-dir-registry.h (+2/-0)
include/helper/metadata.h (+1/-0)
include/helper/registry.h (+1/-0)
include/helper/restore-helper.h (+62/-0)
src/helper/CMakeLists.txt (+2/-0)
src/helper/data-dir-registry.cpp (+43/-11)
src/helper/metadata.cpp (+1/-0)
src/helper/restore-helper.cpp (+366/-0)
src/service/CMakeLists.txt (+1/-0)
src/service/keeper-helper.cpp (+5/-2)
src/service/keeper-task-restore.cpp (+130/-0)
src/service/keeper-task-restore.h (+46/-0)
src/service/keeper-user.cpp (+8/-2)
src/service/keeper.cpp (+79/-33)
src/service/keeper.h (+6/-0)
src/service/manifest.cpp (+3/-3)
src/service/task-manager.cpp (+30/-3)
src/service/task-manager.h (+2/-0)
src/storage-framework/CMakeLists.txt (+3/-0)
src/storage-framework/downloader.h (+46/-0)
src/storage-framework/sf-downloader.cpp (+56/-0)
src/storage-framework/sf-downloader.h (+48/-0)
src/storage-framework/storage_framework_client.cpp (+21/-20)
src/storage-framework/storage_framework_client.h (+2/-1)
tests/CMakeLists.txt (+6/-0)
tests/fakes/CMakeLists.txt (+9/-0)
tests/fakes/fake-restore-helper.cpp (+130/-0)
tests/fakes/fake-restore-helper.h (+27/-0)
tests/integration/helpers/CMakeLists.txt (+4/-0)
tests/integration/helpers/helpers-test.cc (+34/-0)
tests/integration/helpers/test-helpers-base.cpp (+8/-3)
tests/unit/manifest/manifest-test.cpp (+1/-1)
tests/unit/storage-framework/create-uploader-test.cpp (+6/-7)
tests/utils/file-utils.cpp (+1/-1)
tests/utils/storage-framework-local.cpp (+34/-0)
tests/utils/storage-framework-local.h (+4/-0)
To merge this branch: bzr merge lp:~xavi-garcia-mena/keeper/restore
Reviewer Review Type Date Requested Status
unity-api-1-bot continuous-integration Approve
Charles Kerr (community) Approve
Review via email: mp+310653@code.launchpad.net

Commit message

First version of restore.

Description of the change

This branch adds the first version of restore, with a fake restore helper to test it.

The branch adds the restore-task-restore, restore-helper and modifies the registry in order to add the restore entries.

It also includes a fake restore helper that just reads the contents of the restore file that keeper reads from storage-framework, so we can compare if the download process is fine.

The integration test now runs a backup, checks it, and finally executes a restore based on the restore entries that we get from keeper using the dbus calls.

To post a comment you must log in.
Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

FAILED: Continuous integration, rev:121
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/115/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/1059/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1066
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/855/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/855/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/855
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/855/artifact/output/*zip*/output.zip
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/855/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/855/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/855/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/855
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/855/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/855
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/855/artifact/output/*zip*/output.zip
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/855/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/115/rebuild

review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
122. By Xavi Garcia

commit just to test on jenkins as it does not fail on the destop

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

FAILED: Continuous integration, rev:122
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/116/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/1066/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1073
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/862/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/862/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/862
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/862/artifact/output/*zip*/output.zip
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/862/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/862/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/862/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/862/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/862/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/862/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/116/rebuild

review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
123. By Xavi Garcia

more pointers to integration test

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

FAILED: Continuous integration, rev:123
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/117/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/1067/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1074
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/863/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/863/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/863/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/863/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/863/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/863/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/863/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/863/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/863
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/863/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/117/rebuild

review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
124. By Xavi Garcia

More tests, this is not intended to be the final solution

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

FAILED: Continuous integration, rev:124
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/119/
Executed test runs:

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/119/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
125. By Xavi Garcia

test reading in separated thread

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
126. By Xavi Garcia

Added simple downloader test

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
127. By Xavi Garcia

a litle tidy up

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
128. By Xavi Garcia

Fix copyright issue

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
129. By Xavi Garcia

Changed the way we create the socket between keeper and restore helper

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
130. By Xavi Garcia

Removed test exec

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

PASSED: Continuous integration, rev:130
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/127/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1096
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1103
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/894
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/894/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/894
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/894/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/894
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/894/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/894
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/894/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/894
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/894/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/894
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/894/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/127/rebuild

review: Approve (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
131. By Xavi Garcia

Modified back to original version, without QLocalServer

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
132. By Xavi Garcia

clean up the code a bit

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
133. By Xavi Garcia

More cleanup and decreased file sizes to transfers to avoid race conditions in slow boxes

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
134. By Xavi Garcia

Added extra log information

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :
review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
135. By Xavi Garcia

Changed sockets to be in a shared pointer

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

FAILED: Continuous integration, rev:135
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/133/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/1120/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1127
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/918
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/918/artifact/output/*zip*/output.zip
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/918/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/918
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/918/artifact/output/*zip*/output.zip
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/918/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/918
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/918/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/918
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/918/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/133/rebuild

review: Needs Fixing (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
136. By Xavi Garcia

Closing socket after setting it to the dbus reply

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

PASSED: Continuous integration, rev:136
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/134/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1121
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1128
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/919
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/919/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/919
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/919/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/919
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/919/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/919
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/919/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/919
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/919/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/919
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/919/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/134/rebuild

review: Approve (continuous-integration)
Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

PASSED: Continuous integration, rev:136
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/135/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1122
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1129
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/920
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/920/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/920
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/920/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/920
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/920/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/920
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/920/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/920
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/920/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/920
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/920/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/135/rebuild

review: Approve (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
137. By Xavi Garcia

Removed QLocalSocket to store the socket to send to the helper as QLocalSocket buffers data and makes that the helper misses packets

138. By Xavi Garcia

Removed test dbus method for waiting to the helper to be ready

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

PASSED: Continuous integration, rev:138
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/136/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1137
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1144
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/935
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/935/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/935
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/935/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/935
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/935/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/935
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/935/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/935
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/935/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/935
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/935/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/136/rebuild

review: Approve (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
139. By Xavi Garcia

Removed extra debug statements

140. By Xavi Garcia

Removed extra lines

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

PASSED: Continuous integration, rev:139
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/137/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1138
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1145
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/936
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/936/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/936
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/936/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/936
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/936/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/936
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/936/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/936
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/936/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/936
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/936/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/137/rebuild

review: Approve (continuous-integration)
Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

PASSED: Continuous integration, rev:140
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/138/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1139
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1146
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/937
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/937/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/937
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/937/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/937
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/937/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/937
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/937/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/937
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/937/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/937
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/937/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/138/rebuild

review: Approve (continuous-integration)
Revision history for this message
Charles Kerr (charlesk) wrote :

inline: one bugfix, some questions, several meddling suggestions

review: Needs Information
lp:~xavi-garcia-mena/keeper/restore updated
141. By Xavi Garcia

Changed as suggested after review

Revision history for this message
Xavi Garcia (xavi-garcia-mena) wrote :

Thanks for the review, Charles...
I'm sorry the branch was a bit messy, specially the test part and too many debug statements.

Comments inline....

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

PASSED: Continuous integration, rev:141
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/139/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1149
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1156
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/944
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/944/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/944
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/944/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/944
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/944/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/944
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/944/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/944
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/944/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/944
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/944/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/139/rebuild

review: Approve (continuous-integration)
lp:~xavi-garcia-mena/keeper/restore updated
142. By Xavi Garcia

Storage framework client bug fixed. A couple of variables needed to be captured

Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

FAILED: Continuous integration, rev:142
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/140/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/1158/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1165
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/952
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/952/artifact/output/*zip*/output.zip
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/952/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/952
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/952/artifact/output/*zip*/output.zip
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/952/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/952
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/952/artifact/output/*zip*/output.zip
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/952/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/140/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Charles Kerr (charlesk) wrote :

Thanks for the cleanups Xavi!

review: Approve
Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

PASSED: Continuous integration, rev:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/142/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/1198
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/1205
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/991
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/991/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/991
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=zesty/991/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/991
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/991/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/991
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=zesty/991/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/991
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/991/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/991
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=zesty/991/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/142/rebuild

review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'data/helper-registry.json.in'
--- data/helper-registry.json.in 2016-08-09 04:20:33 +0000
+++ data/helper-registry.json.in 2016-11-24 14:48:54 +0000
@@ -4,5 +4,10 @@
4 "@FOLDER_BACKUP_EXEC@",4 "@FOLDER_BACKUP_EXEC@",
5 "${subtype}"5 "${subtype}"
6 ]6 ]
7 }7 ,
8 "restore-urls": [
9 "@FOLDER_RESTORE_EXEC@",
10 "${subtype}"
11 ]
12 }
8}13}
914
=== modified file 'include/helper/data-dir-registry.h'
--- include/helper/data-dir-registry.h 2016-08-09 05:44:25 +0000
+++ include/helper/data-dir-registry.h 2016-11-24 14:48:54 +0000
@@ -39,6 +39,8 @@
3939
40 QStringList get_backup_helper_urls(Metadata const& metadata) override;40 QStringList get_backup_helper_urls(Metadata const& metadata) override;
4141
42 QStringList get_restore_helper_urls(Metadata const& metadata) override;
43
42private:44private:
43 class Impl;45 class Impl;
44 friend class Impl;46 friend class Impl;
4547
=== modified file 'include/helper/metadata.h'
--- include/helper/metadata.h 2016-10-14 09:23:11 +0000
+++ include/helper/metadata.h 2016-11-24 14:48:54 +0000
@@ -42,6 +42,7 @@
42 static QString const TITLE_KEY;42 static QString const TITLE_KEY;
43 static QString const VERSION_KEY;43 static QString const VERSION_KEY;
44 static QString const FILE_NAME_KEY;44 static QString const FILE_NAME_KEY;
45 static QString const DIR_NAME_KEY;
45 static QString const DISPLAY_NAME_KEY;46 static QString const DISPLAY_NAME_KEY;
4647
47 // metadata values48 // metadata values
4849
=== modified file 'include/helper/registry.h'
--- include/helper/registry.h 2016-08-08 04:56:35 +0000
+++ include/helper/registry.h 2016-11-24 14:48:54 +0000
@@ -30,6 +30,7 @@
30 Q_DISABLE_COPY(HelperRegistry)30 Q_DISABLE_COPY(HelperRegistry)
3131
32 virtual QStringList get_backup_helper_urls(Metadata const& task) =0;32 virtual QStringList get_backup_helper_urls(Metadata const& task) =0;
33 virtual QStringList get_restore_helper_urls(Metadata const& task) =0;
3334
34protected:35protected:
35 HelperRegistry() =default;36 HelperRegistry() =default;
3637
=== added file 'include/helper/restore-helper.h'
--- include/helper/restore-helper.h 1970-01-01 00:00:00 +0000
+++ include/helper/restore-helper.h 2016-11-24 14:48:54 +0000
@@ -0,0 +1,62 @@
1/*
2 * Copyright (C) 2016 Canonical, Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3, as published
6 * by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranties of
10 * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11 * PURPOSE. See the GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License along
14 * with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authors:
17 * Xavi Garcia <xavi.garcia.mena@canonical.com>
18 * Charles Kerr <charles.kerr@canonical.com>
19 */
20
21#pragma once
22
23#include "storage-framework/downloader.h"
24#include "helper/helper.h" // parent class
25#include "helper/registry.h"
26
27#include <QObject>
28#include <QScopedPointer>
29#include <QString>
30
31#include <memory>
32
33class RestoreHelperPrivate;
34class RestoreHelper final: public Helper
35{
36 Q_OBJECT
37 Q_DECLARE_PRIVATE(RestoreHelper)
38
39public:
40 RestoreHelper(
41 QString const & appid,
42 clock_func const & clock=Helper::default_clock,
43 QObject * parent=nullptr
44 );
45 virtual ~RestoreHelper();
46 Q_DISABLE_COPY(RestoreHelper)
47
48 static constexpr int MAX_INACTIVITY_TIME = 15000;
49
50 void set_downloader(std::shared_ptr<Downloader> const& downloader);
51
52 void start(QStringList const& urls) override;
53 void stop() override;
54 int get_helper_socket() const;
55 QString to_string(Helper::State state) const override;
56 void set_state(State) override;
57protected:
58 void on_helper_finished() override;
59
60private:
61 QScopedPointer<RestoreHelperPrivate> const d_ptr;
62};
063
=== modified file 'src/helper/CMakeLists.txt'
--- src/helper/CMakeLists.txt 2016-09-06 01:31:59 +0000
+++ src/helper/CMakeLists.txt 2016-11-24 14:48:54 +0000
@@ -48,10 +48,12 @@
48 ${HELPER_LIB}48 ${HELPER_LIB}
49 STATIC49 STATIC
50 backup-helper.cpp50 backup-helper.cpp
51 restore-helper.cpp
51 data-dir-registry.cpp52 data-dir-registry.cpp
52 helper.cpp53 helper.cpp
53 metadata.cpp54 metadata.cpp
54 ${CMAKE_SOURCE_DIR}/include/helper/backup-helper.h55 ${CMAKE_SOURCE_DIR}/include/helper/backup-helper.h
56 ${CMAKE_SOURCE_DIR}/include/helper/restore-helper.h
55 ${CMAKE_SOURCE_DIR}/include/helper/data-dir-registry.h57 ${CMAKE_SOURCE_DIR}/include/helper/data-dir-registry.h
56 ${CMAKE_SOURCE_DIR}/include/helper/helper.h58 ${CMAKE_SOURCE_DIR}/include/helper/helper.h
57 ${CMAKE_SOURCE_DIR}/include/helper/registry.h59 ${CMAKE_SOURCE_DIR}/include/helper/registry.h
5860
=== modified file 'src/helper/data-dir-registry.cpp'
--- src/helper/data-dir-registry.cpp 2016-08-10 02:29:17 +0000
+++ src/helper/data-dir-registry.cpp 2016-11-24 14:48:54 +0000
@@ -49,15 +49,27 @@
4949
50 QStringList get_backup_helper_urls(Metadata const& task)50 QStringList get_backup_helper_urls(Metadata const& task)
51 {51 {
52 return get_helper_urls(task, "backup");
53 }
54
55 QStringList get_restore_helper_urls(Metadata const& task)
56 {
57 return get_helper_urls(task, "restore");
58 }
59
60private:
61
62 QStringList get_helper_urls(Metadata const& task, QString const & prop)
63 {
52 QStringList ret;64 QStringList ret;
5365
54 QString type;66 QString type;
55 if (task.get_property(Metadata::TYPE_KEY, type))67 if (task.get_property(Metadata::TYPE_KEY, type))
56 {68 {
57 auto it = registry_.find(std::make_pair(type,QStringLiteral("backup")));69 auto it = registry_.find(std::make_pair(type,prop));
58 if (it == registry_.end())70 if (it == registry_.end())
59 {71 {
60 qCritical() << "can't get backup helper urls for unhandled type" << type;72 qCritical() << "can't get " << prop << " helper urls for unhandled type" << type;
61 }73 }
62 else74 else
63 {75 {
@@ -73,8 +85,6 @@
73 return ret;85 return ret;
74 }86 }
7587
76private:
77
78 // replace "${key}" with task.get_property("key")88 // replace "${key}" with task.get_property("key")
79 QStringList perform_url_substitution(Metadata const& task, QStringList const& urls_in)89 QStringList perform_url_substitution(Metadata const& task, QStringList const& urls_in)
80 {90 {
@@ -139,6 +149,10 @@
139 * "backup-urls": [149 * "backup-urls": [
140 * "/path/to/helper.sh",150 * "/path/to/helper.sh",
141 * "${subtype}"151 * "${subtype}"
152 * ],
153 * "restore-urls": [
154 * "/path/to/helper.sh",
155 * "${subtype}"
142 * ]156 * ]
143 * }157 * }
144 * }158 * }
@@ -156,25 +170,37 @@
156 if (error.error != QJsonParseError::NoError)170 if (error.error != QJsonParseError::NoError)
157 qCritical() << path << "parse error at offset" << error.offset << error.errorString();171 qCritical() << path << "parse error at offset" << error.offset << error.errorString();
158172
159 auto obj = doc.object();173 const auto obj = doc.object();
160 for (auto tit=obj.begin(), tend=obj.end(); tit!=tend; ++tit)174 for (auto tit=obj.begin(), tend=obj.end(); tit!=tend; ++tit)
161 {175 {
162 auto const type = tit.key();176 auto const type = tit.key();
163 auto& info = registry_[std::make_pair(type,QStringLiteral("backup"))];
164
165 auto const props = tit.value().toObject();177 auto const props = tit.value().toObject();
166 auto const urls_jsonval = props["backup-urls"];178
179 auto const &urls_jsonval = props["backup-urls"];
167 if (urls_jsonval.isArray())180 if (urls_jsonval.isArray())
168 {181 {
182 auto& info = registry_[std::make_pair(type,QStringLiteral("backup"))];
169 for (auto url_jsonval : urls_jsonval.toArray())183 for (auto url_jsonval : urls_jsonval.toArray())
170 {184 {
171 info.urls.push_back(url_jsonval.toString());185 info.urls.push_back(url_jsonval.toString());
172 }186 }
187 qDebug() << "loaded" << type << "backup urls from" << path;
188 for(auto const& url : info.urls)
189 qDebug() << "\turl:" << url;
173 }190 }
174191
175 qDebug() << "loaded" << type << "backup urls from" << path;192 auto const &urls_jsonval_restore = props["restore-urls"];
176 for(auto const& url : info.urls)193 if (urls_jsonval_restore.isArray())
177 qDebug() << "\turl:" << url;194 {
195 auto& info = registry_[std::make_pair(type,QStringLiteral("restore"))];
196 for (auto url_jsonval : urls_jsonval_restore.toArray())
197 {
198 info.urls.push_back(url_jsonval.toString());
199 }
200 qDebug() << "loaded" << type << "restore urls from" << path;
201 for(auto const& url : info.urls)
202 qDebug() << "\turl:" << url;
203 }
178 }204 }
179 }205 }
180 }206 }
@@ -198,3 +224,9 @@
198{224{
199 return impl_->get_backup_helper_urls(task);225 return impl_->get_backup_helper_urls(task);
200}226}
227
228QStringList
229DataDirRegistry::get_restore_helper_urls(Metadata const& task)
230{
231 return impl_->get_restore_helper_urls(task);
232}
201233
=== modified file 'src/helper/metadata.cpp'
--- src/helper/metadata.cpp 2016-10-14 09:23:11 +0000
+++ src/helper/metadata.cpp 2016-11-24 14:48:54 +0000
@@ -42,6 +42,7 @@
42const QString Metadata::TITLE_KEY = QStringLiteral("title");42const QString Metadata::TITLE_KEY = QStringLiteral("title");
43const QString Metadata::VERSION_KEY = QStringLiteral("version");43const QString Metadata::VERSION_KEY = QStringLiteral("version");
44const QString Metadata::FILE_NAME_KEY = QStringLiteral("file-name");44const QString Metadata::FILE_NAME_KEY = QStringLiteral("file-name");
45const QString Metadata::DIR_NAME_KEY = QStringLiteral("dir-name");
45const QString Metadata::DISPLAY_NAME_KEY = QStringLiteral("display-name");46const QString Metadata::DISPLAY_NAME_KEY = QStringLiteral("display-name");
4647
47// Metadata values48// Metadata values
4849
=== added file 'src/helper/restore-helper.cpp'
--- src/helper/restore-helper.cpp 1970-01-01 00:00:00 +0000
+++ src/helper/restore-helper.cpp 2016-11-24 14:48:54 +0000
@@ -0,0 +1,366 @@
1/*
2 * Copyright (C) 2016 Canonical, Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3, as published
6 * by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranties of
10 * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11 * PURPOSE. See the GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License along
14 * with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authors:
17 * Xavi Garcia <xavi.garcia.mena@canonical.com>
18 * Charles Kerr <charles.kerr@canonical.com>
19 */
20
21#include "util/connection-helper.h"
22#include "helper/restore-helper.h"
23#include "service/app-const.h" // HELPER_TYPE
24
25#include <QByteArray>
26#include <QDebug>
27#include <QLocalSocket>
28#include <QMap>
29#include <QObject>
30#include <QString>
31#include <QTimer>
32#include <QVector>
33
34#include <fcntl.h>
35#include <sys/types.h>
36#include <sys/socket.h>
37
38#include <functional> // std::bind()
39
40
41class RestoreHelperPrivate
42{
43public:
44
45 explicit RestoreHelperPrivate(
46 RestoreHelper* backup_helper
47 )
48 : q_ptr(backup_helper)
49 {
50 // listen for inactivity from storage framework
51 QObject::connect(&timer_, &QTimer::timeout,
52 std::bind(&RestoreHelperPrivate::on_inactivity_detected, this)
53 );
54
55 // fire up the sockets
56 int fds[2];
57 int rc = socketpair(AF_UNIX, SOCK_STREAM | SOCK_NONBLOCK, 0, fds);
58 if (rc == -1)
59 {
60 // TODO throw exception.
61 qWarning() << "RestoreHelperPrivate: error creating socket pair to communicate with helper ";
62 return;
63 }
64
65 // helper socket is for the client.
66 // We don't use a QLocalSocket here as it buffers data and it makes the helper miss packets.
67 helper_socket_ = fds[1];
68
69 write_socket_.setSocketDescriptor(fds[0], QLocalSocket::ConnectedState, QIODevice::WriteOnly);
70 }
71
72 ~RestoreHelperPrivate() = default;
73
74 Q_DISABLE_COPY(RestoreHelperPrivate)
75
76 void start(QStringList const& urls)
77 {
78 q_ptr->Helper::start(urls);
79 reset_inactivity_timer();
80 }
81
82 void set_downloader(std::shared_ptr<Downloader> const& downloader)
83 {
84 n_read_ = 0;
85 n_uploaded_ = 0;
86 read_error_ = false;
87 write_error_ = false;
88 cancelled_ = false;
89
90 q_ptr->set_expected_size(downloader->file_size());
91 downloader_ = downloader;
92
93 connections_.remember(QObject::connect(
94 &write_socket_, &QLocalSocket::bytesWritten,
95 std::bind(&RestoreHelperPrivate::on_data_uploaded, this, std::placeholders::_1)
96 ));
97
98 // listen for data ready to read
99 QObject::connect(downloader_->socket().get(), &QLocalSocket::readyRead,
100 std::bind(&RestoreHelperPrivate::on_ready_read, this)
101 );
102
103 // maybe there's data already to be read
104 process_more();
105
106 reset_inactivity_timer();
107 }
108
109 void stop()
110 {
111 write_socket_.disconnectFromServer();
112 cancelled_ = true;
113 q_ptr->Helper::stop();
114 }
115
116 int get_helper_socket() const
117 {
118 return helper_socket_;
119 }
120
121 QString to_string(Helper::State state) const
122 {
123 return state == Helper::State::STARTED
124 ? QStringLiteral("restoring")
125 : q_ptr->Helper::to_string(state);
126 }
127
128 void on_state_changed(Helper::State state)
129 {
130 switch (state)
131 {
132 case Helper::State::CANCELLED:
133 case Helper::State::FAILED:
134 qDebug() << "cancelled/failed, calling downloader_.reset()";
135 downloader_.reset();
136 break;
137
138 case Helper::State::DATA_COMPLETE: {
139 qDebug() << "Restore helper finished, calling downloader_.finish()";
140 write_socket_.disconnectFromServer();
141 downloader_->finish();
142 downloader_.reset();
143 break;
144 }
145
146 //case Helper::State::NOT_STARTED:
147 //case Helper::State::STARTED:
148 default:
149 break;
150 }
151 }
152
153 void on_helper_finished()
154 {
155 stop_inactivity_timer();
156 check_for_done();
157 }
158
159private:
160
161 void on_inactivity_detected()
162 {
163 stop_inactivity_timer();
164 qWarning() << "Inactivity detected in the helper...stopping it";
165 stop();
166 }
167
168 void on_ready_read()
169 {
170 process_more();
171 }
172
173 void on_data_uploaded(qint64 n)
174 {
175 n_uploaded_ += n;
176 q_ptr->record_data_transferred(n);
177 qDebug("n_read %zu n_uploaded %zu (newly uploaded %zu)", size_t(n_read_), size_t(n_uploaded_), size_t(n));
178 process_more();
179 check_for_done();
180 }
181
182 void process_more()
183 {
184 if (!downloader_)
185 return;
186
187 char readbuf[UPLOAD_BUFFER_MAX_];
188 auto socket = downloader_->socket();
189 while(socket->bytesAvailable() || upload_buffer_.size())
190 {
191 if (socket->bytesAvailable())
192 {
193 // try to fill the upload buf
194 int max_bytes = UPLOAD_BUFFER_MAX_ - upload_buffer_.size();
195 if (max_bytes > 0) {
196 const auto n = socket->read(readbuf, max_bytes);
197 if (n > 0) {
198 n_read_ += n;
199 upload_buffer_.append(readbuf, int(n));
200 qDebug("upload_buffer_.size() is %zu after reading %zu from helper", size_t(upload_buffer_.size()), size_t(n));
201 }
202 else if (n < 0) {
203 read_error_ = true;
204 qDebug() << "Read error in restore helper: " << socket->errorString();
205 stop();
206 return;
207 }
208 }
209 }
210
211 if (upload_buffer_.size())
212 {
213 // try to empty the upload buf
214 const auto n = write_socket_.write(upload_buffer_);
215 if (n > 0) {
216 upload_buffer_.remove(0, int(n));
217 qDebug("upload_buffer_.size() is %zu after writing %zu to cloud", size_t(upload_buffer_.size()), size_t(n));
218 continue;
219 }
220 else {
221 if (n < 0) {
222 write_error_ = true;
223 qWarning() << "Write error:" << write_socket_.errorString();
224 stop();
225 }
226 break;
227 }
228 }
229 }
230
231 reset_inactivity_timer();
232 }
233
234 void reset_inactivity_timer()
235 {
236 static constexpr int MAX_TIME_WAITING_FOR_DATA {RestoreHelper::MAX_INACTIVITY_TIME};
237 timer_.start(MAX_TIME_WAITING_FOR_DATA);
238 }
239
240 void stop_inactivity_timer()
241 {
242 timer_.stop();
243 }
244
245 void check_for_done()
246 {
247 if (cancelled_)
248 {
249 q_ptr->set_state(Helper::State::CANCELLED);
250 }
251 else if (read_error_ || write_error_ || n_uploaded_ > q_ptr->expected_size())
252 {
253 if (!q_ptr->is_helper_running())
254 {
255 q_ptr->set_state(Helper::State::FAILED);
256 }
257 }
258 else if (n_uploaded_ == q_ptr->expected_size())
259 {
260 if (downloader_)
261 {
262 if (q_ptr->is_helper_running())
263 {
264 // only in the case that the helper process finished we move to the next state
265 // this is to prevent to start the next task too early
266 q_ptr->set_state(Helper::State::DATA_COMPLETE);
267 stop_inactivity_timer();
268 }
269 }
270 else
271 q_ptr->set_state(Helper::State::COMPLETE);
272 }
273 }
274
275 /***
276 ****
277 ***/
278
279 static constexpr int UPLOAD_BUFFER_MAX_ {1024*16};
280
281 RestoreHelper * const q_ptr;
282 QTimer timer_;
283 std::shared_ptr<Downloader> downloader_;
284 int helper_socket_ = -1;
285 QLocalSocket write_socket_;
286 QByteArray upload_buffer_;
287 qint64 n_read_ = 0;
288 qint64 n_uploaded_ = 0;
289 bool read_error_ = false;
290 bool write_error_ = false;
291 bool cancelled_ = false;
292 ConnectionHelper connections_;
293};
294
295/***
296****
297***/
298
299RestoreHelper::RestoreHelper(
300 QString const & appid,
301 clock_func const & clock,
302 QObject * parent
303)
304 : Helper(appid, clock, parent)
305 , d_ptr(new RestoreHelperPrivate(this))
306{
307}
308
309RestoreHelper::~RestoreHelper() =default;
310
311void
312RestoreHelper::start(QStringList const& url)
313{
314 Q_D(RestoreHelper);
315
316 d->start(url);
317}
318
319void
320RestoreHelper::stop()
321{
322 Q_D(RestoreHelper);
323
324 d->stop();
325}
326
327void
328RestoreHelper::set_downloader(std::shared_ptr<Downloader> const& downloader)
329{
330 Q_D(RestoreHelper);
331
332 d->set_downloader(downloader);
333}
334
335int
336RestoreHelper::get_helper_socket() const
337{
338 Q_D(const RestoreHelper);
339
340 return d->get_helper_socket();
341}
342
343QString
344RestoreHelper::to_string(Helper::State state) const
345{
346 Q_D(const RestoreHelper);
347
348 return d->to_string(state);
349}
350
351void
352RestoreHelper::set_state(Helper::State state)
353{
354 Q_D(RestoreHelper);
355
356 Helper::set_state(state);
357 d->on_state_changed(state);
358}
359
360void RestoreHelper::on_helper_finished()
361{
362 Q_D(RestoreHelper);
363
364 Helper::on_helper_finished();
365 d->on_helper_finished();
366}
0367
=== modified file 'src/service/CMakeLists.txt'
--- src/service/CMakeLists.txt 2016-10-28 15:11:21 +0000
+++ src/service/CMakeLists.txt 2016-11-24 14:48:54 +0000
@@ -15,6 +15,7 @@
15 task-manager.cpp15 task-manager.cpp
16 keeper-task.cpp16 keeper-task.cpp
17 keeper-task-backup.cpp17 keeper-task-backup.cpp
18 keeper-task-restore.cpp
18 manifest.cpp19 manifest.cpp
19 metadata-provider.h20 metadata-provider.h
20)21)
2122
=== modified file 'src/service/keeper-helper.cpp'
--- src/service/keeper-helper.cpp 2016-08-10 02:06:08 +0000
+++ src/service/keeper-helper.cpp 2016-11-24 14:48:54 +0000
@@ -44,8 +44,11 @@
4444
45QDBusUnixFileDescriptor KeeperHelper::StartRestore()45QDBusUnixFileDescriptor KeeperHelper::StartRestore()
46{46{
47 // TODO get the file descriptor of the item in storage framework47 // pass it back to Keeper to do the work
48 return QDBusUnixFileDescriptor();48 Q_ASSERT(calledFromDBus());
49 auto bus = connection();
50 auto& msg = message();
51 return keeper_.StartRestore(bus, msg);
49}52}
5053
51void KeeperHelper::UpdateStatus(const QString &app_id, const QString &status, double percentage)54void KeeperHelper::UpdateStatus(const QString &app_id, const QString &status, double percentage)
5255
=== added file 'src/service/keeper-task-restore.cpp'
--- src/service/keeper-task-restore.cpp 1970-01-01 00:00:00 +0000
+++ src/service/keeper-task-restore.cpp 2016-11-24 14:48:54 +0000
@@ -0,0 +1,130 @@
1/*
2 * Copyright (C) 2016 Canonical, Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3, as published
6 * by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranties of
10 * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11 * PURPOSE. See the GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License along
14 * with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authors:
17 * Xavi Garcia <xavi.garcia.mena@canonical.com>
18 * Charles Kerr <charles.kerr@canonical.com>
19 */
20
21#include "util/connection-helper.h"
22#include "storage-framework/storage_framework_client.h"
23#include "helper/restore-helper.h"
24#include "service/app-const.h" // DEKKO_APP_ID
25#include "service/keeper-task-restore.h"
26#include "service/keeper-task.h"
27#include "service/private/keeper-task_p.h"
28
29namespace sf = unity::storage::qt::client;
30
31class KeeperTaskRestorePrivate : public KeeperTaskPrivate
32{
33 Q_DECLARE_PUBLIC(KeeperTaskRestore)
34public:
35 KeeperTaskRestorePrivate(KeeperTask * keeper_task,
36 KeeperTask::TaskData & task_data,
37 QSharedPointer<HelperRegistry> const & helper_registry,
38 QSharedPointer<StorageFrameworkClient> const & storage)
39 : KeeperTaskPrivate(keeper_task, task_data, helper_registry, storage)
40 {
41 }
42
43 ~KeeperTaskRestorePrivate() = default;
44
45 QStringList get_helper_urls() const
46 {
47 return helper_registry_->get_restore_helper_urls(task_data_.metadata);
48 }
49
50 void init_helper()
51 {
52 qDebug() << "Initializing a restore helper";
53 helper_.reset(new RestoreHelper(DEKKO_APP_ID), [](Helper *h){h->deleteLater();});
54 qDebug() << "Helper " << static_cast<void*>(helper_.data()) << " was created";
55 }
56
57 void ask_for_downloader()
58 {
59 qDebug() << "asking storage framework for a socket for reading";
60
61 QString file_name;
62 task_data_.metadata.get_property(Metadata::FILE_NAME_KEY, file_name);
63 if (file_name.isEmpty())
64 {
65 qWarning() << "ERROR: the restore task does not provide a valid file name to read from.";
66 return;
67 }
68
69 QString dir_name;
70 task_data_.metadata.get_property(Metadata::DIR_NAME_KEY, dir_name);
71 if (dir_name.isEmpty())
72 {
73 qWarning() << "ERROR: the restore task does not provide a valid directory name.";
74 return;
75 }
76
77 // extract the dir_name.
78 connections_.connect_future(
79 storage_->get_new_downloader(dir_name, file_name),
80 std::function<void(std::shared_ptr<Downloader> const&)>{
81 [this](std::shared_ptr<Downloader> const& downloader){
82 qDebug() << "Downloader is" << static_cast<void*>(downloader.get());
83 int fd {-1};
84 if (downloader) {
85 qDebug() << "Helper is " << static_cast<void*>(helper_.data());
86 auto restore_helper = qSharedPointerDynamicCast<RestoreHelper>(helper_);
87 restore_helper->set_downloader(downloader);
88 fd = restore_helper->get_helper_socket();
89 }
90 qDebug("emitting task_socket_ready(socket=%d)", fd);
91 Q_EMIT(q_ptr->task_socket_ready(fd));
92 }
93 }
94 );
95 }
96
97private:
98 ConnectionHelper connections_;
99};
100
101KeeperTaskRestore::KeeperTaskRestore(TaskData & task_data,
102 QSharedPointer<HelperRegistry> const & helper_registry,
103 QSharedPointer<StorageFrameworkClient> const & storage,
104 QObject *parent)
105 : KeeperTask(*new KeeperTaskRestorePrivate(this, task_data, helper_registry, storage), parent)
106{
107}
108
109KeeperTaskRestore::~KeeperTaskRestore() = default;
110
111QStringList KeeperTaskRestore::get_helper_urls() const
112{
113 Q_D(const KeeperTaskRestore);
114
115 return d->get_helper_urls();
116}
117
118void KeeperTaskRestore::init_helper()
119{
120 Q_D(KeeperTaskRestore);
121
122 d->init_helper();
123}
124
125void KeeperTaskRestore::ask_for_downloader()
126{
127 Q_D(KeeperTaskRestore);
128
129 d->ask_for_downloader();
130}
0131
=== added file 'src/service/keeper-task-restore.h'
--- src/service/keeper-task-restore.h 1970-01-01 00:00:00 +0000
+++ src/service/keeper-task-restore.h 2016-11-24 14:48:54 +0000
@@ -0,0 +1,46 @@
1/*
2 * Copyright (C) 2016 Canonical, Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3, as published
6 * by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranties of
10 * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11 * PURPOSE. See the GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License along
14 * with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authors:
17 * Xavi Garcia <xavi.garcia.mena@canonical.com>
18 * Charles Kerr <charles.kerr@canonical.com>
19 */
20#pragma once
21
22#include "keeper-task.h"
23
24class KeeperTaskRestorePrivate;
25
26class KeeperTaskRestore : public KeeperTask
27{
28 Q_OBJECT
29 Q_DECLARE_PRIVATE(KeeperTaskRestore)
30public:
31
32 KeeperTaskRestore(TaskData & task_data,
33 QSharedPointer<HelperRegistry> const & helper_registry,
34 QSharedPointer<StorageFrameworkClient> const & storage,
35 QObject *parent = nullptr);
36 virtual ~KeeperTaskRestore();
37
38 Q_DISABLE_COPY(KeeperTaskRestore)
39
40 void ask_for_downloader();
41
42protected:
43 QStringList get_helper_urls() const override;
44 void init_helper() override;
45
46};
047
=== modified file 'src/service/keeper-user.cpp'
--- src/service/keeper-user.cpp 2016-10-28 15:11:21 +0000
+++ src/service/keeper-user.cpp 2016-11-24 14:48:54 +0000
@@ -70,9 +70,15 @@
70void70void
71KeeperUser::StartRestore (const QStringList& keys)71KeeperUser::StartRestore (const QStringList& keys)
72{72{
73 // FIXME: writeme73 Q_ASSERT(calledFromDBus());
7474
75 qDebug() << keys;75 auto bus = connection();
76 auto& msg = message();
77 // if we start a restore right after a backup the uuid
78 // will be found as a backup uuid.
79 // Just clear the backup cache to avoid that.
80 keeper_.invalidate_choices_cache();
81 keeper_.start_tasks(keys, bus, msg);
76}82}
7783
78QVariantDictMap84QVariantDictMap
7985
=== modified file 'src/service/keeper.cpp'
--- src/service/keeper.cpp 2016-11-24 14:48:54 +0000
+++ src/service/keeper.cpp 2016-11-24 14:48:54 +0000
@@ -32,6 +32,7 @@
32#include <QVector>32#include <QVector>
3333
34#include <algorithm> // std::find_if34#include <algorithm> // std::find_if
35#include <unistd.h>
3536
36namespace37namespace
37{38{
@@ -83,8 +84,8 @@
83 Q_DISABLE_COPY(KeeperPrivate)84 Q_DISABLE_COPY(KeeperPrivate)
8485
85 void start_tasks(QStringList const & uuids,86 void start_tasks(QStringList const & uuids,
86 QDBusConnection bus,87 QDBusConnection bus,
87 QDBusMessage const & msg)88 QDBusMessage const & msg)
88 {89 {
89 auto get_tasks = [](const QVector<Metadata>& pool, QStringList const& keys){90 auto get_tasks = [](const QVector<Metadata>& pool, QStringList const& keys){
90 QMap<QString,Metadata> tasks;91 QMap<QString,Metadata> tasks;
@@ -97,9 +98,10 @@
97 };98 };
9899
99 // async part100 // async part
101 qDebug() << "Looking for backup options....";
100 connections_.connect_oneshot(102 connections_.connect_oneshot(
101 this,103 this,
102 &KeeperPrivate::choices_ready,104 &KeeperPrivate::backup_choices_ready,
103 std::function<void()>{[this, uuids, msg, bus, get_tasks](){105 std::function<void()>{[this, uuids, msg, bus, get_tasks](){
104 auto tasks = get_tasks(cached_backup_choices_, uuids);106 auto tasks = get_tasks(cached_backup_choices_, uuids);
105 if (!tasks.empty())107 if (!tasks.empty())
@@ -116,12 +118,15 @@
116 }118 }
117 else // restore119 else // restore
118 {120 {
121 qDebug() << "Looking for restore options....";
119 connections_.connect_oneshot(122 connections_.connect_oneshot(
120 this,123 this,
121 &KeeperPrivate::choices_ready,124 &KeeperPrivate::restore_choices_ready,
122 std::function<void()>{[this, uuids, msg, bus, get_tasks](){125 std::function<void()>{[this, uuids, msg, bus, get_tasks](){
126 qDebug() << "Choices ready";
123 auto unhandled = QSet<QString>::fromList(uuids);127 auto unhandled = QSet<QString>::fromList(uuids);
124 auto restore_tasks = get_tasks(cached_restore_choices_, uuids);128 auto restore_tasks = get_tasks(cached_restore_choices_, uuids);
129 qDebug() << "After getting tasks...";
125 if (!restore_tasks.empty() && task_manager_.start_restore(restore_tasks.values()))130 if (!restore_tasks.empty() && task_manager_.start_restore(restore_tasks.values()))
126 unhandled.subtract(QSet<QString>::fromList(restore_tasks.keys()));131 unhandled.subtract(QSet<QString>::fromList(restore_tasks.keys()));
127132
@@ -141,6 +146,18 @@
141 msg.setDelayedReply(true);146 msg.setDelayedReply(true);
142 }147 }
143148
149 void emit_choices_ready(ChoicesType type)
150 {
151 switch(type)
152 {
153 case KeeperPrivate::ChoicesType::BACKUP_CHOICES:
154 Q_EMIT(backup_choices_ready());
155 break;
156 case KeeperPrivate::ChoicesType::RESTORES_CHOICES:
157 Q_EMIT(restore_choices_ready());
158 break;
159 }
160 }
144 void get_choices(const QSharedPointer<MetadataProvider> & provider, ChoicesType type)161 void get_choices(const QSharedPointer<MetadataProvider> & provider, ChoicesType type)
145 {162 {
146 bool check_empty = (type == KeeperPrivate::ChoicesType::BACKUP_CHOICES)163 bool check_empty = (type == KeeperPrivate::ChoicesType::BACKUP_CHOICES)
@@ -161,7 +178,7 @@
161 cached_restore_choices_ = provider->get_backups();178 cached_restore_choices_ = provider->get_backups();
162 break;179 break;
163 };180 };
164 Q_EMIT(choices_ready());181 emit_choices_ready(type);
165 }}182 }}
166 );183 );
167184
@@ -169,7 +186,7 @@
169 }186 }
170 else187 else
171 {188 {
172 Q_EMIT(choices_ready());189 emit_choices_ready(type);
173 }190 }
174 }191 }
175192
@@ -178,7 +195,7 @@
178 {195 {
179 connections_.connect_oneshot(196 connections_.connect_oneshot(
180 this,197 this,
181 &KeeperPrivate::choices_ready,198 &KeeperPrivate::backup_choices_ready,
182 std::function<void()>{[this, msg, bus](){199 std::function<void()>{[this, msg, bus](){
183 qDebug() << "Backup choices are ready";200 qDebug() << "Backup choices are ready";
184 // reply now to the dbus call201 // reply now to the dbus call
@@ -198,7 +215,7 @@
198 {215 {
199 connections_.connect_oneshot(216 connections_.connect_oneshot(
200 this,217 this,
201 &KeeperPrivate::choices_ready,218 &KeeperPrivate::restore_choices_ready,
202 std::function<void()>{[this, msg, bus](){219 std::function<void()>{[this, msg, bus](){
203 qDebug() << "Restore choices are ready";220 qDebug() << "Restore choices are ready";
204 // reply now to the dbus call221 // reply now to the dbus call
@@ -213,29 +230,6 @@
213 return QVariantDictMap();230 return QVariantDictMap();
214 }231 }
215232
216 // TODO REFACTOR THIS TO USE THE SAME METHOD POR RESTORES AND BACKUPS
217 void get_restore_choices()
218 {
219 if (cached_restore_choices_.isEmpty())
220 {
221 connections_.connect_oneshot(
222 restore_choices_.data(),
223 &MetadataProvider::finished,
224 std::function<void()>{[this](){
225 qDebug() << "Get restores finished";
226 cached_restore_choices_ = restore_choices_->get_backups();
227 Q_EMIT(choices_ready());
228 }}
229 );
230
231 restore_choices_->get_backups_async();
232 }
233 else
234 {
235 Q_EMIT(choices_ready());
236 }
237 }
238
239 QVariantDictMap get_state() const233 QVariantDictMap get_state() const
240 {234 {
241 return task_manager_.get_state();235 return task_manager_.get_state();
@@ -260,7 +254,7 @@
260 }254 }
261 );255 );
262256
263 qDebug() << "Asking for an storage framework socket to the task manager";257 qDebug() << "Asking for a storage framework socket from the task manager";
264 task_manager_.ask_for_uploader(n_bytes);258 task_manager_.ask_for_uploader(n_bytes);
265259
266 // tell the caller that we'll be responding async260 // tell the caller that we'll be responding async
@@ -268,12 +262,47 @@
268 return QDBusUnixFileDescriptor(0);262 return QDBusUnixFileDescriptor(0);
269 }263 }
270264
265
266 QDBusUnixFileDescriptor start_restore(QDBusConnection bus,
267 QDBusMessage const & msg)
268 {
269 qDebug() << "Keeper::StartRestore()";
270
271 connections_.connect_oneshot(
272 &task_manager_,
273 &TaskManager::socket_ready,
274 std::function<void(int)>{
275 [bus,msg](int fd){
276 qDebug("RestoreManager returned socket %d", fd);
277 auto reply = msg.createReply();
278 reply << QVariant::fromValue(QDBusUnixFileDescriptor(fd));
279 close(fd);
280 bus.send(reply);
281 }
282 }
283 );
284
285 qDebug() << "Asking for a storage framework socket from the task manager";
286 task_manager_.ask_for_downloader();
287
288 // tell the caller that we'll be responding async
289 msg.setDelayedReply(true);
290 return QDBusUnixFileDescriptor(0);
291 }
292
271 void cancel()293 void cancel()
272 {294 {
273 task_manager_.cancel();295 task_manager_.cancel();
274 }296 }
297
298 void invalidate_choices_cache()
299 {
300 cached_backup_choices_.clear();
301 }
302
275Q_SIGNALS:303Q_SIGNALS:
276 void choices_ready();304 void backup_choices_ready();
305 void restore_choices_ready();
277306
278private:307private:
279308
@@ -334,6 +363,15 @@
334 return d->start_backup(bus, msg, n_bytes);363 return d->start_backup(bus, msg, n_bytes);
335}364}
336365
366QDBusUnixFileDescriptor
367Keeper::StartRestore(QDBusConnection bus,
368 QDBusMessage const & msg)
369{
370 Q_D(Keeper);
371
372 return d->start_restore(bus, msg);
373}
374
337QVariantDictMap375QVariantDictMap
338Keeper::get_backup_choices_var_dict_map(QDBusConnection bus,376Keeper::get_backup_choices_var_dict_map(QDBusConnection bus,
339 QDBusMessage const & msg)377 QDBusMessage const & msg)
@@ -368,4 +406,12 @@
368 return d->cancel();406 return d->cancel();
369}407}
370408
409void
410Keeper::invalidate_choices_cache()
411{
412 Q_D(Keeper);
413
414 d->invalidate_choices_cache();
415}
416
371#include "keeper.moc"417#include "keeper.moc"
372418
=== modified file 'src/service/keeper.h'
--- src/service/keeper.h 2016-10-28 15:11:21 +0000
+++ src/service/keeper.h 2016-11-24 14:48:54 +0000
@@ -59,6 +59,10 @@
59 QDBusMessage const & message,59 QDBusMessage const & message,
60 quint64 nbytes);60 quint64 nbytes);
6161
62
63 QDBusUnixFileDescriptor StartRestore(QDBusConnection,
64 QDBusMessage const & message);
65
62 void start_tasks(QStringList const & uuids,66 void start_tasks(QStringList const & uuids,
63 QDBusConnection bus,67 QDBusConnection bus,
64 QDBusMessage const & msg);68 QDBusMessage const & msg);
@@ -67,6 +71,8 @@
6771
68 void cancel();72 void cancel();
6973
74 void invalidate_choices_cache();
75
70private:76private:
71 QScopedPointer<KeeperPrivate> const d_ptr;77 QScopedPointer<KeeperPrivate> const d_ptr;
72};78};
7379
=== modified file 'src/service/manifest.cpp'
--- src/service/manifest.cpp 2016-10-13 12:22:31 +0000
+++ src/service/manifest.cpp 2016-11-24 14:48:54 +0000
@@ -110,8 +110,8 @@
110 {110 {
111 connections_.connect_future(111 connections_.connect_future(
112 storage_->get_new_downloader(dir_, MANIFEST_FILE_NAME),112 storage_->get_new_downloader(dir_, MANIFEST_FILE_NAME),
113 std::function<void(sf::Downloader::SPtr const&)>{113 std::function<void(std::shared_ptr<Downloader> const&)>{
114 [this](sf::Downloader::SPtr const& downloader){114 [this](std::shared_ptr<Downloader> const& downloader){
115 qDebug() << "Manifest downloader is" << static_cast<void*>(downloader.get());115 qDebug() << "Manifest downloader is" << static_cast<void*>(downloader.get());
116 if (downloader)116 if (downloader)
117 {117 {
@@ -125,7 +125,7 @@
125 }125 }
126 auto json_content = socket->readAll();126 auto json_content = socket->readAll();
127 from_json(json_content);127 from_json(json_content);
128 downloader->finish_download();128 downloader->finish();
129 finish();129 finish();
130 }130 }
131 else131 else
132132
=== modified file 'src/service/task-manager.cpp'
--- src/service/task-manager.cpp 2016-11-24 14:48:54 +0000
+++ src/service/task-manager.cpp 2016-11-24 14:48:54 +0000
@@ -20,6 +20,7 @@
2020
21#include "helper/metadata.h"21#include "helper/metadata.h"
22#include "keeper-task-backup.h"22#include "keeper-task-backup.h"
23#include "keeper-task-restore.h"
23#include "manifest.h"24#include "manifest.h"
24#include "storage-framework/storage_framework_client.h"25#include "storage-framework/storage_framework_client.h"
25#include "task-manager.h"26#include "task-manager.h"
@@ -50,6 +51,7 @@
5051
51 bool start_restore(QList<Metadata> const& tasks)52 bool start_restore(QList<Metadata> const& tasks)
52 {53 {
54 qDebug() << "Starting restore...";
53 return start_tasks(tasks, Mode::RESTORE);55 return start_tasks(tasks, Mode::RESTORE);
54 }56 }
5557
@@ -78,6 +80,22 @@
78 }80 }
79 }81 }
8082
83 void ask_for_downloader()
84 {
85 qDebug() << "Starting restore";
86 if (task_)
87 {
88 auto restore_task_ = qSharedPointerDynamicCast<KeeperTaskRestore>(task_);
89 if (!restore_task_)
90 {
91 qWarning() << "Only restore tasks are allowed to ask for storage framework downloaders";
92 // TODO Mark this as an error at the current task and move to the next task
93 return;
94 }
95 restore_task_->ask_for_downloader();
96 }
97 }
98
81 void cancel()99 void cancel()
82 {100 {
83 if (task_)101 if (task_)
@@ -158,7 +176,6 @@
158176
159 void on_helper_state_changed(Helper::State state)177 void on_helper_state_changed(Helper::State state)
160 {178 {
161 qDebug() << "Task State changed";
162 auto backup_task_ = qSharedPointerDynamicCast<KeeperTaskBackup>(task_);179 auto backup_task_ = qSharedPointerDynamicCast<KeeperTaskBackup>(task_);
163 auto& td = task_data_[current_task_];180 auto& td = task_data_[current_task_];
164 update_task_state(td);181 update_task_state(td);
@@ -174,6 +191,7 @@
174 {191 {
175 qDebug() << "Backup task finished. The file created in storage framework is: [" << backup_task_->get_file_name() << "]";192 qDebug() << "Backup task finished. The file created in storage framework is: [" << backup_task_->get_file_name() << "]";
176 td.metadata.set_property(Metadata::FILE_NAME_KEY, backup_task_->get_file_name());193 td.metadata.set_property(Metadata::FILE_NAME_KEY, backup_task_->get_file_name());
194 td.metadata.set_property(Metadata::DIR_NAME_KEY, backup_dir_name_);
177 active_manifest_->add_entry(td.metadata);195 active_manifest_->add_entry(td.metadata);
178 }196 }
179 if (remaining_tasks_.size())197 if (remaining_tasks_.size())
@@ -217,14 +235,16 @@
217 qDebug() << "Creating task for uuid = " << uuid;235 qDebug() << "Creating task for uuid = " << uuid;
218 // initialize a new task236 // initialize a new task
219237
220 task_.data()->disconnect();238 if (task_)
239 task_.data()->disconnect();
240
221 if (mode_ == Mode::BACKUP)241 if (mode_ == Mode::BACKUP)
222 {242 {
223 task_.reset(new KeeperTaskBackup(td, helper_registry_, storage_));243 task_.reset(new KeeperTaskBackup(td, helper_registry_, storage_));
224 }244 }
225 else245 else
226 {246 {
227 // TODO initialize a Restore task247 task_.reset(new KeeperTaskRestore(td, helper_registry_, storage_));
228 }248 }
229249
230 qDebug() << "task created: " << state_;250 qDebug() << "task created: " << state_;
@@ -396,6 +416,13 @@
396 d->ask_for_uploader(n_bytes);416 d->ask_for_uploader(n_bytes);
397}417}
398418
419void TaskManager::ask_for_downloader()
420{
421 Q_D(TaskManager);
422
423 d->ask_for_downloader();
424}
425
399void TaskManager::cancel()426void TaskManager::cancel()
400{427{
401 Q_D(TaskManager);428 Q_D(TaskManager);
402429
=== modified file 'src/service/task-manager.h'
--- src/service/task-manager.h 2016-09-28 13:42:21 +0000
+++ src/service/task-manager.h 2016-11-24 14:48:54 +0000
@@ -58,6 +58,8 @@
5858
59 void ask_for_uploader(quint64 n_bytes);59 void ask_for_uploader(quint64 n_bytes);
6060
61 void ask_for_downloader();
62
61 void cancel();63 void cancel();
6264
63Q_SIGNALS:65Q_SIGNALS:
6466
=== modified file 'src/storage-framework/CMakeLists.txt'
--- src/storage-framework/CMakeLists.txt 2016-09-05 17:24:18 +0000
+++ src/storage-framework/CMakeLists.txt 2016-11-24 14:48:54 +0000
@@ -11,6 +11,9 @@
11 uploader.h11 uploader.h
12 sf-uploader.cpp12 sf-uploader.cpp
13 sf-uploader.h13 sf-uploader.h
14 downloader.h
15 sf-downloader.cpp
16 sf-downloader.h
14)17)
1518
16set_target_properties(19set_target_properties(
1720
=== added file 'src/storage-framework/downloader.h'
--- src/storage-framework/downloader.h 1970-01-01 00:00:00 +0000
+++ src/storage-framework/downloader.h 2016-11-24 14:48:54 +0000
@@ -0,0 +1,46 @@
1/*
2 * Copyright (C) 2016 Canonical, Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3, as published
6 * by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranties of
10 * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11 * PURPOSE. See the GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License along
14 * with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authors:
17 * Charles Kerr <charles.kerr@canonical.com>
18 * Xavi Garcia Mena <xavi.garcia.mena@canonical.com>
19 */
20
21#pragma once
22
23#include <QLocalSocket>
24#include <QObject>
25
26#include <memory>
27
28class Downloader: public QObject
29{
30 Q_OBJECT
31
32public:
33
34 Q_DISABLE_COPY(Downloader)
35
36 Downloader(QObject *parent=nullptr): QObject(parent) {}
37 virtual ~Downloader() =default;
38
39 virtual std::shared_ptr<QLocalSocket> socket() =0;
40 virtual void finish() =0;
41 virtual qint64 file_size() const =0;
42
43Q_SIGNALS:
44
45 void download_finished();
46};
047
=== added file 'src/storage-framework/sf-downloader.cpp'
--- src/storage-framework/sf-downloader.cpp 1970-01-01 00:00:00 +0000
+++ src/storage-framework/sf-downloader.cpp 2016-11-24 14:48:54 +0000
@@ -0,0 +1,56 @@
1/*
2 * Copyright (C) 2016 Canonical, Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3, as published
6 * by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranties of
10 * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11 * PURPOSE. See the GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License along
14 * with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authors:
17 * Charles Kerr <charles.kerr@canonical.com>
18 * Xavi Garcia Mena <xavi.garcia.mena@canonical.com>
19 */
20
21#include "storage-framework/sf-downloader.h"
22
23#include <QFuture>
24#include <QFutureWatcher>
25
26StorageFrameworkDownloader::StorageFrameworkDownloader(
27 unity::storage::qt::client::Downloader::SPtr const& downloader,
28 qint64 file_size,
29 QObject *parent
30):
31 Downloader{parent},
32 downloader_{downloader},
33 file_size_{file_size}
34{
35 qDebug() << "StorageFrameworkDownloader";
36}
37
38std::shared_ptr<QLocalSocket>
39StorageFrameworkDownloader::socket()
40{
41 return downloader_->socket();
42}
43
44void
45StorageFrameworkDownloader::finish()
46{
47 qDebug() << Q_FUNC_INFO << "is finishing";
48 downloader_->finish_download();
49 Q_EMIT(download_finished()); // TODO add the code to call finish_download
50}
51
52qint64
53StorageFrameworkDownloader::file_size() const
54{
55 return file_size_;
56}
057
=== added file 'src/storage-framework/sf-downloader.h'
--- src/storage-framework/sf-downloader.h 1970-01-01 00:00:00 +0000
+++ src/storage-framework/sf-downloader.h 2016-11-24 14:48:54 +0000
@@ -0,0 +1,48 @@
1/*
2 * Copyright (C) 2016 Canonical, Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3, as published
6 * by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranties of
10 * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11 * PURPOSE. See the GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License along
14 * with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authors:
17 * Charles Kerr <charles.kerr@canonical.com>
18 */
19
20#pragma once
21
22#include "util/connection-helper.h"
23#include "storage-framework/downloader.h"
24
25#include <unity/storage/qt/client/client-api.h>
26
27#include <QLocalSocket>
28
29#include <memory>
30
31class StorageFrameworkDownloader final: public Downloader
32{
33public:
34
35 StorageFrameworkDownloader(unity::storage::qt::client::Downloader::SPtr const& uploader,
36 qint64 file_size,
37 QObject * parent = nullptr);
38 std::shared_ptr<QLocalSocket> socket() override;
39 void finish() override;
40 qint64 file_size() const override;
41
42private:
43
44 unity::storage::qt::client::Downloader::SPtr const downloader_;
45 qint64 file_size_;
46
47 ConnectionHelper connections_;
48};
049
=== modified file 'src/storage-framework/storage_framework_client.cpp'
--- src/storage-framework/storage_framework_client.cpp 2016-11-03 10:32:49 +0000
+++ src/storage-framework/storage_framework_client.cpp 2016-11-24 14:48:54 +0000
@@ -19,6 +19,7 @@
19 */19 */
2020
21#include "storage-framework/storage_framework_client.h"21#include "storage-framework/storage_framework_client.h"
22#include "storage-framework/sf-downloader.h"
22#include "storage-framework/sf-uploader.h"23#include "storage-framework/sf-uploader.h"
2324
24#include <QDateTime>25#include <QDateTime>
@@ -39,7 +40,7 @@
39{40{
40}41}
4142
42StorageFrameworkClient::~StorageFrameworkClient() =default;43StorageFrameworkClient::~StorageFrameworkClient() = default;
4344
44/***45/***
45****46****
@@ -115,8 +116,8 @@
115 connection_helper_.connect_future(116 connection_helper_.connect_future(
116 get_keeper_folder(root, dir_name, true),117 get_keeper_folder(root, dir_name, true),
117 std::function<void(sf::Folder::SPtr const&)>{118 std::function<void(sf::Folder::SPtr const&)>{
118 [this, fi, n_bytes, file_name](sf::Folder::SPtr const& keeper_root){119 [this, fi, n_bytes, file_name,root](sf::Folder::SPtr const& keeper_folder){
119 if (!keeper_root)120 if (!keeper_folder)
120 {121 {
121 qWarning() << "Error creating keeper root folder";122 qWarning() << "Error creating keeper root folder";
122 std::shared_ptr<Uploader> ret;123 std::shared_ptr<Uploader> ret;
@@ -127,9 +128,9 @@
127 else128 else
128 {129 {
129 connection_helper_.connect_future(130 connection_helper_.connect_future(
130 keeper_root->create_file(file_name, n_bytes),131 keeper_folder->create_file(file_name, n_bytes),
131 std::function<void(std::shared_ptr<sf::Uploader> const&)>{132 std::function<void(std::shared_ptr<sf::Uploader> const&)>{
132 [this, fi](std::shared_ptr<sf::Uploader> const& sf_uploader){133 [this, fi, keeper_folder](std::shared_ptr<sf::Uploader> const& sf_uploader){
133 qDebug() << "keeper_root->create_file() finished";134 qDebug() << "keeper_root->create_file() finished";
134 std::shared_ptr<Uploader> ret;135 std::shared_ptr<Uploader> ret;
135 if (sf_uploader) {136 if (sf_uploader) {
@@ -154,10 +155,10 @@
154 return fi.future();155 return fi.future();
155}156}
156157
157QFuture<sf::Downloader::SPtr>158QFuture<std::shared_ptr<Downloader>>
158StorageFrameworkClient::get_new_downloader(QString const & dir_name, QString const & file_name)159StorageFrameworkClient::get_new_downloader(QString const & dir_name, QString const & file_name)
159{160{
160 QFutureInterface<sf::Downloader::SPtr> fi;161 QFutureInterface<std::shared_ptr<Downloader>> fi;
161162
162 add_roots_task([this, fi, dir_name, file_name](QVector<sf::Root::SPtr> const& roots)163 add_roots_task([this, fi, dir_name, file_name](QVector<sf::Root::SPtr> const& roots)
163 {164 {
@@ -171,7 +172,7 @@
171 if (!keeper_root)172 if (!keeper_root)
172 {173 {
173 qWarning() << "Error accessing keeper root folder";174 qWarning() << "Error accessing keeper root folder";
174 sf::Downloader::SPtr ret;175 std::shared_ptr<Downloader> ret;
175 QFutureInterface<decltype(ret)> qfi(fi);176 QFutureInterface<decltype(ret)> qfi(fi);
176 qfi.reportResult(ret);177 qfi.reportResult(ret);
177 qfi.reportFinished();178 qfi.reportFinished();
@@ -183,27 +184,27 @@
183 get_storage_framework_file(keeper_root, file_name),184 get_storage_framework_file(keeper_root, file_name),
184 std::function<void(sf::File::SPtr const&)>{185 std::function<void(sf::File::SPtr const&)>{
185 [this, fi](sf::File::SPtr const& sf_file){186 [this, fi](sf::File::SPtr const& sf_file){
186 sf::Downloader::SPtr ret_null;
187 if (sf_file) {187 if (sf_file) {
188 connection_helper_.connect_future(188 connection_helper_.connect_future(
189 sf_file->create_downloader(),189 sf_file->create_downloader(),
190 std::function<void(sf::Downloader::SPtr const&)>{190 std::function<void(sf::Downloader::SPtr const&)>{
191 [this, fi, ret_null](sf::Downloader::SPtr const& sf_downloader){191 [this, fi, sf_file](sf::Downloader::SPtr const& sf_downloader){
192 QFutureInterface<decltype(ret_null)> qfi(fi);192 std::shared_ptr<Downloader> ret;
193 if (sf_downloader)193 if (sf_downloader)
194 {194 {
195 qfi.reportResult(sf_downloader);195 ret.reset(
196 qfi.reportFinished();196 new StorageFrameworkDownloader(sf_downloader, sf_file->size(), this),
197 }197 [](Downloader* d){d->deleteLater();}
198 else198 );
199 {199 }
200 qfi.reportResult(ret_null);200 QFutureInterface<decltype(ret)> qfi(fi);
201 qfi.reportFinished();201 qfi.reportResult(ret);
202 }202 qfi.reportFinished();
203 }203 }
204 }204 }
205 );205 );
206 } else {206 } else {
207 std::shared_ptr<Downloader> ret_null;
207 QFutureInterface<decltype(ret_null)> qfi(fi);208 QFutureInterface<decltype(ret_null)> qfi(fi);
208 qfi.reportResult(ret_null);209 qfi.reportResult(ret_null);
209 qfi.reportFinished();210 qfi.reportFinished();
@@ -341,7 +342,7 @@
341 connection_helper_.connect_future(342 connection_helper_.connect_future(
342 root->create_folder(dir_name),343 root->create_folder(dir_name),
343 std::function<void(sf::Folder::SPtr const &)>{344 std::function<void(sf::Folder::SPtr const &)>{
344 [this, fi, res](sf::Folder::SPtr const & folder){345 [this, fi, res, root](sf::Folder::SPtr const & folder){
345 QFutureInterface<decltype(res)> qfi(fi);346 QFutureInterface<decltype(res)> qfi(fi);
346 qfi.reportResult(folder);347 qfi.reportResult(folder);
347 qfi.reportFinished();348 qfi.reportFinished();
348349
=== modified file 'src/storage-framework/storage_framework_client.h'
--- src/storage-framework/storage_framework_client.h 2016-11-03 10:32:49 +0000
+++ src/storage-framework/storage_framework_client.h 2016-11-24 14:48:54 +0000
@@ -22,6 +22,7 @@
2222
23#include "util/connection-helper.h"23#include "util/connection-helper.h"
24#include "storage-framework/uploader.h"24#include "storage-framework/uploader.h"
25#include "storage-framework/downloader.h"
2526
26#include <unity/storage/qt/client/client-api.h>27#include <unity/storage/qt/client/client-api.h>
2728
@@ -43,7 +44,7 @@
43 virtual ~StorageFrameworkClient();44 virtual ~StorageFrameworkClient();
4445
45 QFuture<std::shared_ptr<Uploader>> get_new_uploader(int64_t n_bytes, QString const & dir_name, QString const & file_name);46 QFuture<std::shared_ptr<Uploader>> get_new_uploader(int64_t n_bytes, QString const & dir_name, QString const & file_name);
46 QFuture<unity::storage::qt::client::Downloader::SPtr> get_new_downloader(QString const & dir_name, QString const & file_name);47 QFuture<std::shared_ptr<Downloader>> get_new_downloader(QString const & dir_name, QString const & file_name);
47 QFuture<QVector<QString>> get_keeper_dirs();48 QFuture<QVector<QString>> get_keeper_dirs();
4849
49 static QString const KEEPER_FOLDER;50 static QString const KEEPER_FOLDER;
5051
=== modified file 'tests/CMakeLists.txt'
--- tests/CMakeLists.txt 2016-09-05 13:54:15 +0000
+++ tests/CMakeLists.txt 2016-11-24 14:48:54 +0000
@@ -20,6 +20,11 @@
20)20)
2121
22set(22set(
23 RESTORE_HELPER
24 fake-restore-helper
25)
26
27set(
23 BACKUP_HELPER_FAILURE28 BACKUP_HELPER_FAILURE
24 fake-backup-helper-failure29 fake-backup-helper-failure
25)30)
@@ -27,6 +32,7 @@
27set(KEEPER_TAR_CREATE_BIN ${CMAKE_BINARY_DIR}/src/tar/keeper-tar-create)32set(KEEPER_TAR_CREATE_BIN ${CMAKE_BINARY_DIR}/src/tar/keeper-tar-create)
28set(KEEPER_HELPER_TEST_LOCATION ${CMAKE_BINARY_DIR}/tests/fakes/helpers-test.sh)33set(KEEPER_HELPER_TEST_LOCATION ${CMAKE_BINARY_DIR}/tests/fakes/helpers-test.sh)
29set(BACKUP_HELPER_FAILURE_LOCATION ${CMAKE_BINARY_DIR}/tests/fakes/${BACKUP_HELPER_FAILURE})34set(BACKUP_HELPER_FAILURE_LOCATION ${CMAKE_BINARY_DIR}/tests/fakes/${BACKUP_HELPER_FAILURE})
35set(RESTORE_HELPER_TEST_LOCATION ${CMAKE_BINARY_DIR}/tests/fakes/${RESTORE_HELPER})
3036
31add_definitions(37add_definitions(
32 -DCMAKE_SOURCE_DIR="${CMAKE_CURRENT_SOURCE_DIR}"38 -DCMAKE_SOURCE_DIR="${CMAKE_CURRENT_SOURCE_DIR}"
3339
=== modified file 'tests/fakes/CMakeLists.txt'
--- tests/fakes/CMakeLists.txt 2016-08-30 12:50:46 +0000
+++ tests/fakes/CMakeLists.txt 2016-11-24 14:48:54 +0000
@@ -30,6 +30,15 @@
30)30)
3131
32add_executable(32add_executable(
33 ${RESTORE_HELPER}
34 fake-restore-helper.cpp
35)
36target_link_libraries(
37 ${RESTORE_HELPER}
38 ${LINK_LIBS}
39)
40
41add_executable(
33 ${BACKUP_HELPER_FAILURE}42 ${BACKUP_HELPER_FAILURE}
34 fake-backup-helper.cpp43 fake-backup-helper.cpp
35)44)
3645
=== added file 'tests/fakes/fake-restore-helper.cpp'
--- tests/fakes/fake-restore-helper.cpp 1970-01-01 00:00:00 +0000
+++ tests/fakes/fake-restore-helper.cpp 2016-11-24 14:48:54 +0000
@@ -0,0 +1,130 @@
1/*
2 * Copyright (C) 2016 Canonical, Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3, as published
6 * by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranties of
10 * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11 * PURPOSE. See the GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License along
14 * with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authors:
17 * Xavi Garcia <xavi.garcia.mena@canonical.com>
18 */
19
20#include "fake-restore-helper.h"
21
22#include <qdbus-stubs/dbus-types.h>
23#include <qdbus-stubs/keeper_helper_interface.h>
24
25#include <QCoreApplication>
26#include <QDebug>
27#include <QDBusConnection>
28#include <QDBusInterface>
29#include <QDBusReply>
30#include <QProcessEnvironment>
31#include <QLocalSocket>
32#include <QtGlobal>
33
34#include <unistd.h>
35#include <sys/ioctl.h>
36
37constexpr int UPLOAD_BUFFER_MAX_ = 64 * 1024;
38
39void myMessageHandler(QtMsgType type, const QMessageLogContext &, const QString & msg)
40{
41 QString txt;
42 switch (type) {
43 case QtDebugMsg:
44 txt = QString("Debug: %1").arg(msg);
45 break;
46 case QtWarningMsg:
47 txt = QString("Warning: %1").arg(msg);
48 break;
49 case QtCriticalMsg:
50 txt = QString("Critical: %1").arg(msg);
51 break;
52 case QtFatalMsg:
53 txt = QString("Fatal: %1").arg(msg);
54 abort();
55 }
56 QFile outFile(TEST_RESTORE_LOG_FILE_PATH);
57 outFile.open(QIODevice::WriteOnly);
58 QTextStream ts(&outFile);
59 ts << txt << endl;
60}
61
62int
63main(int argc, char **argv)
64{
65 QCoreApplication app(argc, argv);
66 qInstallMessageHandler(myMessageHandler);
67
68 // dump the inputs to stdout
69 qDebug() << "argc:" << argc;
70 for(int i=0; i<argc; ++i)
71 qDebug() << "argv[" << i << "] is" << argv[i];
72 const auto env = QProcessEnvironment::systemEnvironment();
73 for(const auto& key : env.keys())
74 qDebug() << "env" << qPrintable(key) << "is" << qPrintable(env.value(key));
75
76 qDebug() << "Retrieving connection";
77
78 // ask the service for a socket
79 auto conn = QDBusConnection::connectToBus(QDBusConnection::SessionBus, DBusTypes::KEEPER_SERVICE);
80 const auto object_path = QString::fromUtf8(DBusTypes::KEEPER_HELPER_PATH);
81 QSharedPointer<DBusInterfaceKeeperHelper> helper_iface (new DBusInterfaceKeeperHelper(DBusTypes::KEEPER_SERVICE, object_path, conn));
82
83 qDebug() << "Is valid:" << helper_iface->isValid();
84
85 auto fd_reply = helper_iface->StartRestore();
86 fd_reply.waitForFinished();
87 if (fd_reply.isError())
88 {
89 qFatal("Call to '%s.StartRestore() at '%s' call failed: %s",
90 DBusTypes::KEEPER_SERVICE,
91 qPrintable(object_path),
92 qPrintable(fd_reply.error().message())
93 );
94 }
95 const auto ufd = fd_reply.value();
96 const auto fd = ufd.fileDescriptor();
97 qDebug() << "The file descriptor obtained is: " << fd;
98
99 char buffer[UPLOAD_BUFFER_MAX_];
100 int n_bytes_read = 0;
101 QFile file(TEST_RESTORE_FILE_PATH);
102 file.open(QIODevice::WriteOnly);
103 for(;;)
104 {
105 // Read data into buffer. We may not have enough to fill up buffer, so we
106 // store how many bytes were actually read in bytes_read.
107 int bytes_read = read(fd, buffer, sizeof(buffer));
108 if (bytes_read == 0) // We're done reading from the file
109 {
110 qDebug() << "Returned 0 bytes read";
111 QCoreApplication::exit(0);
112 break;
113 }
114
115 else if (bytes_read < 0)
116 {
117 if (errno != EAGAIN)
118 qWarning() << "Error reading from the socket: " << strerror(errno);
119 }
120 else
121 {
122 n_bytes_read += bytes_read;
123 qDebug() << "Read: " << bytes_read << " Total: " << n_bytes_read;
124 file.write(buffer, bytes_read);
125 file.flush();
126 }
127 }
128 file.close();
129 return 0;
130}
0131
=== added file 'tests/fakes/fake-restore-helper.h'
--- tests/fakes/fake-restore-helper.h 1970-01-01 00:00:00 +0000
+++ tests/fakes/fake-restore-helper.h 2016-11-24 14:48:54 +0000
@@ -0,0 +1,27 @@
1/*
2 * Copyright (C) 2016 Canonical, Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3, as published
6 * by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranties of
10 * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11 * PURPOSE. See the GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License along
14 * with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authors:
17 * Charles Kerr <charles.kerr@canonical.com>
18 */
19
20#pragma once
21
22namespace
23{
24static char const TEST_RESTORE_FILE_PATH[] = "/tmp/test-restore-helper";
25static char const TEST_RESTORE_LOG_FILE_PATH[] = "/tmp/restore-helper-output";
26
27}
028
=== modified file 'tests/integration/helpers/CMakeLists.txt'
--- tests/integration/helpers/CMakeLists.txt 2016-10-28 15:11:21 +0000
+++ tests/integration/helpers/CMakeLists.txt 2016-11-24 14:48:54 +0000
@@ -65,6 +65,10 @@
65 FOLDER_BACKUP_EXEC65 FOLDER_BACKUP_EXEC
66 ${KEEPER_HELPER_TEST_LOCATION}66 ${KEEPER_HELPER_TEST_LOCATION}
67)67)
68set(
69 FOLDER_RESTORE_EXEC
70 ${RESTORE_HELPER_TEST_LOCATION}
71)
68configure_file(72configure_file(
69 ${CMAKE_SOURCE_DIR}/data/${HELPER_REGISTRY_FILENAME}.in73 ${CMAKE_SOURCE_DIR}/data/${HELPER_REGISTRY_FILENAME}.in
70 ${HELPERS_TEST}-registry.json74 ${HELPERS_TEST}-registry.json
7175
=== modified file 'tests/integration/helpers/helpers-test.cc'
--- tests/integration/helpers/helpers-test.cc 2016-11-02 14:01:23 +0000
+++ tests/integration/helpers/helpers-test.cc 2016-11-24 14:48:54 +0000
@@ -20,6 +20,7 @@
20 */20 */
2121
22#include "test-helpers-base.h"22#include "test-helpers-base.h"
23#include "tests/fakes/fake-restore-helper.h"
2324
24class TestHelpers: public TestHelpersBase25class TestHelpers: public TestHelpersBase
25{26{
@@ -140,6 +141,39 @@
140141
141 // finally check that we have a valid manifest file.142 // finally check that we have a valid manifest file.
142 EXPECT_TRUE(check_manifest_file(backup_items));143 EXPECT_TRUE(check_manifest_file(backup_items));
144
145 QDBusPendingReply<QVariantDictMap> restore_choices_reply = user_iface->call("GetRestoreChoices");
146 restore_choices_reply.waitForFinished();
147 EXPECT_TRUE(restore_choices_reply.isValid()) << qPrintable(choices.error().message());
148
149 const auto restore_choices = restore_choices_reply.value();
150 EXPECT_EQ(2, restore_choices.size());
151
152 // check that we have the first uuid that we did the backup
153 const auto iter_restore = restore_choices.find(user_folder_uuid);
154 EXPECT_NE(iter_restore, restore_choices.end());
155 EXPECT_EQ(user_folder_uuid, iter_restore.key());
156
157 // ask to restore that uuid
158 QDBusPendingReply<void> restore_reply = user_iface->call("StartRestore", QStringList{iter_restore.key()});
159 restore_reply.waitForFinished();
160 ASSERT_TRUE(restore_reply.isValid()) << qPrintable(dbus_test_runner.sessionConnection().lastError().message());
161
162 // waits until all tasks are complete, recording PropertiesChanged signals
163 // and checks all the recorded values
164 EXPECT_TRUE(capture_and_check_state_until_all_tasks_complete(spy, {iter_restore.key()}, "complete"));
165
166 // verify that the file that the fake restore helper creates is one of the ones in storage framework
167 QString storage_framework_file_path;
168 EXPECT_TRUE(StorageFrameworkLocalUtils::get_storage_frameowork_file_equal_to(TEST_RESTORE_FILE_PATH, storage_framework_file_path));
169
170 // Finally check that the storage framework file that matched is the right one
171 // Keeper uses the display name plus .keeper extension for the files it creates
172 QFileInfo sf_file_info(storage_framework_file_path);
173 EXPECT_EQ(sf_file_info.fileName(), QStringLiteral("%1.keeper").arg(get_display_name_for_xdg_folder_path(user_dir, choices.value())));
174
175 auto show_helper_ouput_cmd = QStringLiteral("cat %1").arg(TEST_RESTORE_LOG_FILE_PATH);
176 system(show_helper_ouput_cmd.toStdString().c_str());
143}177}
144178
145TEST_F(TestHelpers, StartFullTestCancelling)179TEST_F(TestHelpers, StartFullTestCancelling)
146180
=== modified file 'tests/integration/helpers/test-helpers-base.cpp'
--- tests/integration/helpers/test-helpers-base.cpp 2016-11-03 08:58:51 +0000
+++ tests/integration/helpers/test-helpers-base.cpp 2016-11-24 14:48:54 +0000
@@ -179,15 +179,19 @@
179 {179 {
180 return previous == "saving" || previous == "queued";180 return previous == "saving" || previous == "queued";
181 }181 }
182 else if (current == "restoring")
183 {
184 return previous == "restoring" || previous == "queued";
185 }
182 else if (current == "finishing")186 else if (current == "finishing")
183 {187 {
184 return previous == "finishing" || previous == "saving";188 return previous == "finishing" || previous == "saving" || previous == "restoring";
185 }189 }
186 else if (current == "complete")190 else if (current == "complete")
187 {191 {
188 // we may pass from "saving" to "complete" if we don't have enough time192 // we may pass from "saving" to "complete" if we don't have enough time
189 // to emit the "finishing" state change193 // to emit the "finishing" state change
190 return previous == "complete" || previous == "finishing" || previous == "saving";194 return previous == "complete" || previous == "finishing" || previous == "saving" || previous == "restoring";
191 }195 }
192 else if (current == "failed")196 else if (current == "failed")
193 {197 {
@@ -704,7 +708,7 @@
704 return false;708 return false;
705 }709 }
706710
707 QSharedPointer<StorageFrameworkClient> sf_client(new StorageFrameworkClient);711 QSharedPointer<StorageFrameworkClient> sf_client(new StorageFrameworkClient, [](StorageFrameworkClient* sf){sf->deleteLater();});
708 Manifest manifest_read(sf_client, dir_name);712 Manifest manifest_read(sf_client, dir_name);
709 QSignalSpy spy_read(&manifest_read, &Manifest::finished);713 QSignalSpy spy_read(&manifest_read, &Manifest::finished);
710714
@@ -716,6 +720,7 @@
716 if (!spy_read.count())720 if (!spy_read.count())
717 {721 {
718 qWarning() << "Failed reading manifest file";722 qWarning() << "Failed reading manifest file";
723 sf_client.reset();
719 return false;724 return false;
720 }725 }
721726
722727
=== modified file 'tests/unit/manifest/manifest-test.cpp'
--- tests/unit/manifest/manifest-test.cpp 2016-10-06 14:53:44 +0000
+++ tests/unit/manifest/manifest-test.cpp 2016-11-24 14:48:54 +0000
@@ -78,7 +78,7 @@
7878
79 g_setenv("XDG_DATA_HOME", tmp_dir.path().toLatin1().data(), true);79 g_setenv("XDG_DATA_HOME", tmp_dir.path().toLatin1().data(), true);
8080
81 QSharedPointer<StorageFrameworkClient> sf_client(new StorageFrameworkClient);81 QSharedPointer<StorageFrameworkClient> sf_client(new StorageFrameworkClient, [](StorageFrameworkClient* sf){sf->deleteLater();});
82 Manifest manifest(sf_client, test_dir);82 Manifest manifest(sf_client, test_dir);
8383
84 auto objects_to_test = 10;84 auto objects_to_test = 10;
8585
=== modified file 'tests/unit/storage-framework/create-uploader-test.cpp'
--- tests/unit/storage-framework/create-uploader-test.cpp 2016-11-02 11:31:09 +0000
+++ tests/unit/storage-framework/create-uploader-test.cpp 2016-11-24 14:48:54 +0000
@@ -86,7 +86,7 @@
86 // create a downloader86 // create a downloader
87 auto downloader_fut = sf_client.get_new_downloader(test_dir, test_file_name);87 auto downloader_fut = sf_client.get_new_downloader(test_dir, test_file_name);
88 {88 {
89 QFutureWatcher<sf::Downloader::SPtr> w;89 QFutureWatcher<std::shared_ptr<Downloader>> w;
90 QSignalSpy spy(&w, &decltype(w)::finished);90 QSignalSpy spy(&w, &decltype(w)::finished);
91 w.setFuture(downloader_fut);91 w.setFuture(downloader_fut);
92 assert(spy.wait());92 assert(spy.wait());
@@ -107,14 +107,13 @@
107107
108 EXPECT_EQ(downloader_content, test_content);108 EXPECT_EQ(downloader_content, test_content);
109109
110 auto finish_downloader_fut = downloader->finish_download();110 QSignalSpy spy_downloader(downloader.get(), &Downloader::download_finished);
111 downloader->finish();
112 if (!spy_downloader.count())
111 {113 {
112 QFutureWatcher<void> w;114 spy_downloader.wait();
113 QSignalSpy spy(&w, &decltype(w)::finished);
114 w.setFuture(finish_downloader_fut);
115 assert(spy.wait());
116 ASSERT_EQ(spy.count(), 1);
117 }115 }
116 EXPECT_EQ(1, spy_downloader.count());
118117
119 // get another uploader118 // get another uploader
120 QString test_file_name_2 = QStringLiteral("test_file2");119 QString test_file_name_2 = QStringLiteral("test_file2");
121120
=== modified file 'tests/utils/file-utils.cpp'
--- tests/utils/file-utils.cpp 2016-11-02 10:43:30 +0000
+++ tests/utils/file-utils.cpp 2016-11-24 14:48:54 +0000
@@ -199,7 +199,7 @@
199 return false;199 return false;
200 }200 }
201 auto checksum1 = calculate_checksum(filePath1, QCryptographicHash::Md5);201 auto checksum1 = calculate_checksum(filePath1, QCryptographicHash::Md5);
202 auto checksum2 = calculate_checksum(filePath1, QCryptographicHash::Md5);202 auto checksum2 = calculate_checksum(filePath2, QCryptographicHash::Md5);
203 if (checksum1 != checksum2)203 if (checksum1 != checksum2)
204 {204 {
205 qWarning() << "Checksum for file:" << filePath1 << "differ";205 qWarning() << "Checksum for file:" << filePath1 << "differ";
206206
=== modified file 'tests/utils/storage-framework-local.cpp'
--- tests/utils/storage-framework-local.cpp 2016-10-28 15:11:21 +0000
+++ tests/utils/storage-framework-local.cpp 2016-11-24 14:48:54 +0000
@@ -170,4 +170,38 @@
170 : "";170 : "";
171}171}
172172
173bool get_storage_frameowork_file_equal_to(QString const & file_path, QString & path)
174{
175 auto const backups = get_storage_framework_files();
176 for (auto const& backup : backups)
177 {
178 auto const backup_filename = backup.absoluteFilePath();
179 if (FileUtils::compareFiles(file_path, backup_filename))
180 {
181 path = backup_filename;
182 return true;
183 }
184 }
185 return false;
186}
187
188bool get_storage_frameowork_file_equal_in_size_to(QString const & file_path, QString & path)
189{
190 auto const backups = get_storage_framework_files();
191 for (auto const& backup : backups)
192 {
193 auto const backup_filename = backup.absoluteFilePath();
194 QFileInfo info1(backup_filename);
195 QFileInfo info2(file_path);
196 qDebug() << "File 1 size = " << info1.size();
197 qDebug() << "File 2 size = " << info2.size();
198 if (info1.size() == info2.size())
199 {
200 path = backup_filename;
201 return true;
202 }
203 }
204 return false;
205}
206
173} // namespace StorageFrameworkLocalUtils207} // namespace StorageFrameworkLocalUtils
174208
=== modified file 'tests/utils/storage-framework-local.h'
--- tests/utils/storage-framework-local.h 2016-10-28 15:11:21 +0000
+++ tests/utils/storage-framework-local.h 2016-11-24 14:48:54 +0000
@@ -41,4 +41,8 @@
41 QFileInfoList get_storage_framework_files();41 QFileInfoList get_storage_framework_files();
4242
43 QString get_storage_framework_dir_name();43 QString get_storage_framework_dir_name();
44
45 bool get_storage_frameowork_file_equal_to(QString const & file_path, QString & path);
46
47 bool get_storage_frameowork_file_equal_in_size_to(QString const & file_path, QString & path);
44}48}

Subscribers

People subscribed via source and target branches

to all changes: