Ubuntu Notes app

Merge lp:~mzanetti/reminders-app/two-job-queues into lp:reminders-app

  1. two-job-queues
  2. Merge into trunk
Proposed by Michael Zanetti
Status: Merged
Approved by: Riccardo Padovani
Approved revision: 388
Merged at revision: 380
Proposed branch: lp:~mzanetti/reminders-app/two-job-queues
Merge into: lp:reminders-app
Prerequisite: lp:~mzanetti/reminders-app/cleanup-debug
Diff against target: 887 lines (+298/-127)
15 files modified
src/app/qml/reminders.qml (+2/-0)
src/app/qml/ui/NotesPage.qml (+4/-0)
src/libqtevernote/evernoteconnection.cpp (+107/-30)
src/libqtevernote/evernoteconnection.h (+19/-2)
src/libqtevernote/jobs/evernotejob.cpp (+10/-3)
src/libqtevernote/jobs/evernotejob.h (+8/-3)
src/libqtevernote/jobs/fetchnotesjob.cpp (+3/-1)
src/libqtevernote/note.cpp (+38/-44)
src/libqtevernote/note.h (+7/-5)
src/libqtevernote/notesstore.cpp (+29/-21)
src/libqtevernote/resource.cpp (+20/-9)
src/libqtevernote/resource.h (+3/-1)
src/libqtevernote/resourceimageprovider.cpp (+44/-8)
src/libqtevernote/resourceimageprovider.h (+3/-0)
src/libqtevernote/utils/enmldocument.cpp (+1/-0)
To merge this branch: bzr merge lp:~mzanetti/reminders-app/two-job-queues
Reviewer Review Type Date Requested Status
Ubuntu Phone Apps Jenkins Bot continuous-integration Approve
Riccardo Padovani Approve
Review via email: mp+252175@code.launchpad.net

This proposal supersedes a proposal from 2015-03-04.

Commit message

Further improve the jobqueue by splitting it up into high, medium and low priority queues and optimizing the reply rate.

To post a comment you must log in.
Revision history for this message
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote : Posted in a previous version of this proposal

PASSED: Continuous integration, rev:372
http://91.189.93.70:8080/job/reminders-app-ci/694/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2238
        deb: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2238/artifact/work/output/*zip*/output.zip
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-utopic-amd64-ci/296
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-vivid-amd64-ci/115

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/reminders-app-ci/694/rebuild

review: Approve (continuous-integration)
Revision history for this message
Riccardo Padovani (rpadovani) wrote : Posted in a previous version of this proposal

Looks good to me, but I left a couple of comments, just to be sure changes you made are wanted :-)

review: Needs Information
Revision history for this message
Michael Zanetti (mzanetti) wrote : Posted in a previous version of this proposal

Thanks for those comments. While I think this branch is ok, I found an issue with the attachToDuplicate() mechanism. I will fix that in another branch, as that issue is not introduced by this branch.

Revision history for this message
Michael Zanetti (mzanetti) wrote :

Ok. I've fixed the attachToDuplicate issue too and rebased it on cleanup-debug as they were conflicting. Please give this a good portion of testing.

Revision history for this message
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :

PASSED: Continuous integration, rev:374
http://91.189.93.70:8080/job/reminders-app-ci/704/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2285
        deb: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2285/artifact/work/output/*zip*/output.zip
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-utopic-amd64-ci/306
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-vivid-amd64-ci/125

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/reminders-app-ci/704/rebuild

review: Approve (continuous-integration)
Revision history for this message
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :

PASSED: Continuous integration, rev:375
http://91.189.93.70:8080/job/reminders-app-ci/705/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2288
        deb: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2288/artifact/work/output/*zip*/output.zip
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-utopic-amd64-ci/307
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-vivid-amd64-ci/126

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/reminders-app-ci/705/rebuild

review: Approve (continuous-integration)
Revision history for this message
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :

PASSED: Continuous integration, rev:380
http://91.189.93.70:8080/job/reminders-app-ci/706/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2289
        deb: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2289/artifact/work/output/*zip*/output.zip
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-utopic-amd64-ci/308
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-vivid-amd64-ci/127

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/reminders-app-ci/706/rebuild

review: Approve (continuous-integration)
Revision history for this message
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :

PASSED: Continuous integration, rev:381
http://91.189.93.70:8080/job/reminders-app-ci/707/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2290
        deb: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2290/artifact/work/output/*zip*/output.zip
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-utopic-amd64-ci/309
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-vivid-amd64-ci/128

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/reminders-app-ci/707/rebuild

review: Approve (continuous-integration)
Revision history for this message
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :

PASSED: Continuous integration, rev:382
http://91.189.93.70:8080/job/reminders-app-ci/708/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2293
        deb: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2293/artifact/work/output/*zip*/output.zip
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-utopic-amd64-ci/310
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-vivid-amd64-ci/129

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/reminders-app-ci/708/rebuild

review: Approve (continuous-integration)
Revision history for this message
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :

PASSED: Continuous integration, rev:386
http://91.189.93.70:8080/job/reminders-app-ci/709/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2309
        deb: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2309/artifact/work/output/*zip*/output.zip
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-utopic-amd64-ci/311
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-vivid-amd64-ci/130

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/reminders-app-ci/709/rebuild

review: Approve (continuous-integration)
Revision history for this message
Riccardo Padovani (rpadovani) wrote :

Tested, looks good to me, just 3 small inline comments

review: Needs Fixing
lp:~mzanetti/reminders-app/two-job-queues updated
387. By Michael Zanetti

fix issues from reviews

Revision history for this message
Michael Zanetti (mzanetti) wrote :

> Tested, looks good to me, just 3 small inline comments

Very good catches. All fixed.

lp:~mzanetti/reminders-app/two-job-queues updated
388. By Michael Zanetti

use qCDebug

Revision history for this message
Riccardo Padovani (rpadovani) wrote :

Now looks perfect :-)

review: Approve
Revision history for this message
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :

PASSED: Continuous integration, rev:387
http://91.189.93.70:8080/job/reminders-app-ci/712/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2315
        deb: http://91.189.93.70:8080/job/generic-mediumtests-utopic/2315/artifact/work/output/*zip*/output.zip
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-utopic-amd64-ci/314
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-vivid-amd64-ci/133

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/reminders-app-ci/712/rebuild

review: Approve (continuous-integration)
Revision history for this message
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/app/qml/reminders.qml'
2--- src/app/qml/reminders.qml 2015-03-06 12:16:37 +0000
3+++ src/app/qml/reminders.qml 2015-03-09 10:03:17 +0000
4@@ -111,6 +111,7 @@
5
6 function displayNote(note) {
7 print("displayNote:", note.guid)
8+ note.load(true);
9 if (root.narrowMode) {
10 print("creating noteview");
11 var component = Qt.createComponent(Qt.resolvedUrl("ui/NotePage.qml"));
12@@ -127,6 +128,7 @@
13 }
14
15 function switchToEditMode(note) {
16+ note.load(true)
17 if (root.narrowMode) {
18 if (pagestack.depth > 1) {
19 pagestack.pop();
20
21=== modified file 'src/app/qml/ui/NotesPage.qml'
22--- src/app/qml/ui/NotesPage.qml 2015-03-01 22:32:41 +0000
23+++ src/app/qml/ui/NotesPage.qml 2015-03-09 10:03:17 +0000
24@@ -177,6 +177,10 @@
25 syncError: model.syncError
26 conflicting: model.conflicting
27
28+ Component.onCompleted: {
29+ notes.note(model.guid).load(false);
30+ }
31+
32 onItemClicked: {
33 if (!model.conflicting) {
34 root.selectedNote = NotesStore.note(guid);
35
36=== modified file 'src/libqtevernote/evernoteconnection.cpp'
37--- src/libqtevernote/evernoteconnection.cpp 2015-03-06 00:47:45 +0000
38+++ src/libqtevernote/evernoteconnection.cpp 2015-03-09 10:03:17 +0000
39@@ -153,11 +153,23 @@
40 return;
41 }
42
43- foreach (EvernoteJob *job, m_jobQueue) {
44- job->emitJobDone(EvernoteConnection::ErrorCodeConnectionLost, "Disconnected from Evernote");
45- job->deleteLater();
46- }
47- m_jobQueue.clear();
48+ foreach (EvernoteJob *job, m_highPriorityJobQueue) {
49+ job->emitJobDone(EvernoteConnection::ErrorCodeConnectionLost, "Disconnected from Evernote");
50+ job->deleteLater();
51+ }
52+ m_highPriorityJobQueue.clear();
53+
54+ foreach (EvernoteJob *job, m_mediumPriorityJobQueue) {
55+ job->emitJobDone(EvernoteConnection::ErrorCodeConnectionLost, "Disconnected from Evernote");
56+ job->deleteLater();
57+ }
58+ m_mediumPriorityJobQueue.clear();
59+
60+ foreach (EvernoteJob *job, m_lowPriorityJobQueue) {
61+ job->emitJobDone(EvernoteConnection::ErrorCodeConnectionLost, "Disconnected from Evernote");
62+ job->deleteLater();
63+ }
64+ m_lowPriorityJobQueue.clear();
65
66 m_errorMessage.clear();
67 emit errorChanged();
68@@ -340,11 +352,38 @@
69 return false;
70 }
71
72-EvernoteJob* EvernoteConnection::findDuplicate(EvernoteJob *job)
73-{
74- foreach (EvernoteJob *queuedJob, m_jobQueue) {
75- // explicitly use custom operator==()
76- if (job->operator ==(queuedJob)) {
77+void EvernoteConnection::attachDuplicate(EvernoteJob *original, EvernoteJob *duplicate)
78+{
79+ if (duplicate->originatingObject() && duplicate->originatingObject() != original->originatingObject()) {
80+ duplicate->attachToDuplicate(m_currentJob);
81+ }
82+ connect(original, &EvernoteJob::jobFinished, duplicate, &EvernoteJob::deleteLater);
83+}
84+
85+EvernoteJob* EvernoteConnection::findExistingDuplicate(EvernoteJob *job)
86+{
87+ qCDebug(dcJobQueue) << "Length:"
88+ << m_highPriorityJobQueue.count() + m_mediumPriorityJobQueue.count() + m_lowPriorityJobQueue.count()
89+ << "(High:" << m_highPriorityJobQueue.count() << "Medium:" << m_mediumPriorityJobQueue.count() << "Low:" << m_lowPriorityJobQueue.count() << ")";
90+
91+ foreach (EvernoteJob *queuedJob, m_highPriorityJobQueue) {
92+ // explicitly use custom operator==()
93+ if (job->operator ==(queuedJob)) {
94+ qCDebug(dcJobQueue) << "Found duplicate in high priority queue";
95+ return queuedJob;
96+ }
97+ }
98+ foreach (EvernoteJob *queuedJob, m_mediumPriorityJobQueue) {
99+ // explicitly use custom operator==()
100+ if (job->operator ==(queuedJob)) {
101+ qCDebug(dcJobQueue) << "Found duplicate in medium priority queue";
102+ return queuedJob;
103+ }
104+ }
105+ foreach (EvernoteJob *queuedJob, m_lowPriorityJobQueue) {
106+ // explicitly use custom operator==()
107+ if (job->operator ==(queuedJob)) {
108+ qCDebug(dcJobQueue) << "Found duplicate in low priority queue";
109 return queuedJob;
110 }
111 }
112@@ -359,26 +398,56 @@
113 job->deleteLater();
114 return;
115 }
116- EvernoteJob *duplicate = findDuplicate(job);
117- if (duplicate) {
118- job->attachToDuplicate(duplicate);
119- connect(duplicate, &EvernoteJob::finished, job, &EvernoteJob::deleteLater);
120+ if (m_currentJob && m_currentJob->operator ==(job)) {
121+ qCDebug(dcJobQueue) << "Duplicate of new job request already running:" << job->toString();
122+ if (m_currentJob->isFinished()) {
123+ qCWarning(dcJobQueue) << "Job seems to be stuck in a loop. Deleting it:" << job->toString();
124+ job->deleteLater();
125+ } else {
126+ attachDuplicate(m_currentJob, job);
127+ }
128+ return;
129+ }
130+ EvernoteJob *existingJob = findExistingDuplicate(job);
131+ if (existingJob) {
132+ qCDebug(dcJobQueue) << "Duplicate job already queued:" << job->toString();
133+ attachDuplicate(existingJob, job);
134 // reprioritze the repeated request
135- qCDebug(dcJobQueue) << "Duplicate job already queued:" << job->toString();
136 if (job->jobPriority() == EvernoteJob::JobPriorityHigh) {
137- qCDebug(dcJobQueue) << "Reprioritising duplicate job:" << job->toString();
138- duplicate->setJobPriority(job->jobPriority());
139- m_jobQueue.prepend(m_jobQueue.takeAt(m_jobQueue.indexOf(duplicate)));
140+ qCDebug(dcJobQueue) << "Reprioritising duplicate job in high priority queue:" << job->toString();
141+ existingJob->setJobPriority(job->jobPriority());
142+ if (m_highPriorityJobQueue.contains(existingJob)) {
143+ m_highPriorityJobQueue.prepend(m_highPriorityJobQueue.takeAt(m_highPriorityJobQueue.indexOf(existingJob)));
144+ } else if (m_mediumPriorityJobQueue.contains(existingJob)){
145+ m_highPriorityJobQueue.prepend(m_mediumPriorityJobQueue.takeAt(m_mediumPriorityJobQueue.indexOf(existingJob)));
146+ } else {
147+ m_highPriorityJobQueue.prepend(m_lowPriorityJobQueue.takeAt(m_lowPriorityJobQueue.indexOf(existingJob)));
148+ }
149+ } else if (job->jobPriority() == EvernoteJob::JobPriorityMedium){
150+ if (m_mediumPriorityJobQueue.contains(existingJob)) {
151+ qCDebug(dcJobQueue) << "Reprioritising duplicate job in medium priority queue:" << job->toString();
152+ m_mediumPriorityJobQueue.prepend(m_mediumPriorityJobQueue.takeAt(m_mediumPriorityJobQueue.indexOf(existingJob)));
153+ } else if (m_lowPriorityJobQueue.contains(existingJob)) {
154+ m_mediumPriorityJobQueue.prepend(m_lowPriorityJobQueue.takeAt(m_lowPriorityJobQueue.indexOf(existingJob)));
155+ }
156+ } else if (job->jobPriority() == EvernoteJob::JobPriorityLow) {
157+ if (m_lowPriorityJobQueue.contains(existingJob)) {
158+ qCDebug(dcJobQueue) << "Reprioritising duplicate job in low priority queue:" << job->toString();
159+ m_lowPriorityJobQueue.prepend(m_lowPriorityJobQueue.takeAt(m_lowPriorityJobQueue.indexOf(existingJob)));
160+ }
161 }
162 } else {
163- connect(job, &EvernoteJob::finished, job, &EvernoteJob::deleteLater);
164- connect(job, &EvernoteJob::finished, this, &EvernoteConnection::startNextJob);
165+ connect(job, &EvernoteJob::jobFinished, job, &EvernoteJob::deleteLater);
166+ connect(job, &EvernoteJob::jobFinished, this, &EvernoteConnection::startNextJob);
167 if (job->jobPriority() == EvernoteJob::JobPriorityHigh) {
168- qCDebug(dcJobQueue) << "Prepending high priority job request:" << job->toString();
169- m_jobQueue.prepend(job);
170+ qCDebug(dcJobQueue) << "Adding high priority job request:" << job->toString();
171+ m_highPriorityJobQueue.prepend(job);
172+ } else if (job->jobPriority() == EvernoteJob::JobPriorityMedium){
173+ qCDebug(dcJobQueue) << "Adding medium priority job request:" << job->toString();
174+ m_mediumPriorityJobQueue.prepend(job);
175 } else {
176- qCDebug(dcJobQueue) << "Appending low priority job request:" << job->toString();
177- m_jobQueue.append(job);
178+ qCDebug(dcJobQueue) << "Adding low priority job request:" << job->toString();
179+ m_lowPriorityJobQueue.prepend(job);
180 }
181 startJobQueue();
182 }
183@@ -401,16 +470,24 @@
184
185 void EvernoteConnection::startJobQueue()
186 {
187- if (m_jobQueue.isEmpty()) {
188- return;
189- }
190-
191 if (m_currentJob) {
192 return;
193 }
194
195- m_currentJob = m_jobQueue.takeFirst();
196- qCDebug(dcJobQueue) << "Starting job:" << m_currentJob->toString();
197+ if (!m_highPriorityJobQueue.isEmpty()) {
198+ m_currentJob = m_highPriorityJobQueue.takeFirst();
199+ } else if (!m_mediumPriorityJobQueue.isEmpty()){
200+ m_currentJob = m_mediumPriorityJobQueue.takeFirst();
201+ } else if (!m_lowPriorityJobQueue.isEmpty()){
202+ m_currentJob = m_lowPriorityJobQueue.takeFirst();
203+ }
204+
205+ if (!m_currentJob) {
206+ qCDebug(dcJobQueue) << "Queue empty. Nothing to do.";
207+ return;
208+ }
209+
210+ qCDebug(dcJobQueue) << QString("Starting job (Priority: %1):").arg(m_currentJob->jobPriority()) << m_currentJob->toString();
211 m_currentJob->start();
212 }
213
214
215=== modified file 'src/libqtevernote/evernoteconnection.h'
216--- src/libqtevernote/evernoteconnection.h 2015-02-26 22:47:10 +0000
217+++ src/libqtevernote/evernoteconnection.h 2015-03-09 10:03:17 +0000
218@@ -73,6 +73,18 @@
219 QString token() const;
220 void setToken(const QString &token);
221
222+ // This will add the job to the job queue. The job queue will take ownership of the object
223+ // and manage it's lifetime.
224+ // * If there is an identical job already existing in the queue, the duplicate will be
225+ // attached to original job and not actually fetched a second time from the network in
226+ // order to reduce network traffic.
227+ // * If the new job has a higher priority than the existing one, the existing one will
228+ // reprioritized to the higher priorty.
229+ // * If the jobs have different originatingObjects, each job will emit the jobDone signal,
230+ // if instead the originatingObject is the same in both jobs, only one of them will emit
231+ // a jobDone signal. This is useful if you want to reschedule a job with higher priority
232+ // without having to track previously queued jobs and avoid invoking the connected slot
233+ // multiple times.
234 void enqueue(EvernoteJob *job);
235
236 bool isConnected() const;
237@@ -104,7 +116,10 @@
238 bool connectUserStore();
239 bool connectNotesStore();
240
241- EvernoteJob* findDuplicate(EvernoteJob *job);
242+ EvernoteJob* findExistingDuplicate(EvernoteJob *job);
243+
244+ // "duplicate" will be attached to "original"
245+ void attachDuplicate(EvernoteJob *original, EvernoteJob *duplicate);
246
247 bool m_useSSL;
248 bool m_isConnected;
249@@ -115,7 +130,9 @@
250
251 // There must be only one job running at a time
252 // Do not start jobs other than with startJobQueue()
253- QList<EvernoteJob*> m_jobQueue;
254+ QList<EvernoteJob*> m_highPriorityJobQueue;
255+ QList<EvernoteJob*> m_mediumPriorityJobQueue;
256+ QList<EvernoteJob*> m_lowPriorityJobQueue;
257 EvernoteJob *m_currentJob;
258
259 // Those need to be mutexed
260
261=== modified file 'src/libqtevernote/jobs/evernotejob.cpp'
262--- src/libqtevernote/jobs/evernotejob.cpp 2015-03-06 00:42:42 +0000
263+++ src/libqtevernote/jobs/evernotejob.cpp 2015-03-09 10:03:17 +0000
264@@ -38,11 +38,13 @@
265 using namespace apache::thrift::protocol;
266 using namespace apache::thrift::transport;
267
268-EvernoteJob::EvernoteJob(QObject *parent, JobPriority jobPriority) :
269- QThread(parent),
270+EvernoteJob::EvernoteJob(QObject *originatingObject, JobPriority jobPriority) :
271+ QThread(nullptr),
272 m_token(EvernoteConnection::instance()->token()),
273- m_jobPriority(jobPriority)
274+ m_jobPriority(jobPriority),
275+ m_originatingObject(originatingObject)
276 {
277+ connect(this, &QThread::finished, this, &EvernoteJob::jobFinished);
278 }
279
280 EvernoteJob::~EvernoteJob()
281@@ -198,6 +200,11 @@
282 return metaObject()->className();
283 }
284
285+QObject *EvernoteJob::originatingObject() const
286+{
287+ return m_originatingObject;
288+}
289+
290 QString EvernoteJob::token()
291 {
292 return m_token;
293
294=== modified file 'src/libqtevernote/jobs/evernotejob.h'
295--- src/libqtevernote/jobs/evernotejob.h 2015-02-26 22:47:10 +0000
296+++ src/libqtevernote/jobs/evernotejob.h 2015-03-09 10:03:17 +0000
297@@ -37,8 +37,7 @@
298 * your job won't be executed but you should instead forward the other's job results.
299 *
300 * Jobs can be enqueue()d in NotesStore.
301- * They will destroy themselves when finished.
302- * The jobqueue will take care about starting them.
303+ * The jobqueue will take care about starting them and deleting them.
304 */
305 class EvernoteJob : public QThread
306 {
307@@ -46,10 +45,11 @@
308 public:
309 enum JobPriority {
310 JobPriorityHigh,
311+ JobPriorityMedium,
312 JobPriorityLow
313 };
314
315- explicit EvernoteJob(QObject *parent = 0, JobPriority jobPriority = JobPriorityHigh);
316+ explicit EvernoteJob(QObject *originatingObject = 0, JobPriority jobPriority = JobPriorityHigh);
317 virtual ~EvernoteJob();
318
319 JobPriority jobPriority() const;
320@@ -63,9 +63,13 @@
321
322 virtual QString toString() const;
323
324+ QObject* originatingObject() const;
325+
326 signals:
327 void connectionLost(const QString &errorMessage);
328
329+ void jobFinished();
330+
331 protected:
332 virtual void resetConnection() = 0;
333 virtual void startJob() = 0;
334@@ -76,6 +80,7 @@
335 private:
336 QString m_token;
337 JobPriority m_jobPriority;
338+ QObject *m_originatingObject;
339
340 friend class EvernoteConnection;
341 };
342
343=== modified file 'src/libqtevernote/jobs/fetchnotesjob.cpp'
344--- src/libqtevernote/jobs/fetchnotesjob.cpp 2015-03-06 00:47:45 +0000
345+++ src/libqtevernote/jobs/fetchnotesjob.cpp 2015-03-09 10:03:17 +0000
346@@ -41,7 +41,9 @@
347 return false;
348 }
349 return this->m_filterNotebookGuid == otherJob->m_filterNotebookGuid
350- && this->m_searchWords == otherJob->m_searchWords;
351+ && this->m_searchWords == otherJob->m_searchWords
352+ && this->m_startIndex == otherJob->m_startIndex
353+ && this->m_chunkSize == otherJob->m_chunkSize;
354 }
355
356 void FetchNotesJob::attachToDuplicate(const EvernoteJob *other)
357
358=== modified file 'src/libqtevernote/note.cpp'
359--- src/libqtevernote/note.cpp 2015-03-08 16:01:23 +0000
360+++ src/libqtevernote/note.cpp 2015-03-09 10:03:17 +0000
361@@ -38,7 +38,6 @@
362 m_isSearchResult(false),
363 m_updateSequenceNumber(updateSequenceNumber),
364 m_loading(false),
365- m_loadingHighPriority(false),
366 m_loaded(false),
367 m_needsContentSync(false),
368 m_syncError(false),
369@@ -64,15 +63,9 @@
370
371 infoFile.beginGroup("resources");
372 foreach (const QString &hash, infoFile.childGroups()) {
373- if (Resource::isCached(hash)) {
374- infoFile.beginGroup(hash);
375- // Assuming the resource is already cached...
376- addResource(QByteArray(), hash, infoFile.value("fileName").toString(), infoFile.value("type").toString());
377- infoFile.endGroup();
378- } else {
379- // uh oh... have a resource description without file... reset sequence number to indicate we need a sync
380- qCWarning(dcNotesStore) << "Have a resource description but no resource file for it";
381- }
382+ infoFile.beginGroup(hash);
383+ addResource(hash, infoFile.value("fileName").toString(), infoFile.value("type").toString());
384+ infoFile.endGroup();
385 }
386 infoFile.endGroup();
387
388@@ -243,7 +236,6 @@
389
390 QString Note::enmlContent() const
391 {
392- load();
393 return m_content.enml();
394 }
395
396@@ -263,13 +255,11 @@
397
398 QString Note::htmlContent() const
399 {
400- load();
401 return m_content.toHtml(m_guid);
402 }
403
404 QString Note::richTextContent() const
405 {
406- load();
407 return m_content.toRichText(m_guid);
408 }
409
410@@ -286,15 +276,11 @@
411
412 QString Note::plaintextContent() const
413 {
414- load();
415 return m_content.toPlaintext().trimmed();
416 }
417
418 QString Note::tagline() const
419 {
420- if (m_tagline.isEmpty()) {
421- load(false);
422- }
423 return m_tagline;
424 }
425
426@@ -488,11 +474,12 @@
427 QStringList Note::resourceUrls() const
428 {
429 QList<QString> ret;
430- foreach (const QString &hash, m_resources.keys()) {
431- QUrl url("image://resource/" + m_resources.value(hash)->type());
432+ foreach (Resource *resource, m_resources) {
433+ QUrl url("image://resource/" + resource->type());
434 QUrlQuery arguments;
435 arguments.addQueryItem("noteGuid", m_guid);
436- arguments.addQueryItem("hash", hash);
437+ arguments.addQueryItem("hash", resource->hash());
438+ arguments.addQueryItem("loaded", resource->isCached() ? "true" : "false");
439 url.setQuery(arguments);
440 ret << url.toString();
441 }
442@@ -505,25 +492,29 @@
443 }
444
445
446-Resource* Note::addResource(const QByteArray &data, const QString &hash, const QString &fileName, const QString &type)
447+Resource* Note::addResource(const QString &hash, const QString &fileName, const QString &type, const QByteArray &data)
448 {
449+ Resource *resource;
450 if (m_resources.contains(hash)) {
451- return m_resources.value(hash);
452+ resource = m_resources.value(hash);
453+ if (!data.isEmpty()) {
454+ resource->setData(data);
455+ }
456+ } else {
457+ resource = new Resource(data, hash, fileName, type, this);
458+ m_resources.insert(hash, resource);
459+ QSettings infoFile(m_infoFile, QSettings::IniFormat);
460+ infoFile.beginGroup("resources");
461+ infoFile.beginGroup(hash);
462+ infoFile.setValue("fileName", fileName);
463+ infoFile.setValue("type", type);
464+ infoFile.endGroup();
465+ infoFile.endGroup();
466 }
467
468- Resource *resource = new Resource(data, hash, fileName, type, this);
469- m_resources.insert(hash, resource);
470 emit resourcesChanged();
471 emit contentChanged();
472
473- QSettings infoFile(m_infoFile, QSettings::IniFormat);
474- infoFile.beginGroup("resources");
475- infoFile.beginGroup(hash);
476- infoFile.setValue("fileName", fileName);
477- infoFile.setValue("type", type);
478- infoFile.endGroup();
479- infoFile.endGroup();
480-
481 return resource;
482 }
483
484@@ -593,7 +584,7 @@
485 note->setUpdateSequenceNumber(m_updateSequenceNumber);
486 note->setDeleted(m_deleted);
487 foreach (Resource *resource, m_resources) {
488- note->addResource(resource->data(), resource->hash(), resource->fileName(), resource->type());
489+ note->addResource(resource->hash(), resource->fileName(), resource->type(), resource->data());
490 }
491 note->m_needsContentSync = m_needsContentSync;
492
493@@ -620,17 +611,11 @@
494 NotesStore::instance()->deleteNote(m_guid);
495 }
496
497-void Note::setLoading(bool loading, bool highPriority)
498+void Note::setLoading(bool loading)
499 {
500 if (m_loading != loading) {
501 m_loading = loading;
502 emit loadingChanged();
503-
504- if (!m_loading) {
505- m_loadingHighPriority = false;
506- } else {
507- m_loadingHighPriority = highPriority;
508- }
509 }
510 }
511
512@@ -670,13 +655,22 @@
513 }
514 }
515
516-void Note::load(bool priorityHigh) const
517+void Note::load(bool priorityHigh)
518 {
519 if (!m_loaded && isCached()) {
520 loadFromCacheFile();
521- } else if (!m_loaded) {
522- if (!m_loading || (priorityHigh && !m_loadingHighPriority)) {
523- NotesStore::instance()->refreshNoteContent(m_guid, FetchNoteJob::LoadContent, priorityHigh ? EvernoteJob::JobPriorityHigh : EvernoteJob::JobPriorityLow);
524+ }
525+
526+ if (!m_loaded) {
527+ NotesStore::instance()->refreshNoteContent(m_guid, FetchNoteJob::LoadContent, priorityHigh ? EvernoteJob::JobPriorityHigh : EvernoteJob::JobPriorityMedium);
528+ return;
529+ }
530+
531+ // Check if resources are loaded
532+ foreach (Resource *resource, m_resources) {
533+ if (!resource->isCached()) {
534+ NotesStore::instance()->refreshNoteContent(m_guid, FetchNoteJob::LoadResources, priorityHigh ? EvernoteJob::JobPriorityHigh : EvernoteJob::JobPriorityLow);
535+ break;
536 }
537 }
538 }
539
540=== modified file 'src/libqtevernote/note.h'
541--- src/libqtevernote/note.h 2015-03-08 16:01:23 +0000
542+++ src/libqtevernote/note.h 2015-03-09 10:03:17 +0000
543@@ -157,7 +157,7 @@
544 QStringList resourceUrls() const;
545 Q_INVOKABLE Resource* resource(const QString &hash);
546 QList<Resource*> resources() const;
547- Resource *addResource(const QByteArray &data, const QString &hash, const QString &fileName, const QString &type);
548+
549
550 Q_INVOKABLE void markTodo(const QString &todoId, bool checked);
551 Q_INVOKABLE void attachFile(int position, const QUrl &fileName);
552@@ -167,6 +167,8 @@
553 int renderWidth() const;
554 void setRenderWidth(int renderWidth);
555
556+ Q_INVOKABLE void load(bool highPriority = false);
557+
558 public slots:
559 void save();
560 void remove();
561@@ -201,7 +203,7 @@
562
563 private:
564 // Those should only be called from NotesStore, which is a friend
565- void setLoading(bool loading, bool highPriority = false);
566+ void setLoading(bool loading);
567 void setSyncError(bool syncError);
568 void setDeleted(bool deleted);
569 void syncToCacheFile();
570@@ -210,9 +212,10 @@
571 void setUpdateSequenceNumber(qint32 updateSequenceNumber);
572 void setLastSyncedSequenceNumber(qint32 lastSyncedSequenceNumber);
573 void setConflicting(bool conflicting);
574+ Resource *addResource(const QString &hash, const QString &fileName, const QString &type, const QByteArray &data = QByteArray());
575+ void addMissingResource();
576+ void setMissingResources(int missingResources);
577
578- // const because we want to load on demand in getters. Keep this private!
579- void load(bool priorityHigh = true) const;
580 void loadFromCacheFile() const;
581
582 private:
583@@ -236,7 +239,6 @@
584 QString m_infoFile;
585
586 bool m_loading;
587- bool m_loadingHighPriority;
588 mutable bool m_loaded;
589 bool m_synced;
590 bool m_needsContentSync;
591
592=== modified file 'src/libqtevernote/notesstore.cpp'
593--- src/libqtevernote/notesstore.cpp 2015-03-08 16:01:23 +0000
594+++ src/libqtevernote/notesstore.cpp 2015-03-09 10:03:17 +0000
595@@ -677,7 +677,7 @@
596 if (note->updateSequenceNumber() < result.updateSequenceNum) {
597 qCDebug(dcSync) << "refreshing note from network. suequence number changed: " << note->updateSequenceNumber() << "->" << result.updateSequenceNum;
598 changedRoles = updateFromEDAM(result, note);
599- refreshNoteContent(note->guid(), FetchNoteJob::LoadContent, EvernoteJob::JobPriorityLow);
600+ refreshNoteContent(note->guid(), FetchNoteJob::LoadContent, EvernoteJob::JobPriorityMedium);
601 syncToCacheFile(note);
602 }
603 } else {
604@@ -798,15 +798,17 @@
605 return;
606 }
607 if (EvernoteConnection::instance()->isConnected()) {
608- qCDebug(dcNotesStore) << "Fetching note content from network for note" << guid << (what == FetchNoteJob::LoadContent ? "content" : "image");
609+ qCDebug(dcNotesStore) << "Fetching note content from network for note" << guid << (what == FetchNoteJob::LoadContent ? "Content" : "Resource") << "Priority:" << priority;
610 FetchNoteJob *job = new FetchNoteJob(guid, what, this);
611 job->setJobPriority(priority);
612 connect(job, &FetchNoteJob::resultReady, this, &NotesStore::fetchNoteJobDone);
613 EvernoteConnection::instance()->enqueue(job);
614
615- note->setLoading(true, priority == EvernoteJob::JobPriorityHigh);
616- int idx = m_notes.indexOf(note);
617- emit dataChanged(index(idx), index(idx), QVector<int>() << RoleLoading);
618+ if (!note->loading()) {
619+ note->setLoading(true);
620+ int idx = m_notes.indexOf(note);
621+ emit dataChanged(index(idx), index(idx), QVector<int>() << RoleLoading);
622+ }
623 }
624 }
625
626@@ -818,16 +820,13 @@
627 qCWarning(dcSync) << "can't find note for this update... ignoring...";
628 return;
629 }
630+
631 QModelIndex noteIndex = index(m_notes.indexOf(note));
632 QVector<int> roles;
633
634- note->setLoading(false);
635- roles << RoleLoading;
636-
637 switch (errorCode) {
638 case EvernoteConnection::ErrorCodeNoError:
639 // All is well
640- emit dataChanged(noteIndex, noteIndex, roles);
641 break;
642 case EvernoteConnection::ErrorCodeUserException:
643 qCWarning(dcSync) << "FetchNoteJobDone: EDAMUserException:" << errorMessage;
644@@ -877,15 +876,17 @@
645 QString mime = QString::fromStdString(resource.mime);
646
647 if (what == FetchNoteJob::LoadResources) {
648- qCDebug(dcSync) << "Resource fetched for note:" << note->guid() << "Filename:" << fileName << "Mimetype:" << mime << "Hash:" << hash;
649+ qCDebug(dcSync) << "Resource content fetched for note:" << note->guid() << "Filename:" << fileName << "Mimetype:" << mime << "Hash:" << hash;
650 QByteArray resourceData = QByteArray(resource.data.body.data(), resource.data.size);
651- note->addResource(resourceData, hash, fileName, mime);
652- } else if (Resource::isCached(hash)) {
653- qCDebug(dcSync) << "Resource already cached for note:" << note->guid() << "Filename:" << fileName << "Mimetype:" << mime << "Hash:" << hash;
654- note->addResource(QByteArray(), hash, fileName, mime);
655+ note->addResource(hash, fileName, mime, resourceData);
656 } else {
657- qCDebug(dcSync) << "Resource not yet fetched for note:" << note->guid() << "Filename:" << fileName << "Mimetype:" << mime << "Hash:" << hash;
658- refreshWithResourceData = true;
659+ qCDebug(dcSync) << "Adding resource info to note:" << note->guid() << "Filename:" << fileName << "Mimetype:" << mime << "Hash:" << hash;
660+ Resource *resource = note->addResource(hash, fileName, mime);
661+
662+ if (!resource->isCached()) {
663+ qCDebug(dcSync) << "Resource not yet fetched for note:" << note->guid() << "Filename:" << fileName << "Mimetype:" << mime << "Hash:" << hash;
664+ refreshWithResourceData = true;
665+ }
666 }
667 roles << RoleHtmlContent << RoleEnmlContent << RoleResourceUrls;
668 }
669@@ -915,17 +916,20 @@
670 note->setReminderDoneTime(reminderDoneTime);
671 roles << RoleReminderDone << RoleReminderDoneTime;
672 }
673+
674+ note->setLoading(false);
675+ roles << RoleLoading;
676+
677 emit noteChanged(note->guid(), note->notebookGuid());
678-
679 emit dataChanged(noteIndex, noteIndex, roles);
680
681 if (refreshWithResourceData) {
682 qCDebug(dcSync) << "Fetching Note resources:" << note->guid();
683- refreshNoteContent(note->guid(), FetchNoteJob::LoadResources, job->jobPriority());
684- } else {
685- syncToCacheFile(note); // Syncs into the list cache
686- note->syncToCacheFile(); // Syncs note's content into notes cache
687+ EvernoteJob::JobPriority newPriority = job->jobPriority() == EvernoteJob::JobPriorityMedium ? EvernoteJob::JobPriorityLow : job->jobPriority();
688+ refreshNoteContent(note->guid(), FetchNoteJob::LoadResources, newPriority);
689 }
690+ syncToCacheFile(note); // Syncs into the list cache
691+ note->syncToCacheFile(); // Syncs note's content into notes cache
692 }
693
694 void NotesStore::refreshNotebooks()
695@@ -1004,6 +1008,8 @@
696 }
697 }
698
699+ qCDebug(dcSync) << "Remote notebooks merged into storage. Merging local changes to server.";
700+
701 foreach (Notebook *notebook, unhandledNotebooks) {
702 if (notebook->lastSyncedSequenceNumber() == 0) {
703 qCDebug(dcSync) << "Have a local notebook that doesn't exist on Evernote. Creating on server:" << notebook->guid();
704@@ -1027,6 +1033,8 @@
705 notebook->deleteLater();
706 }
707 }
708+
709+ qCDebug(dcSync) << "Notebooks merged.";
710 }
711
712 void NotesStore::refreshTags()
713
714=== modified file 'src/libqtevernote/resource.cpp'
715--- src/libqtevernote/resource.cpp 2015-03-06 00:47:45 +0000
716+++ src/libqtevernote/resource.cpp 2015-03-09 10:03:17 +0000
717@@ -52,15 +52,16 @@
718 }
719 }
720
721-bool Resource::isCached(const QString &hash)
722-{
723- QDir cacheDir(NotesStore::instance()->storageLocation());
724- foreach (const QString fi, cacheDir.entryList()) {
725- if (fi.contains(hash)) {
726- return true;
727- }
728- }
729- return false;
730+Resource::Resource(const QString &hash, const QString &fileName, const QString &type, QObject *parent):
731+ Resource(QByteArray(), hash, fileName, type, parent)
732+{
733+
734+}
735+
736+bool Resource::isCached()
737+{
738+ QFileInfo fi(m_filePath);
739+ return fi.exists();
740 }
741
742 Resource::Resource(const QString &path, QObject *parent):
743@@ -159,3 +160,13 @@
744 }
745 return QByteArray();
746 }
747+
748+void Resource::setData(const QByteArray &data)
749+{
750+ QFile file(m_filePath);
751+ if (file.open(QFile::WriteOnly | QFile::Truncate)) {
752+ file.write(data);
753+ } else {
754+ qCDebug(dcNotesStore) << "Error saving data for resource:" << m_hash;
755+ }
756+}
757
758=== modified file 'src/libqtevernote/resource.h'
759--- src/libqtevernote/resource.h 2015-02-16 21:57:16 +0000
760+++ src/libqtevernote/resource.h 2015-03-09 10:03:17 +0000
761@@ -37,10 +37,12 @@
762 public:
763 Resource(const QString &path, QObject *parent = 0);
764 Resource(const QByteArray &data, const QString &hash, const QString &fileName, const QString &type, QObject *parent = 0);
765+ Resource(const QString &hash, const QString &fileName, const QString &type, QObject *parent = 0);
766
767- static bool isCached(const QString &hash);
768+ bool isCached();
769
770 QByteArray data() const;
771+ void setData(const QByteArray &data);
772 QString hash() const;
773 QString fileName() const;
774 QString type() const;
775
776=== modified file 'src/libqtevernote/resourceimageprovider.cpp'
777--- src/libqtevernote/resourceimageprovider.cpp 2015-03-06 00:47:45 +0000
778+++ src/libqtevernote/resourceimageprovider.cpp 2015-03-09 10:03:17 +0000
779@@ -5,11 +5,13 @@
780 #include <note.h>
781
782 #include <QUrlQuery>
783+#include <QFileInfo>
784+#include <QStandardPaths>
785+#include <QDir>
786
787 ResourceImageProvider::ResourceImageProvider():
788 QQuickImageProvider(QQuickImageProvider::Image)
789 {
790-
791 }
792
793 QImage ResourceImageProvider::requestImage(const QString &id, QSize *size, const QSize &requestedSize)
794@@ -18,6 +20,7 @@
795 QUrlQuery arguments(id.split('?').last());
796 QString noteGuid = arguments.queryItemValue("noteGuid");
797 QString resourceHash = arguments.queryItemValue("hash");
798+ bool isLoaded = arguments.queryItemValue("loaded") == "true";
799 Note *note = NotesStore::instance()->note(noteGuid);
800 if (!note) {
801 qCWarning(dcNotesStore) << "Unable to find note for resource:" << id;
802@@ -26,19 +29,52 @@
803
804 QImage image;
805 if (mediaType.startsWith("image")) {
806- QSize tmpSize = requestedSize;
807- if (!requestedSize.isValid() || requestedSize.width() > 1024 || requestedSize.height() > 1024) {
808- tmpSize = QSize(1024, 1024);
809+ if (isLoaded) {
810+ QSize tmpSize = requestedSize;
811+ if (!requestedSize.isValid() || requestedSize.width() > 1024 || requestedSize.height() > 1024) {
812+ tmpSize = QSize(1024, 1024);
813+ }
814+ image = QImage::fromData(NotesStore::instance()->note(noteGuid)->resource(resourceHash)->imageData(tmpSize));
815+ } else {
816+ image = loadIcon("image-x-generic-symbolic", requestedSize);
817 }
818- image = QImage::fromData(NotesStore::instance()->note(noteGuid)->resource(resourceHash)->imageData(tmpSize));
819 } else if (mediaType.startsWith("audio")) {
820- image.load("/usr/share/icons/suru/mimetypes/scalable/audio-x-generic-symbolic.svg");
821+ image = loadIcon("audio-x-generic-symbolic", requestedSize);
822 } else if (mediaType == "application/pdf") {
823- image.load("/usr/share/icons/suru/mimetypes/scalable/application-pdf-symbolic.svg");
824+ image = loadIcon("application-pdf-symbolic", requestedSize);
825 } else {
826- image.load("/usr/share/icons/suru/mimetypes/scalable/empty-symbolic.svg");
827+ image = loadIcon("empty-symbolic", requestedSize);
828 }
829
830 *size = image.size();
831 return image;
832 }
833+
834+QImage ResourceImageProvider::loadIcon(const QString &name, const QSize &size)
835+{
836+ QString cachePath = QStandardPaths::standardLocations(QStandardPaths::CacheLocation).first();
837+ QString path = QString(cachePath + "/%1_%2x%3.png").arg(name).arg(size.width()).arg(size.height());
838+ QFileInfo fi(path);
839+ if (fi.exists()) {
840+ QImage image;
841+ image.load(path);
842+ return image;
843+ }
844+
845+ QString svgPath = "/usr/share/icons/suru/mimetypes/scalable/" + name + ".svg";
846+ QImage image;
847+ image.load(svgPath);
848+ if (size.height() > 0 && size.width() > 0) {
849+ image = image.scaled(size, Qt::KeepAspectRatio, Qt::SmoothTransformation);
850+ } else if (size.height() > 0) {
851+ image = image.scaledToHeight(size.height(), Qt::SmoothTransformation);
852+ } else if (size.width() > 0) {
853+ image = image.scaledToWidth(size.width(), Qt::SmoothTransformation);
854+ }
855+ QDir dir(cachePath);
856+ if (!dir.exists()) {
857+ dir.mkpath(cachePath);
858+ }
859+ image.save(path);
860+ return image;
861+}
862
863=== modified file 'src/libqtevernote/resourceimageprovider.h'
864--- src/libqtevernote/resourceimageprovider.h 2014-10-23 21:27:46 +0000
865+++ src/libqtevernote/resourceimageprovider.h 2015-03-09 10:03:17 +0000
866@@ -9,6 +9,9 @@
867 explicit ResourceImageProvider();
868
869 QImage requestImage(const QString &id, QSize *size, const QSize &requestedSize);
870+
871+ void scale(QImage &image, const QSize &size);
872+ QImage loadIcon(const QString &name, const QSize &size);
873 };
874
875 #endif // RESOURCEIMAGEPROVIDER_H
876
877=== modified file 'src/libqtevernote/utils/enmldocument.cpp'
878--- src/libqtevernote/utils/enmldocument.cpp 2015-03-08 16:01:23 +0000
879+++ src/libqtevernote/utils/enmldocument.cpp 2015-03-09 10:03:17 +0000
880@@ -296,6 +296,7 @@
881 QUrlQuery arguments;
882 arguments.addQueryItem("noteGuid", noteGuid);
883 arguments.addQueryItem("hash", hash);
884+ arguments.addQueryItem("loaded", NotesStore::instance()->note(noteGuid)->resource(hash)->isCached() ? "true" : "false");
885 url.setQuery(arguments);
886 return url.toString();
887 }

Subscribers

People subscribed via source and target branches