Merge lp:~renatofilho/account-polld/query-calendars into lp:~ubuntu-push-hackers/account-polld/trunk
- query-calendars
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Jonas G. Drange |
Approved revision: | 181 |
Merged at revision: | 170 |
Proposed branch: | lp:~renatofilho/account-polld/query-calendars |
Merge into: | lp:~ubuntu-push-hackers/account-polld/trunk |
Prerequisite: | lp:~artmello/account-polld/account-polld-notifications |
Diff against target: |
214 lines (+107/-41) 2 files modified
plugins/gcalendar/gcalendar.go (+81/-37) plugins/gcalendar/syncmonitor.go (+26/-4) |
To merge this branch: | bzr merge lp:~renatofilho/account-polld/query-calendars |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
system-apps-ci-bot | continuous-integration | Needs Fixing | |
PS Jenkins bot | continuous-integration | Pending | |
Jonas G. Drange | Pending | ||
Review via email: mp+300468@code.launchpad.net |
This proposal supersedes a proposal from 2016-05-30.
Commit message
Check for changes on each calendar.
Description of the change
system-apps-ci-bot (system-apps-ci-bot) wrote : Posted in a previous version of this proposal | # |
system-apps-ci-bot (system-apps-ci-bot) wrote : Posted in a previous version of this proposal | # |
PASSED: Continuous integration, rev:171
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
system-apps-ci-bot (system-apps-ci-bot) wrote : Posted in a previous version of this proposal | # |
PASSED: Continuous integration, rev:172
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
system-apps-ci-bot (system-apps-ci-bot) wrote : Posted in a previous version of this proposal | # |
PASSED: Continuous integration, rev:173
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
system-apps-ci-bot (system-apps-ci-bot) wrote : Posted in a previous version of this proposal | # |
PASSED: Continuous integration, rev:174
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
system-apps-ci-bot (system-apps-ci-bot) wrote : Posted in a previous version of this proposal | # |
PASSED: Continuous integration, rev:175
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
Click here to trigger a rebuild:
https:/
system-apps-ci-bot (system-apps-ci-bot) wrote : Posted in a previous version of this proposal | # |
PASSED: Continuous integration, rev:176
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
Click here to trigger a rebuild:
https:/
system-apps-ci-bot (system-apps-ci-bot) wrote : Posted in a previous version of this proposal | # |
PASSED: Continuous integration, rev:177
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
Click here to trigger a rebuild:
https:/
system-apps-ci-bot (system-apps-ci-bot) wrote : Posted in a previous version of this proposal | # |
PASSED: Continuous integration, rev:178
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
Click here to trigger a rebuild:
https:/
Jonas G. Drange (jonas-drange) wrote : Posted in a previous version of this proposal | # |
L128, 136, 164: Maybe prefix these with “calendar:” as you have done elsewhere, so it becomes easier to understand the log output.
I also saw that the calendar did not display the correct timezone: http://
One other thing was that it seems the sync got in a bad state. Compare http://
Renato Araujo Oliveira Filho (renatofilho) wrote : Posted in a previous version of this proposal | # |
Thanks jonas,
> L128, 136, 164: Maybe prefix these with “calendar:” as you have done
> elsewhere, so it becomes easier to understand the log output.
Fixed on rev 179.
>
> I also saw that the calendar did not display the correct timezone:
> http://
Are you using the latest application, there is a click package on silo 73 that should be used with this branch. If the error still could you report a bug. This looks like not related with these changes.
>
> One other thing was that it seems the sync got in a bad state. Compare
> http://
> the same calendar, and I only have that one calendar installed.
What is the diff? I do not see the problem, could you point that? And make sure that you are using the latest app.
Thanks.
Jonas G. Drange (jonas-drange) wrote : Posted in a previous version of this proposal | # |
LGTM!
system-apps-ci-bot (system-apps-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:181
https:/
Executed test runs:
SUCCESS: https:/
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
Preview Diff
1 | === modified file 'plugins/gcalendar/gcalendar.go' |
2 | --- plugins/gcalendar/gcalendar.go 2016-04-14 15:03:27 +0000 |
3 | +++ plugins/gcalendar/gcalendar.go 2016-07-19 14:10:41 +0000 |
4 | @@ -32,7 +32,7 @@ |
5 | pluginName = "gcalendar" |
6 | ) |
7 | |
8 | -var baseUrl, _ = url.Parse("https://www.googleapis.com/calendar/v3/calendars/primary/") |
9 | +var baseUrl,_ = url.Parse("https://www.googleapis.com/calendar/v3/calendars/") |
10 | |
11 | type GCalendarPlugin struct { |
12 | accountId uint |
13 | @@ -49,42 +49,83 @@ |
14 | func (p *GCalendarPlugin) Poll(authData *accounts.AuthData) ([]*plugins.PushMessageBatch, error) { |
15 | // This envvar check is to ease testing. |
16 | if token := os.Getenv("ACCOUNT_POLLD_TOKEN_GCALENDAR"); token != "" { |
17 | + log.Print("calendar: Using token from: ACCOUNT_POLLD_TOKEN_GCALENDAR env var") |
18 | authData.AccessToken = token |
19 | } |
20 | |
21 | + log.Print("calendar: Check calendar changes for account:", p.accountId) |
22 | + |
23 | syncMonitor := NewSyncMonitor() |
24 | if syncMonitor == nil { |
25 | - log.Print("Sync monitor not available yet.") |
26 | - return nil, nil |
27 | - } |
28 | - |
29 | - lastSyncDate, err := syncMonitor.LastSyncDate(p.accountId, "calendar") |
30 | - if err != nil { |
31 | - log.Print("calendar plugin ", p.accountId, ": cannot load previous sync date: ", err, ". Try next time.") |
32 | - return nil, nil |
33 | - } else { |
34 | - log.Print("calendar plugin ", p.accountId, ": last sync date: ", lastSyncDate) |
35 | - } |
36 | - |
37 | - resp, err := p.requestChanges(authData.AccessToken, lastSyncDate) |
38 | - if err != nil { |
39 | - return nil, err |
40 | - } |
41 | - |
42 | - messages, err := p.parseChangesResponse(resp) |
43 | - if err != nil { |
44 | - return nil, err |
45 | - } |
46 | - |
47 | - if len(messages) > 0 { |
48 | - // Update last sync date |
49 | - log.Print("Request calendar sync") |
50 | - err = syncMonitor.SyncAccount(p.accountId, "calendar") |
51 | - if err != nil { |
52 | - log.Print("Fail to start calendar sync ", p.accountId, " error: ", err) |
53 | - } |
54 | - } else { |
55 | - log.Print("Found no calendar updates for account: ", p.accountId) |
56 | + log.Print("calendar: Sync monitor not available yet.") |
57 | + return nil, nil |
58 | + } |
59 | + |
60 | + state, err := syncMonitor.State() |
61 | + if err != nil { |
62 | + log.Print("calendar: Fail to retrieve sync monitor state ", err) |
63 | + return nil, nil |
64 | + } |
65 | + if state != "idle" { |
66 | + log.Print("calendar: Sync monitor is not on 'idle' state, try later!") |
67 | + return nil, nil |
68 | + } |
69 | + |
70 | + calendars, err := syncMonitor.ListCalendarsByAccount(p.accountId) |
71 | + if err != nil { |
72 | + log.Print("calendar: Calendar plugin ", p.accountId, ": cannot load calendars: ", err) |
73 | + return nil, nil |
74 | + } |
75 | + |
76 | + var calendarsToSync []string |
77 | + log.Print("calendar: Number of calendars for account:", p.accountId, " size:", len(calendars)) |
78 | + |
79 | + for id, calendar := range calendars { |
80 | + lastSyncDate, err := syncMonitor.LastSyncDate(p.accountId, id) |
81 | + if err != nil { |
82 | + log.Print("\tcalendar: ", calendar, ", cannot load previous sync date: ", err, ". Try next time.") |
83 | + continue |
84 | + } else { |
85 | + log.Print("\tcalendar: ", calendar, " Id: ", id, ": last sync date: ", lastSyncDate) |
86 | + } |
87 | + |
88 | + var needSync bool |
89 | + needSync = (len(lastSyncDate) == 0) |
90 | + |
91 | + if !needSync { |
92 | + resp, err := p.requestChanges(authData.AccessToken, id, lastSyncDate) |
93 | + if err != nil { |
94 | + log.Print("\tcalendar: ERROR: Fail to query for changes: ", err) |
95 | + continue |
96 | + } |
97 | + |
98 | + messages, err := p.parseChangesResponse(resp) |
99 | + if err != nil { |
100 | + log.Print("\tcalendar: ERROR: Fail to parse changes: ", err) |
101 | + if err == plugins.ErrTokenExpired { |
102 | + log.Print("\t\tcalendar: Abort poll") |
103 | + return nil, err |
104 | + } else { |
105 | + continue |
106 | + } |
107 | + } |
108 | + needSync = (len(messages) > 0) |
109 | + } |
110 | + |
111 | + if needSync { |
112 | + log.Print("\tcalendar: Calendar needs sync: ", calendar) |
113 | + calendarsToSync = append(calendarsToSync, id) |
114 | + } else { |
115 | + log.Print("\tcalendar: Found no calendar updates for account: ", p.accountId, " calendar: ", calendar) |
116 | + } |
117 | + } |
118 | + |
119 | + if len(calendarsToSync) > 0 { |
120 | + log.Print("calendar: Request account sync") |
121 | + err = syncMonitor.SyncAccount(p.accountId, calendarsToSync) |
122 | + if err != nil { |
123 | + log.Print("calendar: ERROR: Fail to start account sync ", p.accountId, " message: ", err) |
124 | + } |
125 | } |
126 | |
127 | return nil, nil |
128 | @@ -96,6 +137,7 @@ |
129 | |
130 | if resp.StatusCode != http.StatusOK { |
131 | var errResp errorResp |
132 | + log.Print("calendar: Invalid response:", errResp.Err.Code) |
133 | if err := decoder.Decode(&errResp); err != nil { |
134 | return nil, err |
135 | } |
136 | @@ -107,24 +149,26 @@ |
137 | |
138 | var events eventList |
139 | if err := decoder.Decode(&events); err != nil { |
140 | + log.Print("calendar: Fail to decode") |
141 | return nil, err |
142 | } |
143 | |
144 | for _, ev := range events.Events { |
145 | - log.Print("Found event: ", ev.Etag, ev.Summary) |
146 | + log.Print("calendar: Found event: ", ev.Etag, ev.Summary) |
147 | } |
148 | |
149 | return events.Events, nil |
150 | } |
151 | |
152 | -func (p *GCalendarPlugin) requestChanges(accessToken string, lastSyncDate string) (*http.Response, error) { |
153 | - u, err := baseUrl.Parse("events") |
154 | +func (p *GCalendarPlugin) requestChanges(accessToken string, calendar string, lastSyncDate string) (*http.Response, error) { |
155 | + u, err := baseUrl.Parse("") |
156 | if err != nil { |
157 | return nil, err |
158 | } |
159 | + u.Path += calendar + "/events" |
160 | |
161 | - //GET https://www.googleapis.com/calendar/v3/calendars/primary/events?showDeleted=true&singleEvents=true&updatedMin=2016-04-06T10%3A00%3A00.00Z&fields=description%2Citems(description%2Cetag%2Csummary)&key={YOUR_API_KEY} |
162 | - query := baseUrl.Query() |
163 | + //GET https://www.googleapis.com/calendar/v3/calendars/<calendar>/events?showDeleted=true&singleEvents=true&updatedMin=2016-04-06T10%3A00%3A00.00Z&fields=description%2Citems(description%2Cetag%2Csummary)&key={YOUR_API_KEY} |
164 | + query := u.Query() |
165 | query.Add("showDeleted", "true") |
166 | query.Add("singleEvents", "true") |
167 | query.Add("fields", "description,items(summary,etag)") |
168 | |
169 | === modified file 'plugins/gcalendar/syncmonitor.go' |
170 | --- plugins/gcalendar/syncmonitor.go 2016-04-14 15:03:27 +0000 |
171 | +++ plugins/gcalendar/syncmonitor.go 2016-07-19 14:10:41 +0000 |
172 | @@ -55,8 +55,19 @@ |
173 | } |
174 | } |
175 | |
176 | -func (p *SyncMonitor) LastSyncDate(accountId uint, serviceName string) (lastSyncDate string, err error) { |
177 | - message, err := p.obj.Call(busInterface, "lastSuccessfulSyncDate", uint32(accountId), serviceName) |
178 | +func (p *SyncMonitor) ListCalendarsByAccount(accountId uint) (calendars map[string]string, err error) { |
179 | + message, err := p.obj.Call(busInterface, "listCalendarsByAccount", uint32(accountId)) |
180 | + if err != nil { |
181 | + var calendars map[string]string |
182 | + return calendars, err |
183 | + } else { |
184 | + err = message.Args(&calendars) |
185 | + return calendars, err |
186 | + } |
187 | +} |
188 | + |
189 | +func (p *SyncMonitor) LastSyncDate(accountId uint, sourceId string) (lastSyncDate string, err error) { |
190 | + message, err := p.obj.Call(busInterface, "lastSuccessfulSyncDate", uint32(accountId), sourceId) |
191 | if err != nil { |
192 | return "", err |
193 | } else { |
194 | @@ -66,7 +77,18 @@ |
195 | } |
196 | } |
197 | |
198 | -func (p *SyncMonitor) SyncAccount(accountId uint, serviceName string) (err error) { |
199 | - _, err = p.obj.Call(busInterface, "syncAccount", uint32(accountId), serviceName) |
200 | +func (p *SyncMonitor) SyncAccount(accountId uint, sources []string) (err error) { |
201 | + _, err = p.obj.Call(busInterface, "syncAccount", uint32(accountId), sources) |
202 | return err |
203 | } |
204 | + |
205 | +func (p *SyncMonitor) State() (state string, err error) { |
206 | + message, err := p.obj.Call(busInterface, "state") |
207 | + if err != nil { |
208 | + return "", err |
209 | + } else { |
210 | + err = message.Args(&state) |
211 | + return state, err |
212 | + } |
213 | +} |
214 | + |
PASSED: Continuous integration, rev:170 /jenkins. canonical. com/system- apps/job/ lp-account- polld-ci/ 1/ /jenkins. canonical. com/system- apps/job/ build/691 /jenkins. canonical. com/system- apps/job/ test-0- autopkgtest/ label=phone- armhf,release= vivid+overlay, testname= default/ 97 /jenkins. canonical. com/system- apps/job/ build-0- fetch/691 /jenkins. canonical. com/system- apps/job/ build-1- sourcepkg/ release= vivid+overlay/ 649 /jenkins. canonical. com/system- apps/job/ build-1- sourcepkg/ release= xenial+ overlay/ 649 /jenkins. canonical. com/system- apps/job/ build-1- sourcepkg/ release= yakkety/ 649 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=amd64, release= vivid+overlay/ 641 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=amd64, release= vivid+overlay/ 641/artifact/ output/ *zip*/output. zip /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=amd64, release= xenial+ overlay/ 641 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=amd64, release= xenial+ overlay/ 641/artifact/ output/ *zip*/output. zip /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=amd64, release= yakkety/ 641 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=amd64, release= yakkety/ 641/artifact/ output/ *zip*/output. zip /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=armhf, release= vivid+overlay/ 641 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=armhf, release= vivid+overlay/ 641/artifact/ output/ *zip*/output. zip /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=armhf, release= xenial+ overlay/ 641 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=armhf, release= xenial+ overlay/ 641/artifact/ output/ *zip*/output. zip /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=armhf, release= yakkety/ 641 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=armhf, release= yakkety/ 641/artifact/ output/ *zip*/output. zip /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=i386, release= vivid+overlay/ 641 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=i386, release= vivid+overlay/ 641/artifact/ output/ *zip*/output. zip /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=i386, release= xenial+ overlay/ 641 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=i386, release= xenial+ overlay/ 641/artifact/ output/ *zip*/output. zip /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=i386, release= yakkety/ 641 /jenkins. canonical. com/system- apps/job/ build-2- binpkg/ arch=i386, release= yakkety/ 641/artifact/ output/ *zip*/output. zip
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild: /jenkins. canonical. com/system- apps/job/ lp-account- polld-ci/ 1/rebuild
https:/