poll-daemon

Merge lp:~renatofilho/account-polld/account-polld-caldav into lp:~ubuntu-push-hackers/account-polld/trunk

account-polld-caldav
Merge into trunk

Proposed by Renato Araujo Oliveira Filho on 2016-07-12

Status:	Superseded
Proposed branch:	lp:~renatofilho/account-polld/account-polld-caldav
Merge into:	lp:~ubuntu-push-hackers/account-polld/trunk
Prerequisite:	lp:~renatofilho/account-polld/query-calendars
Diff against target:	1606 lines (+978/-244) 14 files modified accounts/account-watcher.c (+79/-90) accounts/account-watcher.h (+9/-3) accounts/accounts.c (+3/-2) accounts/accounts.go (+25/-9) cmd/account-polld/account_service.go (+11/-13) cmd/account-polld/main.go (+24/-28) cmd/account-watcher-test/main.go (+1/-3) plugins/caldav/api.go (+54/-0) plugins/caldav/caldav.go (+203/-0) plugins/dekko/api.go (+127/-0) plugins/dekko/dekko.go (+346/-0) plugins/gcalendar/gcalendar.go (+3/-2) plugins/gcalendar/syncmonitor.go (+0/-94) syncmonitor/syncmonitor.go (+93/-0)
To merge this branch:	bzr merge lp:~renatofilho/account-polld/account-polld-caldav
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
system-apps-ci-bot	continuous-integration		Approve on 2016-08-02
PS Jenkins bot	continuous-integration	2016-07-12	Pending
Ubuntu Push Hackers		2016-07-12	Pending
Review via email: mp+299868@code.launchpad.net

This proposal supersedes a proposal from 2016-07-12.

This proposal has been superseded by a proposal from 2016-08-02.

Commit message

Implement the caldav plugin to query for changes on caldav sources.

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-07-12:

Download full text (3.3 KiB)

FAILED: Continuous integration, rev:179
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~renatofilho/account-polld/account-polld-caldav/+merge/299868/+edit-commit-message

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/11/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-07-27:

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/16/rebuild

review: Approve (continuous-integration)

lp:~renatofilho/account-polld/account-polld-caldav updated on 2016-07-30

180. By Renato Araujo Oliveira Filho on 2016-07-29: Do not use hard coded urls.
181. By Renato Araujo Oliveira Filho on 2016-07-30: print debug message.
182. By Renato Araujo Oliveira Filho on 2016-07-30: Trunk merged.

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-07-30:

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/17/rebuild

review: Approve (continuous-integration)

lp:~renatofilho/account-polld/account-polld-caldav updated on 2016-07-30

183. By Renato Araujo Oliveira Filho on 2016-07-30: Use UTC date on end date.

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-07-30:

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/18/rebuild

review: Needs Fixing (continuous-integration)

lp:~renatofilho/account-polld/account-polld-caldav updated on 2016-07-30

184. By Renato Araujo Oliveira Filho on 2016-07-30: Fix utc conversion.

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-07-30:

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/19/rebuild

review: Approve (continuous-integration)

lp:~renatofilho/account-polld/account-polld-caldav updated on 2016-07-31

185. By Renato Araujo Oliveira Filho on 2016-07-31: Retrieve changed events one miture before last sync.

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-07-31:

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/20/rebuild

review: Approve (continuous-integration)

lp:~renatofilho/account-polld/account-polld-caldav updated on 2016-07-31

186. By Renato Araujo Oliveira Filho on 2016-07-31: Fix startDate.

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-07-31:

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/21/rebuild

review: Approve (continuous-integration)

lp:~renatofilho/account-polld/account-polld-caldav updated on 2016-07-31

187. By Renato Araujo Oliveira Filho on 2016-07-31: Query for changes in localtime format if UTC returns empty values.

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-07-31:

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/22/rebuild

review: Approve (continuous-integration)

lp:~renatofilho/account-polld/account-polld-caldav updated on 2016-08-01

188. By Renato Araujo Oliveira Filho on 2016-08-01: Query events using UTC time.

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-08-01:

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/23/rebuild

review: Approve (continuous-integration)

lp:~renatofilho/account-polld/account-polld-caldav updated on 2016-08-02

189. By Renato Araujo Oliveira Filho on 2016-08-02: Rever changes on account_manager.go
190. By Renato Araujo Oliveira Filho on 2016-08-02: Format code.
191. By Renato Araujo Oliveira Filho on 2016-08-02: Merged: lp:~mardy/account-polld/dekko-gmail

Revision history for this message

system-apps-ci-bot (system-apps-ci-bot) wrote on 2016-08-02:

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-apps/job/lp-account-polld-ci/24/rebuild

review: Approve (continuous-integration)

lp:~renatofilho/account-polld/account-polld-caldav updated on 2016-08-03

192. By Renato Araujo Oliveira Filho on 2016-08-03: Remove debug messages.

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk

Subscribers

People subscribed via source and target branches

to all changes:

Renato Araujo Oliveira Filho

Ubuntu Push Hackers

 === modified file 'accounts/account-watcher.c'
 --- accounts/account-watcher.c	2016-04-19 19:58:25 +0000
 +++ accounts/account-watcher.c	2016-08-02 14:37:44 +0000
@@ -31,12 +31,12 @@
  struct _AccountWatcher {
      AgManager *manager;
      /* A hash table of the enabled accounts we know of.
--     * Keys are account ID integers, and AccountInfo structs as values.
++     * Keys are "<accountId>/<serviceName>", and AccountInfo structs as values.
       */
      GHashTable *services;
--    gulong enabled_event_signal_id;
--    gulong account_deleted_signal_id;
++    /* List of supported services' IDs */
++    GSList *supported_services;
      AccountEnabledCallback callback;
      void *user_data;
@@ -51,9 +51,7 @@
      GVariant *auth_params;
      GVariant *session_data;
--    gulong enabled_signal_id;
      AgAccountId account_id;
--    gboolean enabled; /* the last known state of the account */
  };
  static void account_info_clear_login(AccountInfo *info) {
@@ -74,11 +72,6 @@
  static void account_info_free(AccountInfo *info) {
      account_info_clear_login(info);
--    if (info->enabled_signal_id != 0) {
--        g_signal_handler_disconnect(
--            info->account_service, info->enabled_signal_id);
--    }
--    info->enabled_signal_id = 0;
      if (info->account_service) {
          g_object_unref(info->account_service);
          info->account_service = NULL;
@@ -94,6 +87,8 @@
      char *client_secret = NULL;
      char *access_token = NULL;
      char *token_secret = NULL;
++    char *secret = NULL;
++    char *user_name = NULL;
      if (info->auth_params != NULL) {
          /* Look up OAuth 2 parameters, falling back to OAuth 1 names */
@@ -109,6 +104,8 @@
      if (info->session_data != NULL) {
          g_variant_lookup(info->session_data, "AccessToken", "&s", &access_token);
          g_variant_lookup(info->session_data, "TokenSecret", "&s", &token_secret);
++        g_variant_lookup(info->session_data, "Secret", "&s", &secret);
++        g_variant_lookup(info->session_data, "UserName", "&s", &user_name);
+     }
      info->watcher->callback(info->watcher,
@@ -116,11 +113,13 @@
                              service_type,
                              service_name,
                              error,
--                            info->enabled,
++                            TRUE,
                              client_id,
                              client_secret,
                              access_token,
                              token_secret,
++                            user_name,
++                            secret,
                              info->watcher->user_data);
+ }
@@ -178,24 +177,6 @@
      ag_auth_data_unref(auth_data);
+ }
--static void account_info_enabled_cb(
--    AgAccountService *account_service, gboolean enabled, AccountInfo *info) {
--    trace("account_info_enabled_cb for %u, enabled=%d\n", info->account_id, enabled);
--    if (info->enabled == enabled) {
--        /* no change */
--        return;
--    }
--    info->enabled = enabled;
--
--    if (enabled) {
--        account_info_login(info);
--    } else {
--        account_info_clear_login(info);
--        // Send notification that account has been disabled */
--        account_info_notify(info, NULL);
--    }
--}
--
  static AccountInfo *account_info_new(AccountWatcher *watcher, AgAccountService *account_service) {
      AccountInfo *info = g_new0(AccountInfo, 1);
      info->watcher = watcher;
@@ -204,99 +185,105 @@
      AgAccount *account = ag_account_service_get_account(account_service);
      g_object_get(account, "id", &info->account_id, NULL);
--    info->enabled_signal_id = g_signal_connect(
--        account_service, "enabled",
--        G_CALLBACK(account_info_enabled_cb), info);
--    // Set initial state
--    account_info_enabled_cb(account_service, ag_account_service_get_enabled(account_service), info);
--
      return info;
+ }
--static void account_watcher_enabled_event_cb(
--    AgManager *manager, AgAccountId account_id, AccountWatcher *watcher) {
--    trace("enabled-event for %u\n", account_id);
--    if (g_hash_table_contains(watcher->services, GUINT_TO_POINTER(account_id))) {
--        /* We are already tracking this account */
--        return;
--    }
--    AgAccount *account = ag_manager_get_account(manager, account_id);
--    if (account == NULL) {
--        /* There was a problem looking up the account */
--        return;
--    }
--    /* Since our AgManager is restricted to a particular service type,
--     * pick the first service for the account. */
--    GList *services = ag_account_list_services(account);
--    if (services != NULL) {
--        AgService *service = services->data;
--        AgAccountService *account_service = ag_account_service_new(
--            account, service);
--        AccountInfo *info = account_info_new(watcher, account_service);
--        g_object_unref(account_service);
--        g_hash_table_insert(watcher->services, GUINT_TO_POINTER(account_id), info);
--    }
--    ag_service_list_free(services);
--    g_object_unref(account);
--}
--
--static void account_watcher_account_deleted_cb(
--    AgManager *manager, AgAccountId account_id, AccountWatcher *watcher) {
--    trace("account-deleted for %u\n", account_id);
--    /* A disabled event should have been sent prior to this, so no
--     * need to send any notification. */
--    g_hash_table_remove(watcher->services, GUINT_TO_POINTER(account_id));
++static gboolean service_is_supported(AccountWatcher *watcher,
++                                     const char *service_id)
++{
++    GSList *node = g_slist_find_custom(watcher->supported_services,
++                                       service_id,
++                                       (GCompareFunc)g_strcmp0);
++    return node != NULL;
+ }
  static gboolean account_watcher_setup(void *user_data) {
      AccountWatcher *watcher = (AccountWatcher *)user_data;
--    /* Track changes to accounts */
--    watcher->enabled_event_signal_id = g_signal_connect(
--        watcher->manager, "enabled-event",
--        G_CALLBACK(account_watcher_enabled_event_cb), watcher);
--    watcher->account_deleted_signal_id = g_signal_connect(
--        watcher->manager, "account-deleted",
--        G_CALLBACK(account_watcher_account_deleted_cb), watcher);
--
      /* Now check initial state */
--    GList *enabled_accounts = ag_manager_list(watcher->manager);
++    GList *enabled_accounts =
++        ag_manager_get_enabled_account_services(watcher->manager);
++    GList *old_services = g_hash_table_get_keys(watcher->services);
++
++    /* Update the services table */
      GList *l;
      for (l = enabled_accounts; l != NULL; l = l->next) {
--        AgAccountId account_id = GPOINTER_TO_UINT(l->data);
--        account_watcher_enabled_event_cb(watcher->manager, account_id, watcher);
--    }
--    ag_manager_list_free(enabled_accounts);
++        AgAccountService *account_service = l->data;
++        AgAccountId id = ag_account_service_get_account(account_service)->id;
++        AgService *service = ag_account_service_get_service(account_service);
++        const char *service_id = ag_service_get_name(service);
++
++        if (!service_is_supported(watcher, service_id)) continue;
++
++        char *key = g_strdup_printf("%d/%s", id, service_id);
++
++        AccountInfo *info = g_hash_table_lookup(watcher->services, key);
++        if (info) {
++            GList *node = g_list_find_custom(old_services, key,
++                                             (GCompareFunc)g_strcmp0);
++            old_services = g_list_remove_link(old_services, node);
++            g_free(key);
++        } else {
++            trace("adding account %s\n", key);
++            info = account_info_new(watcher, account_service);
++            g_hash_table_insert(watcher->services, key, info);
++        }
++        account_info_login(info);
++    }
++    g_list_free_full(enabled_accounts, g_object_unref);
++
++    /* Remove from the table the accounts which are no longer enabled */
++    for (l = old_services; l != NULL; l = l->next) {
++        char *key = l->data;
++        trace("removing account %s\n", key);
++        g_hash_table_remove(watcher->services, key);
++    }
++    g_list_free(old_services);
      return G_SOURCE_REMOVE;
+ }
--AccountWatcher *account_watcher_new(const char *service_type,
--                                    AccountEnabledCallback callback,
++AccountWatcher *account_watcher_new(AccountEnabledCallback callback,
                                      void *user_data) {
      AccountWatcher *watcher = g_new0(AccountWatcher, 1);
--    watcher->manager = ag_manager_new_for_service_type(service_type);
++    watcher->manager = ag_manager_new();
      watcher->services = g_hash_table_new_full(
--        g_direct_hash, g_direct_equal, NULL, (GDestroyNotify)account_info_free);
++        g_str_hash, g_str_equal, g_free, (GDestroyNotify)account_info_free);
++    watcher->supported_services = NULL;
      watcher->callback = callback;
      watcher->user_data = user_data;
++    return watcher;
++}
++
++void account_watcher_add_service(AccountWatcher *watcher,
++                                 char *serviceId) {
++    watcher->supported_services =
++        g_slist_prepend(watcher->supported_services, serviceId);
++}
++
++void account_watcher_run(AccountWatcher *watcher) {
      /* Make sure main setup occurs within the mainloop thread */
      g_idle_add(account_watcher_setup, watcher);
--    return watcher;
+ }
  struct refresh_info {
      AccountWatcher *watcher;
      AgAccountId account_id;
++    char *service_name;
  };
++static void refresh_info_free(struct refresh_info *data) {
++    g_free(data->service_name);
++    g_free(data);
++}
++
  static gboolean account_watcher_refresh_cb(void *user_data) {
      struct refresh_info *data = (struct refresh_info *)user_data;
--    AccountInfo *info = g_hash_table_lookup(
--        data->watcher->services, GUINT_TO_POINTER(data->account_id));
++    char *key = g_strdup_printf("%d/%s", data->account_id, data->service_name);
++    AccountInfo *info = g_hash_table_lookup(data->watcher->services, key);
      if (info != NULL) {
          account_info_login(info);
+     }
@@ -304,10 +291,12 @@
      return G_SOURCE_REMOVE;
+ }
--void account_watcher_refresh(AccountWatcher *watcher, unsigned int account_id) {
++void account_watcher_refresh(AccountWatcher *watcher, unsigned int account_id,
++                             const char *service_name) {
      struct refresh_info *data = g_new(struct refresh_info, 1);
      data->watcher = watcher;
      data->account_id = account_id;
++    data->service_name = g_strdup(service_name);
      g_idle_add_full(G_PRIORITY_DEFAULT_IDLE, account_watcher_refresh_cb,
--                    data, g_free);
++                    data, (GDestroyNotify)refresh_info_free);
+ }
 === modified file 'accounts/account-watcher.h'
 --- accounts/account-watcher.h	2016-04-19 19:58:25 +0000
 +++ accounts/account-watcher.h	2016-08-02 14:37:44 +0000
@@ -30,12 +30,18 @@
                                         const char *client_secret,
                                         const char *access_token,
                                         const char *token_secret,
++                                       const char *user_name,
++                                       const char *secret,
                                         void *user_data);
--AccountWatcher *account_watcher_new(const char *service_type,
--                                    AccountEnabledCallback callback,
++AccountWatcher *account_watcher_new(AccountEnabledCallback callback,
                                      void *user_data);
++void account_watcher_add_service(AccountWatcher *watcher,
++                                 char *serviceId);
++void account_watcher_run(AccountWatcher *watcher);
--void account_watcher_refresh(AccountWatcher *watcher, unsigned int account_id);
++void account_watcher_refresh(AccountWatcher *watcher,
++                             unsigned int account_id,
++                             const char *service_name);
  #endif
 === modified file 'accounts/accounts.c'
 --- accounts/accounts.c	2016-04-19 19:58:25 +0000
 +++ accounts/accounts.c	2016-08-02 14:37:44 +0000
@@ -1,6 +1,6 @@
  #include "_cgo_export.h"
--AccountWatcher *watch_for_service_type(const char *service_type) {
++AccountWatcher *watch() {
      /* Transfer service names to hash table */
      if (FALSE) {
          /* The Go callback doesn't quite match the
@@ -16,10 +16,11 @@
                         GError *error, int enabled,
                         char *client_id, char *client_secret,
                         char *access_token, char *token_secret,
++                       char *user_name, char *secret,
                         void *user_data) = authCallback;
+     }
      AccountWatcher *watcher = account_watcher_new(
--        service_type, (AccountEnabledCallback)authCallback, NULL);
++        (AccountEnabledCallback)authCallback, NULL);
      return watcher;
+ }
 === modified file 'accounts/accounts.go'
 --- accounts/accounts.go	2016-04-19 19:58:25 +0000
 +++ accounts/accounts.go	2016-08-02 14:37:44 +0000
@@ -22,7 +22,7 @@
  #include <glib.h>
  #include "account-watcher.h"
--AccountWatcher *watch_for_service_type(const char *service_type);
++AccountWatcher *watch();
  */
  import "C"
  import (
@@ -47,6 +47,8 @@
  	ClientSecret string
  	AccessToken  string
  	TokenSecret  string
++	Secret       string
++	UserName     string
+ }
  var (
@@ -54,12 +56,10 @@
  	authChannelsLock sync.Mutex
+ )
--// NewWatcher creates a new account watcher for the given service names
--func NewWatcher(serviceType string) *Watcher {
++// NewWatcher creates a new account watcher
++func NewWatcher() *Watcher {
  	w := new(Watcher)
--	cServiceType := C.CString(serviceType)
--	defer C.free(unsafe.Pointer(cServiceType))
--	w.watcher = C.watch_for_service_type(cServiceType)
++	w.watcher = C.watch()
  	ch := make(chan AuthData)
  	w.C = ch
@@ -70,14 +70,24 @@
  	return w
+ }
++func (w *Watcher) AddService(serviceId string) {
++	C.account_watcher_add_service(w.watcher, C.CString(serviceId))
++}
++
++// Walk through the enabled accounts, and get auth tokens for each of them.
++// The new access token will be delivered over the watcher's channel.
++func (w *Watcher) Run() {
++	C.account_watcher_run(w.watcher)
++}
++
  // Refresh requests that the token for the given account be refreshed.
  // The new access token will be delivered over the watcher's channel.
--func (w *Watcher) Refresh(accountId uint) {
--	C.account_watcher_refresh(w.watcher, C.uint(accountId))
++func (w *Watcher) Refresh(accountId uint, serviceName string) {
++	C.account_watcher_refresh(w.watcher, C.uint(accountId), C.CString(serviceName))
+ }
  //export authCallback
--func authCallback(watcher unsafe.Pointer, accountId C.uint, serviceType *C.char, serviceName *C.char, error *C.GError, enabled C.int, clientId, clientSecret, accessToken, tokenSecret *C.char, userData unsafe.Pointer) {
++func authCallback(watcher unsafe.Pointer, accountId C.uint, serviceType *C.char, serviceName *C.char, error *C.GError, enabled C.int, clientId, clientSecret, accessToken, tokenSecret *C.char, userName *C.char, secret *C.char, userData unsafe.Pointer) {
  	// Ideally the first argument would be of type
  	// *C.AccountWatcher, but that fails with Go 1.2.
  	authChannelsLock.Lock()
@@ -110,5 +120,11 @@
  	if tokenSecret != nil {
  		data.TokenSecret = C.GoString(tokenSecret)
+ 	}
++	if secret != nil {
++		data.Secret = C.GoString(secret)
++	}
++	if userName != nil {
++		data.UserName = C.GoString(userName)
++	}
  	ch <- data
+ }
 === renamed file 'cmd/account-polld/account_manager.go' => 'cmd/account-polld/account_service.go'
 --- cmd/account-polld/account_manager.go	2016-06-30 19:21:46 +0000
 +++ cmd/account-polld/account_service.go	2016-08-02 14:37:44 +0000
@@ -26,7 +26,7 @@
  	"launchpad.net/ubuntu-push/click"
+ )
--type AccountManager struct {
++type AccountService struct {
  	watcher          *accounts.Watcher
  	authData         accounts.AuthData
  	plugin           plugins.Plugin
@@ -51,8 +51,8 @@
  	clickNotInstalledError = errors.New("Click not installed")
+ )
--func NewAccountManager(watcher *accounts.Watcher, postWatch chan *PostWatch, plugin plugins.Plugin) *AccountManager {
--	return &AccountManager{
++func NewAccountService(watcher *accounts.Watcher, postWatch chan *PostWatch, plugin plugins.Plugin) *AccountService {
++	return &AccountService{
  		watcher:   watcher,
  		plugin:    plugin,
  		postWatch: postWatch,
@@ -61,20 +61,18 @@
+ 	}
+ }
--func (a *AccountManager) Delete() {
++func (a *AccountService) Delete() {
  	close(a.authChan)
  	close(a.doneChan)
+ }
  // Poll() always needs to be called asynchronously as otherwise qtcontacs' GetAvatar()
  // will raise an error: "QSocketNotifier: Can only be used with threads started with QThread"
--func (a *AccountManager) Poll(bootstrap bool) {
++func (a *AccountService) Poll(bootstrap bool) {
  	gotNewAuthData := false
--	if !a.authData.Enabled {
--		if a.authData, gotNewAuthData = <-a.authChan; !gotNewAuthData {
--			log.Println("Account", a.authData.AccountId, "no longer enabled")
--			return
--		}
++	if a.authData, gotNewAuthData = <-a.authChan; !gotNewAuthData {
++		log.Println("Account", a.authData.AccountId, "no longer enabled")
++		return
+ 	}
  	if a.penaltyCount > 0 {
@@ -126,7 +124,7 @@
  					// and mark the data as disabled.
  					// Do not refresh immediately when we just got new (faulty) auth data as immediately trying
  					// again is probably not going to help. Instead, we wait for the next poll cycle.
--					a.watcher.Refresh(a.authData.AccountId)
++					a.watcher.Refresh(a.authData.AccountId, a.authData.ServiceName)
  					a.authData.Enabled = false
  					a.authData.Error = err
+ 				}
@@ -139,7 +137,7 @@
  	log.Printf("Ending poll for account %d", a.authData.AccountId)
+ }
--func (a *AccountManager) poll() {
++func (a *AccountService) poll() {
  	log.Println("Polling account", a.authData.AccountId)
  	if !isClickInstalled(a.plugin.ApplicationId()) {
  		log.Println(
@@ -167,7 +165,7 @@
+ 	}
+ }
--func (a *AccountManager) updateAuthData(authData accounts.AuthData) {
++func (a *AccountService) updateAuthData(authData accounts.AuthData) {
  	a.authChan <- authData
+ }
 === modified file 'cmd/account-polld/main.go'
 --- cmd/account-polld/main.go	2016-04-21 23:33:48 +0000
 +++ cmd/account-polld/main.go	2016-08-02 14:37:44 +0000
@@ -27,6 +27,8 @@
  	"launchpad.net/account-polld/accounts"
  	"launchpad.net/account-polld/gettext"
  	"launchpad.net/account-polld/plugins"
++	"launchpad.net/account-polld/plugins/caldav"
++	"launchpad.net/account-polld/plugins/dekko"
  	"launchpad.net/account-polld/plugins/gcalendar"
  	"launchpad.net/account-polld/plugins/gmail"
  	"launchpad.net/account-polld/plugins/twitter"
@@ -41,19 +43,18 @@
+ }
  type AccountKey struct {
--	serviceType string
++	serviceId   string
  	accountId   uint
+ }
  /* Use identifiers and API keys provided by the respective webapps which are the official
     end points for the notifications */
  const (
--	SERVICETYPE_WEBAPPS  = "webapps"
--	SERVICETYPE_CALENDAR = "calendar"
--
++	SERVICENAME_DEKKO     = "dekko.dekkoproject_dekko"
  	SERVICENAME_GMAIL     = "com.ubuntu.developer.webapps.webapp-gmail_webapp-gmail"
  	SERVICENAME_TWITTER   = "com.ubuntu.developer.webapps.webapp-twitter_webapp-twitter"
  	SERVICENAME_GCALENDAR = "google-caldav"
++	SERVICENAME_OCALENDAR = "owncloud-caldav"
+ )
  const (
@@ -101,16 +102,18 @@
+ }
  func monitorAccounts(postWatch chan *PostWatch, pollBus *pollbus.PollBus) {
--	watchers := make(map[string]*accounts.Watcher)
--	watchers[SERVICETYPE_WEBAPPS] = accounts.NewWatcher(SERVICETYPE_WEBAPPS)
--	watchers[SERVICETYPE_CALENDAR] = accounts.NewWatcher(SERVICETYPE_CALENDAR)
++	watcher := accounts.NewWatcher()
++	watcher.AddService(SERVICENAME_DEKKO)
++	watcher.AddService(SERVICENAME_GMAIL)
++	watcher.AddService(SERVICENAME_GCALENDAR)
++	watcher.AddService(SERVICENAME_TWITTER)
--	mgr := make(map[AccountKey]*AccountManager)
++	mgr := make(map[AccountKey]*AccountService)
  	var wg sync.WaitGroup
  	pullAccount := func(data accounts.AuthData) bool {
--		accountKey := AccountKey{data.ServiceType, data.AccountId}
++		accountKey := AccountKey{data.ServiceName, data.AccountId}
  		if account, ok := mgr[accountKey]; ok {
  			if data.Enabled {
  				log.Println("New account data for existing account with id", data.AccountId)
@@ -131,8 +134,11 @@
+ 			}
  		} else if data.Enabled {
  			var plugin plugins.Plugin
--			log.Println("Creat plugin for service: ", data.ServiceName)
++			log.Println("Creating plugin for service: ", data.ServiceName)
  			switch data.ServiceName {
++			case SERVICENAME_DEKKO:
++				log.Println("Creating account with id", data.AccountId, "for", data.ServiceName)
++				plugin = dekko.New(data.AccountId)
  			case SERVICENAME_GMAIL:
  				log.Println("Creating account with id", data.AccountId, "for", data.ServiceName)
  				plugin = gmail.New(data.AccountId)
@@ -143,11 +149,14 @@
  				// This is just stubbed until the plugin exists.
  				log.Println("Creating account with id", data.AccountId, "for", data.ServiceName)
  				plugin = twitter.New()
++			case SERVICENAME_OCALENDAR:
++				log.Println("Creating account with id", data.AccountId, "for", data.ServiceName)
++				plugin = caldav.New(data.AccountId)
  			default:
  				log.Println("Unhandled account with id", data.AccountId, "for", data.ServiceName)
  				return false
+ 			}
--			mgr[accountKey] = NewAccountManager(watchers[data.ServiceType], postWatch, plugin)
++			mgr[accountKey] = NewAccountService(watcher, postWatch, plugin)
  			mgr[accountKey].updateAuthData(data)
  			wg.Add(1)
  			go func() {
@@ -165,27 +174,14 @@
  L:
  	for {
  		select {
--		case data := <-watchers[SERVICETYPE_CALENDAR].C:
--			if pullAccount(data) == false {
--				continue L
--			}
--		case data := <-watchers[SERVICETYPE_WEBAPPS].C:
--			if pullAccount(data) == false {
++		case data := <-watcher.C:
++			if pullAccount(data) == false {
++				log.Println("pullAccount returned false, continuing")
  				continue L
+ 			}
  		case <-pollBus.PollChan:
  			wg.Wait() // Finish all running Poll() calls before potentially polling the same accounts again
--			for _, v := range mgr {
--				if v.authData.Error != plugins.ErrTokenExpired { // Do not poll if the new token hasn't been loaded yet
--					wg.Add(1)
--					go func(accountManager *AccountManager) {
--						defer wg.Done()
--						accountManager.Poll(false)
--					}(v)
--				} else {
--					log.Println("Skipping account with id", v.authData.AccountId, "as it is refreshing its token")
--				}
--			}
++			watcher.Run()
  			wg.Wait()
  			pollBus.SignalDone()
+ 		}
 === modified file 'cmd/account-watcher-test/main.go'
 --- cmd/account-watcher-test/main.go	2014-07-24 05:38:51 +0000
 +++ cmd/account-watcher-test/main.go	2016-08-02 14:37:44 +0000
@@ -2,14 +2,12 @@
  import (
  	"fmt"
--	"os"
  	"launchpad.net/account-polld/accounts"
+ )
  func main() {
--	// Expects a list of service names as command line arguments
--	for data := range accounts.NewWatcher(os.Args[1]).C {
++	for data := range accounts.NewWatcher().C {
  		if data.Error != nil {
  			fmt.Println("Failed to authenticate account", data.AccountId, ":", data.Error)
  		} else {
 === added directory 'plugins/caldav'
 === added file 'plugins/caldav/api.go'
 --- plugins/caldav/api.go	1970-01-01 00:00:00 +0000
 +++ plugins/caldav/api.go	2016-08-02 14:37:44 +0000
@@ -0,0 +1,54 @@
++/*
++ Copyright 2014 Canonical Ltd.
++
++ This program is free software: you can redistribute it and/or modify it
++ under the terms of the GNU General Public License version 3, as published
++ by the Free Software Foundation.
++
++ This program is distributed in the hope that it will be useful, but
++ WITHOUT ANY WARRANTY; without even the implied warranties of
++ MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
++ PURPOSE.  See the GNU General Public License for more details.
++
++ You should have received a copy of the GNU General Public License along
++ with this program.  If not, see <http://www.gnu.org/licenses/>.
++*/
++
++package caldav
++
++import (
++	"fmt"
++)
++
++// eventList holds a response to call to Calendar.events: list
++// defined in https://developers.google.com/google-apps/calendar/v3/reference/events/list#response
++type eventList struct {
++	// Messages holds a list of message.
++	Events []event `json:"items"`
++}
++
++// event holds the event data response for a Calendar.event.
++// The full definition of a message is defined in
++// https://developers.google.com/google-apps/calendar/v3/reference/events#resource-representations
++type event struct {
++	// Id is the immutable ID of the message.
++	Etag string `json:"etag"`
++	// ThreadId is the ID of the thread the message belongs to.
++	Summary string `json:"summary"`
++}
++
++func (e event) String() string {
++	return fmt.Sprintf("Id: %s, snippet: '%s'\n", e.Etag, e.Summary)
++}
++
++type errorResp struct {
++	Err struct {
++		Code    uint64 `json:"code"`
++		Message string `json:"message"`
++		Errors  []struct {
++			Domain  string `json:"domain"`
++			Reason  string `json:"reason"`
++			Message string `json:"message"`
++		} `json:"errors"`
++	} `json:"error"`
++}
 === added file 'plugins/caldav/caldav.go'
 --- plugins/caldav/caldav.go	1970-01-01 00:00:00 +0000
 +++ plugins/caldav/caldav.go	2016-08-02 14:37:44 +0000
@@ -0,0 +1,203 @@
++/*
++ Copyright 2016 Canonical Ltd.
++
++ This program is free software: you can redistribute it and/or modify it
++ under the terms of the GNU General Public License version 3, as published
++ by the Free Software Foundation.
++
++ This program is distributed in the hope that it will be useful, but
++ WITHOUT ANY WARRANTY; without even the implied warranties of
++ MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
++ PURPOSE.  See the GNU General Public License for more details.
++
++ You should have received a copy of the GNU General Public License along
++ with this program.  If not, see <http://www.gnu.org/licenses/>.
++*/
++
++package caldav
++
++import (
++	"bytes"
++	"fmt"
++	"io/ioutil"
++	"log"
++	"net/http"
++	"net/url"
++	"os"
++	"strings"
++	"time"
++
++	"launchpad.net/account-polld/accounts"
++	"launchpad.net/account-polld/plugins"
++	"launchpad.net/account-polld/syncmonitor"
++)
++
++const (
++	APP_ID     = "com.ubuntu.calendar_calendar"
++	pluginName = "caldav"
++)
++
++type CalDavPlugin struct {
++	accountId uint
++}
++
++func New(accountId uint) *CalDavPlugin {
++	return &CalDavPlugin{accountId: accountId}
++}
++
++func (p *CalDavPlugin) ApplicationId() plugins.ApplicationId {
++	return plugins.ApplicationId(APP_ID)
++}
++
++func (p *CalDavPlugin) Poll(authData *accounts.AuthData) ([]*plugins.PushMessageBatch, error) {
++	// This envvar check is to ease testing.
++	if token := os.Getenv("ACCOUNT_POLLD_TOKEN_CALDAV"); token != "" {
++		log.Print("Using token from: ACCOUNT_POLLD_TOKEN_CALDAV env var")
++		authData.AccessToken = token
++	}
++
++	log.Print("Check calendar changes for account:", p.accountId)
++
++	syncMonitor := syncmonitor.NewSyncMonitor()
++	if syncMonitor == nil {
++		log.Print("Sync monitor not available yet.")
++		return nil, nil
++	}
++
++	state, err := syncMonitor.State()
++	if err != nil {
++		log.Print("Fail to retrieve sync monitor state ", err)
++		return nil, nil
++	}
++	if state != "idle" {
++		log.Print("Sync monitor is not on 'idle' state, try later!")
++		return nil, nil
++	}
++
++	calendars, err := syncMonitor.ListCalendarsByAccount(p.accountId)
++	if err != nil {
++		log.Print("Calendar plugin ", p.accountId, ": cannot load calendars: ", err)
++		return nil, nil
++	}
++
++	var calendarsToSync []string
++	log.Print("Number of calendars for account:", p.accountId, " size:", len(calendars))
++
++	for id, calendar := range calendars {
++		lastSyncDate, err := syncMonitor.LastSyncDate(p.accountId, id)
++		if err != nil {
++			log.Print("\tcalendar: ", id, ", cannot load previous sync date: ", err, ". Try next time.")
++			continue
++		} else {
++			log.Print("\tcalendar: ", id, " Url: ", calendar, " last sync date: ", lastSyncDate)
++		}
++
++		var needSync bool
++		needSync = (len(lastSyncDate) == 0)
++
++		if !needSync {
++			resp, err := p.requestChanges(authData, calendar, lastSyncDate)
++			if err != nil {
++				log.Print("\tERROR: Fail to query for changes: ", err)
++				continue
++			}
++
++			needSync, err = p.containEvents(resp)
++			if err != nil {
++				log.Print("\tERROR: Fail to parse changes: ", err)
++				if err == plugins.ErrTokenExpired {
++					log.Print("\t\tAbort poll")
++					return nil, err
++				} else {
++					continue
++				}
++			}
++		}
++
++		if needSync {
++			log.Print("\tCalendar needs sync: ", id)
++			calendarsToSync = append(calendarsToSync, id)
++		} else {
++			log.Print("\tFound no calendar updates for account: ", p.accountId, " calendar: ", id)
++		}
++	}
++
++	if len(calendarsToSync) > 0 {
++		log.Print("Request account sync")
++		err = syncMonitor.SyncAccount(p.accountId, calendarsToSync)
++		if err != nil {
++			log.Print("ERROR: Fail to start account sync ", p.accountId, " message: ", err)
++		}
++	}
++
++	return nil, nil
++}
++
++func (p *CalDavPlugin) containEvents(resp *http.Response) (bool, error) {
++	defer resp.Body.Close()
++	log.Print("RESPONSE CODE ----:", resp.StatusCode)
++
++	if resp.StatusCode != 207 {
++		var errResp errorResp
++		log.Print("Invalid response:", errResp.Err.Code)
++		return false, nil
++	} else {
++		data, err := ioutil.ReadAll(resp.Body)
++		if err != nil {
++			return false, err
++		}
++		fmt.Printf("DATA: %s", data)
++		return strings.Contains(string(data), "BEGIN:VEVENT"), nil
++	}
++
++	return false, nil
++}
++
++func (p *CalDavPlugin) requestChanges(authData *accounts.AuthData, calendar string, lastSyncDate string) (*http.Response, error) {
++	u, err := url.Parse(calendar)
++	if err != nil {
++		return nil, err
++	}
++	startDate, err := time.Parse(time.RFC3339, lastSyncDate)
++	if err != nil {
++		log.Print("Fail to parse date: ", lastSyncDate)
++		return nil, err
++	}
++
++	// Start date will be one minute before last sync
++	startDate = startDate.Add(time.Duration(-1) * time.Minute)
++
++	// End Date will be one year in the future from now
++	endDate := time.Now().AddDate(1, 0, 0).UTC()
++
++	log.Print("Calendar Url:", calendar)
++	//u.Path += "/remote.php/caldav/calendars/renatox@gmail.com/" + calendar
++
++	//GET https://my.owndrive.com:443/remote.php/caldav/calendars/renatox%40gmail.com/teste/
++	query := "<c:calendar-query xmlns:d=\"DAV:\" xmlns:c=\"urn:ietf:params:xml:ns:caldav\">\n"
++	query += "<d:prop>\n"
++	query += "<d:getetag />\n"
++	query += "<c:calendar-data />\n"
++	query += "</d:prop>\n"
++	query += "<c:filter>\n"
++	query += "<c:comp-filter name=\"VCALENDAR\">\n"
++	query += "<c:comp-filter name=\"VEVENT\">\n"
++	query += "<c:prop-filter name=\"LAST-MODIFIED\">\n"
++	query += "<c:time-range start=\"" + startDate.Format("20060102T150405Z") + "\" end=\"" + endDate.Format("20060102T150405Z") + "\"/>\n"
++	query += "</c:prop-filter>\n"
++	query += "</c:comp-filter>\n"
++	query += "</c:comp-filter>\n"
++	query += "</c:filter>\n"
++	query += "</c:calendar-query>\n"
++	log.Print("Query: ", query)
++	req, err := http.NewRequest("REPORT", u.String(), bytes.NewBufferString(query))
++	if err != nil {
++		return nil, err
++	}
++	req.Header.Set("Depth", "1")
++	req.Header.Set("Prefer", "return-minimal")
++	req.Header.Set("Content-Type", "application/xml; charset=utf-8")
++	req.SetBasicAuth(authData.UserName, authData.Secret)
++
++	return http.DefaultClient.Do(req)
++}
 === added directory 'plugins/dekko'
 === added file 'plugins/dekko/api.go'
 --- plugins/dekko/api.go	1970-01-01 00:00:00 +0000
 +++ plugins/dekko/api.go	2016-08-02 14:37:44 +0000
@@ -0,0 +1,127 @@
++/*
++ Copyright 2014 Canonical Ltd.
++
++ This program is free software: you can redistribute it and/or modify it
++ under the terms of the GNU General Public License version 3, as published
++ by the Free Software Foundation.
++
++ This program is distributed in the hope that it will be useful, but
++ WITHOUT ANY WARRANTY; without even the implied warranties of
++ MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
++ PURPOSE.  See the GNU General Public License for more details.
++
++ You should have received a copy of the GNU General Public License along
++ with this program.  If not, see <http://www.gnu.org/licenses/>.
++*/
++
++package dekko
++
++import (
++	"fmt"
++	"time"
++
++	"launchpad.net/account-polld/plugins"
++)
++
++const gmailTime = "Mon, 2 Jan 2006 15:04:05 -0700"
++
++type pushes map[string]*plugins.PushMessage
++type headers map[string]string
++
++// messageList holds a response to call to Users.messages: list
++// defined in https://developers.google.com/gmail/api/v1/reference/users/messages/list
++type messageList struct {
++	// Messages holds a list of message.
++	Messages []message `json:"messages"`
++	// NextPageToken is used to retrieve the next page of results in the list.
++	NextPageToken string `json:"nextPageToken"`
++	// ResultSizeEstimage is the estimated total number of results.
++	ResultSizeEstimage uint64 `json:"resultSizeEstimate"`
++}
++
++// message holds a partial response for a Users.messages.
++// The full definition of a message is defined in
++// https://developers.google.com/gmail/api/v1/reference/users/messages#resource
++type message struct {
++	// Id is the immutable ID of the message.
++	Id string `json:"id"`
++	// ThreadId is the ID of the thread the message belongs to.
++	ThreadId string `json:"threadId"`
++	// HistoryId is the ID of the last history record that modified
++	// this message.
++	HistoryId string `json:"historyId"`
++	// Snippet is a short part of the message text. This text is
++	// used for the push message summary.
++	Snippet string `json:"snippet"`
++	// Payload represents the message payload.
++	Payload payload `json:"payload"`
++}
++
++func (m message) String() string {
++	return fmt.Sprintf("Id: %d, snippet: '%s'\n", m.Id, m.Snippet[:10])
++}
++
++// ById implements sort.Interface for []message based on
++// the Id field.
++type byId []message
++
++func (m byId) Len() int           { return len(m) }
++func (m byId) Swap(i, j int)      { m[i], m[j] = m[j], m[i] }
++func (m byId) Less(i, j int) bool { return m[i].Id < m[j].Id }
++
++// payload represents the message payload.
++type payload struct {
++	Headers []messageHeader `json:"headers"`
++}
++
++func (p *payload) mapHeaders() headers {
++	headers := make(map[string]string)
++	for _, hdr := range p.Headers {
++		headers[hdr.Name] = hdr.Value
++	}
++	return headers
++}
++
++func (hdr headers) getTimestamp() time.Time {
++	timestamp, ok := hdr[hdrDATE]
++	if !ok {
++		return time.Now()
++	}
++
++	if t, err := time.Parse(gmailTime, timestamp); err == nil {
++		return t
++	}
++	return time.Now()
++}
++
++func (hdr headers) getEpoch() int64 {
++	return hdr.getTimestamp().Unix()
++}
++
++// messageHeader represents the message headers.
++type messageHeader struct {
++	Name  string `json:"name"`
++	Value string `json:"value"`
++}
++
++type errorResp struct {
++	Err struct {
++		Code    uint64 `json:"code"`
++		Message string `json:"message"`
++		Errors  []struct {
++			Domain  string `json:"domain"`
++			Reason  string `json:"reason"`
++			Message string `json:"message"`
++		} `json:"errors"`
++	} `json:"error"`
++}
++
++func (err *errorResp) Error() string {
++	return fmt.Sprint("backend response:", err.Err.Message)
++}
++
++const (
++	hdrDATE    = "Date"
++	hdrSUBJECT = "Subject"
++	hdrFROM    = "From"
++)
 === added file 'plugins/dekko/dekko.go'
 --- plugins/dekko/dekko.go	1970-01-01 00:00:00 +0000
 +++ plugins/dekko/dekko.go	2016-08-02 14:37:44 +0000
@@ -0,0 +1,346 @@
++/*
++ Copyright 2014 Canonical Ltd.
++
++ This program is free software: you can redistribute it and/or modify it
++ under the terms of the GNU General Public License version 3, as published
++ by the Free Software Foundation.
++
++ This program is distributed in the hope that it will be useful, but
++ WITHOUT ANY WARRANTY; without even the implied warranties of
++ MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
++ PURPOSE.  See the GNU General Public License for more details.
++
++ You should have received a copy of the GNU General Public License along
++ with this program.  If not, see <http://www.gnu.org/licenses/>.
++*/
++
++package dekko
++
++import (
++	"encoding/json"
++	"fmt"
++	"net/http"
++	"net/mail"
++	"net/url"
++	"os"
++	"regexp"
++	"sort"
++	"strings"
++	"time"
++
++	"log"
++
++	"launchpad.net/account-polld/accounts"
++	"launchpad.net/account-polld/gettext"
++	"launchpad.net/account-polld/plugins"
++	"launchpad.net/account-polld/qtcontact"
++)
++
++const (
++	APP_ID           = "dekko.dekkoproject_dekko"
++	dekkoDispatchUrl = "dekko://notify/%d/%s/%s"
++	// If there's more than 10 emails in one batch, we don't show 10 notification
++	// bubbles, but instead show one summary. We always show all notifications in the
++	// indicator.
++	individualNotificationsLimit = 10
++	pluginName                   = "dekko"
++)
++
++type reportedIdMap map[string]time.Time
++
++var baseUrl, _ = url.Parse("https://www.googleapis.com/gmail/v1/users/me/")
++
++// timeDelta defines how old messages can be to be reported.
++var timeDelta = time.Duration(time.Hour * 24)
++
++// trackDelta defines how old messages can be before removed from tracking
++var trackDelta = time.Duration(time.Hour * 24 * 7)
++
++// relativeTimeDelta is the same as timeDelta
++var relativeTimeDelta string = "1d"
++
++// regexp for identifying non-ascii characters
++var nonAsciiChars, _ = regexp.Compile("[^\x00-\x7F]")
++
++type GmailPlugin struct {
++	// reportedIds holds the messages that have already been notified. This
++	// approach is taken against timestamps as it avoids needing to call
++	// get on the message.
++	reportedIds reportedIdMap
++	accountId   uint
++}
++
++func idsFromPersist(accountId uint) (ids reportedIdMap, err error) {
++	err = plugins.FromPersist(pluginName, accountId, &ids)
++	if err != nil {
++		return nil, err
++	}
++	// discard old ids
++	timestamp := time.Now()
++	for k, v := range ids {
++		delta := timestamp.Sub(v)
++		if delta > trackDelta {
++			log.Print("gmail plugin ", accountId, ": deleting ", k, " as ", delta, " is greater than ", trackDelta)
++			delete(ids, k)
++		}
++	}
++	return ids, nil
++}
++
++func (ids reportedIdMap) persist(accountId uint) (err error) {
++	err = plugins.Persist(pluginName, accountId, ids)
++	if err != nil {
++		log.Print("gmail plugin ", accountId, ": failed to save state: ", err)
++	}
++	return nil
++}
++
++func New(accountId uint) *GmailPlugin {
++	reportedIds, err := idsFromPersist(accountId)
++	if err != nil {
++		log.Print("gmail plugin ", accountId, ": cannot load previous state from storage: ", err)
++	} else {
++		log.Print("gmail plugin ", accountId, ": last state loaded from storage")
++	}
++	return &GmailPlugin{reportedIds: reportedIds, accountId: accountId}
++}
++
++func (p *GmailPlugin) ApplicationId() plugins.ApplicationId {
++	return plugins.ApplicationId(APP_ID)
++}
++
++func (p *GmailPlugin) Poll(authData *accounts.AuthData) ([]*plugins.PushMessageBatch, error) {
++	// This envvar check is to ease testing.
++	if token := os.Getenv("ACCOUNT_POLLD_TOKEN_GMAIL"); token != "" {
++		authData.AccessToken = token
++	}
++
++	resp, err := p.requestMessageList(authData.AccessToken)
++	if err != nil {
++		return nil, err
++	}
++	messages, err := p.parseMessageListResponse(resp)
++	if err != nil {
++		return nil, err
++	}
++
++	// TODO use the batching API defined in https://developers.google.com/gmail/api/guides/batch
++	for i := range messages {
++		resp, err := p.requestMessage(messages[i].Id, authData.AccessToken)
++		if err != nil {
++			return nil, err
++		}
++		messages[i], err = p.parseMessageResponse(resp)
++		if err != nil {
++			return nil, err
++		}
++	}
++	notif, err := p.createNotifications(messages)
++	if err != nil {
++		return nil, err
++	}
++	return []*plugins.PushMessageBatch{
++		&plugins.PushMessageBatch{
++			Messages:        notif,
++			Limit:           individualNotificationsLimit,
++			OverflowHandler: p.handleOverflow,
++			Tag:             "dekko",
++		}}, nil
++
++}
++
++func (p *GmailPlugin) reported(id string) bool {
++	_, ok := p.reportedIds[id]
++	return ok
++}
++
++func (p *GmailPlugin) createNotifications(messages []message) ([]*plugins.PushMessage, error) {
++	timestamp := time.Now()
++	pushMsgMap := make(pushes)
++
++	for _, msg := range messages {
++		hdr := msg.Payload.mapHeaders()
++
++		from := hdr[hdrFROM]
++		var avatarPath string
++
++		emailAddress, err := mail.ParseAddress(from)
++		if err != nil {
++			// If the email address contains non-ascii characters, we get an
++			// error so we're going to try again, this time mangling the name
++			// by removing all non-ascii characters. We only care about the email
++			// address here anyway.
++			// XXX: We can't check the error message due to [1]: the error
++			// message is different in go < 1.3 and > 1.5.
++			// [1] https://github.com/golang/go/issues/12492
++			mangledAddr := nonAsciiChars.ReplaceAllString(from, "")
++			mangledEmail, mangledParseError := mail.ParseAddress(mangledAddr)
++			if mangledParseError == nil {
++				emailAddress = mangledEmail
++			}
++		} else if emailAddress.Name != "" {
++			// We only want the Name if the first ParseAddress
++			// call was successful. I.e. we do not want the name
++			// from a mangled email address.
++			from = emailAddress.Name
++		}
++
++		if emailAddress != nil {
++			avatarPath = qtcontact.GetAvatar(emailAddress.Address)
++			// If icon path starts with a path separator, assume local file path,
++			// encode it and prepend file scheme defined in RFC 1738.
++			if strings.HasPrefix(avatarPath, string(os.PathSeparator)) {
++				avatarPath = url.QueryEscape(avatarPath)
++				avatarPath = "file://" + avatarPath
++			}
++		}
++
++		msgStamp := hdr.getTimestamp()
++
++		if _, ok := pushMsgMap[msg.ThreadId]; ok {
++			// TRANSLATORS: the %s is an appended "from" corresponding to an specific email thread
++			pushMsgMap[msg.ThreadId].Notification.Card.Summary += fmt.Sprintf(gettext.Gettext(", %s"), from)
++		} else if timestamp.Sub(msgStamp) < timeDelta {
++			// TRANSLATORS: the %s is the "from" header corresponding to a specific email
++			summary := fmt.Sprintf(gettext.Gettext("%s"), from)
++			// TRANSLATORS: the first %s refers to the email "subject", the second %s refers "from"
++			body := fmt.Sprintf(gettext.Gettext("%s\n%s"), hdr[hdrSUBJECT], msg.Snippet)
++			// fmt with label personal and threadId
++			action := fmt.Sprintf(dekkoDispatchUrl, p.accountId, "INBOX", msg.Id)
++			epoch := hdr.getEpoch()
++			pushMsgMap[msg.ThreadId] = plugins.NewStandardPushMessage(summary, body, action, avatarPath, epoch)
++		} else {
++			log.Print("gmail plugin ", p.accountId, ": skipping message id ", msg.Id, " with date ", msgStamp, " older than ", timeDelta)
++		}
++	}
++	pushMsg := make([]*plugins.PushMessage, 0, len(pushMsgMap))
++	for _, v := range pushMsgMap {
++		pushMsg = append(pushMsg, v)
++	}
++	return pushMsg, nil
++
++}
++func (p *GmailPlugin) handleOverflow(pushMsg []*plugins.PushMessage) *plugins.PushMessage {
++	// TODO it would probably be better to grab the estimate that google returns in the message list.
++	approxUnreadMessages := len(pushMsg)
++
++	// TRANSLATORS: the %d refers to the number of new email messages.
++	summary := fmt.Sprintf(gettext.Gettext("You have %d new messages"), approxUnreadMessages)
++
++	body := ""
++
++	// fmt with label personal and no threadId
++	action := fmt.Sprintf(dekkoDispatchUrl, p.accountId, "INBOX")
++	epoch := time.Now().Unix()
++
++	return plugins.NewStandardPushMessage(summary, body, action, "", epoch)
++}
++
++func (p *GmailPlugin) parseMessageListResponse(resp *http.Response) ([]message, error) {
++	defer resp.Body.Close()
++	decoder := json.NewDecoder(resp.Body)
++
++	if resp.StatusCode != http.StatusOK {
++		var errResp errorResp
++		if err := decoder.Decode(&errResp); err != nil {
++			return nil, err
++		}
++		if errResp.Err.Code == 401 {
++			return nil, plugins.ErrTokenExpired
++		}
++		return nil, &errResp
++	}
++
++	var messages messageList
++	if err := decoder.Decode(&messages); err != nil {
++		return nil, err
++	}
++
++	filteredMsg := p.messageListFilter(messages.Messages)
++
++	return filteredMsg, nil
++}
++
++// messageListFilter returns a subset of unread messages where the subset
++// depends on not being in reportedIds. Before returning, reportedIds is
++// updated with the new list of unread messages.
++func (p *GmailPlugin) messageListFilter(messages []message) []message {
++	sort.Sort(byId(messages))
++	var reportMsg []message
++	var ids = make(reportedIdMap)
++
++	for _, msg := range messages {
++		if !p.reported(msg.Id) {
++			reportMsg = append(reportMsg, msg)
++		}
++		ids[msg.Id] = time.Now()
++	}
++	p.reportedIds = ids
++	p.reportedIds.persist(p.accountId)
++	return reportMsg
++}
++
++func (p *GmailPlugin) parseMessageResponse(resp *http.Response) (message, error) {
++	defer resp.Body.Close()
++	decoder := json.NewDecoder(resp.Body)
++
++	if resp.StatusCode != http.StatusOK {
++		var errResp errorResp
++		if err := decoder.Decode(&errResp); err != nil {
++			return message{}, err
++		}
++		return message{}, &errResp
++	}
++
++	var msg message
++	if err := decoder.Decode(&msg); err != nil {
++		return message{}, err
++	}
++
++	return msg, nil
++}
++
++func (p *GmailPlugin) requestMessage(id, accessToken string) (*http.Response, error) {
++	u, err := baseUrl.Parse("messages/" + id)
++	if err != nil {
++		return nil, err
++	}
++
++	query := u.Query()
++	// only request specific fields
++	query.Add("fields", "snippet,threadId,id,payload/headers")
++	// get the full message to get From and Subject from headers
++	query.Add("format", "full")
++	u.RawQuery = query.Encode()
++
++	req, err := http.NewRequest("GET", u.String(), nil)
++	if err != nil {
++		return nil, err
++	}
++	req.Header.Set("Authorization", "Bearer "+accessToken)
++
++	return http.DefaultClient.Do(req)
++}
++
++func (p *GmailPlugin) requestMessageList(accessToken string) (*http.Response, error) {
++	u, err := baseUrl.Parse("messages")
++	if err != nil {
++		return nil, err
++	}
++
++	query := u.Query()
++
++	// get all unread inbox emails received after
++	// the last time we checked. If this is the first
++	// time we check, get unread emails after timeDelta
++	query.Add("q", fmt.Sprintf("is:unread in:inbox newer_than:%s", relativeTimeDelta))
++	u.RawQuery = query.Encode()
++
++	req, err := http.NewRequest("GET", u.String(), nil)
++	if err != nil {
++		return nil, err
++	}
++	req.Header.Set("Authorization", "Bearer "+accessToken)
++
++	return http.DefaultClient.Do(req)
++}
 === modified file 'plugins/gcalendar/gcalendar.go'
 --- plugins/gcalendar/gcalendar.go	2016-07-12 14:36:52 +0000
 +++ plugins/gcalendar/gcalendar.go	2016-08-02 14:37:44 +0000
@@ -25,6 +25,7 @@
  	"launchpad.net/account-polld/accounts"
  	"launchpad.net/account-polld/plugins"
++	"launchpad.net/account-polld/syncmonitor"
+ )
  const (
@@ -32,7 +33,7 @@
  	pluginName = "gcalendar"
+ )
--var baseUrl,_ = url.Parse("https://www.googleapis.com/calendar/v3/calendars/")
++var baseUrl, _ = url.Parse("https://www.googleapis.com/calendar/v3/calendars/")
  type GCalendarPlugin struct {
  	accountId uint
@@ -55,7 +56,7 @@
  	log.Print("calendar: Check calendar changes for account:", p.accountId)
--	syncMonitor := NewSyncMonitor()
++	syncMonitor := syncmonitor.NewSyncMonitor()
  	if syncMonitor == nil {
  		log.Print("calendar: Sync monitor not available yet.")
  		return nil, nil
 === removed file 'plugins/gcalendar/syncmonitor.go'
 --- plugins/gcalendar/syncmonitor.go	2016-06-23 18:06:35 +0000
 +++ plugins/gcalendar/syncmonitor.go	1970-01-01 00:00:00 +0000
@@ -1,94 +0,0 @@
--/*
-- Copyright 2016 Canonical Ltd.
--
-- This program is free software: you can redistribute it and/or modify it
-- under the terms of the GNU General Public License version 3, as published
-- by the Free Software Foundation.
--
-- This program is distributed in the hope that it will be useful, but
-- WITHOUT ANY WARRANTY; without even the implied warranties of
-- MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
-- PURPOSE.  See the GNU General Public License for more details.
--
-- You should have received a copy of the GNU General Public License along
-- with this program.  If not, see <http://www.gnu.org/licenses/>.
--*/
--
--package gcalendar
--
--import (
--	"log"
--	"runtime"
--
--	"launchpad.net/go-dbus/v1"
--)
--
--const (
--	busInterface = "com.canonical.SyncMonitor"
--	busPath      = "/com/canonical/SyncMonitor"
--	busName      = "com.canonical.SyncMonitor"
--)
--
--type SyncMonitor struct {
--	conn *dbus.Connection
--	obj  *dbus.ObjectProxy
--}
--
--func NewSyncMonitor() *SyncMonitor {
--	conn, err := dbus.Connect(dbus.SessionBus)
--	if err != nil {
--		log.Print("Fail to connect with session bus: ", err)
--		return nil
--	}
--
--	p := &SyncMonitor{
--		conn: conn,
--		obj:  conn.Object(busName, busPath),
--	}
--	runtime.SetFinalizer(p, clean)
--	return p
--}
--
--func clean(p *SyncMonitor) {
--	if p.conn != nil {
--		p.conn.Close()
--	}
--}
--
--func (p *SyncMonitor) ListCalendarsByAccount(accountId uint) (calendars map[string]string, err error) {
--	message, err := p.obj.Call(busInterface, "listCalendarsByAccount", uint32(accountId))
--	if err != nil {
--		var calendars map[string]string
--		return calendars, err
--	} else {
--		err = message.Args(&calendars)
--		return calendars, err
--	}
--}
--
--func (p *SyncMonitor) LastSyncDate(accountId uint, sourceId string) (lastSyncDate string, err error) {
--	message, err := p.obj.Call(busInterface, "lastSuccessfulSyncDate", uint32(accountId), sourceId)
--	if err != nil {
--		return "", err
--	} else {
--		var lastSyncDate string
--		err = message.Args(&lastSyncDate)
--		return lastSyncDate, err
--	}
--}
--
--func (p *SyncMonitor) SyncAccount(accountId uint, sources []string) (err error) {
--	_, err = p.obj.Call(busInterface, "syncAccount", uint32(accountId), sources)
--	return err
--}
--
--func (p *SyncMonitor) State() (state string, err error) {
--	message, err := p.obj.Call(busInterface, "state")
--	if err != nil {
--		return "", err
--	} else {
--		err = message.Args(&state)
--		return state, err
--	}
--}
--
 === added directory 'syncmonitor'
 === added file 'syncmonitor/syncmonitor.go'
 --- syncmonitor/syncmonitor.go	1970-01-01 00:00:00 +0000
 +++ syncmonitor/syncmonitor.go	2016-08-02 14:37:44 +0000
@@ -0,0 +1,93 @@
++/*
++ Copyright 2016 Canonical Ltd.
++
++ This program is free software: you can redistribute it and/or modify it
++ under the terms of the GNU General Public License version 3, as published
++ by the Free Software Foundation.
++
++ This program is distributed in the hope that it will be useful, but
++ WITHOUT ANY WARRANTY; without even the implied warranties of
++ MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
++ PURPOSE.  See the GNU General Public License for more details.
++
++ You should have received a copy of the GNU General Public License along
++ with this program.  If not, see <http://www.gnu.org/licenses/>.
++*/
++
++package syncmonitor
++
++import (
++	"log"
++	"runtime"
++
++	"launchpad.net/go-dbus/v1"
++)
++
++const (
++	busInterface = "com.canonical.SyncMonitor"
++	busPath      = "/com/canonical/SyncMonitor"
++	busName      = "com.canonical.SyncMonitor"
++)
++
++type SyncMonitor struct {
++	conn *dbus.Connection
++	obj  *dbus.ObjectProxy
++}
++
++func NewSyncMonitor() *SyncMonitor {
++	conn, err := dbus.Connect(dbus.SessionBus)
++	if err != nil {
++		log.Print("Fail to connect with session bus: ", err)
++		return nil
++	}
++
++	p := &SyncMonitor{
++		conn: conn,
++		obj:  conn.Object(busName, busPath),
++	}
++	runtime.SetFinalizer(p, clean)
++	return p
++}
++
++func clean(p *SyncMonitor) {
++	if p.conn != nil {
++		p.conn.Close()
++	}
++}
++
++func (p *SyncMonitor) ListCalendarsByAccount(accountId uint) (calendars map[string]string, err error) {
++	message, err := p.obj.Call(busInterface, "listCalendarsByAccount", uint32(accountId))
++	if err != nil {
++		var calendars map[string]string
++		return calendars, err
++	} else {
++		err = message.Args(&calendars)
++		return calendars, err
++	}
++}
++
++func (p *SyncMonitor) LastSyncDate(accountId uint, sourceId string) (lastSyncDate string, err error) {
++	message, err := p.obj.Call(busInterface, "lastSuccessfulSyncDate", uint32(accountId), sourceId)
++	if err != nil {
++		return "", err
++	} else {
++		var lastSyncDate string
++		err = message.Args(&lastSyncDate)
++		return lastSyncDate, err
++	}
++}
++
++func (p *SyncMonitor) SyncAccount(accountId uint, sources []string) (err error) {
++	_, err = p.obj.Call(busInterface, "syncAccount", uint32(accountId), sources)
++	return err
++}
++
++func (p *SyncMonitor) State() (state string, err error) {
++	message, err := p.obj.Call(busInterface, "state")
++	if err != nil {
++		return "", err
++	} else {
++		err = message.Args(&state)
++		return state, err
++	}
++}

poll-daemon

Merge lp:~renatofilho/account-polld/account-polld-caldav into lp:~ubuntu-push-hackers/account-polld/trunk

Commit message

Description of the change

Unmerged revisions

Preview Diff

Subscribers