Merge lp:~barry/gwibber/bug-990145 into lp:gwibber

[Barry Warsaw]

It's a release goal for Quantal to ship only Python 3 on the desktop CD, so we
need to port Gwibber to Python 3. This branch doesn't do that. Instead, it
removes a dependency which we know will not be ported upstream any time soon.
I've heard from the mx project that they have no plans to port to Python 3.

This branch changes the uses of mx.DateTime to the built-in datetime module.
A 'make check' seems to pass locally, but it's entirely possible that I'm not
running the full suite correctly. I'm happy to make any additional changes
that might be necessary.

[Barry Warsaw (barry)]

On May 10, 2012, at 06:09 PM, Ken VanDine wrote:

>Review: Needs Fixing
>
>Actually my last test didn't download any updates for facebook, so I purged
>all my facebook messages from the database. Now facebook refresh gives me
>this traceback:

Ah, so the FB data does have a timezone. Seems like a shallow bug, should be
easy to fix. I'll do that asap.

-Barry

>
>
>Dispatcher Thread-6 : ERROR <facebook:receive> Operation failed
>Dispatcher Thread-6 : DEBUG Traceback:
>Traceback (most recent call last):
> File "/home/ken/src/gwibber/trunk/gwibber/microblog/dispatcher.py", line 81, in run
> message_data = PROTOCOLS[account["service"]].Client(account)(opname, **args)
> File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/facebook/__init__.py", line 276, in __call__
> return getattr(self, opname)(**args)
> File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/facebook/__init__.py", line 292, in receive
> return [self._message(post) for post in data["data"]]
> File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/facebook/__init__.py", line 151, in _message
> m["time"] = convert_time(data)
> File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/facebook/__init__.py", line 80, in convert_time
> as_datetime = datetime.strptime(iso8601, ISO8601FORMAT)
> File "/usr/lib/python2.7/_strptime.py", line 328, in _strptime
> data_string[found.end():])
>ValueError: unconverted data remains: +0000
>

[Ken VanDine (ken-vandine)]

Thanks for the branch, review looks good and the unit tests pass. However it does fail with twitter, unfortunately we don't have tests that use real data from the services. It looks like it needs special handling like you did in facebook. Here is the traceback:

Dispatcher Thread-1 : ERROR <twitter:receive> Operation failed
Dispatcher Thread-1 : DEBUG Traceback:
Traceback (most recent call last):
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/dispatcher.py", line 81, in run
    message_data = PROTOCOLS[account["service"]].Client(account)(opname, **args)
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/twitter/__init__.py", line 411, in __call__
    return getattr(self, opname)(**args)
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/twitter/__init__.py", line 414, in receive
    return self._get("statuses/home_timeline.json", include_entities=1, count=count, since_id=since)
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/twitter/__init__.py", line 397, in _get
    if parse: return [getattr(self, "_%s" % parse)(m) for m in data]
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/twitter/__init__.py", line 235, in _message
    n["time"] = util.parsetime(data["created_at"])
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/util/__init__.py", line 36, in parsetime
    dt = datetime.strptime(t, ISO8601FORMAT)
  File "/usr/lib/python2.7/_strptime.py", line 325, in _strptime
    (data_string, format))
ValueError: time data 'Thu May 10 17:52:38 +0000 2012' does not match format '%Y-%m-%dT%H:%M:%S'

review: Needs Fixing

[Ken VanDine (ken-vandine)]

Actually my last test didn't download any updates for facebook, so I purged all my facebook messages from the database. Now facebook refresh gives me this traceback:

Dispatcher Thread-6 : ERROR <facebook:receive> Operation failed
Dispatcher Thread-6 : DEBUG Traceback:
Traceback (most recent call last):
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/dispatcher.py", line 81, in run
    message_data = PROTOCOLS[account["service"]].Client(account)(opname, **args)
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/facebook/__init__.py", line 276, in __call__
    return getattr(self, opname)(**args)
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/facebook/__init__.py", line 292, in receive
    return [self._message(post) for post in data["data"]]
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/facebook/__init__.py", line 151, in _message
    m["time"] = convert_time(data)
  File "/home/ken/src/gwibber/trunk/gwibber/microblog/plugins/facebook/__init__.py", line 80, in convert_time
    as_datetime = datetime.strptime(iso8601, ISO8601FORMAT)
  File "/usr/lib/python2.7/_strptime.py", line 328, in _strptime
    data_string[found.end():])
ValueError: unconverted data remains: +0000

review: Needs Fixing

[lp:~barry/gwibber/bug-990145]

1345. By Barry Warsaw on 2012-05-11

 - Add the most minimal of unittests, with instructions for running them in
   the README file.

 - Add unittests for parsetime().

 - Support more formats in parsetime(), e.g. timezone aware strings (but only
   for UTC, i.e. +0000), and also for alternative ISO 8601 (space instead of
   'T').

[Barry Warsaw (barry)]

I've pushed an update which should fix the parsing in the last comment. I also added the most minimal of unittests, based on our out-of-band discussion. See the README for details. It's ugly and not tied into `make check` but at least it's a start. ;)

[lp:~barry/gwibber/bug-990145]

1346. By Barry Warsaw on 2012-05-11

trunk merge

[Barry Warsaw (barry)]

I think the last update may not completely fix this problem. I'm working on an update.

[lp:~barry/gwibber/bug-990145]

1347. By Barry Warsaw on 2012-05-11

 - Remove some unused imports, and other import cleanups.
 - Avoid circular imports by moving parsetime() to new module gwibber.time
 - Refactor convert_time() to use parsetime() in order to fix the bugs
   everywhere.

[Barry Warsaw (barry)]

Latest update pushed, which should work (seems to locally anyway ;). You'll see the diff got bigger because I had to fiddle with the imports to prevent some circular import problems.

[lp:~barry/gwibber/bug-990145]

1348. By Barry Warsaw on 2012-05-11

 - Handle Twitter and Facebook nonconformity to standards.
 - Refactor the code to be more readable.
 - Improve the comments.

[Ken VanDine (ken-vandine)]

twitter and facebook confirmed to work, however I tested with identica and status.net and they aren't using a UTC time and give the same ValueError. Here is an example time string:

Wed Mar 21 02:30:31 -0400 2012

There is also a typo in the flickr plugin:
- timetup = datetime.utcfromtimestap(int(t)).timetuple()
+ timetup = datetime.utcfromtimestamp(int(t)).timetuple()

The good news is that will be the last of the supported plugins, so once identica and statusnet are working this is good to go!

Thanks!

review: Needs Fixing

[Barry Warsaw (barry)]

On May 11, 2012, at 11:42 PM, Ken VanDine wrote:

>Review: Needs Fixing
>
>twitter and facebook confirmed to work, however I tested with identica and
>status.net and they aren't using a UTC time and give the same ValueError.
>Here is an example time string:
>
>Wed Mar 21 02:30:31 -0400 2012

Madness!

What does Gwibber expect us to do with non-UTC timezones? Just dropping the
tz doesn't seem right. OTOH, we have to convert them to naive datetimes. So
I'm guessing we parse them into non-UTC tz-aware datetimes, then convert them
to UTC, then make them tz-naive. Does that sound right?

>There is also a typo in the flickr plugin:
>- timetup = datetime.utcfromtimestap(int(t)).timetuple()
>+ timetup = datetime.utcfromtimestamp(int(t)).timetuple()
>
>The good news is that will be the last of the supported plugins, so once
>identica and statusnet are working this is good to go!

Yay! :)

Cheers,
-Barry

[Ken VanDine (ken-vandine)]

On Fri, 2012-05-11 at 16:52 -0700, Barry Warsaw wrote:
> On May 11, 2012, at 11:42 PM, Ken VanDine wrote:
>
> >Review: Needs Fixing
> >
> >twitter and facebook confirmed to work, however I tested with identica and
> >status.net and they aren't using a UTC time and give the same ValueError.
> >Here is an example time string:
> >
> >Wed Mar 21 02:30:31 -0400 2012
>
> Madness!
>
> What does Gwibber expect us to do with non-UTC timezones? Just dropping the
> tz doesn't seem right. OTOH, we have to convert them to naive datetimes. So
> I'm guessing we parse them into non-UTC tz-aware datetimes, then convert them
> to UTC, then make them tz-naive. Does that sound right?

Yeah, I think that is the only thing we can do. Sure wish they were
consistent :(

Thanks!

>
> >There is also a typo in the flickr plugin:
> >- timetup = datetime.utcfromtimestap(int(t)).timetuple()
> >+ timetup = datetime.utcfromtimestamp(int(t)).timetuple()
> >
> >The good news is that will be the last of the supported plugins, so once
> >identica and statusnet are working this is good to go!
>
> Yay! :)
>
> Cheers,
> -Barry
>

--
Ken VanDine
Ubuntu Desktop Integration Engineer
Canonical, Ltd.

[lp:~barry/gwibber/bug-990145]

1349. By Barry Warsaw on 2012-05-12

 - Change the flickr plugin to use the parsetime() utility.
 - Remove some unused imports from the flickr plugin.
 - parsetime(): Elaborate so that non-UTC timezones are handled properly,
   which is used by identica and status.net. We have to use regexps here
   though since %z isn't universally supported until Python 3.

[Barry Warsaw (barry)]

On May 11, 2012, at 11:42 PM, Ken VanDine wrote:

>twitter and facebook confirmed to work, however I tested with identica and
>status.net and they aren't using a UTC time and give the same ValueError.
>Here is an example time string:
>
>Wed Mar 21 02:30:31 -0400 2012

Okay, I think I'm handling this case now. Manual timezone math is always fun
(you have to subtract the value from your local time to get you to UTC).
Please do check my math! (And the test cases. :)

>There is also a typo in the flickr plugin:
>- timetup = datetime.utcfromtimestap(int(t)).timetuple()
>+ timetup = datetime.utcfromtimestamp(int(t)).timetuple()

I changed this to use gwibber.time.parsetime().

>The good news is that will be the last of the supported plugins, so once
>identica and statusnet are working this is good to go!

Great! Hopefully this time <wink> will do it.

Update pushed.

[Ken VanDine (ken-vandine)]

Looks great, thanks!

review: Approve

=== modified file 'README'
--- README	2012-05-10 17:20:35 +0000
+++ README	2012-05-12 02:14:17 +0000
@@ -0,0 +1,50 @@
+Installing Gwibber
+==================
+
+  Requirements
+  ------------
+
+	Please note that the version numbers listed below only reflect my test
+	environment. Gwibber is known to work with those specific versions, but
+	will probably work fine on most current desktop distributions that include
+	Python's WebKit GTK+ bindings.
+
+    * python (2.5)
+    * python-dbus (0.80.2)
+    * python-gtk2 (2.10.4)
+    * python-gconf (2.18.0)
+    * python-imaging (1.1.6)
+    * python-notify (0.1.1)
+    * python-webkitgtk (1.0.1)
+    * python-simplejson (1.9.1)
+    * python-distutils-extra
+    * python-feedparser (4.1)
+    * python-xdg (0.15)
+    * python-mako (0.2.2)
+    * python-pycurl
+
+  Installation
+  ------------
+
+	Gwibber uses Python's distutils framework for installation. In order to
+	install Gwibber, you will need root access. To install Gwibber, perform
+	the following command as root:
+
+      $ python setup.py install
+
+  Run Gwibber
+  -----------
+
+	If you installed Gwibber using the setup.py script, you can launch the
+	program by typing "gwibber" at the command line. If you want to run
+	Gwibber without installing it, start "bin/gwibber" from within the
+	Gwibber directory.
+
+  Testing
+  -------
+
+        You can run the dbus isolated tests by cd'ing into the tests directory
+        and running `make check`.
+
+        You can run the Python unittests by cd'ing into tests/python/unittests
+        and running `PYTHONPATH=../../.. python -m unittest discover -v`

=== modified file 'gwibber/microblog/dispatcher.py'
--- gwibber/microblog/dispatcher.py	2012-03-27 15:48:44 +0000
+++ gwibber/microblog/dispatcher.py	2012-05-12 02:14:17 +0000
@@ -3,7 +3,7 @@
 
 import traceback, json, random, string
 import dbus, dbus.service
-import sqlite3, mx.DateTime, re, uuid
+import sqlite3, re, uuid
 import urlshorter, storage, network, util, uploader
 from gettext import lgettext as _
 
@@ -943,7 +943,7 @@
           operations.append(o)
     
       if operations:
-        logger.debug("** Starting Refresh - %s **", mx.DateTime.now())
+        logger.debug("** Starting Refresh - %s **", datetime.now())
         self.LoadingStarted()
         self.perform_async_operation(operations)
 

=== modified file 'gwibber/microblog/plugins/facebook/__init__.py'
--- gwibber/microblog/plugins/facebook/__init__.py	2012-03-09 19:34:33 +0000
+++ gwibber/microblog/plugins/facebook/__init__.py	2012-05-12 02:14:17 +0000
@@ -2,8 +2,7 @@
 
 from gwibber.microblog import network, util
 from gwibber.microblog.util import resources
-import hashlib, mx.DateTime, time
-from os.path import join, getmtime, exists
+import hashlib, time
 from gettext import lgettext as _
 from gwibber.microblog.util.const import *
 # Try to import * from custom, install custom.py to include packaging 
@@ -12,11 +11,15 @@
   from gwibber.microblog.util.custom import *
 except:
   pass
+from gwibber.time import parsetime
+
+from datetime import datetime, timedelta
 
 import logging
 logger = logging.getLogger("Facebook")
 logger.debug("Initializing.")
 
+
 PROTOCOL_INFO = {
   "name": "Facebook",
   "version": "1.1",
@@ -56,6 +59,21 @@
 URL_PREFIX = "https://graph.facebook.com/"
 POST_URL = "http://www.facebook.com/profile.php?id=%s&v=feed&story_fbid=%s&ref=mf"
 
+
+def convert_time(data):
+    """Extract and convert the time to a timestamp (seconds since Epoch).
+
+    First, look for the 'updated_time' key in the data, falling back to
+    'created_time' if the former is missing.
+    """
+    # Convert from an ISO 8601 format time string to a Epoch timestamp.
+    # Assume standard ISO 8601 'T' separator, no microseconds, and naive
+    # time zone as per:
+    # https://developers.facebook.com/docs/reference/api/event/
+    time_string = data.get('updated_time', data['created_time'])
+    return parsetime(time_string)
+
+
 class Client:
   def __init__(self, acct):
     self.account = acct
@@ -122,7 +140,7 @@
     m["service"] = "facebook"
     m["account"] = self.account["id"]
 
-    m["time"] = int(mx.DateTime.DateTimeFrom(str(data.get("updated_time", data["created_time"]))))
+    m["time"] = convert_time(data)
     m["url"] = "https://facebook.com/" + data["id"].split("_")[0] + "/posts/" + data["id"].split("_")[1]
     
 
@@ -233,7 +251,7 @@
         for item in data["comments"]["data"]:
           m["comments"].append({
               "text": item["message"],
-              "time": int(mx.DateTime.DateTimeFrom(str(data.get("updated_time", data["created_time"])))),
+              "time": convert_time(data),
               "sender": self._sender(item["from"]),
             })
 
@@ -250,15 +268,16 @@
     return getattr(self, opname)(**args)
 
   def receive(self, since=None):
-    if not since:
-      since = int(mx.DateTime.DateTimeFromTicks(mx.DateTime.localtime()) - mx.DateTime.TimeDelta(hours=240.0))
+    if since is None:
+      past = datetime.now() - timedelta(hours=240.0)
     else:
-      since = int(mx.DateTime.DateTimeFromTicks(since).localtime())
+      past = datetime.fromtimestamp(since)
 
+    since = int(time.mktime(past.timetuple()))
     data = self._get("me/home", since=since, limit=100)
     
     logger.debug("<STATS> facebook:receive account:%s since:%s size:%s",
-        self.account["id"], mx.DateTime.DateTimeFromTicks(since), len(str(data)))
+        self.account["id"], datetime.fromtimestamp(since), len(str(data)))
     
     if not self._check_error(data):
       try:

=== modified file 'gwibber/microblog/plugins/flickr/__init__.py'
--- gwibber/microblog/plugins/flickr/__init__.py	2012-02-10 10:06:59 +0000
+++ gwibber/microblog/plugins/flickr/__init__.py	2012-05-12 02:14:17 +0000
@@ -1,11 +1,10 @@
-from gwibber.microblog import network, util
-from gwibber.microblog.util import resources
+from gwibber.microblog import network
+from gwibber.time import parsetime
 
 import logging
 logger = logging.getLogger("Flickr")
 logger.debug("Initializing.")
 
-import re, mx.DateTime
 from gettext import lgettext as _
 
 PROTOCOL_INFO = {
@@ -37,8 +36,6 @@
 IMAGE_URL = "http://farm%s.static.flickr.com/%s/%s_%s_%s.jpg"
 IMAGE_PAGE_URL = "http://www.flickr.com/photos/%s/%s"
 
-def parse_time(t):
-  return mx.DateTime.DateTimeFromTicks(int(t)).gmtime().ticks()
 
 class Client:
   def __init__(self, acct):
@@ -49,7 +46,7 @@
     m["mid"] = str(data["id"])
     m["service"] = "flickr"
     m["account"] = self.account["id"]
-    m["time"] = parse_time(data["dateupload"])
+    m["time"] = parsetime(data["dateupload"])
     m["source"] = False
     m["text"] = data["title"]
 

=== modified file 'gwibber/microblog/plugins/friendfeed/__init__.py'
--- gwibber/microblog/plugins/friendfeed/__init__.py	2012-02-10 10:06:59 +0000
+++ gwibber/microblog/plugins/friendfeed/__init__.py	2012-05-12 02:14:17 +0000
@@ -1,5 +1,6 @@
 from gwibber.microblog import network, util
 from gwibber.microblog.util import resources
+from gwibber.time import parsetime
 
 import logging
 logger = logging.getLogger("FriendFeed")
@@ -61,7 +62,7 @@
       "mid": data["id"],
       "service": "friendfeed",
       "account": self.account["id"],
-      "time": util.parsetime(data["published"]),
+      "time": parsetime(data["published"]),
       "source": data.get("via", {}).get("name", None),
       "text": data["title"],
       "html": util.linkify(data["title"]),
@@ -86,7 +87,7 @@
       for item in data["comments"][-3:]:
         m["comments"].append({
           "text": item["body"],
-          "time": util.parsetime(item["date"]),
+          "time": parsetime(item["date"]),
           "sender": self._sender(item["user"]),
         })
 

=== modified file 'gwibber/microblog/plugins/identica/__init__.py'
--- gwibber/microblog/plugins/identica/__init__.py	2012-02-13 20:39:02 +0000
+++ gwibber/microblog/plugins/identica/__init__.py	2012-05-12 02:14:17 +0000
@@ -1,7 +1,9 @@
+from gettext import lgettext as _
+from oauth import oauth
+
 from gwibber.microblog import network, util
-from oauth import oauth
 from gwibber.microblog.util import resources
-from gettext import lgettext as _
+from gwibber.time import parsetime
 
 import logging
 logger = logging.getLogger("Identica")
@@ -71,7 +73,7 @@
       m["service"] = "identica"
       m["account"] = self.account["id"]
       if data.has_key("created_at"):
-        m["time"] = util.parsetime(data["created_at"])
+        m["time"] = parsetime(data["created_at"])
       m["source"] = data.get("source", False)
       m["text"] = util.unescape(data["text"])
       m["to_me"] = ("@%s" % self.account["username"]) in data["text"]
@@ -130,12 +132,12 @@
     if data.has_key("retweeted_status"):
       n["retweeted_by"] = self._user(data["user"] if "user" in data else data["sender"])
       if data.has_key("created_at"):
-        n["time"] = util.parsetime(data["created_at"])
+        n["time"] = parsetime(data["created_at"])
       data = data["retweeted_status"]
     else:
       n["retweeted_by"] = None
       if data.has_key("created_at"):
-        n["time"] = util.parsetime(data["created_at"])
+        n["time"] = parsetime(data["created_at"])
 
     m = self._common(data)
 

=== modified file 'gwibber/microblog/plugins/qaiku/__init__.py'
--- gwibber/microblog/plugins/qaiku/__init__.py	2012-02-10 10:06:59 +0000
+++ gwibber/microblog/plugins/qaiku/__init__.py	2012-05-12 02:14:17 +0000
@@ -1,5 +1,6 @@
 from gwibber.microblog import network, util
 from gwibber.microblog.util import resources
+from gwibber.time import parsetime
 
 import logging
 logger = logging.getLogger("Qaiku")
@@ -46,7 +47,7 @@
     m["mid"] = str(data["id"])
     m["service"] = "qaiku"
     m["account"] = self.account["id"]
-    m["time"] = util.parsetime(data["created_at"])
+    m["time"] = parsetime(data["created_at"])
     m["text"] = data["text"]
     m["to_me"] = ("@%s" % self.account["username"]) in data["text"]
 

=== modified file 'gwibber/microblog/plugins/statusnet/__init__.py'
--- gwibber/microblog/plugins/statusnet/__init__.py	2012-02-13 20:39:02 +0000
+++ gwibber/microblog/plugins/statusnet/__init__.py	2012-05-12 02:14:17 +0000
@@ -1,8 +1,9 @@
-import re
+from gettext import lgettext as _
+from oauth import oauth
+
 from gwibber.microblog import network, util
-from oauth import oauth
 from gwibber.microblog.util import resources
-from gettext import lgettext as _
+from gwibber.time import parsetime
 
 import logging
 logger = logging.getLogger("StatusNet")
@@ -71,7 +72,7 @@
       m["service"] = "statusnet"
       m["account"] = self.account["id"]
       if data.has_key("created_at"):
-        m["time"] = util.parsetime(data["created_at"])
+        m["time"] = parsetime(data["created_at"])
       m["source"] = data.get("source", False)
       m["text"] = util.unescape(data["text"])
       m["to_me"] = ("@%s" % self.account["username"]) in data["text"]
@@ -130,12 +131,12 @@
     if data.has_key("retweeted_status"):
       n["retweeted_by"] = self._user(data["user"] if "user" in data else data["sender"])
       if data.has_key("created_at"):
-        n["time"] = util.parsetime(data["created_at"])
+        n["time"] = parsetime(data["created_at"])
       data = data["retweeted_status"]
     else:
       n["retweeted_by"] = None
       if data.has_key("created_at"):
-        n["time"] = util.parsetime(data["created_at"])
+        n["time"] = parsetime(data["created_at"])
 
     m = self._common(data)
     for k in n:

=== modified file 'gwibber/microblog/plugins/twitter/__init__.py'
--- gwibber/microblog/plugins/twitter/__init__.py	2012-03-30 14:32:20 +0000
+++ gwibber/microblog/plugins/twitter/__init__.py	2012-05-12 02:14:17 +0000
@@ -1,8 +1,10 @@
-from gwibber.microblog import network, util
 import cgi
+from gettext import lgettext as _
 from oauth import oauth
+
+from gwibber.microblog import network, util
 from gwibber.microblog.util import resources
-from gettext import lgettext as _
+from gwibber.time import parsetime
 
 import logging
 logger = logging.getLogger("Twitter")
@@ -79,7 +81,7 @@
       m["service"] = "twitter"
       m["account"] = self.account["id"]
       if data.has_key("created_at"):
-        m["time"] = util.parsetime(data["created_at"])
+        m["time"] = parsetime(data["created_at"])
       m["text"] = util.unescape(data["text"])
       m["text"] = cgi.escape(m["text"])
       m["content"] = m["text"]
@@ -227,12 +229,12 @@
     if data.has_key("retweeted_status"):
       n["retweeted_by"] = self._user(data["user"] if "user" in data else data["sender"])
       if data.has_key("created_at"):
-        n["time"] = util.parsetime(data["created_at"])
+        n["time"] = parsetime(data["created_at"])
       data = data["retweeted_status"]
     else:
       n["retweeted_by"] = None
       if data.has_key("created_at"):
-        n["time"] = util.parsetime(data["created_at"])
+        n["time"] = parsetime(data["created_at"])
 
     m = self._common(data)
     for k in n:

=== modified file 'gwibber/microblog/util/__init__.py'
--- gwibber/microblog/util/__init__.py	2012-04-20 21:16:37 +0000
+++ gwibber/microblog/util/__init__.py	2012-05-12 02:14:17 +0000
@@ -1,7 +1,7 @@
-import os, locale, re, mx.DateTime, cgi, httplib2
-import resources
+import re, cgi, httplib2
+from gwibber.microblog.util import resources
 import dbus
-from const import *
+from gwibber.microblog.util.const import *
 from htmlentitydefs import name2codepoint
 
 import logging
@@ -16,12 +16,6 @@
 
 COUNT = 200
 
-def parsetime(t):
-  locale.setlocale(locale.LC_TIME, 'C')
-  result = mx.DateTime.Parser.DateTimeFromString(t)
-  locale.setlocale(locale.LC_TIME, '')
-  return result.ticks()
-
 URL_SCHEMES = ('http', 'https', 'ftp', 'mailto', 'news', 'gopher',
                'nntp', 'telnet', 'wais', 'prospero', 'aim', 'webcal')
 

=== modified file 'gwibber/microblog/util/resources.py'
--- gwibber/microblog/util/resources.py	2012-03-20 16:31:03 +0000
+++ gwibber/microblog/util/resources.py	2012-05-12 02:14:17 +0000
@@ -9,7 +9,7 @@
 from os import makedirs, remove, environ
 from os.path import join, isdir, realpath, exists
 import Image
-import mx.DateTime
+from datetime import datetime
 from gwibber.microblog import network
 from gwibber.microblog.util.const import *
 import inspect
@@ -211,7 +211,7 @@
     dump_cache_dir = realpath(join(CACHE_BASE_DIR, "gwibber", "dump", service))
     if not isdir(dump_cache_dir):
       makedirs(dump_cache_dir)
-    dump_cache_file = join(dump_cache_dir, (aid + "." + str(mx.DateTime.now()) + "." + operation))
+    dump_cache_file = join(dump_cache_dir, (aid + "." + str(datetime.now()) + "." + operation))
 
     if not exists(dump_cache_file) or len(open(dump_cache_file, "r").read()) < 1:
       logger.debug("Dumping test data %s - %s - %s", service, aid, operation)

=== added file 'gwibber/time.py'
--- gwibber/time.py	1970-01-01 00:00:00 +0000
+++ gwibber/time.py	2012-05-12 02:14:17 +0000
@@ -0,0 +1,96 @@
+from __future__ import absolute_import
+
+import re
+import time
+import locale
+
+from contextlib import contextmanager
+from datetime import datetime, timedelta
+
+
+# Date time formats.  Assume no microseconds and no timezone.
+ISO8601_FORMAT = '%Y-%m-%dT%H:%M:%S'
+TWITTER_FORMAT = '%a %b %d %H:%M:%S %Y'
+
+
+@contextmanager
+def c_locale():
+    locale.setlocale(locale.LC_TIME, 'C')
+    try:
+        yield
+    finally:
+        locale.setlocale(locale.LC_TIME, '')
+
+
+def iso8601(t):
+    return datetime.strptime(t, ISO8601_FORMAT)
+
+def iso8601alt(t):
+    return datetime.strptime(t, ISO8601_FORMAT.replace('T', ' '))
+
+def twitter(t):
+    return datetime.strptime(t, TWITTER_FORMAT)
+
+
+def parsetime(t):
+    """Parse an ISO 8601 datetime string and return seconds since epoch.
+
+    This accepts either a naive (i.e. timezone-less) string or a timezone
+    aware string.  The timezone must start with a + or - and must be followed
+    by exactly four digits.  This string is parsed and converted to UTC.  This
+    value is then converted to an integer seconds since epoch.
+    """
+    with c_locale():
+        # In order to parse the UTC timezone (e.g. +0000), you'd think we
+        # could just append %z on the format, but that doesn't work
+        # across Python versions.  On some platforms in Python 2.7, %z is not
+        # a valid format directive, and its use will raise a ValueError.
+        #
+        # In Python 3.2, strptime() is implemented in Python, so it *is*
+        # supported everywhere, but we still can't rely on it because of the
+        # non-ISO 8601 formats that some APIs use (I'm looking at you Twitter
+        # and Facebook).  We'll use a regular expression to tear out the
+        # timezone string and do the conversion ourselves.
+        #
+        # tz_offset is a list for two reasons.  First, so we can play games
+        # with scoped access to the variable inside capture_tz() without the
+        # use of `nonlocal` which doesn't exist in Python 2.  Also, we'll use
+        # this as a cheap way of ensuring there aren't multiple matching
+        # timezone strings in a single input string.
+        tz_offset = []
+        def capture_tz(match_object):
+            tz_string = match_object.group('tz')
+            if tz_string is not None:
+                # It's possible that we'll see more than one substring
+                # matching the timezone pattern.  It should be highly unlikely
+                # so we won't test for that here, at least not now.
+                #
+                # The tz_offset is positive, so it must be subtracted from the
+                # naive datetime in order to return it to UTC.  E.g.
+                #   13:00 -0400 is 17:00 +0000
+                # or
+                # 1300 - (-0400 / 100)
+                tz_offset.append(timedelta(hours=int(tz_string) / 100))
+            # Return the empty string so as to remove the timezone pattern
+            # from the string we're going to parse.
+            return ''
+        naive_t = re.sub(r'[ ]*(?P<tz>[-+]\d{4})', capture_tz, t)
+        if len(tz_offset) == 0:
+            tz_offset = timedelta()
+        elif len(tz_offset) == 1:
+            tz_offset = tz_offset[0]
+        else:
+            raise ValueError('Unsupported time string: {0}'.format(t))
+        for parser in (iso8601, iso8601alt, twitter):
+            try:
+                dt = parser(naive_t) - tz_offset
+            except ValueError:
+                pass
+            else:
+                break
+        else:
+            # Nothing matched.
+            raise ValueError('Unsupported time string: {0}'.format(t))
+        # We must have gotten a valid datetime.  Convert it to Epoch seconds.
+        timetup = dt.timetuple()
+        return int(time.mktime(timetup))

=== modified file 'gwibber/util.py'
--- gwibber/util.py	2012-02-13 19:59:49 +0000
+++ gwibber/util.py	2012-05-12 02:14:17 +0000
@@ -1,4 +1,4 @@
-import dbus, os, mx.DateTime, webbrowser
+import dbus, os, webbrowser
 import subprocess
 from gi.repository import GLib, Gdk, GdkPixbuf, Gtk
 from microblog.util import resources
@@ -105,8 +105,8 @@
 
 def generate_time_string(t):
   if isinstance(t, str): return t
-  t = mx.DateTime.TimestampFromTicks(t)
-  d = mx.DateTime.gmt() - t
+  t = datetime.fromtimestap(t)
+  d = datetime.utcnow() - t
 
   # Aliasing the function doesn't work here with intltool...
   if d.days >= 365:

=== modified file 'tests/plugins/test/__init__.py'
--- tests/plugins/test/__init__.py	2012-02-10 10:06:59 +0000
+++ tests/plugins/test/__init__.py	2012-05-12 02:14:17 +0000
@@ -1,13 +1,11 @@
+import json
 from gwibber.microblog import util
-import json
+from gwibber.time import parsetime
 
 import logging
 logger = logging.getLogger("Plugin Test")
 logger.debug("Initializing.")
 
-import re, mx.DateTime
-from gettext import lgettext as _
-
 PROTOCOL_INFO = {
   "name": "Test",
   "version": 0.1,
@@ -40,7 +38,7 @@
     m["service"] = "test"
     m["account"] = self.account["id"]
     if data.has_key("created_at"):
-      m["time"] = util.parsetime(data["created_at"])
+      m["time"] = parsetime(data["created_at"])
     m["source"] = data["source"]
     m["text"] = data["text"]
 

=== added directory 'tests/python/unittests'
=== added file 'tests/python/unittests/__init__.py'
=== added file 'tests/python/unittests/test_time.py'
--- tests/python/unittests/test_time.py	1970-01-01 00:00:00 +0000
+++ tests/python/unittests/test_time.py	2012-05-12 02:14:17 +0000
@@ -0,0 +1,53 @@
+import unittest
+
+from gwibber.time import parsetime
+
+
+class TimeParseTest(unittest.TestCase):
+    def test_type(self):
+        # parsetime() should always return int seconds since the epoch.
+        self.assertTrue(isinstance(parsetime('2012-05-10T13:36:45'), int))
+
+    def test_parse_naive(self):
+        # ISO 8601 standard format without timezone.
+        self.assertEqual(parsetime('2012-05-10T13:36:45'), 1336682205)
+
+    def test_parse_utctz(self):
+        # ISO 8601 standard format with UTC timezone.
+        self.assertEqual(parsetime('2012-05-10T13:36:45 +0000'), 1336682205)
+
+    def test_parse_naive_altsep(self):
+        # ISO 8601 alternative format without timezone.
+        self.assertEqual(parsetime('2012-05-10 13:36:45'), 1336682205)
+
+    def test_parse_utctz_altsep(self):
+        # ISO 8601 alternative format with UTC timezone.
+        self.assertEqual(parsetime('2012-05-10T13:36:45 +0000'), 1336682205)
+
+    def test_bad_time_string(self):
+        # Odd unsupported format.
+        self.assertRaises(ValueError, parsetime, '2012/05/10 13:36:45')
+
+    def test_non_utc(self):
+        # Non-UTC timezones are get converted to UTC, before conversion to
+        # epoch seconds.
+        self.assertEqual(parsetime('2012-05-10T13:36:45 -0400'), 1336696605)
+
+    def test_nonstandard_twitter(self):
+        # Sigh.  Twitter has to be different.
+        self.assertEqual(parsetime('Thu May 10 13:36:45 +0000 2012'),
+                         1336682205)
+
+    def test_nonstandard_twitter_non_utc(self):
+        # Sigh.  Twitter has to be different.
+        self.assertEqual(parsetime('Thu May 10 13:36:45 -0400 2012'),
+                         1336696605)
+
+    def test_nonstandard_facebook(self):
+        # Sigh.  Facebook gets close, but no cigar.
+        self.assertEqual(parsetime('2012-05-10T13:36:45+0000'), 1336682205)
+
+    def test_multiple_timezones(self):
+        # Multiple timezone strings are not supported.
+        self.assertRaises(ValueError, parsetime,
+                          '2012-05-10T13:36:45 +0000 -0400')

=== modified file 'tests/python/utils/__init__.py'
--- tests/python/utils/__init__.py	2012-03-12 23:47:12 +0000
+++ tests/python/utils/__init__.py	2012-05-12 02:14:17 +0000
@@ -1,4 +1,7 @@
-import os, unittest, gettext, mx.DateTime
+import time
+from datetime import datetime, timedelta
+
+import os, unittest
 from gi.repository import Gwibber
 from hashlib import sha1
 
@@ -7,34 +10,40 @@
         os.environ["XDG_DATA_HOME"] = os.path.realpath (os.path.join (os.path.curdir, "..", "data"))
         os.environ["XDG_CACHE_HOME"] = os.path.realpath (os.path.join (os.path.curdir, "..", "data"))
         self.utils = Gwibber.Utils ()
-        self.now = mx.DateTime.gmt()
+        self.now = datetime.utcnow()
+
+    def _ago(self, **kws):
+        # Convert keywords into a timedelta, subtract that from utcnow, and
+        # return seconds since epoch.
+        timetup = (self.now - timedelta(**kws)).timetuple()
+        return time.mktime(timetup)
 
     def test_time_string_seconds (self):
-        ts = self.utils.generate_time_string (self.now - 59 * mx.DateTime.oneSecond)
+        ts = self.utils.generate_time_string(self._ago(seconds=59))
         self.assertEqual (ts, 'a few seconds ago')
 
     def test_time_string_minute (self):
-        ts = self.utils.generate_time_string (self.now - 60 * mx.DateTime.oneSecond)
+        ts = self.utils.generate_time_string(self._ago(seconds=60))
         self.assertEqual (ts, '1 minute ago')
 
     def test_time_string_minutes (self):
-        ts = self.utils.generate_time_string (self.now - 3559 * mx.DateTime.oneSecond)
+        ts = self.utils.generate_time_string(self._ago(seconds=3559))
         self.assertIn ('minutes ago', ts)
 
     def test_time_string_hour (self):
-        ts = self.utils.generate_time_string (self.now - 3601 * mx.DateTime.oneSecond)
+        ts = self.utils.generate_time_string(self._ago(seconds=3601))
         self.assertEqual (ts, '1 hour ago')
 
     def test_time_string_hours (self):
-        ts = self.utils.generate_time_string (self.now - 7201 * mx.DateTime.oneSecond)
+        ts = self.utils.generate_time_string(self._ago(seconds=7201))
         self.assertIn ('hours ago', ts)
 
     def test_time_string_day (self):
-        ts = self.utils.generate_time_string (self.now - 86400 * mx.DateTime.oneSecond)
+        ts = self.utils.generate_time_string(self._ago(seconds=86400))
         self.assertEqual (ts, '1 day ago')
 
     def test_time_string_days (self):
-        ts = self.utils.generate_time_string (self.now - 2 * mx.DateTime.oneDay)
+        ts = self.utils.generate_time_string(self._ago(days=2))
         self.assertIn ('days ago', ts)
 
 class AvatarPathTestCase (unittest.TestCase):