Merge lp:~alecu/ubuntu-sso-client/and-post into lp:ubuntu-sso-client

[Alejandro J. Cura]

Restfulclient calls are now POST; response.headers are now case-insensitive dicts; OAuth timestamp sync with the server (LP: #916034)

Restfulclient calls are now POST; response.headers are now case-insensitive dicts; OAuth timestamp sync with the server (LP: #916034)

[Manuel de la Peña (mandel)]

The header dictionary looks very similar to the InsensitiveDict found in twisted.python.util. Can we reuse the twisted implementation?

review: Needs Fixing

[Alejandro J. Cura (alecu)]

> The header dictionary looks very similar to the InsensitiveDict found in
> twisted.python.util. Can we reuse the twisted implementation?

The HeaderDict is slightly different from the twisted InsensitiveDict in that the former is a specialization of defaultdict from the stdlib.

The thing is that *we could* simulate a defaultdict from the code in each webclient, by checking for existence of the element and creating it just before using it. But I think the code would be repeated but slightly different in each webclient, so I think it looks better this way.

[lp:~alecu/ubuntu-sso-client/and-post]

842. By Alejandro J. Cura on 2012-01-18

change the assert to assertEqual

[Manuel de la Peña (mandel)]

+1

review: Approve

[Natalia Bidart (nataliabidart)]

Can headers be in "any" language? I was wondering since lowering a string may collide two different words in some languages. I know Turkish has several i's with unexpected upper/lower values.

Also, I think we need to call self.content.close() on connectionLost() (inside StringProtocol).

review: Needs Information

[Alejandro J. Cura (alecu)]

> Can headers be in "any" language? I was wondering since lowering a string may
> collide two different words in some languages. I know Turkish has several i's
> with unexpected upper/lower values.

The "name" of the headers must be ASCII according to the http spec, so it's not an issue in this case.

> Also, I think we need to call self.content.close() on connectionLost() (inside
> StringProtocol).

I don't think it's necessary to do .close() on StringIO objects, since the resources used by them are freed as soon as the objects are collected.

[Natalia Bidart (nataliabidart)]

Looks good!

One tiny note: copyright for timestamp checker files is 2011, perhaps we should make that 2012?

review: Approve

[lp:~alecu/ubuntu-sso-client/and-post]

843. By Alejandro J. Cura on 2012-01-19

fix copyright date on files

=== modified file 'ubuntu_sso/utils/webclient/common.py'
--- ubuntu_sso/utils/webclient/common.py	2012-01-06 20:13:11 +0000
+++ ubuntu_sso/utils/webclient/common.py	2012-01-19 00:42:23 +0000
@@ -15,11 +15,12 @@
 # with this program.  If not, see <http://www.gnu.org/licenses/>.
 """The common bits of a webclient."""
 
-import time
+import collections
 
 from httplib2 import iri2uri
 from oauth import oauth
-from twisted.internet import defer
+
+from ubuntu_sso.utils.webclient.timestamp import TimestampChecker
 
 
 class WebClientError(Exception):
@@ -39,25 +40,60 @@
         self.headers = headers
 
 
+class HeaderDict(collections.defaultdict):
+    """A case insensitive dict for headers."""
+
+    # pylint: disable=E1002
+    def __init__(self, *args, **kwargs):
+        """Handle case-insensitive keys."""
+        super(HeaderDict, self).__init__(list, *args, **kwargs)
+        # pylint: disable=E1101
+        for key, value in self.items():
+            super(HeaderDict, self).__delitem__(key)
+            self[key] = value
+
+    def __setitem__(self, key, value):
+        """Set the value with a case-insensitive key."""
+        super(HeaderDict, self).__setitem__(key.lower(), value)
+
+    def __getitem__(self, key):
+        """Get the value with a case-insensitive key."""
+        return super(HeaderDict, self).__getitem__(key.lower())
+
+    def __delitem__(self, key):
+        """Delete the item with the case-insensitive key."""
+        super(HeaderDict, self).__delitem__(key.lower())
+
+    def __contains__(self, key):
+        """Check the containment with a case-insensitive key."""
+        return super(HeaderDict, self).__contains__(key.lower())
+
+
 class BaseWebClient(object):
     """The webclient base class, to be extended by backends."""
 
+    timestamp_checker = None
+
     def __init__(self, username=None, password=None):
         """Initialize this instance."""
         self.username = username
         self.password = password
 
     def request(self, iri, method="GET", extra_headers=None,
-                oauth_credentials=None):
+                oauth_credentials=None, post_content=None):
         """Return a deferred that will be fired with a Response object."""
         raise NotImplementedError
 
+    @classmethod
+    def get_timestamp_checker(cls):
+        """Get the timestamp checker for this class of webclient."""
+        if cls.timestamp_checker is None:
+            cls.timestamp_checker = TimestampChecker(cls())
+        return cls.timestamp_checker
+
     def get_timestamp(self):
         """Get a timestamp synchronized with the server."""
-        # pylint: disable=W0511
-        # TODO: get the synchronized timestamp
-        timestamp = time.time()
-        return defer.succeed(int(timestamp))
+        return self.get_timestamp_checker().get_faithful_time()
 
     def force_use_proxy(self, settings):
         """Setup this webclient to use the given proxy settings."""

=== modified file 'ubuntu_sso/utils/webclient/libsoup.py'
--- ubuntu_sso/utils/webclient/libsoup.py	2012-01-04 21:26:50 +0000
+++ ubuntu_sso/utils/webclient/libsoup.py	2012-01-19 00:42:23 +0000
@@ -17,12 +17,11 @@
 
 import httplib
 
-from collections import defaultdict
-
 from twisted.internet import defer
 
 from ubuntu_sso.utils.webclient.common import (
     BaseWebClient,
+    HeaderDict,
     Response,
     UnauthorizedError,
     WebClientError,
@@ -48,7 +47,7 @@
     def _on_message(self, session, message, d):
         """Handle the result of an http message."""
         if message.status_code == httplib.OK:
-            headers = defaultdict(list)
+            headers = HeaderDict()
             response_headers = message.get_property("response-headers")
             add_header = lambda key, value, _: headers[key].append(value)
             response_headers.foreach(add_header, None)
@@ -68,7 +67,7 @@
 
     @defer.inlineCallbacks
     def request(self, iri, method="GET", extra_headers=None,
-                oauth_credentials=None):
+                oauth_credentials=None, post_content=None):
         """Return a deferred that will be fired with a Response object."""
         uri = self.iri_to_uri(iri)
         if extra_headers:
@@ -88,6 +87,9 @@
         for key, value in headers.iteritems():
             message.request_headers.append(key, value)
 
+        if post_content:
+            message.request_body.append(post_content)
+
         self.session.queue_message(message, self._on_message, d)
         response = yield d
         defer.returnValue(response)

=== modified file 'ubuntu_sso/utils/webclient/qtnetwork.py'
--- ubuntu_sso/utils/webclient/qtnetwork.py	2012-01-12 17:46:59 +0000
+++ ubuntu_sso/utils/webclient/qtnetwork.py	2012-01-19 00:42:23 +0000
@@ -17,9 +17,8 @@
 
 import sys
 
-from collections import defaultdict
-
 from PyQt4.QtCore import (
+    QBuffer,
     QCoreApplication,
     QUrl,
 )
@@ -33,6 +32,7 @@
 
 from ubuntu_sso.utils.webclient.common import (
     BaseWebClient,
+    HeaderDict,
     Response,
     UnauthorizedError,
     WebClientError,
@@ -62,7 +62,7 @@
 
     @defer.inlineCallbacks
     def request(self, iri, method="GET", extra_headers=None,
-                oauth_credentials=None):
+                oauth_credentials=None, post_content=None):
         """Return a deferred that will be fired with a Response object."""
         uri = self.iri_to_uri(iri)
         request = QNetworkRequest(QUrl(uri))
@@ -87,7 +87,9 @@
         elif method == "HEAD":
             reply = self.nam.head(request)
         else:
-            reply = self.nam.sendCustomRequest(request, method)
+            post_buffer = QBuffer()
+            post_buffer.setData(post_content)
+            reply = self.nam.sendCustomRequest(request, method, post_buffer)
         self.replies[reply] = d
         result = yield d
         defer.returnValue(result)
@@ -104,7 +106,7 @@
         error = reply.error()
         content = reply.readAll()
         if not error:
-            headers = defaultdict(list)
+            headers = HeaderDict()
             for key, value in reply.rawHeaderPairs():
                 headers[str(key)].append(str(value))
             response = Response(bytes(content), headers)

=== modified file 'ubuntu_sso/utils/webclient/restful.py'
--- ubuntu_sso/utils/webclient/restful.py	2012-01-06 18:57:43 +0000
+++ ubuntu_sso/utils/webclient/restful.py	2012-01-19 00:42:23 +0000
@@ -22,6 +22,10 @@
 
 from ubuntu_sso.utils import webclient
 
+POST_HEADERS = {
+    "content-type": "application/x-www-form-urlencoded",
+}
+
 
 class RestfulClient(object):
     """A proxy-enabled restful client."""
@@ -39,14 +43,18 @@
     def restcall(self, method, **kwargs):
         """Make a restful call."""
         assert isinstance(method, unicode)
+        params = {}
         for key, value in kwargs.items():
             if isinstance(value, basestring):
                 assert isinstance(value, unicode)
-                kwargs[key] = value.encode("utf-8")
+            params[key] = json.dumps(value)
         namespace, operation = method.split(".")
-        kwargs["ws.op"] = operation
-        encoded_args = urllib.urlencode(kwargs)
-        iri = self.service_iri + namespace + "?" + encoded_args
+        params["ws.op"] = operation
+        encoded_args = urllib.urlencode(params)
+        iri = self.service_iri + namespace
         creds = self.oauth_credentials
-        result = yield self.webclient.request(iri, oauth_credentials=creds)
+        result = yield self.webclient.request(iri, method="POST",
+                                              oauth_credentials=creds,
+                                              post_content=encoded_args,
+                                              extra_headers=POST_HEADERS)
         defer.returnValue(json.loads(result.content))

=== modified file 'ubuntu_sso/utils/webclient/tests/test_restful.py'
--- ubuntu_sso/utils/webclient/tests/test_restful.py	2012-01-06 18:57:43 +0000
+++ ubuntu_sso/utils/webclient/tests/test_restful.py	2012-01-19 00:42:23 +0000
@@ -87,40 +87,51 @@
         self.assertEqual(len(self.wc.called), 1)
 
     @defer.inlineCallbacks
-    def get_parsed_url(self):
-        """Call the sample operation, and return the url used."""
+    def test_restful_namespace_added_to_url(self):
+        """The restful namespace is added to the url."""
         yield self.rc.restcall(SAMPLE_OPERATION, **SAMPLE_ARGS)
         iri, _, _ = self.wc.called[0]
         uri = iri.encode("ascii")
-        defer.returnValue(urlparse.urlparse(uri))
-
-    @defer.inlineCallbacks
-    def test_restful_namespace_added_to_url(self):
-        """The restful namespace is added to the url."""
-        url = yield self.get_parsed_url()
+        url = urlparse.urlparse(uri)
+        # pylint: disable=E1101
         self.assertTrue(url.path.endswith(SAMPLE_NAMESPACE),
                         "The namespace is included in url")
 
     @defer.inlineCallbacks
-    def test_restful_method_added_to_url(self):
-        """The restful method is added to the url."""
-        url = yield self.get_parsed_url()
-        url_query = urlparse.parse_qs(url.query)
-        self.assertEqual(url_query["ws.op"][0], SAMPLE_METHOD)
-
-    @defer.inlineCallbacks
-    def test_arguments_added_to_call(self):
-        """The keyword arguments are used in the called url."""
-        url = yield self.get_parsed_url()
-        url_query = dict(urlparse.parse_qsl(url.query))
-        del(url_query["ws.op"])
-        expected = {}
-        for key, value in SAMPLE_ARGS.items():
-            if isinstance(value, unicode):
-                expected[key] = value.encode("utf-8")
-            else:
-                expected[key] = str(value)
-        self.assertEqual(url_query, expected)
+    def test_restful_method_added_to_params(self):
+        """The restful method is added to the params."""
+        yield self.rc.restcall(SAMPLE_OPERATION, **SAMPLE_ARGS)
+        _, _, webcall_kwargs = self.wc.called[0]
+        wc_params = urlparse.parse_qs(webcall_kwargs["post_content"])
+        self.assertEqual(wc_params["ws.op"][0], SAMPLE_METHOD)
+
+    @defer.inlineCallbacks
+    def test_arguments_added_as_json_to_webcall(self):
+        """The keyword arguments are used as json in the webcall."""
+        yield self.rc.restcall(SAMPLE_OPERATION, **SAMPLE_ARGS)
+        _, _, webcall_kwargs = self.wc.called[0]
+        params = urlparse.parse_qsl(webcall_kwargs["post_content"])
+        result = {}
+        for key, value in params:
+            if key == "ws.op":
+                continue
+            result[key] = restful.json.loads(value)
+        self.assertEqual(result, SAMPLE_ARGS)
+
+    @defer.inlineCallbacks
+    def test_post_header_sent(self):
+        """A header is sent specifying the contents of the post."""
+        yield self.rc.restcall(SAMPLE_OPERATION, **SAMPLE_ARGS)
+        _, _, webcall_kwargs = self.wc.called[0]
+        self.assertEqual(restful.POST_HEADERS,
+                         webcall_kwargs["extra_headers"])
+
+    @defer.inlineCallbacks
+    def test_post_method_set(self):
+        """The method of the webcall is set to POST."""
+        yield self.rc.restcall(SAMPLE_OPERATION, **SAMPLE_ARGS)
+        _, _, webcall_kwargs = self.wc.called[0]
+        self.assertEqual("POST", webcall_kwargs["method"])
 
     @defer.inlineCallbacks
     def test_return_value_json_parsed(self):

=== added file 'ubuntu_sso/utils/webclient/tests/test_timestamp.py'
--- ubuntu_sso/utils/webclient/tests/test_timestamp.py	1970-01-01 00:00:00 +0000
+++ ubuntu_sso/utils/webclient/tests/test_timestamp.py	2012-01-19 00:42:23 +0000
@@ -0,0 +1,140 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright 2011-2012 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Tests for the timestamp sync classes."""
+
+from twisted.application import internet, service
+from twisted.internet import defer
+from twisted.trial.unittest import TestCase
+from twisted.web import server, resource
+
+from ubuntu_sso.utils.webclient import timestamp, webclient_factory
+
+
+class FakedError(Exception):
+    """Stub to replace Request.error."""
+
+
+class RootResource(resource.Resource):
+    """A root resource that logs the number of calls."""
+
+    isLeaf = True
+
+    def __init__(self, *args, **kwargs):
+        """Initialize this fake instance."""
+        resource.Resource.__init__(self, *args, **kwargs)
+        self.count = 0
+        self.request_headers = []
+
+    # pylint: disable=C0103
+    def render_HEAD(self, request):
+        """Increase the counter on each render."""
+        self.request_headers.append(request.requestHeaders)
+        self.count += 1
+        return ""
+
+
+class MockWebServer(object):
+    """A mock webserver for testing."""
+
+    # pylint: disable=E1101
+    def __init__(self):
+        """Start up this instance."""
+        self.root = RootResource()
+        site = server.Site(self.root)
+        application = service.Application('web')
+        self.service_collection = service.IServiceCollection(application)
+        self.tcpserver = internet.TCPServer(0, site)
+        self.tcpserver.setServiceParent(self.service_collection)
+        self.service_collection.startService()
+
+    def get_iri(self):
+        """Build the url for this mock server."""
+        # pylint: disable=W0212
+        port_num = self.tcpserver._port.getHost().port
+        return u"http://localhost:%d/" % port_num
+
+    def stop(self):
+        """Shut it down."""
+        self.service_collection.stopService()
+
+
+class TimestampCheckerTestCase(TestCase):
+    """Tests for the timestamp checker."""
+
+    @defer.inlineCallbacks
+    def setUp(self):
+        yield super(TimestampCheckerTestCase, self).setUp()
+        self.ws = MockWebServer()
+        self.addCleanup(self.ws.stop)
+        self.wc = webclient_factory()
+        self.addCleanup(self.wc.shutdown)
+        self.patch(timestamp.TimestampChecker, "SERVER_IRI", self.ws.get_iri())
+
+    @defer.inlineCallbacks
+    def test_returned_value_is_int(self):
+        """The returned value is an integer."""
+        checker = timestamp.TimestampChecker(self.wc)
+        result = yield checker.get_faithful_time()
+        self.assertEqual(type(result), int)
+
+    @defer.inlineCallbacks
+    def test_first_call_does_head(self):
+        """The first call gets the clock from our web."""
+        checker = timestamp.TimestampChecker(self.wc)
+        yield checker.get_faithful_time()
+        self.assertEqual(self.ws.root.count, 1)
+
+    @defer.inlineCallbacks
+    def test_second_call_is_cached(self):
+        """For the second call, the time is cached."""
+        checker = timestamp.TimestampChecker(self.wc)
+        yield checker.get_faithful_time()
+        yield checker.get_faithful_time()
+        self.assertEqual(self.ws.root.count, 1)
+
+    @defer.inlineCallbacks
+    def test_after_timeout_cache_expires(self):
+        """After some time, the cache expires."""
+        fake_timestamp = 1
+        self.patch(timestamp.time, "time", lambda: fake_timestamp)
+        checker = timestamp.TimestampChecker(self.wc)
+        yield checker.get_faithful_time()
+        fake_timestamp += timestamp.TimestampChecker.CHECKING_INTERVAL
+        yield checker.get_faithful_time()
+        self.assertEqual(self.ws.root.count, 2)
+
+    @defer.inlineCallbacks
+    def test_server_error_means_skew_not_updated(self):
+        """When server can't be reached, the skew is not updated."""
+        fake_timestamp = 1
+        self.patch(timestamp.time, "time", lambda: fake_timestamp)
+        checker = timestamp.TimestampChecker(self.wc)
+        failing_get_server_time = lambda _: defer.fail(FakedError())
+        self.patch(checker, "get_server_time", failing_get_server_time)
+        yield checker.get_faithful_time()
+        self.assertEqual(checker.skew, 0)
+        self.assertEqual(checker.next_check,
+                    fake_timestamp + timestamp.TimestampChecker.ERROR_INTERVAL)
+
+    @defer.inlineCallbacks
+    def test_server_date_sends_nocache_headers(self):
+        """Getting the server date sends the no-cache headers."""
+        checker = timestamp.TimestampChecker(self.wc)
+        yield checker.get_server_date_header(self.ws.get_iri())
+        self.assertEqual(len(self.ws.root.request_headers), 1)
+        headers = self.ws.root.request_headers[0]
+        result = headers.getRawHeaders("Cache-Control")
+        self.assertEqual(result, ["no-cache"])

=== modified file 'ubuntu_sso/utils/webclient/tests/test_webclient.py'
--- ubuntu_sso/utils/webclient/tests/test_webclient.py	2012-01-12 13:55:30 +0000
+++ ubuntu_sso/utils/webclient/tests/test_webclient.py	2012-01-19 00:42:23 +0000
@@ -28,6 +28,7 @@
 from ubuntuone.devtools.testcases.squid import SquidTestCase
 
 from ubuntu_sso.utils import webclient
+from ubuntu_sso.utils.webclient.common import HeaderDict
 
 ANY_VALUE = object()
 SAMPLE_KEY = "result"
@@ -42,12 +43,15 @@
     token_secret="the token secret",
 )
 SAMPLE_HEADERS = {SAMPLE_KEY: SAMPLE_VALUE}
+SAMPLE_POST_PARAMS = {"param1": "value1", "param2": "value2"}
 
 SIMPLERESOURCE = "simpleresource"
+POSTABLERESOURECE = "postableresourece"
 THROWERROR = "throwerror"
 UNAUTHORIZED = "unauthorized"
 HEADONLY = "headonly"
 VERIFYHEADERS = "verifyheaders"
+VERIFYPOSTPARAMS = "verifypostparams"
 GUARDED = "guarded"
 OAUTHRESOURCE = "oauthresource"
 
@@ -70,6 +74,14 @@
         return SAMPLE_RESOURCE
 
 
+class PostableResource(resource.Resource):
+    """A resource that only answers to POST requests."""
+
+    def render_POST(self, request):
+        """Make a bit of html out of these resource's content."""
+        return SAMPLE_RESOURCE
+
+
 class HeadOnlyResource(resource.Resource):
     """A resource that fails if called with a method other than HEAD."""
 
@@ -87,6 +99,29 @@
         if headers != [SAMPLE_VALUE]:
             request.setResponseCode(http.BAD_REQUEST)
             return "ERROR: Expected header not present."
+        request.setHeader(SAMPLE_KEY, SAMPLE_VALUE)
+        return SAMPLE_RESOURCE
+
+
+class VerifyPostParameters(resource.Resource):
+    """A resource that answers to POST requests with some parameters."""
+
+    def fetch_post_args_only(self, request):
+        """Fetch only the POST arguments, not the args in the url."""
+        request.process = lambda: None
+        request.requestReceived(request.method, request.path,
+                                request.clientproto)
+        return request.args
+
+    def render_POST(self, request):
+        """Verify the parameters that we've been called with."""
+        post_params = self.fetch_post_args_only(request)
+        expected = dict((key, [val]) for key, val
+                                      in SAMPLE_POST_PARAMS.items())
+        if post_params != expected:
+            request.setResponseCode(http.BAD_REQUEST)
+            return "ERROR: Expected arguments not present, %r != %r" % (
+                    post_params, expected)
         return SAMPLE_RESOURCE
 
 
@@ -142,6 +177,7 @@
         """Start up this instance."""
         root = resource.Resource()
         root.putChild(SIMPLERESOURCE, SimpleResource())
+        root.putChild(POSTABLERESOURECE, PostableResource())
 
         root.putChild(THROWERROR, resource.NoResource())
 
@@ -150,6 +186,7 @@
         root.putChild(UNAUTHORIZED, unauthorized_resource)
         root.putChild(HEADONLY, HeadOnlyResource())
         root.putChild(VERIFYHEADERS, VerifyHeadersResource())
+        root.putChild(VERIFYPOSTPARAMS, VerifyPostParameters())
         root.putChild(OAUTHRESOURCE, OAuthCheckerResource())
 
         db = checkers.InMemoryUsernamePasswordDatabaseDontUse()
@@ -259,6 +296,25 @@
                                  webclient.WebClientError)
 
     @defer.inlineCallbacks
+    def test_post(self):
+        """Test a post request."""
+        result = yield self.wc.request(self.base_iri + POSTABLERESOURECE,
+                                       method="POST")
+        self.assertEqual(SAMPLE_RESOURCE, result.content)
+
+    @defer.inlineCallbacks
+    def test_post_with_args(self):
+        """Test a post request with arguments."""
+        args = urllib.urlencode(SAMPLE_POST_PARAMS)
+        iri = self.base_iri + VERIFYPOSTPARAMS + "?" + args
+        headers = {
+            "content-type": "application/x-www-form-urlencoded",
+        }
+        result = yield self.wc.request(iri, method="POST",
+                                      extra_headers=headers, post_content=args)
+        self.assertEqual(SAMPLE_RESOURCE, result.content)
+
+    @defer.inlineCallbacks
     def test_unauthorized(self):
         """Detect when a request failed with the UNAUTHORIZED http code."""
         yield self.assertFailure(self.wc.request(self.base_iri + UNAUTHORIZED),
@@ -275,7 +331,8 @@
         """The extra_headers are sent to the server."""
         result = yield self.wc.request(self.base_iri + VERIFYHEADERS,
                                        extra_headers=SAMPLE_HEADERS)
-        self.assertEqual(SAMPLE_RESOURCE, result.content)
+        self.assertIn(SAMPLE_KEY, result.headers)
+        self.assertEqual(result.headers[SAMPLE_KEY], [SAMPLE_VALUE])
 
     @defer.inlineCallbacks
     def test_send_basic_auth(self):
@@ -294,6 +351,22 @@
         self.assertEqual(SAMPLE_RESOURCE, result.content)
 
     @defer.inlineCallbacks
+    def test_oauth_signing_uses_timestamp(self):
+        """OAuth signing uses the timestamp."""
+        called = []
+
+        def fake_get_faithful_time():
+            """A fake get_timestamp"""
+            called.append(True)
+            return defer.succeed(1)
+
+        tsc = self.wc.get_timestamp_checker()
+        self.patch(tsc, "get_faithful_time", fake_get_faithful_time)
+        yield self.wc.request(self.base_iri + OAUTHRESOURCE,
+                              oauth_credentials=SAMPLE_CREDENTIALS)
+        self.assertTrue(called, "The timestamp must be retrieved.")
+
+    @defer.inlineCallbacks
     def test_returned_content_are_bytes(self):
         """The returned content are bytes."""
         result = yield self.wc.request(self.base_iri + OAUTHRESOURCE,
@@ -302,6 +375,35 @@
                         "The type of %r must be bytes" % result.content)
 
 
+class TimestampCheckerTestCase(TestCase):
+    """Tests for the timestampchecker classmethod."""
+
+    @defer.inlineCallbacks
+    def setUp(self):
+        """Initialize this testcase."""
+        yield super(TimestampCheckerTestCase, self).setUp()
+        self.wc = webclient.webclient_factory()
+        self.patch(self.wc.__class__, "timestamp_checker", None)
+
+    def test_timestamp_checker_has_the_same_class_as_the_creator(self):
+        """The TimestampChecker has the same class."""
+        tsc = self.wc.get_timestamp_checker()
+        self.assertEqual(tsc.webclient.__class__, self.wc.__class__)
+
+    def test_timestamp_checker_uses_different_webclient_than_the_creator(self):
+        """The TimestampChecker uses a different webclient than the creator."""
+        tsc = self.wc.get_timestamp_checker()
+        self.assertNotEqual(tsc.webclient, self.wc)
+
+    def test_timestamp_checker_is_the_same_for_all_webclients(self):
+        """The TimestampChecker is the same for all webclients."""
+        tsc1 = self.wc.get_timestamp_checker()
+        wc2 = webclient.webclient_factory()
+        tsc2 = wc2.get_timestamp_checker()
+        # pylint: disable=E1101
+        self.assertIs(tsc1, tsc2)
+
+
 class BasicProxyTestCase(SquidTestCase):
     """Test that the proxy works at all."""
 
@@ -340,6 +442,31 @@
         test_authenticated_proxy_is_used.skip = reason
 
 
+class HeaderDictTestCase(TestCase):
+    """Tests for the case insensitive header dictionary."""
+
+    def test_constructor_handles_keys(self):
+        """The constructor handles case-insensitive keys."""
+        hd = HeaderDict({"ClAvE": "value"})
+        self.assertIn("clave", hd)
+
+    def test_can_set_get_items(self):
+        """The item is set/getted."""
+        hd = HeaderDict()
+        hd["key"] = "value"
+        hd["KEY"] = "value2"
+        self.assertEqual(hd["key"], "value2")
+
+    def test_can_test_presence(self):
+        """The presence of an item is found."""
+        hd = HeaderDict()
+        self.assertNotIn("cLaVe", hd)
+        hd["CLAVE"] = "value1"
+        self.assertIn("cLaVe", hd)
+        del(hd["cLAVe"])
+        self.assertNotIn("cLaVe", hd)
+
+
 class OAuthTestCase(TestCase):
     """Test for the oauth signing code."""
 

=== added file 'ubuntu_sso/utils/webclient/timestamp.py'
--- ubuntu_sso/utils/webclient/timestamp.py	1970-01-01 00:00:00 +0000
+++ ubuntu_sso/utils/webclient/timestamp.py	2012-01-19 00:42:23 +0000
@@ -0,0 +1,74 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright 2011-2012 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Timestamp synchronization with the server."""
+
+import time
+
+from twisted.internet import defer
+from twisted.web import http
+
+from ubuntu_sso.logger import setup_logging
+
+logger = setup_logging("ubuntu_sso.utils.webclient.timestamp")
+NOCACHE_HEADERS = {"Cache-Control": "no-cache"}
+
+
+class TimestampChecker(object):
+    """A timestamp that's regularly checked with a server."""
+
+    CHECKING_INTERVAL = 60 * 60  # in seconds
+    ERROR_INTERVAL = 30  # in seconds
+    SERVER_IRI = u"http://one.ubuntu.com/api/time"
+
+    def __init__(self, webclient):
+        """Initialize this instance."""
+        self.next_check = time.time()
+        self.skew = 0
+        self.webclient = webclient
+
+    @defer.inlineCallbacks
+    def get_server_date_header(self, server_iri):
+        """Get the server date using twisted webclient."""
+        response = yield self.webclient.request(server_iri, method="HEAD",
+                                                extra_headers=NOCACHE_HEADERS)
+        defer.returnValue(response.headers["Date"][0])
+
+    @defer.inlineCallbacks
+    def get_server_time(self):
+        """Get the time at the server."""
+        date_string = yield self.get_server_date_header(self.SERVER_IRI)
+        timestamp = http.stringToDatetime(date_string)
+        defer.returnValue(timestamp)
+
+    @defer.inlineCallbacks
+    def get_faithful_time(self):
+        """Get an accurate timestamp."""
+        local_time = time.time()
+        if local_time >= self.next_check:
+            try:
+                server_time = yield self.get_server_time()
+                self.next_check = local_time + self.CHECKING_INTERVAL
+                self.skew = server_time - local_time
+                logger.debug("Calculated server-local time skew:", self.skew)
+            # We just log all exceptions while trying to get the server time
+            # pylint: disable=W0703
+            except Exception, e:
+                logger.debug("Error while verifying the server time skew:", e)
+                self.next_check = local_time + self.ERROR_INTERVAL
+        logger.debug("Using corrected timestamp:",
+                  http.datetimeToString(local_time + self.skew))
+        defer.returnValue(int(local_time + self.skew))
+# timestamp_checker = TimestampChecker()

=== modified file 'ubuntu_sso/utils/webclient/txweb.py'
--- ubuntu_sso/utils/webclient/txweb.py	2012-01-04 21:26:27 +0000
+++ ubuntu_sso/utils/webclient/txweb.py	2012-01-19 00:42:23 +0000
@@ -17,25 +17,70 @@
 
 import base64
 
-from twisted.web import client, error, http
-from twisted.internet import defer
+from StringIO import StringIO
+
+from twisted.internet import defer, protocol, reactor
+from twisted.web import client, http, http_headers, iweb
+from zope.interface import implements
 
 from ubuntu_sso.utils.webclient.common import (
     BaseWebClient,
+    HeaderDict,
     Response,
     UnauthorizedError,
     WebClientError,
 )
 
 
+class StringProtocol(protocol.Protocol):
+    """Hold the stuff received in a StringIO."""
+
+    # pylint: disable=C0103
+    def __init__(self):
+        """Initialize this instance."""
+        self.deferred = defer.Deferred()
+        self.content = StringIO()
+
+    def dataReceived(self, data):
+        """Some more blocks received."""
+        self.content.write(data)
+
+    def connectionLost(self, reason=protocol.connectionDone):
+        """No more bytes available."""
+        self.deferred.callback(self.content.getvalue())
+
+
+class StringProducer(object):
+    """Simple implementation of IBodyProducer."""
+    implements(iweb.IBodyProducer)
+
+    def __init__(self, body):
+        """Initialize this instance with some bytes."""
+        self.body = body
+        self.length = len(body)
+
+    # pylint: disable=C0103
+    def startProducing(self, consumer):
+        """Start producing to the given IConsumer provider."""
+        consumer.write(self.body)
+        return defer.succeed(None)
+
+    def pauseProducing(self):
+        """In our case, do nothing."""
+
+    def stopProducing(self):
+        """In our case, do nothing."""
+
+
 class WebClient(BaseWebClient):
     """A simple web client that does not support proxies, yet."""
 
     @defer.inlineCallbacks
     def request(self, iri, method="GET", extra_headers=None,
-                oauth_credentials=None):
+                oauth_credentials=None, post_content=None):
         """Get the page, or fail trying."""
         uri = self.iri_to_uri(iri)
+
         if extra_headers:
             headers = dict(extra_headers)
         else:
@@ -52,13 +97,39 @@
             headers["Authorization"] = "Basic " + auth
 
         try:
-            result = yield client.getPage(uri, method=method, headers=headers)
-            response = Response(result)
-            defer.returnValue(response)
-        except error.Error as e:
-            if int(e.status) == http.UNAUTHORIZED:
-                raise UnauthorizedError(e.message)
-            raise WebClientError(e.message)
+            request_headers = http_headers.Headers()
+            for key, value in headers.items():
+                request_headers.addRawHeader(key, value)
+            agent = client.Agent(reactor)
+            if post_content:
+                body_producer = StringProducer(post_content)
+            else:
+                body_producer = None
+            agent_response = yield agent.request(method, uri,
+                                                 headers=request_headers,
+                                                 bodyProducer=body_producer)
+            raw_headers = agent_response.headers.getAllRawHeaders()
+            response_headers = HeaderDict(raw_headers)
+            if method.lower() != "head":
+                response_content = yield self.get_agent_content(agent_response)
+            else:
+                response_content = ""
+            if agent_response.code == http.OK:
+                defer.returnValue(Response(response_content, response_headers))
+            if agent_response.code == http.UNAUTHORIZED:
+                raise UnauthorizedError(agent_response.phrase,
+                                        response_content)
+            raise WebClientError(agent_response.phrase, response_content)
+        except WebClientError:
+            raise
+        except Exception as e:
+            raise WebClientError(e.message, e)
+
+    def get_agent_content(self, agent_response):
+        """Get the contents of an agent response."""
+        string_protocol = StringProtocol()
+        agent_response.deliverBody(string_protocol)
+        return string_protocol.deferred
 
     def force_use_proxy(self, settings):
         """Setup this webclient to use the given proxy settings."""