Merge lp:~alecu/ubuntu-sso-client/proxy-integration-tests into lp:ubuntu-sso-client

[Alejandro J. Cura]

Proxy aware webclient with QtNetwork and libsoup backends, and integration tests. (LP: #884963 LP: #884968 LP: #884970 LP: #884971 LP: #911844)

Proxy aware webclient with QtNetwork and libsoup backends, and integration tests. (LP: #884963 LP: #884968 LP: #884970 LP: #884971)
This branch also forces the webclient to take unicode IRIs (instead of ascii URIs) (LP: #911844)

NOTE: this branch needs the squid support in this branch:
 * lp:~mandel/ubuntuone-dev-tools/proxy-testcase

[Alejandro J. Cura (alecu)]

Proxy aware webclient with QtNetwork and libsoup backends, and integration tests. (LP: #884963 LP: #884968 LP: #884970 LP: #884971)

[Natalia Bidart (nataliabidart)]

I have this PEP8 issue:

./ubuntu_sso/utils/webclient/tests/test_gsettings.py:56:17: E202 whitespace before ')'

review: Needs Fixing

[Manuel de la Peña (mandel)]

With squid3 the tests won't pass. The solution to this is to make the MockServer timeout the connection that the proxy has. Please take a look at ubuntuone-dev-tools for an example.

review: Needs Fixing

[Alejandro J. Cura (alecu)]

> I have this PEP8 issue:
>
> ./ubuntu_sso/utils/webclient/tests/test_gsettings.py:56:17: E202 whitespace
> before ')'

I can't reproduce that in my Precise installation.
Anyway, I'm rewritting that bit of code so it looks better.

[lp:~alecu/ubuntu-sso-client/proxy-integration-tests]

836. By Alejandro J. Cura on 2012-01-09

Forcefully disconnect squid3 that uses HTTP/1.1 and lint fixes

[Manuel de la Peña (mandel)]

I dont understand why GSETTINGS_CMDLINE is stored as a single string if you are going to do .split() with it. Isn't it more clear to store it as it will be used?

review: Needs Information

[Alejandro J. Cura (alecu)]

> I dont understand why GSETTINGS_CMDLINE is stored as a single string if you
> are going to do .split() with it. Isn't it more clear to store it as it will
> be used?

I think it's more clear to store it as just a string; to me using a list of strings for this adds noise to the code.
Also, if we ever need to change that cmdline it's easier to change it this way.

[Natalia Bidart (nataliabidart)]

Module docstring for ubuntu_sso/utils/webclient/gsettings.py says:

"""A webclient backend that uses QtNetwork."""

I would guess this is a copy&paste typo?

review: Needs Information

[lp:~alecu/ubuntu-sso-client/proxy-integration-tests]

837. By Alejandro J. Cura on 2012-01-10

fix for a docstring

[Natalia Bidart (nataliabidart)]

So, I ran the suite in windows and I'm getting:

ubuntu_sso.utils.webclient.tests.test_webclient
  WebClientTestCase
    test_get_iri ... Traceback (most recent call last):
Failure: twisted.internet.defer.TimeoutError: <ubuntu_sso.utils.webclient.tests.
test_webclient.WebClientTestCase testMethod=test_get_iri> (test_get_iri) still r
unning at 8.0 secs
[ERROR]
    test_get_iri_error ... Traceback (most recent call last):
Failure: twisted.internet.defer.TimeoutError: <ubuntu_sso.utils.webclient.tests.
test_webclient.WebClientTestCase testMethod=test_get_iri_error> (test_get_iri_er
ror) still running at 8.0 secs
[ERROR]
    test_method_head ... Traceback (most recent call last):
Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
<DelayedCall 0x425ea08 [0.22000002861s] called=0 cancelled=0 SaveSite._updateLog
DateTime()>
<DelayedCall 0x425eaf8 [5.21399998665s] called=0 cancelled=0 onTimeout(<Deferred
 at 0x425ea30>)>
[ERROR]
Traceback (most recent call last):
Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean.
Selectables:
<<class 'twisted.internet.tcp.Port'> of ubuntu_sso.utils.webclient.tests.test_we
bclient.SaveSite on 50543>
[ERROR]

review: Needs Fixing

[Alejandro J. Cura (alecu)]

I tried increasing the test timeout on that TestCase, from 8 to 28, and now all tests pass.
I don't think it's a clean solution, so tomorrow I'll be looking for a better solution.

[lp:~alecu/ubuntu-sso-client/proxy-integration-tests]

838. By Alejandro J. Cura on 2012-01-11

127.0.0.1 instead of localhost, because tests on windows timeout before resolving

[Natalia Bidart (nataliabidart)]

Looks great now!

review: Approve

[Manuel de la Peña (mandel)]

Ran the tests on Ubuntu P and Windows 7, all pass as expected. Code looks great.

review: Approve

=== modified file 'ubuntu_sso/utils/webclient/common.py'
--- ubuntu_sso/utils/webclient/common.py	2011-12-13 19:26:16 +0000
+++ ubuntu_sso/utils/webclient/common.py	2012-01-11 04:40:28 +0000
@@ -17,6 +17,7 @@
 
 import time
 
+from httplib2 import iri2uri
 from oauth import oauth
 from twisted.internet import defer
 
@@ -46,7 +47,7 @@
         self.username = username
         self.password = password
 
-    def request(self, url, method="GET", extra_headers=None,
+    def request(self, iri, method="GET", extra_headers=None,
                 oauth_credentials=None):
         """Return a deferred that will be fired with a Response object."""
         raise NotImplementedError
@@ -57,8 +58,18 @@
         # TODO: get the synchronized timestamp
         return defer.succeed(time.time())
 
+    def force_use_proxy(self, settings):
+        """Setup this webclient to use the given proxy settings."""
+        raise NotImplementedError
+
+    def iri_to_uri(self, iri):
+        """Transform a unicode iri into a ascii uri."""
+        if not isinstance(iri, unicode):
+            raise TypeError
+        return bytes(iri2uri(iri))
+
     @staticmethod
-    def build_oauth_headers(method, url, credentials, timestamp):
+    def build_oauth_headers(method, uri, credentials, timestamp):
         """Build an oauth request given some credentials."""
         consumer = oauth.OAuthConsumer(credentials["consumer_key"],
                                        credentials["consumer_secret"])
@@ -68,7 +79,7 @@
         if timestamp:
             parameters["oauth_timestamp"] = timestamp
         request = oauth.OAuthRequest.from_consumer_and_token(
-                                            http_url=url,
+                                            http_url=uri,
                                             http_method=method,
                                             parameters=parameters,
                                             oauth_consumer=consumer,

=== added file 'ubuntu_sso/utils/webclient/gsettings.py'
--- ubuntu_sso/utils/webclient/gsettings.py	1970-01-01 00:00:00 +0000
+++ ubuntu_sso/utils/webclient/gsettings.py	2012-01-11 04:40:28 +0000
@@ -0,0 +1,63 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright 2011 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Retrieve the proxy configuration from Gnome."""
+
+import subprocess
+
+GSETTINGS_CMDLINE = "gsettings list-recursively org.gnome.system.proxy"
+
+
+def get_proxy_settings():
+    """Parse the proxy settings as returned by the gsettings executable."""
+    output = subprocess.check_output(GSETTINGS_CMDLINE.split())
+    gsettings = {}
+    base_len = len("org.gnome.system.proxy.")
+    # pylint: disable=E1103
+    for line in output.split("\n"):
+        try:
+            path, key, value = line.split(" ", 2)
+        except ValueError:
+            continue
+        if value.startswith("'"):
+            parsed_value = value[1:-1]
+        elif value.startswith('['):
+            parsed_value = value
+        elif value in ('true', 'false'):
+            parsed_value = (value == 'true')
+        else:
+            parsed_value = int(value)
+        relative_key = (path + "." + key)[base_len:]
+        gsettings[relative_key] = parsed_value
+
+    mode = gsettings["mode"]
+    if mode == "none":
+        settings = {}
+    elif mode == "manual":
+        settings = {
+            "host": gsettings["http.host"],
+            "port": gsettings["http.port"],
+        }
+        if gsettings["http.use-authentication"]:
+            settings.update({
+                "username": gsettings["http.authentication-user"],
+                "password": gsettings["http.authentication-password"],
+            })
+    else:
+        # If mode is automatic the PAC javascript should be interpreted
+        # on each request. That is out of scope so it's ignored for now
+        settings = {}
+
+    return settings

=== modified file 'ubuntu_sso/utils/webclient/libsoup.py'
--- ubuntu_sso/utils/webclient/libsoup.py	2011-12-20 01:46:51 +0000
+++ ubuntu_sso/utils/webclient/libsoup.py	2012-01-11 04:40:28 +0000
@@ -17,6 +17,8 @@
 
 import httplib
 
+from collections import defaultdict
+
 from twisted.internet import defer
 
 from ubuntu_sso.utils.webclient.common import (
@@ -26,6 +28,9 @@
     WebClientError,
 )
 
+URI_ANONYMOUS_TEMPLATE = "http://{host}:{port}/"
+URI_USERNAME_TEMPLATE = "http://{username}:{password}@{host}:{port}/"
+
 
 class WebClient(BaseWebClient):
     """A webclient with a libsoup backend."""
@@ -43,7 +48,11 @@
     def _on_message(self, session, message, d):
         """Handle the result of an http message."""
         if message.status_code == httplib.OK:
-            response = Response(message.response_body.data)
+            headers = defaultdict(list)
+            response_headers = message.get_property("response-headers")
+            add_header = lambda key, value, _: headers[key].append(value)
+            response_headers.foreach(add_header, None)
+            response = Response(message.response_body.data, headers)
             d.callback(response)
         elif message.status_code == httplib.UNAUTHORIZED:
             e = UnauthorizedError(message.reason_phrase)
@@ -58,9 +67,10 @@
             auth.authenticate(self.username, self.password)
 
     @defer.inlineCallbacks
-    def request(self, url, method="GET", extra_headers=None,
+    def request(self, iri, method="GET", extra_headers=None,
                 oauth_credentials=None):
         """Return a deferred that will be fired with a Response object."""
+        uri = self.iri_to_uri(iri)
         if extra_headers:
             headers = dict(extra_headers)
         else:
@@ -68,12 +78,12 @@
 
         if oauth_credentials:
             timestamp = yield self.get_timestamp()
-            oauth_headers = self.build_oauth_headers(method, url,
+            oauth_headers = self.build_oauth_headers(method, uri,
                                                 oauth_credentials, timestamp)
             headers.update(oauth_headers)
 
         d = defer.Deferred()
-        message = self.soup.Message.new(method, url)
+        message = self.soup.Message.new(method, uri)
 
         for key, value in headers.iteritems():
             message.request_headers.append(key, value)
@@ -82,6 +92,24 @@
         response = yield d
         defer.returnValue(response)
 
+    def force_use_proxy(self, settings):
+        """Setup this webclient to use the given proxy settings."""
+        # pylint: disable=W0511
+        proxy_uri = self.get_proxy_uri(settings)
+        self.session.set_property("proxy-uri", proxy_uri)
+
+    def get_proxy_uri(self, settings):
+        """Get a Soup.URI for the proxy, or None if disabled."""
+        if "host" in settings and "port" in settings:
+            template = URI_ANONYMOUS_TEMPLATE
+            if "username" in settings and "password" in settings:
+                template = URI_USERNAME_TEMPLATE
+            uri = template.format(**settings)
+            return self.soup.URI.new(uri)
+        else:
+            # If the proxy host is not set, use no proxy
+            return None
+
     def shutdown(self):
         """End the soup session for this webclient."""
         self.session.abort()

=== modified file 'ubuntu_sso/utils/webclient/qtnetwork.py'
--- ubuntu_sso/utils/webclient/qtnetwork.py	2011-12-13 19:26:16 +0000
+++ ubuntu_sso/utils/webclient/qtnetwork.py	2012-01-11 04:40:28 +0000
@@ -15,12 +15,17 @@
 # with this program.  If not, see <http://www.gnu.org/licenses/>.
 """A webclient backend that uses QtNetwork."""
 
+import sys
+
+from collections import defaultdict
+
 from PyQt4.QtCore import (
     QCoreApplication,
     QUrl,
 )
 from PyQt4.QtNetwork import (
     QNetworkAccessManager,
+    QNetworkProxy,
     QNetworkReply,
     QNetworkRequest,
 )
@@ -32,6 +37,7 @@
     UnauthorizedError,
     WebClientError,
 )
+from ubuntu_sso.utils.webclient import gsettings
 
 
 class WebClient(BaseWebClient):
@@ -44,12 +50,22 @@
         self.nam.finished.connect(self._handle_finished)
         self.nam.authenticationRequired.connect(self._handle_authentication)
         self.replies = {}
+        self.setup_proxy()
+
+    def setup_proxy(self):
+        """Setup the proxy settings if needed."""
+        # QtNetwork knows how to use the system settings on both Win and Mac
+        if sys.platform.startswith("linux"):
+            settings = gsettings.get_proxy_settings()
+            if settings:
+                self.force_use_proxy(settings)
 
     @defer.inlineCallbacks
-    def request(self, url, method="GET", extra_headers=None,
+    def request(self, iri, method="GET", extra_headers=None,
                 oauth_credentials=None):
         """Return a deferred that will be fired with a Response object."""
-        request = QNetworkRequest(QUrl(url))
+        uri = self.iri_to_uri(iri)
+        request = QNetworkRequest(QUrl(uri))
 
         if extra_headers:
             headers = dict(extra_headers)
@@ -58,7 +74,7 @@
 
         if oauth_credentials:
             timestamp = yield self.get_timestamp()
-            oauth_headers = self.build_oauth_headers(method, url,
+            oauth_headers = self.build_oauth_headers(method, uri,
                                                 oauth_credentials, timestamp)
             headers.update(oauth_headers)
 
@@ -87,7 +103,10 @@
         d = self.replies.pop(reply)
         error = reply.error()
         if not error:
-            response = Response(reply.readAll())
+            headers = defaultdict(list)
+            for key, value in reply.rawHeaderPairs():
+                headers[str(key)].append(str(value))
+            response = Response(reply.readAll(), headers)
             d.callback(response)
         else:
             if error == QNetworkReply.AuthenticationRequiredError:
@@ -96,6 +115,15 @@
                 exception = WebClientError(reply.errorString())
             d.errback(exception)
 
+    def force_use_proxy(self, settings):
+        """Setup this webclient to use the given proxy settings."""
+        proxy = QNetworkProxy(QNetworkProxy.HttpProxy,
+                              hostName=settings.get("host", ""),
+                              port=settings.get("port", 0),
+                              user=settings.get("username", ""),
+                              password=settings.get("password", ""))
+        self.nam.setProxy(proxy)
+
     def shutdown(self):
         """Shut down all pending requests (if possible)."""
         self.nam.deleteLater()

=== added file 'ubuntu_sso/utils/webclient/tests/test_gsettings.py'
--- ubuntu_sso/utils/webclient/tests/test_gsettings.py	1970-01-01 00:00:00 +0000
+++ ubuntu_sso/utils/webclient/tests/test_gsettings.py	2012-01-11 04:40:28 +0000
@@ -0,0 +1,131 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright 2011 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Test the gsettings parser."""
+
+from twisted.trial.unittest import TestCase
+
+from ubuntu_sso.utils.webclient import gsettings
+
+# Some settings are not used as described in:
+# https://bugzilla.gnome.org/show_bug.cgi?id=648237
+
+TEMPLATE_GSETTINGS_OUTPUT = """\
+org.gnome.system.proxy autoconfig-url '{autoconfig_url}'
+org.gnome.system.proxy ignore-hosts {ignore_hosts:s}
+org.gnome.system.proxy mode '{mode}'
+org.gnome.system.proxy.ftp host '{ftp_host}'
+org.gnome.system.proxy.ftp port {ftp_port}
+org.gnome.system.proxy.http authentication-password '{auth_password}'
+org.gnome.system.proxy.http authentication-user '{auth_user}'
+org.gnome.system.proxy.http host '{http_host}'
+org.gnome.system.proxy.http port {http_port}
+org.gnome.system.proxy.http use-authentication {http_use_auth}
+org.gnome.system.proxy.https host '{https_host}'
+org.gnome.system.proxy.https port {https_port}
+org.gnome.system.proxy.socks host '{socks_host}'
+org.gnome.system.proxy.socks port {socks_port}
+"""
+
+BASE_GSETTINGS_VALUES = {
+    "autoconfig_url": "",
+    "ignore_hosts": ["localhost", "127.0.0.0/8"],
+    "mode": "none",
+    "ftp_host": "",
+    "ftp_port": 0,
+    "auth_password": "",
+    "auth_user": "",
+    "http_host": "",
+    "http_port": 0,
+    "http_use_auth": "false",
+    "https_host": "",
+    "https_port": 0,
+    "socks_host": "",
+    "socks_port": 0,
+}
+
+
+class ProxySettingsTestCase(TestCase):
+    """Test the getting of the proxy settings."""
+
+    def test_gsettings_cmdline_correct(self):
+        """The command line used to get the proxy settings is the right one."""
+        expected = "gsettings list-recursively org.gnome.system.proxy".split()
+        called = []
+
+        def append_output(args):
+            """Append the output and return some settings."""
+            called.append(args)
+            return TEMPLATE_GSETTINGS_OUTPUT.format(**BASE_GSETTINGS_VALUES)
+
+        self.patch(gsettings.subprocess, "check_output", append_output)
+        gsettings.get_proxy_settings()
+        self.assertEqual(called[0], expected)
+
+    def test_gsettings_parser_none(self):
+        """Test a parser of gsettings."""
+        expected = {}
+        fake_output = TEMPLATE_GSETTINGS_OUTPUT.format(**BASE_GSETTINGS_VALUES)
+        self.patch(gsettings.subprocess, "check_output",
+                   lambda _: fake_output)
+        ps = gsettings.get_proxy_settings()
+        self.assertEqual(ps, expected)
+
+    def test_gsettings_parser_http_anonymous(self):
+        """Test a parser of gsettings."""
+        template_values = dict(BASE_GSETTINGS_VALUES)
+        expected_host = "expected_host"
+        expected_port = 54321
+        expected = {
+            "host": expected_host,
+            "port": expected_port,
+        }
+        template_values.update({
+            "mode": "manual",
+            "http_host": expected_host,
+            "http_port": expected_port,
+        })
+        fake_output = TEMPLATE_GSETTINGS_OUTPUT.format(**template_values)
+        self.patch(gsettings.subprocess, "check_output",
+                   lambda _: fake_output)
+        ps = gsettings.get_proxy_settings()
+        self.assertEqual(ps, expected)
+
+    def test_gsettings_parser_http_authenticated(self):
+        """Test a parser of gsettings."""
+        template_values = dict(BASE_GSETTINGS_VALUES)
+        expected_host = "expected_host"
+        expected_port = 54321
+        expected_user = "carlitos"
+        expected_password = "very secret password"
+        expected = {
+            "host": expected_host,
+            "port": expected_port,
+            "username": expected_user,
+            "password": expected_password,
+        }
+        template_values.update({
+            "mode": "manual",
+            "http_host": expected_host,
+            "http_port": expected_port,
+            "auth_user": expected_user,
+            "auth_password": expected_password,
+            "http_use_auth": "true",
+        })
+        fake_output = TEMPLATE_GSETTINGS_OUTPUT.format(**template_values)
+        self.patch(gsettings.subprocess, "check_output",
+                   lambda _: fake_output)
+        ps = gsettings.get_proxy_settings()
+        self.assertEqual(ps, expected)

=== modified file 'ubuntu_sso/utils/webclient/tests/test_webclient.py'
--- ubuntu_sso/utils/webclient/tests/test_webclient.py	2011-12-14 15:29:02 +0000
+++ ubuntu_sso/utils/webclient/tests/test_webclient.py	2012-01-11 04:40:28 +0000
@@ -25,6 +25,7 @@
 from twisted.web import http, resource, server, guard
 
 from ubuntuone.devtools.testcases import TestCase
+from ubuntuone.devtools.testcases.squid import SquidTestCase
 
 from ubuntu_sso.utils import webclient
 
@@ -50,6 +51,8 @@
 GUARDED = "guarded"
 OAUTHRESOURCE = "oauthresource"
 
+WEBCLIENT_MODULE_NAME = webclient.webclient_module().__name__
+
 
 def sample_get_credentials():
     """Will return the sample credentials right now."""
@@ -109,6 +112,29 @@
         return "ERROR: Expected OAuth header not present."
 
 
+class SaveHTTPChannel(http.HTTPChannel):
+    """A save protocol to be used in tests."""
+
+    protocolInstance = None
+
+    def connectionMade(self):
+        """Keep track of the given protocol."""
+        SaveHTTPChannel.protocolInstance = self
+        http.HTTPChannel.connectionMade(self)
+
+
+class SaveSite(server.Site):
+    """A site that let us know when it closed."""
+
+    protocol = SaveHTTPChannel
+
+    def __init__(self, *args, **kwargs):
+        """Create a new instance."""
+        server.Site.__init__(self, *args, **kwargs)
+        # we disable the timeout in the tests, we will deal with it manually.
+        self.timeOut = None
+
+
 class MockWebServer(object):
     """A mock webserver for testing"""
 
@@ -134,23 +160,25 @@
                                                         [cred_factory])
         root.putChild(GUARDED, guarded_resource)
 
-        site = server.Site(root)
+        self.site = SaveSite(root)
         application = service.Application('web')
         self.service_collection = service.IServiceCollection(application)
         #pylint: disable=E1101
-        self.tcpserver = internet.TCPServer(0, site)
+        self.tcpserver = internet.TCPServer(0, self.site)
         self.tcpserver.setServiceParent(self.service_collection)
         self.service_collection.startService()
 
-    def get_url(self):
-        """Build the url for this mock server."""
+    def get_iri(self):
+        """Build the iri for this mock server."""
         #pylint: disable=W0212
         port_num = self.tcpserver._port.getHost().port
-        return "http://localhost:%d/" % port_num
+        return u"http://127.0.0.1:%d/" % port_num
 
     def stop(self):
         """Shut it down."""
         #pylint: disable=E1101
+        if self.site.protocol.protocolInstance:
+            self.site.protocol.protocolInstance.timeoutConnection()
         return self.service_collection.stopService()
 
 
@@ -206,40 +234,46 @@
         yield super(WebClientTestCase, self).setUp()
         self.ws = MockWebServer()
         self.addCleanup(self.ws.stop)
-        self.base_url = self.ws.get_url()
+        self.base_iri = self.ws.get_iri()
         self.wc = webclient.webclient_factory()
         self.addCleanup(self.wc.shutdown)
         # pylint: disable=W0511
         # TODO: skewed timestamp correction
 
     @defer.inlineCallbacks
-    def test_get_url(self):
-        """A url is successfully retrieved from the mock webserver."""
-        result = yield self.wc.request(self.base_url + SIMPLERESOURCE)
+    def test_request_takes_an_iri(self):
+        """Passing a non-unicode iri fails."""
+        d = self.wc.request(bytes(self.base_iri + SIMPLERESOURCE))
+        yield self.assertFailure(d, TypeError)
+
+    @defer.inlineCallbacks
+    def test_get_iri(self):
+        """Passing in a unicode iri works fine."""
+        result = yield self.wc.request(self.base_iri + SIMPLERESOURCE)
         self.assertEqual(SAMPLE_RESOURCE, result.content)
 
     @defer.inlineCallbacks
-    def test_get_url_error(self):
+    def test_get_iri_error(self):
         """The errback is called when there's some error."""
-        yield self.assertFailure(self.wc.request(self.base_url + THROWERROR),
+        yield self.assertFailure(self.wc.request(self.base_iri + THROWERROR),
                                  webclient.WebClientError)
 
     @defer.inlineCallbacks
     def test_unauthorized(self):
         """Detect when a request failed with the UNAUTHORIZED http code."""
-        yield self.assertFailure(self.wc.request(self.base_url + UNAUTHORIZED),
+        yield self.assertFailure(self.wc.request(self.base_iri + UNAUTHORIZED),
                                  webclient.UnauthorizedError)
 
     @defer.inlineCallbacks
     def test_method_head(self):
         """The HTTP method is used."""
-        result = yield self.wc.request(self.base_url + HEADONLY, method="HEAD")
+        result = yield self.wc.request(self.base_iri + HEADONLY, method="HEAD")
         self.assertEqual("", result.content)
 
     @defer.inlineCallbacks
     def test_send_extra_headers(self):
         """The extra_headers are sent to the server."""
-        result = yield self.wc.request(self.base_url + VERIFYHEADERS,
+        result = yield self.wc.request(self.base_iri + VERIFYHEADERS,
                                        extra_headers=SAMPLE_HEADERS)
         self.assertEqual(SAMPLE_RESOURCE, result.content)
 
@@ -249,17 +283,55 @@
         other_wc = webclient.webclient_factory(username=SAMPLE_USERNAME,
                                                password=SAMPLE_PASSWORD)
         self.addCleanup(other_wc.shutdown)
-        result = yield other_wc.request(self.base_url + GUARDED)
+        result = yield other_wc.request(self.base_iri + GUARDED)
         self.assertEqual(SAMPLE_RESOURCE, result.content)
 
     @defer.inlineCallbacks
     def test_request_is_oauth_signed(self):
         """The request is oauth signed."""
-        result = yield self.wc.request(self.base_url + OAUTHRESOURCE,
+        result = yield self.wc.request(self.base_iri + OAUTHRESOURCE,
                                        oauth_credentials=SAMPLE_CREDENTIALS)
         self.assertEqual(SAMPLE_RESOURCE, result.content)
 
 
+class BasicProxyTestCase(SquidTestCase):
+    """Test that the proxy works at all."""
+
+    @defer.inlineCallbacks
+    def setUp(self):
+        yield super(BasicProxyTestCase, self).setUp()
+        self.ws = MockWebServer()
+        self.addCleanup(self.ws.stop)
+        self.base_iri = self.ws.get_iri()
+        self.wc = webclient.webclient_factory()
+        self.addCleanup(self.wc.shutdown)
+
+    def assert_header_contains(self, headers, expected):
+        """One of the headers matching key must contain a given value."""
+        self.assertTrue(any(expected in value for value in headers))
+
+    @defer.inlineCallbacks
+    def test_anonymous_proxy_is_used(self):
+        """The anonymous proxy is used by the webclient."""
+        settings = self.get_nonauth_proxy_settings()
+        self.wc.force_use_proxy(settings)
+        result = yield self.wc.request(self.base_iri + SIMPLERESOURCE)
+        self.assert_header_contains(result.headers["Via"], "squid")
+
+    @defer.inlineCallbacks
+    def test_authenticated_proxy_is_used(self):
+        """The authenticated proxy is used by the webclient."""
+        settings = self.get_auth_proxy_settings()
+        self.wc.force_use_proxy(settings)
+        result = yield self.wc.request(self.base_iri + SIMPLERESOURCE)
+        self.assert_header_contains(result.headers["Via"], "squid")
+
+    if WEBCLIENT_MODULE_NAME.endswith(".txweb"):
+        reason = "txweb does not support proxies."
+        test_anonymous_proxy_is_used.skip = reason
+        test_authenticated_proxy_is_used.skip = reason
+
+
 class OAuthTestCase(TestCase):
     """Test for the oauth signing code."""
 

=== added file 'ubuntu_sso/utils/webclient/tests/webclient_demo.py'
--- ubuntu_sso/utils/webclient/tests/webclient_demo.py	1970-01-01 00:00:00 +0000
+++ ubuntu_sso/utils/webclient/tests/webclient_demo.py	2012-01-11 04:40:28 +0000
@@ -0,0 +1,51 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright 2011 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Sample program that uses the Qt webclient."""
+
+import sys
+from PyQt4 import QtGui
+# need to create the QApplication before installing the reactor
+QtGui.QApplication(sys.argv)
+import qt4reactor
+qt4reactor.install()
+
+from pprint import pprint
+from ubuntu_sso.utils.webclient import qtnetwork
+from twisted.internet import defer, reactor
+
+
[e-mail]
+def do_web_call():
+    """Wait for the webcall to finish and stop the reactor."""
+    webclient = qtnetwork.WebClient()
+    d = webclient.request(u"http://slashdot.org")
+    try:
+        response = yield d
+        pprint(response.headers.items())
+    finally:
+        reactor.stop()
+
+
+def main():
+    """The main function of this test script."""
+    do_web_call()
+    # pylint: disable=E1101
+    reactor.run()
+
+
+if __name__ == "__main__":
+    main()

=== modified file 'ubuntu_sso/utils/webclient/txweb.py'
--- ubuntu_sso/utils/webclient/txweb.py	2011-12-13 19:26:16 +0000
+++ ubuntu_sso/utils/webclient/txweb.py	2012-01-11 04:40:28 +0000
@@ -32,9 +32,10 @@
     """A simple web client that does not support proxies, yet."""
 
     @defer.inlineCallbacks
-    def request(self, url, method="GET", extra_headers=None,
+    def request(self, iri, method="GET", extra_headers=None,
                 oauth_credentials=None):
         """Get the page, or fail trying."""
+        uri = self.iri_to_uri(iri)
         if extra_headers:
             headers = dict(extra_headers)
         else:
@@ -42,7 +43,7 @@
 
         if oauth_credentials:
             timestamp = yield self.get_timestamp()
-            oauth_headers = self.build_oauth_headers(method, url,
+            oauth_headers = self.build_oauth_headers(method, uri,
                                                 oauth_credentials, timestamp)
             headers.update(oauth_headers)
 
@@ -51,10 +52,14 @@
             headers["Authorization"] = "Basic " + auth
 
         try:
-            result = yield client.getPage(url, method=method, headers=headers)
+            result = yield client.getPage(uri, method=method, headers=headers)
             response = Response(result)
             defer.returnValue(response)
         except error.Error as e:
             if int(e.status) == http.UNAUTHORIZED:
                 raise UnauthorizedError(e.message)
             raise WebClientError(e.message)
+
+    def force_use_proxy(self, settings):
+        """Setup this webclient to use the given proxy settings."""
+        # No proxy support in twisted.web.client