Merge lp:~jml/launchpadlib/ssl-creds into lp:launchpadlib

[Jonathan Lange]

Same thing as https://code.launchpad.net/~jml/lazr.restfulclient/disable-ssl-cert-verification/+merge/110503 but for launchpadlib.

The SSL certificates for launchpad.dev and dogfood are invalid, but we still want to test against them. Add an environment variable to disable certificate checking.

<jml> oh right.
<jml> I can't seem to use the API against launchpad.dev
<jml> I get an error about SSL.
<wgrant> Yeah, cert verification will break that.
<jml> httplib2.SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
<jml> wgrant: is there a work around?
<wgrant> I either hack launchpadlib to not verify the cert, or generate a new cert for launchpad.dev and tell it to trust that.
<cjwatson> I often end up hacking httplib2.
<cjwatson> (search for disable_ssl and change False defaults to True)
<cjwatson> I wish there were an environment variable override.
<cjwatson> dogfood has the same problem.
<jml> wgrant, cjwatson: thanks.
<wgrant> jml: In lazr.restfulclient._browser.RestfulHttp.__init__, add disable_ssl_certificate_validation=True to the super call
<jml> wgrant: even better, thanks.
<cjwatson> There are three sites you need to change, IME.
<cjwatson> I always forget the full list.
<cjwatson> Which is why I've started editing httplib2 instead.
<wgrant> Yeah
<wgrant> Trial and error works for me...
<cjwatson> Because not all the call sites are in the same file.
<wgrant> Yep

[Benji York (benji)]

Looks good.

It's too bad that the environment variable isn't discoverable. Maybe we could annotate SSH certificate errors with a message about how to disable validation. Or we could just whitelist the development sites with known-bad certs and the user won't see the errors to start with.

review: Approve (code)

[Jonathan Lange (jml)]

Good point re discoverability. I've annotated the exception.

[Benji York (benji)]

The extra info in the exception looks great.

=== modified file 'src/launchpadlib/credentials.py'
--- src/launchpadlib/credentials.py	2011-12-05 18:53:25 +0000
+++ src/launchpadlib/credentials.py	2012-06-18 15:52:56 +0000
@@ -61,6 +61,35 @@
 EXPLOSIVE_ERRORS = (MemoryError, KeyboardInterrupt, SystemExit)
 
 
+def _ssl_certificate_validation_disabled():
+    """Whether the user has disabled SSL certificate connection.
+
+    Some testing servers have broken certificates.  Rather than raising an
+    error, we allow an environment variable,
+    ``LP_DISABLE_SSL_CERTIFICATE_VALIDATION`` to disable the check.
+    """
+    # XXX: Copied from lazr/restfulclient/_browser.py.  Once it appears in a
+    # released version of lazr.restfulclient, depend on that new version and
+    # delete this copy.
+    return bool(
+        os.environ.get('LP_DISABLE_SSL_CERTIFICATE_VALIDATION', False))
+
+
+def _http_post(url, headers, params):
+    """POST to ``url`` with ``headers`` and a body of urlencoded ``params``.
+
+    Wraps it up to make sure we avoid the SSL certificate validation if our
+    environment tells us to.  Also, raises an error on non-200 statuses.
+    """
+    cert_disabled = _ssl_certificate_validation_disabled()
+    response, content = httplib2.Http(
+        disable_ssl_certificate_validation=cert_disabled).request(
+        url, method='POST', headers=headers, body=urlencode(params))
+    if response.status != 200:
+        raise HTTPError(response, content)
+    return response, content
+
+
 class Credentials(OAuthAuthorizer):
     """Standard credentials storage and usage class.
 
@@ -131,10 +160,7 @@
         headers = {'Referer': web_root}
         if token_format == self.DICT_TOKEN_FORMAT:
             headers['Accept'] = 'application/json'
-        response, content = httplib2.Http().request(
-            url, method='POST', headers=headers, body=urlencode(params))
-        if response.status != 200:
-            raise HTTPError(response, content)
+        response, content = _http_post(url, headers, params)
         if token_format == self.DICT_TOKEN_FORMAT:
             params = simplejson.loads(content)
             if context is not None:
@@ -172,10 +198,7 @@
             oauth_signature='&%s' % self._request_token.secret)
         url = web_root + access_token_page
         headers = {'Referer': web_root}
-        response, content = httplib2.Http().request(
-            url, method='POST', headers=headers, body=urlencode(params))
-        if response.status != 200:
-            raise HTTPError(response, content)
+        response, content = _http_post(url, headers, params)
         self.access_token = AccessToken.from_string(content)