Merge lp:~brian.curtin/ubuntu-sso-client/py3-urllib into lp:ubuntu-sso-client

Proposed by Brian Curtin
Status: Merged
Approved by: dobey
Approved revision: 982
Merged at revision: 982
Proposed branch: lp:~brian.curtin/ubuntu-sso-client/py3-urllib
Merge into: lp:ubuntu-sso-client
Diff against target: 339 lines (+82/-34)
9 files modified
ubuntu_sso/keyring/__init__.py (+9/-3)
ubuntu_sso/keyring/linux.py (+10/-5)
ubuntu_sso/tests/test_account.py (+2/-2)
ubuntu_sso/utils/__init__.py (+11/-4)
ubuntu_sso/utils/webclient/common.py (+7/-1)
ubuntu_sso/utils/webclient/restful.py (+8/-2)
ubuntu_sso/utils/webclient/tests/test_restful.py (+11/-5)
ubuntu_sso/utils/webclient/tests/test_webclient.py (+14/-8)
ubuntu_sso/utils/webclient/txweb.py (+10/-4)
To merge this branch: bzr merge lp:~brian.curtin/ubuntu-sso-client/py3-urllib
Reviewer Review Type Date Requested Status
dobey (community) Approve
Alejandro J. Cura (community) Approve
Review via email: mp+112429@code.launchpad.net

Commit message

Try Python 3 urllib names and fall back to Python 2 as needed.

Description of the change

Python 3 reorganizes the urllib and urllib2 packages into a new structure, so we need to change our imports to work for both. Using the try/except dance, prefer the Python 3 way and fall back to the Python 2 way on ImportError.

In order to ease compatibility, some of the imports are changed to import names rather than the entire module, as there is no good way to write multi-version compatible code that imports the higher level names urllib or urllib2.

To post a comment you must log in.
977. By Brian Curtin

Tag 1018587

978. By Brian Curtin

Add test_webclient changes

Revision history for this message
Alejandro J. Cura (alecu) wrote :

== Python Lint Notices ==

ubuntu_sso/gtk/tests/test_gui.py:
    450: [W0511] FIXME - New GTK+ 3.5 breaks this check - see bug #1014772

ubuntu_sso/main/darwin.py:
    60: [W0511] TODO

ubuntu_sso/qt/__init__.py:
    50: [W0511] TODO

ubuntu_sso/tests/test_account.py:
    35: [W0611] Unused import urllib2

ubuntu_sso/utils/__init__.py:
    39: [F0401] Unable to import 'urllib.parse'
    39: [E0611] No name 'parse' in module 'urllib'
    40: [F0401] Unable to import 'urllib.request'
    40: [E0611] No name 'request' in module 'urllib'

ubuntu_sso/utils/webclient/common.py:
    284: [W0511] TODO
    40: [F0401] Unable to import 'urllib.parse'
    40: [E0611] No name 'parse' in module 'urllib'

ubuntu_sso/utils/webclient/restful.py:
    34: [F0401] Unable to import 'urllib.parse'
    34: [E0611] No name 'parse' in module 'urllib'

ubuntu_sso/utils/webclient/tests/test_restful.py:
    34: [F0401] Unable to import 'urllib.parse'
    34: [E0611] No name 'parse' in module 'urllib'
    128: [E1103, RestfulClientTestCase.test_restful_namespace_added_to_url] Instance of 'ParseResult' has no 'path' member (but some types could not be inferred)

ubuntu_sso/utils/webclient/tests/test_webclient.py:
    37: [F0401] Unable to import 'urllib.parse'
    37: [E0611] No name 'parse' in module 'urllib'

ubuntu_sso/utils/webclient/txweb.py:
    34: [F0401] Unable to import 'urllib.parse'
    34: [E0611] No name 'parse' in module 'urllib'
    102: [E1103, WebClient.raw_request] Instance of 'ParseResult' has no 'scheme' member (but some types could not be inferred)
    103: [E1103, WebClient.raw_request] Instance of 'ParseResult' has no 'netloc' member (but some types could not be inferred)

review: Needs Fixing
979. By Brian Curtin

Add more pylint disable/enable pairs to please Linux

Revision history for this message
Brian Curtin (brian.curtin) wrote :

This should be fixed.

Revision history for this message
Alejandro J. Cura (alecu) wrote :

== Python Lint Notices ==

ubuntu_sso/gtk/tests/test_gui.py:
    450: [W0511] FIXME - New GTK+ 3.5 breaks this check - see bug #1014772

ubuntu_sso/main/darwin.py:
    60: [W0511] TODO

ubuntu_sso/qt/__init__.py:
    50: [W0511] TODO

ubuntu_sso/tests/test_account.py:
    35: [W0611] Unused import urllib2

ubuntu_sso/utils/webclient/common.py:
    286: [W0511] TODO

ubuntu_sso/utils/webclient/tests/test_restful.py:
    130: [E1103, RestfulClientTestCase.test_restful_namespace_added_to_url] Instance of 'ParseResult' has no 'path' member (but some types could not be inferred)

ubuntu_sso/utils/webclient/txweb.py:
    104: [E1103, WebClient.raw_request] Instance of 'ParseResult' has no 'scheme' member (but some types could not be inferred)
    105: [E1103, WebClient.raw_request] Instance of 'ParseResult' has no 'netloc' member (but some types could not be inferred)

review: Needs Fixing
980. By Brian Curtin

Remove unused urllib2 import

981. By Brian Curtin

disable E1103 for an invalid complaint about a missing member

982. By Brian Curtin

Add disabling of E1103 as both Python 2 and Python 3 way of doing urlparse do have scheme and netloc attributes

Revision history for this message
Alejandro J. Cura (alecu) wrote :

+1

review: Approve
Revision history for this message
dobey (dobey) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'ubuntu_sso/keyring/__init__.py'
2--- ubuntu_sso/keyring/__init__.py 2012-04-19 19:29:15 +0000
3+++ ubuntu_sso/keyring/__init__.py 2012-07-03 21:26:20 +0000
4@@ -30,7 +30,13 @@
5
6 import socket
7 import sys
8-import urllib
9+
10+try:
11+ # pylint: disable=E0611,F0401
12+ from urllib.parse import quote
13+ # pylint: enable=E0611,F0401
14+except ImportError:
15+ from urllib import quote
16
17 from twisted.internet.defer import inlineCallbacks, returnValue
18
19@@ -58,9 +64,9 @@
20
21 def get_old_token_name(app_name):
22 """Build the token name (old style). Return an unicode."""
23- quoted_app_name = urllib.quote(app_name)
24+ quoted_app_name = quote(app_name)
25 computer_name = gethostname()
26- quoted_computer_name = urllib.quote(computer_name)
27+ quoted_computer_name = quote(computer_name)
28
29 assert isinstance(computer_name, unicode)
30 assert isinstance(quoted_computer_name, unicode)
31
32=== modified file 'ubuntu_sso/keyring/linux.py'
33--- ubuntu_sso/keyring/linux.py 2012-04-19 19:29:15 +0000
34+++ ubuntu_sso/keyring/linux.py 2012-07-03 21:26:20 +0000
35@@ -29,8 +29,13 @@
36 # files in the program, then also delete it here.
37 """Handle keys in the local kerying."""
38
39-import urllib
40-import urlparse
41+try:
42+ # pylint: disable=E0611,F0401
43+ from urllib.parse import parse_qsl, urlencode
44+ # pylint: enable=E0611,F0401
45+except ImportError:
46+ from urllib import urlencode
47+ from urlparse import parse_qsl
48
49 from twisted.internet.defer import inlineCallbacks, returnValue
50
51@@ -78,7 +83,7 @@
52 def set_credentials(self, app_name, cred):
53 """Set the credentials of the Ubuntu SSO item."""
54 # Creates the secret from the credentials
55- secret = urllib.urlencode(cred)
56+ secret = urlencode(cred)
57
58 attr = self._get_keyring_attr(app_name)
59 # Add our SSO credentials to the keyring
60@@ -95,7 +100,7 @@
61 item = yield self._find_keyring_item(app_name, attr=attr)
62 if item is not None:
63 yield self.set_credentials(app_name,
64- dict(urlparse.parse_qsl(item.secret)))
65+ dict(parse_qsl(item.secret)))
66 yield item.delete()
67
68 result = yield self._find_keyring_item(app_name)
69@@ -114,7 +119,7 @@
70 if item is not None:
71 logger.debug("Parsing secret.")
72 secret = yield item.get_value()
73- returnValue(dict(urlparse.parse_qsl(secret)))
74+ returnValue(dict(parse_qsl(secret)))
75 else:
76 # if no item found, try getting the old credentials
77 if app_name == U1_APP_NAME:
78
79=== modified file 'ubuntu_sso/tests/test_account.py'
80--- ubuntu_sso/tests/test_account.py 2012-06-22 16:55:35 +0000
81+++ ubuntu_sso/tests/test_account.py 2012-07-03 21:26:20 +0000
82@@ -32,12 +32,12 @@
83 """Tests for the SSO account code."""
84
85 import os
86-import urllib2
87
88 from twisted.trial.unittest import TestCase
89 from twisted.internet import defer
90
91 from ubuntu_sso import account
92+from ubuntu_sso import utils
93 from ubuntu_sso.account import (
94 Account,
95 AuthenticationError,
96@@ -215,7 +215,7 @@
97 self.addCleanup(f.close)
98 return f
99
100- self.patch(urllib2, 'urlopen', fake_urlopen) # fd to the path
101+ self.patch(utils, 'urlopen', fake_urlopen) # fd to the path
102 self.processor = Account()
103 self.register_kwargs = dict(email=EMAIL, password=PASSWORD,
104 displayname=NAME,
105
106=== modified file 'ubuntu_sso/utils/__init__.py'
107--- ubuntu_sso/utils/__init__.py 2012-06-22 16:12:03 +0000
108+++ ubuntu_sso/utils/__init__.py 2012-07-03 21:26:20 +0000
109@@ -32,10 +32,17 @@
110 import os
111 import sys
112 import time
113-import urllib2
114
115 from oauth import oauth
116-from urlparse import urlparse
117+
118+try:
119+ # pylint: disable=E0611,F0401
120+ from urllib.parse import urlparse
121+ from urllib.request import Request, urlopen
122+ # pylint: enable=E0611,F0401
123+except ImportError:
124+ from urlparse import urlparse
125+ from urllib2 import Request, urlopen
126
127 from twisted.internet import defer
128
129@@ -121,7 +128,7 @@
130 return result
131
132
133-class RequestHead(urllib2.Request):
134+class RequestHead(Request):
135 """A request with the method set to HEAD."""
136
137 _request_method = "HEAD"
138@@ -147,7 +154,7 @@
139 """Get the time at the server."""
140 headers = {"Cache-Control": "no-cache"}
141 request = RequestHead(self.SERVER_URL, headers=headers)
142- response = urllib2.urlopen(request)
143+ response = urlopen(request)
144 date_string = response.info()["Date"]
145 # delay import, otherwise a default reactor gets installed
146 from twisted.web import http
147
148=== modified file 'ubuntu_sso/utils/webclient/common.py'
149--- ubuntu_sso/utils/webclient/common.py 2012-06-22 16:12:03 +0000
150+++ ubuntu_sso/utils/webclient/common.py 2012-07-03 21:26:20 +0000
151@@ -35,7 +35,13 @@
152 from httplib2 import iri2uri
153 from oauth import oauth
154 from twisted.internet import defer
155-from urlparse import urlparse
156+
157+try:
158+ # pylint: disable=E0611,F0401
159+ from urllib.parse import urlparse
160+ # pylint: enable=E0611,F0401
161+except ImportError:
162+ from urlparse import urlparse
163
164 from ubuntu_sso import USER_SUCCESS, UI_PROXY_CREDS_DIALOG
165 from ubuntu_sso.logger import setup_logging
166
167=== modified file 'ubuntu_sso/utils/webclient/restful.py'
168--- ubuntu_sso/utils/webclient/restful.py 2012-06-14 20:52:29 +0000
169+++ ubuntu_sso/utils/webclient/restful.py 2012-07-03 21:26:20 +0000
170@@ -29,7 +29,13 @@
171 """A proxy-enabled restful client."""
172
173 import json
174-import urllib
175+
176+try:
177+ # pylint: disable=E0611,F0401
178+ from urllib.parse import urlencode
179+ # pylint: enable=E0611,F0401
180+except ImportError:
181+ from urllib import urlencode
182
183 from twisted.internet import defer
184
185@@ -67,7 +73,7 @@
186 params[key] = json.dumps(value)
187 namespace, operation = method.split(".")
188 params["ws.op"] = operation
189- encoded_args = urllib.urlencode(params)
190+ encoded_args = urlencode(params)
191 iri = self.service_iri + namespace
192 creds = self.oauth_credentials
193 logger.debug('Performing REST call to %r.', iri)
194
195=== modified file 'ubuntu_sso/utils/webclient/tests/test_restful.py'
196--- ubuntu_sso/utils/webclient/tests/test_restful.py 2012-06-14 20:46:24 +0000
197+++ ubuntu_sso/utils/webclient/tests/test_restful.py 2012-07-03 21:26:20 +0000
198@@ -29,7 +29,13 @@
199 """Tests for the proxy-enabled restful client."""
200
201 import logging
202-import urlparse
203+
204+try:
205+ # pylint: disable=E0611,F0401
206+ from urllib.parse import urlparse, parse_qs, parse_qsl
207+ # pylint: enable=E0611,F0401
208+except ImportError:
209+ from urlparse import urlparse, parse_qs, parse_qsl
210
211 from twisted.internet import defer
212 from ubuntuone.devtools.handlers import MementoHandler
213@@ -119,8 +125,8 @@
214 yield self.rc.restcall(SAMPLE_OPERATION, **SAMPLE_ARGS)
215 iri, _, _ = self.wc.called[0]
216 uri = iri.encode("ascii")
217- url = urlparse.urlparse(uri)
218- # pylint: disable=E1101
219+ url = urlparse(uri)
220+ # pylint: disable=E1101,E1103
221 self.assertTrue(url.path.endswith(SAMPLE_NAMESPACE),
222 "The namespace is included in url")
223
224@@ -129,7 +135,7 @@
225 """The restful method is added to the params."""
226 yield self.rc.restcall(SAMPLE_OPERATION, **SAMPLE_ARGS)
227 _, _, webcall_kwargs = self.wc.called[0]
228- wc_params = urlparse.parse_qs(webcall_kwargs["post_content"])
229+ wc_params = parse_qs(webcall_kwargs["post_content"])
230 self.assertEqual(wc_params["ws.op"][0], SAMPLE_METHOD)
231
232 @defer.inlineCallbacks
233@@ -137,7 +143,7 @@
234 """The keyword arguments are used as json in the webcall."""
235 yield self.rc.restcall(SAMPLE_OPERATION, **SAMPLE_ARGS)
236 _, _, webcall_kwargs = self.wc.called[0]
237- params = urlparse.parse_qsl(webcall_kwargs["post_content"])
238+ params = parse_qsl(webcall_kwargs["post_content"])
239 result = {}
240 for key, value in params:
241 if key == "ws.op":
242
243=== modified file 'ubuntu_sso/utils/webclient/tests/test_webclient.py'
244--- ubuntu_sso/utils/webclient/tests/test_webclient.py 2012-06-22 16:51:06 +0000
245+++ ubuntu_sso/utils/webclient/tests/test_webclient.py 2012-07-03 21:26:20 +0000
246@@ -32,15 +32,21 @@
247 import os
248 import shutil
249 import sys
250-import urllib
251+
252+try:
253+ # pylint: disable=E0611,F0401
254+ from urllib.parse import (urlencode, unquote, urlparse,
255+ urljoin, parse_qsl)
256+ # pylint: enable=E0611,F0401
257+except ImportError:
258+ from urllib import urlencode, unquote
259+ from urlparse import urlparse, urljoin, parse_qsl
260
261 from OpenSSL import crypto
262 from socket import gethostname
263 from twisted.cred import checkers, portal
264 from twisted.internet import defer
265 from twisted.web import guard, http, resource
266-from urllib2 import urlparse
267-
268 from ubuntuone.devtools.handlers import MementoHandler
269 from ubuntuone.devtools.testcases import TestCase
270 from ubuntuone.devtools.testcases.squid import SquidTestCase
271@@ -326,7 +332,7 @@
272 @defer.inlineCallbacks
273 def test_post_with_args(self):
274 """Test a post request with arguments."""
275- args = urllib.urlencode(SAMPLE_POST_PARAMS)
276+ args = urlencode(SAMPLE_POST_PARAMS)
277 iri = self.base_iri + VERIFYPOSTPARAMS + "?" + args
278 headers = {
279 "content-type": "application/x-www-form-urlencoded",
280@@ -743,7 +749,7 @@
281 method, params_string = header.split(" ", 1)
282 for p in params_string.split(","):
283 k, v = p.strip().split("=")
284- params[k] = urllib.unquote(v[1:-1])
285+ params[k] = unquote(v[1:-1])
286
287 return method, params
288
289@@ -804,12 +810,12 @@
290 result = yield self.wc.build_signed_iri(iri, SAMPLE_CREDENTIALS,
291 parameters=params)
292
293- scheme, netloc, path, params, query, _ = urlparse.urlparse(result)
294+ scheme, netloc, path, params, query, _ = urlparse(result)
295
296- self.assertEqual(urlparse.urljoin(scheme + '://' + netloc, path), uri)
297+ self.assertEqual(urljoin(scheme + '://' + netloc, path), uri)
298
299 expected = request.parameters
300- actual = dict(urlparse.parse_qsl(query))
301+ actual = dict(parse_qsl(query))
302 for i in ('oauth_nonce', 'oauth_signature'):
303 expected.pop(i)
304 actual.pop(i)
305
306=== modified file 'ubuntu_sso/utils/webclient/txweb.py'
307--- ubuntu_sso/utils/webclient/txweb.py 2012-04-23 17:49:31 +0000
308+++ ubuntu_sso/utils/webclient/txweb.py 2012-07-03 21:26:20 +0000
309@@ -29,7 +29,13 @@
310 """A webclient backend that uses twisted.web.client."""
311
312 import base64
313-import urlparse
314+
315+try:
316+ # pylint: disable=E0611,F0401
317+ from urllib.parse import urlparse
318+ # pylint: enable=E0611,F0401
319+except ImportError:
320+ from urlparse import urlparse
321
322 from twisted.internet import defer
323
324@@ -92,12 +98,12 @@
325 # delay import, otherwise a default reactor gets installed
326 from twisted.web import error
327
328- parsed_url = urlparse.urlparse(uri)
329+ parsed_url = urlparse(uri)
330
331- # pylint: disable=E1101
332+ # pylint: disable=E1101,E1103
333 https = parsed_url.scheme == "https"
334 host = parsed_url.netloc.split(":")[0]
335- # pylint: enable=E1101
336+ # pylint: enable=E1101,E1103
337 if parsed_url.port is None:
338 port = 443 if https else 80
339 else:

Subscribers

People subscribed via source and target branches