Merge lp:~nataliabidart/ubuntuone-client/stable-3-0-update-2.99.91 into lp:ubuntuone-client/stable-3-0
- stable-3-0-update-2.99.91
- Merge into stable-3-0
Status: | Merged | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Approved by: | Natalia Bidart | ||||||||||||||||||||
Approved revision: | 1178 | ||||||||||||||||||||
Merged at revision: | 1177 | ||||||||||||||||||||
Proposed branch: | lp:~nataliabidart/ubuntuone-client/stable-3-0-update-2.99.91 | ||||||||||||||||||||
Merge into: | lp:ubuntuone-client/stable-3-0 | ||||||||||||||||||||
Diff against target: |
2909 lines (+2139/-257) 25 files modified
Makefile.am (+3/-1) bin/u1sdtool (+1/-1) bin/ubuntuone-proxy-tunnel (+24/-0) contrib/testing/testcase.py (+15/-1) data/source_ubuntuone-client.py (+2/-29) docs/Makefile.am (+0/-1) docs/man/u1sdtool.1 (+1/-1) docs/man/ubuntuone-preferences.1 (+0/-16) tests/platform/linux/test_notification.py (+6/-0) tests/proxy/__init__.py (+150/-0) tests/proxy/ssl/dummy.cert (+19/-0) tests/proxy/ssl/dummy.key (+16/-0) tests/proxy/test_tunnel_client.py (+212/-0) tests/proxy/test_tunnel_server.py (+655/-0) tests/syncdaemon/test_action_queue.py (+102/-137) tests/syncdaemon/test_tunnel_runner.py (+128/-0) ubuntuone/platform/constants.py (+5/-0) ubuntuone/platform/linux/notification.py (+3/-0) ubuntuone/proxy/__init__.py (+14/-0) ubuntuone/proxy/common.py (+60/-0) ubuntuone/proxy/logger.py (+36/-0) ubuntuone/proxy/tunnel_client.py (+181/-0) ubuntuone/proxy/tunnel_server.py (+376/-0) ubuntuone/syncdaemon/action_queue.py (+52/-70) ubuntuone/syncdaemon/tunnel_runner.py (+78/-0) |
||||||||||||||||||||
To merge this branch: | bzr merge lp:~nataliabidart/ubuntuone-client/stable-3-0-update-2.99.91 | ||||||||||||||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Alejandro J. Cura (community) | Approve | ||
Roberto Alsina (community) | Approve | ||
Review via email: mp+98535@code.launchpad.net |
Commit message
- Updating from trunk up to revno 1213:
[ Alejandro J. Cura <email address hidden> ]
- Only allow connections that provide the right cookie thru
the tunnel (LP: #929207).
- Use proxy credentials from the keyring (LP: #929207)
- QNetwork must use the proxy built; forward disconnections in the
tunnel client.
- Use the txweb webclient from sso for webcalls, so they can be
proxied too (LP: #929207, LP: #929212).
- Tunnel storage protocol if proxy enabled in system settings
(LP: #929208).
- A proxy tunnel process to be started when proxies are enabled
(LP: #929207).
- A client for the proxy tunnel, to be used by syncdaemon
(LP: #929207).
- The infrastructure for a QtNetwork based process to tunnel
syncdaemon traffic thru proxies (LP: #929207).
[ Roberto Alsina <email address hidden> ]
- Fix tunnel spawning code so that it works on windows.
[ Rodney Dawes <email address hidden> ]
- Don't attach old useless log files, and clean up the report a
little (LP: #956407).
- Fix the typo in the u1sdtool --list-shared help text and man page
(LP: #682954).
- Don't install the really old preferences man file we don't need.
- Set the transient hint on the notifications to True
(LP: #887369).
Description of the change
Roberto Alsina (ralsina) : | # |
Preview Diff
1 | === modified file 'Makefile.am' |
2 | --- Makefile.am 2012-02-21 20:34:55 +0000 |
3 | +++ Makefile.am 2012-03-20 23:05:30 +0000 |
4 | @@ -19,6 +19,7 @@ |
5 | |
6 | libexec_SCRIPTS = \ |
7 | bin/ubuntuone-syncdaemon \ |
8 | + bin/ubuntuone-proxy-tunnel \ |
9 | bin/ubuntuone-login |
10 | |
11 | clientdefsdir = $(pythonpkgdir)/ubuntuone |
12 | @@ -53,7 +54,8 @@ |
13 | test: logging.conf $(clientdefs_DATA) Makefile |
14 | echo "$(PYTHONPATH)" |
15 | if test "x$(builddir)" == "x$(srcdir)"; then \ |
16 | - PYTHONPATH="$(PYTHONPATH)" u1trial -r $(REACTOR) -p tests/platform/windows tests; \ |
17 | + PYTHONPATH="$(PYTHONPATH)" u1trial -r $(REACTOR) -p tests/platform/windows,tests/proxy tests; \ |
18 | + PYTHONPATH="$(PYTHONPATH)" u1trial -r qt4 -p tests/platform/windows tests/proxy; \ |
19 | fi |
20 | rm -rf _trial_temp |
21 | |
22 | |
23 | === modified file 'bin/u1sdtool' |
24 | --- bin/u1sdtool 2012-01-26 00:46:08 +0000 |
25 | +++ bin/u1sdtool 2012-03-20 23:05:30 +0000 |
26 | @@ -242,7 +242,7 @@ |
27 | help="Share PATH to USER. ") |
28 | parser.add_option("", "--list-shared", dest="list_shared", |
29 | action="store_true", |
30 | - help="List the shared path's/shares offered. ") |
31 | + help="List the shared paths/shares offered. ") |
32 | parser.add_option("", "--create-folder", dest="create_folder", |
33 | metavar="PATH", |
34 | help="Create user defined folder in the specified path") |
35 | |
36 | === added file 'bin/ubuntuone-proxy-tunnel' |
37 | --- bin/ubuntuone-proxy-tunnel 1970-01-01 00:00:00 +0000 |
38 | +++ bin/ubuntuone-proxy-tunnel 2012-03-20 23:05:30 +0000 |
39 | @@ -0,0 +1,24 @@ |
40 | +#!/usr/bin/python |
41 | +# |
42 | +# Copyright 2012 Canonical Ltd. |
43 | +# |
44 | +# This program is free software: you can redistribute it and/or modify it |
45 | +# under the terms of the GNU General Public License version 3, as published |
46 | +# by the Free Software Foundation. |
47 | +# |
48 | +# This program is distributed in the hope that it will be useful, but |
49 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
50 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
51 | +# PURPOSE. See the GNU General Public License for more details. |
52 | +# |
53 | +# You should have received a copy of the GNU General Public License along |
54 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
55 | + |
56 | +"""Ubuntu One tunnel for proxy support.""" |
57 | + |
58 | +import sys |
59 | + |
60 | +from ubuntuone.proxy.tunnel_server import main |
61 | + |
62 | +if __name__ == "__main__": |
63 | + main(sys.argv) |
64 | |
65 | === modified file 'contrib/testing/testcase.py' |
66 | --- contrib/testing/testcase.py 2012-02-09 21:57:00 +0000 |
67 | +++ contrib/testing/testcase.py 2012-03-20 23:05:30 +0000 |
68 | @@ -28,7 +28,7 @@ |
69 | from collections import defaultdict |
70 | from functools import wraps |
71 | |
72 | -from twisted.internet import defer |
73 | +from twisted.internet import defer, reactor |
74 | from twisted.trial.unittest import TestCase as TwistedTestCase |
75 | from ubuntuone.devtools.testcases import skipIfOS |
76 | from zope.interface import implements |
77 | @@ -267,6 +267,17 @@ |
78 | return defer.succeed(True) |
79 | |
80 | |
81 | +class FakeTunnelRunner(object): |
82 | + """A fake proxy.tunnel_client.TunnelRunner.""" |
83 | + |
84 | + def __init__(self, *args): |
85 | + """Fake a proxy tunnel.""" |
86 | + |
87 | + def get_client(self): |
88 | + """Always return the reactor.""" |
89 | + return defer.succeed(reactor) |
90 | + |
91 | + |
92 | class BaseTwistedTestCase(TwistedTestCase): |
93 | """Base TestCase with helper methods to handle temp dir. |
94 | |
95 | @@ -276,6 +287,7 @@ |
96 | makedirs(path): support read-only shares |
97 | """ |
98 | MAX_FILENAME = 32 # some platforms limit lengths of filenames |
99 | + tunnel_runner_class = FakeTunnelRunner |
100 | |
101 | def mktemp(self, name='temp'): |
102 | """ Customized mktemp that accepts an optional name argument. """ |
103 | @@ -380,6 +392,8 @@ |
104 | # Patch the user home |
105 | self.home_dir = self.mktemp('ubuntuonehacker') |
106 | self.patch(platform, "xdg_home", self.home_dir) |
107 | + self.patch(action_queue.tunnel_runner, "TunnelRunner", |
108 | + self.tunnel_runner_class) |
109 | |
110 | |
111 | class FakeMainTestCase(BaseTwistedTestCase): |
112 | |
113 | === modified file 'data/source_ubuntuone-client.py' |
114 | --- data/source_ubuntuone-client.py 2011-02-09 14:40:29 +0000 |
115 | +++ data/source_ubuntuone-client.py 2012-03-20 23:05:30 +0000 |
116 | @@ -1,6 +1,6 @@ |
117 | # Apport integration for Ubuntu One client |
118 | # |
119 | -# Copyright 2009 Canonical Ltd. |
120 | +# Copyright 2009-2012 Canonical Ltd. |
121 | # |
122 | # This program is free software: you can redistribute it and/or modify it |
123 | # under the terms of the GNU General Public License version 3, as published |
124 | @@ -17,7 +17,7 @@ |
125 | # pylint: disable-msg=F0401,C0103 |
126 | # shut up about apport. We know. We aren't going to backport it for pqm |
127 | import apport |
128 | -from apport.hookutils import attach_file_if_exists, packaging |
129 | +from apport.hookutils import attach_file_if_exists |
130 | import os.path |
131 | from xdg.BaseDirectory import xdg_cache_home, xdg_config_home |
132 | |
133 | @@ -28,11 +28,8 @@ |
134 | u1_client_log = os.path.join(u1_log_path, "syncdaemon.log") |
135 | u1_except_log = os.path.join(u1_log_path, "syncdaemon-exceptions.log") |
136 | u1_invalidnames_log = os.path.join(u1_log_path, "syncdaemon-invalid-names.log") |
137 | -u1_oauth_log = os.path.join(u1_log_path, "oauth-login.log") |
138 | -u1_prefs_log = os.path.join(u1_log_path, "u1-prefs.log") |
139 | u1_sd_conf = os.path.join("etc", "xdg", "ubuntuone", "syncdaemon.conf") |
140 | u1_usersd_conf = os.path.join(u1_user_config_path, "syncdaemon.conf") |
141 | -u1_user_conf = os.path.join(u1_user_config_path, "ubuntuone-client.conf") |
142 | |
143 | |
144 | def add_info(report): |
145 | @@ -41,35 +38,11 @@ |
146 | "UbuntuOneSyncdaemonExceptionsLog") |
147 | attach_file_if_exists(report, u1_invalidnames_log, |
148 | "UbuntuOneSyncdaemonInvalidNamesLog") |
149 | - attach_file_if_exists(report, u1_oauth_log, |
150 | - "UbuntuOneOAuthLoginLog") |
151 | - attach_file_if_exists(report, u1_prefs_log, |
152 | - "UbuntuOnePreferencesLog") |
153 | attach_file_if_exists(report, u1_usersd_conf, |
154 | "UbuntuOneUserSyncdaemonConfig") |
155 | attach_file_if_exists(report, u1_sd_conf, |
156 | "UbuntuOneSyncdaemonConfig") |
157 | - attach_file_if_exists(report, u1_user_conf, |
158 | - "UbuntuOneClientConfig") |
159 | |
160 | if not apport.packaging.is_distro_package(report['Package'].split()[0]): |
161 | report['ThirdParty'] = 'True' |
162 | report['CrashDB'] = 'ubuntuone' |
163 | - |
164 | - packages = ['ubuntuone-client', |
165 | - 'python-ubuntuone-client', |
166 | - 'ubuntuone-client-tools', |
167 | - 'ubuntuone-client-gnome', |
168 | - 'python-ubuntuone-storageprotocol', |
169 | - 'ubuntuone-ppa-beta'] |
170 | - |
171 | - versions = '' |
172 | - for package in packages: |
173 | - try: |
174 | - version = packaging.get_version(package) |
175 | - except ValueError: |
176 | - version = 'N/A' |
177 | - if version is None: |
178 | - version = 'N/A' |
179 | - versions += '%s %s\n' % (package, version) |
180 | - report['UbuntuOneClientPackages'] = versions |
181 | |
182 | === modified file 'docs/Makefile.am' |
183 | --- docs/Makefile.am 2011-02-09 14:40:29 +0000 |
184 | +++ docs/Makefile.am 2012-03-20 23:05:30 +0000 |
185 | @@ -2,7 +2,6 @@ |
186 | |
187 | manfilesdir = $(mandir)/man1 |
188 | manfiles_DATA = \ |
189 | - man/ubuntuone-preferences.1 \ |
190 | man/u1sdtool.1 |
191 | |
192 | EXTRA_DIST=$(manfiles_DATA) |
193 | |
194 | === modified file 'docs/man/u1sdtool.1' |
195 | --- docs/man/u1sdtool.1 2011-07-02 22:37:52 +0000 |
196 | +++ docs/man/u1sdtool.1 2012-03-20 23:05:30 +0000 |
197 | @@ -131,7 +131,7 @@ |
198 | .RE |
199 | .TP |
200 | \fB\-\-list-shared\fR |
201 | -List the shared path's/shares offered. |
202 | +List the shared paths/shares offered. |
203 | .TP |
204 | \fB\-\-create-folder\fR=\fIPATH\fR |
205 | Create user defined folder in the specified path |
206 | |
207 | === removed file 'docs/man/ubuntuone-preferences.1' |
208 | --- docs/man/ubuntuone-preferences.1 2010-02-04 23:47:35 +0000 |
209 | +++ docs/man/ubuntuone-preferences.1 1970-01-01 00:00:00 +0000 |
210 | @@ -1,16 +0,0 @@ |
211 | -.TH UBUNTUONE-CLIENT-PREFERENCES 1 |
212 | - |
213 | -.SH NAME |
214 | -ubuntuone-client-preferences \- A dialog for configuring Ubuntu One |
215 | - |
216 | -.SH SYNOPSYS |
217 | -.B ubutuone-client-preferences |
218 | - |
219 | -.SH DESCRIPTION |
220 | -This manual page briefly documents the |
221 | -.BR ubuntuone-client-preferences |
222 | -process, which provides a configuration dialog for configuring |
223 | -Ubuntu One file sharing. |
224 | - |
225 | -.SH AUTHOR |
226 | -This manual page was written by Rodney Dawes <rodney.dawes@canonical.com> |
227 | |
228 | === modified file 'tests/platform/linux/test_notification.py' |
229 | --- tests/platform/linux/test_notification.py 2012-01-26 19:28:55 +0000 |
230 | +++ tests/platform/linux/test_notification.py 2012-03-20 23:05:30 +0000 |
231 | @@ -73,6 +73,7 @@ |
232 | self._set_up_mock_notify(FAKE_TITLE, FAKE_MESSAGE, ICON_NAME) |
233 | mock_notification = self.mocker.mock() |
234 | self.mocker.result(mock_notification) |
235 | + mock_notification.set_hint('transient', True) |
236 | mock_notification.show() |
237 | self.mocker.replay() |
238 | Notification(FAKE_APP_NAME).send_notification(FAKE_TITLE, FAKE_MESSAGE) |
239 | @@ -82,9 +83,11 @@ |
240 | self._set_up_mock_notify(FAKE_TITLE, FAKE_MESSAGE, ICON_NAME) |
241 | mock_notification = self.mocker.mock() |
242 | self.mocker.result(mock_notification) |
243 | + mock_notification.set_hint('transient', True) |
244 | mock_notification.show() |
245 | mock_notification.update( |
246 | FAKE_TITLE + '2', FAKE_MESSAGE + '2', ICON_NAME) |
247 | + mock_notification.set_hint('transient', True) |
248 | mock_notification.show() |
249 | self.mocker.replay() |
250 | notifier = Notification(FAKE_APP_NAME) |
251 | @@ -96,6 +99,7 @@ |
252 | self._set_up_mock_notify(FAKE_TITLE, FAKE_MESSAGE, FAKE_ICON) |
253 | mock_notification = self.mocker.mock() |
254 | self.mocker.result(mock_notification) |
255 | + mock_notification.set_hint('transient', True) |
256 | mock_notification.show() |
257 | self.mocker.replay() |
258 | Notification(FAKE_APP_NAME).send_notification( |
259 | @@ -107,9 +111,11 @@ |
260 | mock_notification = self.mocker.mock() |
261 | self.mocker.result(mock_notification) |
262 | mock_notification.set_hint_string('x-canonical-append', '') |
263 | + mock_notification.set_hint('transient', True) |
264 | mock_notification.show() |
265 | mock_notification.update(FAKE_TITLE, FAKE_APPENDAGE, ICON_NAME) |
266 | mock_notification.set_hint_string('x-canonical-append', '') |
267 | + mock_notification.set_hint('transient', True) |
268 | mock_notification.show() |
269 | self.mocker.replay() |
270 | notifier = Notification(FAKE_APP_NAME) |
271 | |
272 | === added directory 'tests/proxy' |
273 | === added file 'tests/proxy/__init__.py' |
274 | --- tests/proxy/__init__.py 1970-01-01 00:00:00 +0000 |
275 | +++ tests/proxy/__init__.py 2012-03-20 23:05:30 +0000 |
276 | @@ -0,0 +1,150 @@ |
277 | +# Copyright 2012 Canonical Ltd. |
278 | +# |
279 | +# This program is free software: you can redistribute it and/or modify it |
280 | +# under the terms of the GNU General Public License version 3, as published |
281 | +# by the Free Software Foundation. |
282 | +# |
283 | +# This program is distributed in the hope that it will be useful, but |
284 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
285 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
286 | +# PURPOSE. See the GNU General Public License for more details. |
287 | +# |
288 | +# You should have received a copy of the GNU General Public License along |
289 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
290 | +"""Tests for the Ubuntu One proxy support.""" |
291 | + |
292 | +from os import path |
293 | +from StringIO import StringIO |
294 | + |
295 | +from twisted.application import internet, service |
296 | +from twisted.internet import defer, ssl |
297 | +from twisted.web import http, resource, server |
298 | + |
299 | +SAMPLE_CONTENT = "hello world!" |
300 | +SIMPLERESOURCE = "simpleresource" |
301 | +DUMMY_KEY_FILENAME = "dummy.key" |
302 | +DUMMY_CERT_FILENAME = "dummy.cert" |
303 | +FAKE_COOKIE = "fa:ke:co:ok:ie" |
304 | + |
305 | + |
306 | +class SaveHTTPChannel(http.HTTPChannel): |
307 | + """A save protocol to be used in tests.""" |
308 | + |
309 | + protocolInstance = None |
310 | + |
311 | + # pylint: disable=C0103 |
312 | + def connectionMade(self): |
313 | + """Keep track of the given protocol.""" |
314 | + SaveHTTPChannel.protocolInstance = self |
315 | + http.HTTPChannel.connectionMade(self) |
316 | + |
317 | + |
318 | +class SaveSite(server.Site): |
319 | + """A site that let us know when it's closed.""" |
320 | + |
321 | + protocol = SaveHTTPChannel |
322 | + |
323 | + def __init__(self, *args, **kwargs): |
324 | + """Create a new instance.""" |
325 | + server.Site.__init__(self, *args, **kwargs) |
326 | + # we disable the timeout in the tests, we will deal with it manually. |
327 | + # pylint: disable=C0103 |
328 | + self.timeOut = None |
329 | + |
330 | + |
331 | +class BaseMockWebServer(object): |
332 | + """A mock webserver for testing""" |
333 | + |
334 | + def __init__(self): |
335 | + """Start up this instance.""" |
336 | + self.root = self.get_root_resource() |
337 | + self.site = SaveSite(self.root) |
338 | + application = service.Application('web') |
339 | + self.service_collection = service.IServiceCollection(application) |
340 | + #pylint: disable=E1101 |
341 | + self.tcpserver = internet.TCPServer(0, self.site) |
342 | + self.tcpserver.setServiceParent(self.service_collection) |
343 | + self.sslserver = internet.SSLServer(0, self.site, self.get_context()) |
344 | + self.sslserver.setServiceParent(self.service_collection) |
345 | + self.service_collection.startService() |
346 | + |
347 | + def get_dummy_path(self, filename): |
348 | + """Path pointing at the dummy certificate files.""" |
349 | + base_path = path.dirname(__file__) |
350 | + return path.join(base_path, "ssl", filename) |
351 | + |
352 | + def get_context(self): |
353 | + """Return an ssl context.""" |
354 | + key_path = self.get_dummy_path(DUMMY_KEY_FILENAME) |
355 | + cert_path = self.get_dummy_path(DUMMY_CERT_FILENAME) |
356 | + return ssl.DefaultOpenSSLContextFactory(key_path, cert_path) |
357 | + |
358 | + def get_root_resource(self): |
359 | + """Get the root resource with all the children.""" |
360 | + raise NotImplementedError |
361 | + |
362 | + def get_iri(self): |
363 | + """Build the iri for this mock server.""" |
364 | + #pylint: disable=W0212 |
365 | + port_num = self.tcpserver._port.getHost().port |
366 | + return u"http://0.0.0.0:%d/" % port_num |
367 | + |
368 | + def get_ssl_iri(self): |
369 | + """Build the iri for the ssl mock server.""" |
370 | + #pylint: disable=W0212 |
371 | + port_num = self.sslserver._port.getHost().port |
372 | + return u"https://0.0.0.0:%d/" % port_num |
373 | + |
374 | + def stop(self): |
375 | + """Shut it down.""" |
376 | + #pylint: disable=E1101 |
377 | + if self.site.protocol.protocolInstance: |
378 | + self.site.protocol.protocolInstance.timeoutConnection() |
379 | + return self.service_collection.stopService() |
380 | + |
381 | + |
382 | +class SimpleResource(resource.Resource): |
383 | + """A simple web resource.""" |
384 | + |
385 | + def __init__(self): |
386 | + """Initialize this mock resource.""" |
387 | + resource.Resource.__init__(self) |
388 | + self.rendered = defer.Deferred() |
389 | + |
390 | + def render_GET(self, request): |
391 | + """Make a bit of html out of the resource's content.""" |
392 | + if not self.rendered.called: |
393 | + self.rendered.callback(None) |
394 | + return SAMPLE_CONTENT |
395 | + |
396 | + |
397 | +class MockWebServer(BaseMockWebServer): |
398 | + """A mock webserver.""" |
399 | + |
400 | + def __init__(self): |
401 | + """Initialize this mock server.""" |
402 | + self.simple_resource = SimpleResource() |
403 | + super(MockWebServer, self).__init__() |
404 | + |
405 | + def get_root_resource(self): |
406 | + """Get the root resource with all the children.""" |
407 | + root = resource.Resource() |
408 | + root.putChild(SIMPLERESOURCE, self.simple_resource) |
409 | + return root |
410 | + |
411 | + |
412 | +class FakeTransport(StringIO): |
413 | + """A fake transport that stores everything written to it.""" |
414 | + |
415 | + connected = True |
416 | + disconnecting = False |
417 | + cookie = None |
418 | + |
419 | + def loseConnection(self): |
420 | + """Mark the connection as lost.""" |
421 | + self.connected = False |
422 | + self.disconnecting = True |
423 | + |
424 | + def getPeer(self): |
425 | + """Return the peer IAddress.""" |
426 | + return None |
427 | |
428 | === added directory 'tests/proxy/ssl' |
429 | === added file 'tests/proxy/ssl/dummy.cert' |
430 | --- tests/proxy/ssl/dummy.cert 1970-01-01 00:00:00 +0000 |
431 | +++ tests/proxy/ssl/dummy.cert 2012-03-20 23:05:30 +0000 |
432 | @@ -0,0 +1,19 @@ |
433 | +-----BEGIN CERTIFICATE----- |
434 | +MIIDEDCCAnmgAwIBAgIJAM/bIJ77awBCMA0GCSqGSIb3DQEBBQUAMIGgMQswCQYD |
435 | +VQQGEwJBUjETMBEGA1UECAwKRmFrZSBTdGF0ZTESMBAGA1UEBwwJRmFrZSBDaXR5 |
436 | +MRUwEwYDVQQKDAxGYWtlIENvbXBhbnkxFjAUBgNVBAsMDUZha2UgRGl2aXNpb24x |
437 | +EjAQBgNVBAMMCUZha2UgTmFtZTElMCMGCSqGSIb3DQEJARYWZmFrZUBlbWFpbC5h |
438 | +ZGRyZXNzLm5vdDAeFw0xMjAyMjIxOTI0MjBaFw0yMjAyMjMxOTI0MjBaMIGgMQsw |
439 | +CQYDVQQGEwJBUjETMBEGA1UECAwKRmFrZSBTdGF0ZTESMBAGA1UEBwwJRmFrZSBD |
440 | +aXR5MRUwEwYDVQQKDAxGYWtlIENvbXBhbnkxFjAUBgNVBAsMDUZha2UgRGl2aXNp |
441 | +b24xEjAQBgNVBAMMCUZha2UgTmFtZTElMCMGCSqGSIb3DQEJARYWZmFrZUBlbWFp |
442 | +bC5hZGRyZXNzLm5vdDCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA0hbliGty |
443 | +HwfZixU609UHBQdbfO+oObrPIrIawWX5FxD6KhX4ei23idmpyYEcXLK4ivNlT4dW |
444 | +27bvhtpf6/FBbu9e1YdwcdDNoXajr9Ia4NZJyANgo9b5UIsnyTc45NlnpZgRg5zc |
445 | +Oz7Vwwr4qf6r1ljK/I2mAO7rlpH5Ak9J+RkCAwEAAaNQME4wHQYDVR0OBBYEFLwr |
446 | +ps/JLNcfpSuuylMnkvImVvkgMB8GA1UdIwQYMBaAFLwrps/JLNcfpSuuylMnkvIm |
447 | +VvkgMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAWYDBAr0MgpnBxIne |
448 | +WRz8MX0/c7IqrEuZCYMSGnU7PoX3GdNk1Lkif1ufELKSoG8jY16CDgEl26GPxA1k |
449 | +Tho7MWSLikLbuQYJs2saF9by0Y/Mrau0auxEnpHZ7pkybeKFnrIqiNKvTVMnjo5T |
450 | +FMET5qEOKKvp9IOnezCYX1nYXyY= |
451 | +-----END CERTIFICATE----- |
452 | |
453 | === added file 'tests/proxy/ssl/dummy.key' |
454 | --- tests/proxy/ssl/dummy.key 1970-01-01 00:00:00 +0000 |
455 | +++ tests/proxy/ssl/dummy.key 2012-03-20 23:05:30 +0000 |
456 | @@ -0,0 +1,16 @@ |
457 | +-----BEGIN PRIVATE KEY----- |
458 | +MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBANIW5Yhrch8H2YsV |
459 | +OtPVBwUHW3zvqDm6zyKyGsFl+RcQ+ioV+Hott4nZqcmBHFyyuIrzZU+HVtu274ba |
460 | +X+vxQW7vXtWHcHHQzaF2o6/SGuDWScgDYKPW+VCLJ8k3OOTZZ6WYEYOc3Ds+1cMK |
461 | ++Kn+q9ZYyvyNpgDu65aR+QJPSfkZAgMBAAECgYBSxFh7TTExjmsjAyMg700LqyFc |
462 | +8CHLVJBkL9ygkqb2cmbMC8nPgJFNSqY8T5Q35OUVQNyJ31zVxJVLAF9H2c0Xy48K |
463 | +IkbS/hntyqlJYK1yfTbTHkDiweToE3Lm+55Do1TX04AyvBrwA1O/jNGi4xIlUEAy |
464 | +1Bs8MrJ1E/j/XDn9/QJBAOuhPTgG3F7bKuBrQzv98CvC5o2Txf3vLY8nL8V24b3l |
465 | +XgqzkDLhUxReBmmkGxZfKAju3+gXFvGGpbP7V8zShg8CQQDkQGs7kArFq/KR/GCh |
466 | +CAmJaDWy4LJkSqzDHoJbTrS7YuqN6X6mW1xPRnWpYSxae38fJsCpG3Vq8Mv1Zl32 |
467 | +VPZXAkEAsAeE9JYri7GwFngLgoXzJr4z/xCmmU5VetyLk7l8a6Eu4E/FKj2rE0wq |
468 | +/kDa+5ubDRFntLuLKGSu5gafUST1gQJABhdmBTfp4a6eEaFPntyNDJq4XCa8/Ao2 |
469 | +JBrrVa57Ckkwg0sI8z2a8A6sUzHhsiR7lwQ8vgaakpkMiGcL+Of5jwJAA/qX3PW+ |
470 | +9JXbjWxpgh7FHnZJNRZ8xSe47REGA7qS/nIlV9iRuf/9M+k3A5VqitfFxrjPwSyI |
471 | +rvKTYkk13dL4hg== |
472 | +-----END PRIVATE KEY----- |
473 | |
474 | === added file 'tests/proxy/test_tunnel_client.py' |
475 | --- tests/proxy/test_tunnel_client.py 1970-01-01 00:00:00 +0000 |
476 | +++ tests/proxy/test_tunnel_client.py 2012-03-20 23:05:30 +0000 |
477 | @@ -0,0 +1,212 @@ |
478 | +# -*- coding: utf8 -*- |
479 | +# |
480 | +# Copyright 2012 Canonical Ltd. |
481 | +# |
482 | +# This program is free software: you can redistribute it and/or modify it |
483 | +# under the terms of the GNU General Public License version 3, as published |
484 | +# by the Free Software Foundation. |
485 | +# |
486 | +# This program is distributed in the hope that it will be useful, but |
487 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
488 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
489 | +# PURPOSE. See the GNU General Public License for more details. |
490 | +# |
491 | +# You should have received a copy of the GNU General Public License along |
492 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
493 | +"""Tests for the proxy tunnel.""" |
494 | + |
495 | +from twisted.internet import defer, protocol, ssl |
496 | +from twisted.trial.unittest import TestCase |
497 | +from twisted.web import client |
498 | + |
499 | +from ubuntuone.devtools.testcases.squid import SquidTestCase |
500 | + |
501 | +from tests.proxy import ( |
502 | + FakeTransport, |
503 | + FAKE_COOKIE, |
504 | + MockWebServer, |
505 | + SAMPLE_CONTENT, |
506 | + SIMPLERESOURCE, |
507 | +) |
508 | +from ubuntuone.proxy import tunnel_client |
509 | +from ubuntuone.proxy.tunnel_client import CRLF, TunnelClient |
510 | +from ubuntuone.proxy.tunnel_server import TunnelServer |
511 | + |
512 | + |
513 | +FAKE_HEADER = ( |
514 | + "HTTP/1.0 200 Connected!" + CRLF + |
515 | + "Header1: value1" + CRLF + |
516 | + "Header2: value2" + CRLF + |
517 | + CRLF |
518 | +) |
519 | + |
520 | + |
521 | +class SavingProtocol(protocol.Protocol): |
522 | + """A protocol that saves all that it receives.""" |
523 | + |
524 | + def __init__(self): |
525 | + """Initialize this protocol.""" |
526 | + self.saved_data = None |
527 | + |
528 | + def connectionMade(self): |
529 | + """The connection was made, start saving.""" |
530 | + self.saved_data = [] |
531 | + |
532 | + def dataReceived(self, data): |
533 | + """Save the data received.""" |
534 | + self.saved_data.append(data) |
535 | + |
536 | + @property |
537 | + def content(self): |
538 | + """All the content so far.""" |
539 | + return "".join(self.saved_data) |
540 | + |
541 | + |
542 | +class TunnelClientProtocolTestCase(TestCase): |
543 | + """Tests for the client side tunnel protocol.""" |
544 | + |
545 | + timeout = 3 |
546 | + |
547 | + @defer.inlineCallbacks |
548 | + def setUp(self): |
549 | + """Initialize this testcase.""" |
550 | + yield super(TunnelClientProtocolTestCase, self).setUp() |
551 | + self.host, self.port = "9.9.9.9", 8765 |
552 | + fake_addr = object() |
553 | + self.cookie = FAKE_COOKIE |
554 | + self.other_proto = SavingProtocol() |
555 | + other_factory = protocol.ClientFactory() |
556 | + other_factory.buildProtocol = lambda _addr: self.other_proto |
557 | + tunnel_client_factory = tunnel_client.TunnelClientFactory(self.host, |
558 | + self.port, other_factory, self.cookie) |
559 | + tunnel_client_proto = tunnel_client_factory.buildProtocol(fake_addr) |
560 | + tunnel_client_proto.transport = FakeTransport() |
561 | + tunnel_client_proto.connectionMade() |
562 | + self.tunnel_client_proto = tunnel_client_proto |
563 | + |
564 | + def test_sends_connect_request(self): |
565 | + """Sends the expected CONNECT request.""" |
566 | + expected = tunnel_client.METHOD_LINE % (self.host, self.port) |
567 | + written = self.tunnel_client_proto.transport.getvalue() |
568 | + first_line = written.split(CRLF)[0] |
569 | + self.assertEqual(first_line + CRLF, expected) |
570 | + self.assertTrue(written.endswith(CRLF * 2), |
571 | + "Ends with a double CRLF") |
572 | + |
573 | + def test_sends_cookie_header(self): |
574 | + """Sends the expected cookie header.""" |
575 | + expected = "%s: %s" % (tunnel_client.TUNNEL_COOKIE_HEADER, self.cookie) |
576 | + written = self.tunnel_client_proto.transport.getvalue() |
577 | + headers = written.split(CRLF)[1:] |
578 | + self.assertIn(expected, headers) |
579 | + |
580 | + def test_handles_successful_connection(self): |
581 | + """A successful connection is handled.""" |
582 | + self.tunnel_client_proto.dataReceived(FAKE_HEADER) |
583 | + self.assertEqual(self.tunnel_client_proto.status_code, "200") |
584 | + |
585 | + def test_protocol_is_switched(self): |
586 | + """The protocol is switched after the headers are received.""" |
587 | + expected = (SAMPLE_CONTENT + CRLF) * 2 |
588 | + self.tunnel_client_proto.dataReceived(FAKE_HEADER + SAMPLE_CONTENT) |
589 | + self.other_proto.dataReceived(CRLF + SAMPLE_CONTENT + CRLF) |
590 | + self.assertEqual(self.other_proto.content, expected) |
591 | + |
592 | + |
593 | +class FakeOtherFactory(object): |
594 | + """A fake factory.""" |
595 | + |
596 | + def __init__(self): |
597 | + """Initialize this fake.""" |
598 | + self.started_called = None |
599 | + self.failed_called = None |
600 | + self.lost_called = None |
601 | + |
602 | + def startedConnecting(self, *args): |
603 | + """Store the call.""" |
604 | + self.started_called = args |
605 | + |
606 | + def clientConnectionFailed(self, *args): |
607 | + """Store the call.""" |
608 | + self.failed_called = args |
609 | + |
610 | + def clientConnectionLost(self, *args): |
611 | + """Store the call.""" |
612 | + self.lost_called = args |
613 | + |
614 | + |
615 | +class TunnelClientFactoryTestCase(TestCase): |
616 | + """Tests for the TunnelClientFactory.""" |
617 | + |
618 | + def test_forwards_started(self): |
619 | + """The factory forwards the startedConnecting call.""" |
620 | + fake_other_factory = FakeOtherFactory() |
621 | + tcf = tunnel_client.TunnelClientFactory(None, None, fake_other_factory, |
622 | + FAKE_COOKIE) |
623 | + fake_connector = object() |
624 | + tcf.startedConnecting(fake_connector) |
625 | + self.assertEqual(fake_other_factory.started_called, (fake_connector,)) |
626 | + |
627 | + def test_forwards_failed(self): |
628 | + """The factory forwards the clientConnectionFailed call.""" |
629 | + fake_reason = object() |
630 | + fake_other_factory = FakeOtherFactory() |
631 | + tcf = tunnel_client.TunnelClientFactory(None, None, fake_other_factory, |
632 | + FAKE_COOKIE) |
633 | + fake_connector = object() |
634 | + tcf.clientConnectionFailed(fake_connector, fake_reason) |
635 | + self.assertEqual(fake_other_factory.failed_called, |
636 | + (fake_connector, fake_reason)) |
637 | + |
638 | + def test_forwards_lost(self): |
639 | + """The factory forwards the clientConnectionLost call.""" |
640 | + fake_reason = object() |
641 | + fake_other_factory = FakeOtherFactory() |
642 | + tcf = tunnel_client.TunnelClientFactory(None, None, fake_other_factory, |
643 | + FAKE_COOKIE) |
644 | + fake_connector = object() |
645 | + tcf.clientConnectionLost(fake_connector, fake_reason) |
646 | + self.assertEqual(fake_other_factory.lost_called, |
647 | + (fake_connector, fake_reason)) |
648 | + |
649 | + |
650 | +class TunnelClientTestCase(SquidTestCase): |
651 | + """Test the client for the tunnel.""" |
652 | + |
653 | + timeout = 3 |
654 | + |
655 | + @defer.inlineCallbacks |
656 | + def setUp(self): |
657 | + """Initialize this testcase.""" |
658 | + yield super(TunnelClientTestCase, self).setUp() |
659 | + self.ws = MockWebServer() |
660 | + self.addCleanup(self.ws.stop) |
661 | + self.dest_url = self.ws.get_iri().encode("utf-8") + SIMPLERESOURCE |
662 | + self.dest_ssl_url = (self.ws.get_ssl_iri().encode("utf-8") + |
663 | + SIMPLERESOURCE) |
664 | + self.cookie = FAKE_COOKIE |
665 | + self.tunnel_server = TunnelServer(self.cookie) |
666 | + self.addCleanup(self.tunnel_server.shutdown) |
667 | + |
668 | + @defer.inlineCallbacks |
669 | + def test_connects_right(self): |
670 | + """Uses the CONNECT method on the tunnel.""" |
671 | + tunnel_client = TunnelClient("0.0.0.0", self.tunnel_server.port, |
672 | + self.cookie) |
673 | + factory = client.HTTPClientFactory(self.dest_url) |
674 | + scheme, host, port, path = client._parse(self.dest_url) |
675 | + tunnel_client.connectTCP(host, port, factory) |
676 | + result = yield factory.deferred |
677 | + self.assertEqual(result, SAMPLE_CONTENT) |
678 | + |
679 | + @defer.inlineCallbacks |
680 | + def test_starts_tls_connection(self): |
681 | + """TLS is started after connecting; control passed to the client.""" |
682 | + tunnel_client = TunnelClient("0.0.0.0", self.tunnel_server.port, |
683 | + self.cookie) |
684 | + factory = client.HTTPClientFactory(self.dest_ssl_url) |
685 | + scheme, host, port, path = client._parse(self.dest_ssl_url) |
686 | + context_factory = ssl.ClientContextFactory() |
687 | + tunnel_client.connectSSL(host, port, factory, context_factory) |
688 | + result = yield factory.deferred |
689 | + self.assertEqual(result, SAMPLE_CONTENT) |
690 | |
691 | === added file 'tests/proxy/test_tunnel_server.py' |
692 | --- tests/proxy/test_tunnel_server.py 1970-01-01 00:00:00 +0000 |
693 | +++ tests/proxy/test_tunnel_server.py 2012-03-20 23:05:30 +0000 |
694 | @@ -0,0 +1,655 @@ |
695 | +# -*- coding: utf8 -*- |
696 | +# |
697 | +# Copyright 2012 Canonical Ltd. |
698 | +# |
699 | +# This program is free software: you can redistribute it and/or modify it |
700 | +# under the terms of the GNU General Public License version 3, as published |
701 | +# by the Free Software Foundation. |
702 | +# |
703 | +# This program is distributed in the hope that it will be useful, but |
704 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
705 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
706 | +# PURPOSE. See the GNU General Public License for more details. |
707 | +# |
708 | +# You should have received a copy of the GNU General Public License along |
709 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
710 | +"""Tests for the proxy tunnel.""" |
711 | + |
712 | +from StringIO import StringIO |
713 | +from urlparse import urlparse |
714 | + |
715 | +from twisted.internet import defer, protocol, reactor |
716 | +from twisted.trial.unittest import TestCase |
717 | +from PyQt4.QtCore import QCoreApplication |
718 | +from PyQt4.QtNetwork import QAuthenticator |
719 | +from ubuntuone.devtools.testcases.squid import SquidTestCase |
720 | + |
721 | +from tests.proxy import ( |
722 | + FakeTransport, |
723 | + FAKE_COOKIE, |
724 | + MockWebServer, |
725 | + SAMPLE_CONTENT, |
726 | + SIMPLERESOURCE, |
727 | +) |
728 | +from ubuntuone.proxy import tunnel_server |
729 | +from ubuntuone.proxy.tunnel_server import CRLF |
730 | + |
731 | + |
732 | +FAKE_SESSION_TEMPLATE = ( |
733 | + "CONNECT %s HTTP/1.0" + CRLF + |
734 | + "Header1: value1" + CRLF + |
735 | + "Header2: value2" + CRLF + |
736 | + tunnel_server.TUNNEL_COOKIE_HEADER + ": %s" + CRLF + |
737 | + CRLF + |
738 | + "GET %s HTTP/1.0" + CRLF + CRLF |
739 | +) |
740 | + |
741 | +FAKE_SETTINGS = { |
742 | + "http": { |
743 | + "host": "myhost", |
744 | + "port": 8888, |
745 | + } |
746 | +} |
747 | + |
748 | +SAMPLE_HOST = "samplehost.com" |
749 | +SAMPLE_PORT = 443 |
750 | + |
751 | +FAKE_CREDS = { |
752 | + "username": "rhea", |
753 | + "password": "caracolcaracola", |
754 | +} |
755 | + |
756 | + |
757 | +class DisconnectingProtocol(protocol.Protocol): |
758 | + """A protocol that just disconnects.""" |
759 | + |
760 | + def connectionMade(self): |
761 | + """Upon connecting: just disconnect.""" |
762 | + self.transport.loseConnection() |
763 | + |
764 | + |
765 | +class DisconnectingClientFactory(protocol.ClientFactory): |
766 | + """A factory that fires a deferred on connection.""" |
767 | + |
768 | + def __init__(self): |
769 | + """Initialize this instance.""" |
770 | + self.connected = defer.Deferred() |
771 | + |
772 | + def buildProtocol(self, addr): |
773 | + """The connection was made.""" |
774 | + proto = DisconnectingProtocol() |
775 | + if not self.connected.called: |
776 | + self.connected.callback(proto) |
777 | + return proto |
778 | + |
779 | + |
780 | +class FakeProtocol(protocol.Protocol): |
781 | + """A protocol that forwards some data.""" |
782 | + |
783 | + def __init__(self, factory, data): |
784 | + """Initialize this fake.""" |
785 | + self.factory = factory |
786 | + self.data = data |
787 | + self.received_data = [] |
788 | + |
789 | + def connectionMade(self): |
790 | + """Upon connection: send the stored data.""" |
791 | + self.transport.write(self.data) |
792 | + |
793 | + def dataReceived(self, data): |
794 | + """Some data was received.""" |
795 | + self.received_data.append(data) |
796 | + |
797 | + def connectionLost(self, reason): |
798 | + """The connection was lost, return the response.""" |
799 | + response = "".join(self.received_data) |
800 | + if not self.factory.response.called: |
801 | + self.factory.response.callback(response) |
802 | + |
803 | + |
804 | +class FakeClientFactory(protocol.ClientFactory): |
805 | + """A factory that forwards some data to the protocol.""" |
806 | + |
807 | + def __init__(self, data): |
808 | + """Initialize this fake.""" |
809 | + self.data = data |
810 | + self.response = defer.Deferred() |
811 | + |
812 | + def buildProtocol(self, addr): |
813 | + """The connection was made.""" |
814 | + return FakeProtocol(self, self.data) |
815 | + |
816 | + |
817 | +class TunnelIntegrationTestCase(SquidTestCase): |
818 | + """Basic tunnel integration tests.""" |
819 | + |
820 | + timeout = 3 |
821 | + |
822 | + @defer.inlineCallbacks |
823 | + def setUp(self): |
824 | + """Initialize this testcase.""" |
825 | + yield super(TunnelIntegrationTestCase, self).setUp() |
826 | + self.ws = MockWebServer() |
827 | + self.addCleanup(self.ws.stop) |
828 | + self.dest_url = self.ws.get_iri().encode("utf-8") + SIMPLERESOURCE |
829 | + self.cookie = FAKE_COOKIE |
830 | + self.tunnel_server = tunnel_server.TunnelServer(self.cookie) |
831 | + self.addCleanup(self.tunnel_server.shutdown) |
832 | + |
833 | + def test_init(self): |
834 | + """The tunnel is started.""" |
835 | + self.assertNotEqual(self.tunnel_server.port, 0) |
836 | + |
837 | + @defer.inlineCallbacks |
838 | + def test_accepts_connections(self): |
839 | + """The tunnel accepts incoming connections.""" |
840 | + ncf = DisconnectingClientFactory() |
841 | + reactor.connectTCP("0.0.0.0", self.tunnel_server.port, ncf) |
842 | + yield ncf.connected |
843 | + |
844 | + @defer.inlineCallbacks |
845 | + def test_complete_connection(self): |
846 | + """Test from the tunnel server down.""" |
847 | + url = urlparse(self.dest_url) |
848 | + fake_session = FAKE_SESSION_TEMPLATE % ( |
849 | + url.netloc, self.cookie, url.path) |
850 | + client = FakeClientFactory(fake_session) |
851 | + reactor.connectTCP("0.0.0.0", self.tunnel_server.port, client) |
852 | + response = yield client.response |
853 | + self.assertIn(SAMPLE_CONTENT, response) |
854 | + |
855 | + |
856 | +class FakeClient(object): |
857 | + """A fake destination client.""" |
858 | + |
859 | + protocol = None |
860 | + connection_result = defer.succeed(True) |
861 | + credentials = None |
862 | + check_credentials = False |
863 | + proxy_domain = None |
864 | + |
865 | + def connect(self, hostport): |
866 | + """Establish a connection with the other end.""" |
867 | + if (self.check_credentials and |
868 | + self.protocol.proxy_credentials != FAKE_CREDS): |
869 | + self.proxy_domain = "fake domain" |
870 | + return defer.fail(tunnel_server.ProxyAuthenticationError()) |
871 | + return self.connection_result |
872 | + |
873 | + def write(self, data): |
874 | + """Write some data to the other end.""" |
875 | + if data == 'GET /simpleresource HTTP/1.0\r\n\r\n': |
876 | + self.protocol.transport.write(SAMPLE_CONTENT) |
877 | + |
878 | + def stop(self): |
879 | + """Stop this fake client.""" |
880 | + |
881 | + def close(self): |
882 | + """Reset this client.""" |
883 | + |
884 | + |
885 | + |
886 | +class ServerTunnelProtocolTestCase(SquidTestCase): |
887 | + """Tests for the ServerTunnelProtocol.""" |
888 | + |
889 | + @defer.inlineCallbacks |
890 | + def setUp(self): |
891 | + """Initialize this test instance.""" |
892 | + yield super(ServerTunnelProtocolTestCase, self).setUp() |
893 | + self.ws = MockWebServer() |
894 | + self.addCleanup(self.ws.stop) |
895 | + self.dest_url = self.ws.get_iri().encode("utf-8") + SIMPLERESOURCE |
896 | + self.transport = FakeTransport() |
897 | + self.transport.cookie = FAKE_COOKIE |
898 | + self.fake_client = FakeClient() |
899 | + self.proto = tunnel_server.ServerTunnelProtocol( |
900 | + lambda _: self.fake_client) |
901 | + self.fake_client.protocol = self.proto |
902 | + self.proto.transport = self.transport |
903 | + self.cookie_line = "%s: %s" % (tunnel_server.TUNNEL_COOKIE_HEADER, |
904 | + FAKE_COOKIE) |
905 | + |
906 | + def test_broken_request(self): |
907 | + """Broken request.""" |
908 | + self.proto.dataReceived("Broken request." + CRLF) |
909 | + self.assertTrue(self.transport.getvalue().startswith("HTTP/1.0 400 "), |
910 | + "A broken request must fail.") |
911 | + |
912 | + def test_wrong_method(self): |
913 | + """Wrong method.""" |
914 | + self.proto.dataReceived("GET http://slashdot.org HTTP/1.0" + CRLF) |
915 | + self.assertTrue(self.transport.getvalue().startswith("HTTP/1.0 405 "), |
916 | + "Using a wrong method fails.") |
917 | + |
918 | + def test_invalid_http_version(self): |
919 | + """Invalid HTTP version.""" |
920 | + self.proto.dataReceived("CONNECT 127.0.0.1:9999 HTTP/1.1" + CRLF) |
921 | + self.assertTrue(self.transport.getvalue().startswith("HTTP/1.0 505 "), |
922 | + "Invalid http version is not allowed.") |
923 | + |
924 | + def test_connection_is_established(self): |
925 | + """The response code is sent.""" |
926 | + expected = "HTTP/1.0 200 Proxy connection established" + CRLF |
927 | + self.proto.dataReceived("CONNECT 127.0.0.1:9999 HTTP/1.0" + CRLF + |
928 | + self.cookie_line + CRLF * 2) |
929 | + self.assertTrue(self.transport.getvalue().startswith(expected), |
930 | + "First line must be the response status") |
931 | + |
932 | + def test_connection_fails(self): |
933 | + """The connection to the other end fails, and it's handled.""" |
934 | + error = tunnel_server.ConnectionError() |
935 | + self.patch(self.fake_client, "connection_result", defer.fail(error)) |
936 | + expected = "HTTP/1.0 500 Connection error" + CRLF |
937 | + self.proto.dataReceived("CONNECT 127.0.0.1:9999 HTTP/1.0" + CRLF + |
938 | + self.cookie_line + CRLF * 2) |
939 | + self.assertTrue(self.transport.getvalue().startswith(expected), |
940 | + "The connection should fail at this point.") |
941 | + |
942 | + def test_headers_stored(self): |
943 | + """The request headers are stored.""" |
944 | + expected = [ |
945 | + ("Header1", "value1"), |
946 | + ("Header2", "value2"), |
947 | + ] |
948 | + self.proto.dataReceived("CONNECT 127.0.0.1:9999 HTTP/1.0" + CRLF + |
949 | + "Header1: value1" + CRLF + |
950 | + "Header2: value2" + CRLF + CRLF) |
951 | + self.assertEqual(self.proto.received_headers, expected) |
952 | + |
953 | + def test_cookie_header_present(self): |
954 | + """The cookie header must be present.""" |
955 | + self.proto.received_headers = [ |
956 | + (tunnel_server.TUNNEL_COOKIE_HEADER, FAKE_COOKIE), |
957 | + ] |
958 | + self.proto.verify_cookie() |
959 | + |
960 | + def test_cookie_header_absent(self): |
961 | + """The tunnel should refuse connections without the cookie.""" |
962 | + self.proto.received_headers = [] |
963 | + exception = self.assertRaises(tunnel_server.ConnectionError, |
964 | + self.proto.verify_cookie) |
965 | + self.assertEqual(exception.code, 418) |
966 | + |
967 | + def test_successful_connect(self): |
968 | + """A successful connect thru the tunnel.""" |
969 | + url = urlparse(self.dest_url) |
970 | + data = FAKE_SESSION_TEMPLATE % (url.netloc, self.transport.cookie, |
971 | + url.path) |
972 | + self.proto.dataReceived(data) |
973 | + lines = self.transport.getvalue().split(CRLF) |
974 | + self.assertEqual(lines[-1], SAMPLE_CONTENT) |
975 | + |
976 | + def test_header_split(self): |
977 | + """Test a header with many colons.""" |
978 | + self.proto.header_line("key: host:port") |
979 | + self.assertIn("key", dict(self.proto.received_headers)) |
980 | + |
981 | + @defer.inlineCallbacks |
982 | + def test_keyring_credentials_are_retried(self): |
983 | + """Wrong credentials are retried with values from keyring.""" |
984 | + self.fake_client.check_credentials = True |
985 | + self.patch(self.proto, "verify_cookie", lambda: None) |
986 | + self.patch(self.proto, "error_response", |
987 | + lambda code, desc: self.fail(desc)) |
988 | + self.proto.proxy_domain = "xxx" |
989 | + self.patch(tunnel_server.Keyring, "get_credentials", |
990 | + lambda _, domain: defer.succeed(FAKE_CREDS)) |
991 | + yield self.proto.headers_done() |
992 | + |
993 | + |
994 | +class FakeServerTunnelProtocol(object): |
995 | + """A fake ServerTunnelProtocol.""" |
996 | + |
997 | + def __init__(self): |
998 | + """Initialize this fake tunnel.""" |
999 | + self.response_received = defer.Deferred() |
1000 | + self.proxy_credentials = None |
1001 | + |
1002 | + def response_data_received(self, data): |
1003 | + """Fire the response deferred.""" |
1004 | + if not self.response_received.called: |
1005 | + self.response_received.callback(data) |
1006 | + |
1007 | + def remote_disconnected(self): |
1008 | + """The remote server disconnected.""" |
1009 | + |
1010 | + def proxy_auth_required(self, proxy, authenticator): |
1011 | + """Proxy credentials are needed.""" |
1012 | + if self.proxy_credentials: |
1013 | + authenticator.setUser(self.proxy_credentials["username"]) |
1014 | + authenticator.setPassword(self.proxy_credentials["password"]) |
1015 | + |
1016 | + |
1017 | +class BuildProxyTestCase(TestCase): |
1018 | + """Tests for the build_proxy function.""" |
1019 | + |
1020 | + def test_socks_is_preferred(self): |
1021 | + """Socks overrides all protocols.""" |
1022 | + settings = { |
1023 | + "http": {"host": "httphost", "port": 3128}, |
1024 | + "https": {"host": "httpshost", "port": 3129}, |
1025 | + "socks": {"host": "sockshost", "port": 1080}, |
1026 | + } |
1027 | + proxy = tunnel_server.build_proxy(settings) |
1028 | + self.assertEqual(proxy.type(), proxy.Socks5Proxy) |
1029 | + self.assertEqual(proxy.hostName(), "sockshost") |
1030 | + self.assertEqual(proxy.port(), 1080) |
1031 | + |
1032 | + def test_https_beats_http(self): |
1033 | + """HTTPS wins over HTTP, since all of SD traffic is https.""" |
1034 | + settings = { |
1035 | + "http": {"host": "httphost", "port": 3128}, |
1036 | + "https": {"host": "httpshost", "port": 3129}, |
1037 | + } |
1038 | + proxy = tunnel_server.build_proxy(settings) |
1039 | + self.assertEqual(proxy.type(), proxy.HttpProxy) |
1040 | + self.assertEqual(proxy.hostName(), "httpshost") |
1041 | + self.assertEqual(proxy.port(), 3129) |
1042 | + |
1043 | + def test_http_if_no_other_choice(self): |
1044 | + """Finally, we use the host configured for HTTP.""" |
1045 | + settings = { |
1046 | + "http": {"host": "httphost", "port": 3128}, |
1047 | + } |
1048 | + proxy = tunnel_server.build_proxy(settings) |
1049 | + self.assertEqual(proxy.type(), proxy.HttpProxy) |
1050 | + self.assertEqual(proxy.hostName(), "httphost") |
1051 | + self.assertEqual(proxy.port(), 3128) |
1052 | + |
1053 | + def test_use_noproxy_as_fallback(self): |
1054 | + """If nothing useful, revert to no proxy.""" |
1055 | + settings = {} |
1056 | + proxy = tunnel_server.build_proxy(settings) |
1057 | + self.assertEqual(proxy.type(), proxy.DefaultProxy) |
1058 | + |
1059 | + |
1060 | +class RemoteSocketTestCase(SquidTestCase): |
1061 | + """Tests for the client that connects to the other side.""" |
1062 | + |
1063 | + timeout = 3 |
1064 | + get_proxy_settings = lambda _: {} |
1065 | + |
1066 | + @defer.inlineCallbacks |
1067 | + def setUp(self): |
1068 | + """Initialize this testcase.""" |
1069 | + yield super(RemoteSocketTestCase, self).setUp() |
1070 | + self.ws = MockWebServer() |
1071 | + self.addCleanup(self.ws.stop) |
1072 | + self.dest_url = self.ws.get_iri().encode("utf-8") + SIMPLERESOURCE |
1073 | + |
1074 | + self.addCleanup(tunnel_server.QNetworkProxy.setApplicationProxy, |
1075 | + tunnel_server.QNetworkProxy.applicationProxy()) |
1076 | + settings = {"http": self.get_proxy_settings()} |
1077 | + proxy = tunnel_server.build_proxy(settings) |
1078 | + tunnel_server.QNetworkProxy.setApplicationProxy(proxy) |
1079 | + |
1080 | + def test_invalid_port(self): |
1081 | + """A request with an invalid port fails with a 400.""" |
1082 | + protocol = tunnel_server.ServerTunnelProtocol( |
1083 | + tunnel_server.RemoteSocket) |
1084 | + protocol.transport = FakeTransport() |
1085 | + protocol.dataReceived("CONNECT 127.0.0.1:wrong_port HTTP/1.0" + |
1086 | + CRLF * 2) |
1087 | + |
1088 | + status_line = protocol.transport.getvalue() |
1089 | + self.assertTrue(status_line.startswith("HTTP/1.0 400 "), |
1090 | + "The port must be an integer.") |
1091 | + |
1092 | + @defer.inlineCallbacks |
1093 | + def test_connection_is_finished_when_stopping(self): |
1094 | + """The client disconnects when requested.""" |
1095 | + fake_protocol = FakeServerTunnelProtocol() |
1096 | + client = tunnel_server.RemoteSocket(fake_protocol) |
1097 | + url = urlparse(self.dest_url) |
1098 | + yield client.connect(url.netloc) |
1099 | + yield client.stop() |
1100 | + |
1101 | + @defer.inlineCallbacks |
1102 | + def test_stop_but_never_connected(self): |
1103 | + """Stop but it was never connected.""" |
1104 | + fake_protocol = FakeServerTunnelProtocol() |
1105 | + client = tunnel_server.RemoteSocket(fake_protocol) |
1106 | + yield client.stop() |
1107 | + |
1108 | + @defer.inlineCallbacks |
1109 | + def test_client_write(self): |
1110 | + """Data written to the client is sent to the other side.""" |
1111 | + fake_protocol = FakeServerTunnelProtocol() |
1112 | + client = tunnel_server.RemoteSocket(fake_protocol) |
1113 | + self.addCleanup(client.stop) |
1114 | + url = urlparse(self.dest_url) |
1115 | + yield client.connect(url.netloc) |
1116 | + client.write("GET /simpleresource HTTP/1.0" + CRLF * 2) |
1117 | + yield self.ws.simple_resource.rendered |
1118 | + |
1119 | + @defer.inlineCallbacks |
1120 | + def test_client_read(self): |
1121 | + """Data received by the client is written into the transport.""" |
1122 | + fake_protocol = FakeServerTunnelProtocol() |
1123 | + client = tunnel_server.RemoteSocket(fake_protocol) |
1124 | + self.addCleanup(client.stop) |
1125 | + url = urlparse(self.dest_url) |
1126 | + yield client.connect(url.netloc) |
1127 | + client.write("GET /simpleresource HTTP/1.0" + CRLF * 2) |
1128 | + yield self.ws.simple_resource.rendered |
1129 | + data = yield fake_protocol.response_received |
1130 | + _headers, content = str(data).split(CRLF * 2, 1) |
1131 | + self.assertEqual(content, SAMPLE_CONTENT) |
1132 | + |
1133 | + |
1134 | +class AnonProxyRemoteSocketTestCase(RemoteSocketTestCase): |
1135 | + """Tests for the client going thru an anonymous proxy.""" |
1136 | + |
1137 | + get_proxy_settings = RemoteSocketTestCase.get_nonauth_proxy_settings |
1138 | + |
1139 | + def parse_headers(self, raw_headers): |
1140 | + """Parse the headers.""" |
1141 | + lines = raw_headers.split(CRLF) |
1142 | + header_lines = lines[1:] |
1143 | + headers_pairs = (l.split(":", 1) for l in header_lines) |
1144 | + return dict((k.lower(), v.strip()) for k, v in headers_pairs) |
1145 | + |
1146 | + @defer.inlineCallbacks |
1147 | + def test_verify_client_uses_proxy(self): |
1148 | + """Verify that the client uses the proxy.""" |
1149 | + fake_protocol = FakeServerTunnelProtocol() |
1150 | + client = tunnel_server.RemoteSocket(fake_protocol) |
1151 | + self.addCleanup(client.stop) |
1152 | + url = urlparse(self.dest_url) |
1153 | + yield client.connect(url.netloc) |
1154 | + client.write("GET /simpleresource HTTP/1.0" + CRLF * 2) |
1155 | + yield self.ws.simple_resource.rendered |
1156 | + data = yield fake_protocol.response_received |
1157 | + raw_headers, _content = str(data).split(CRLF * 2, 1) |
1158 | + self.parse_headers(raw_headers) |
1159 | + |
1160 | + |
1161 | +class AuthenticatedProxyRemoteSocketTestCase(AnonProxyRemoteSocketTestCase): |
1162 | + """Tests for the client going thru an authenticated proxy.""" |
1163 | + |
1164 | + get_proxy_settings = RemoteSocketTestCase.get_auth_proxy_settings |
1165 | + |
1166 | + @defer.inlineCallbacks |
1167 | + def test_proxy_authentication_error(self): |
1168 | + """The proxy credentials were wrong on purpose.""" |
1169 | + settings = {"http": self.get_proxy_settings()} |
1170 | + settings["http"]["password"] = "wrong password!!!" |
1171 | + proxy = tunnel_server.build_proxy(settings) |
1172 | + tunnel_server.QNetworkProxy.setApplicationProxy(proxy) |
1173 | + fake_protocol = FakeServerTunnelProtocol() |
1174 | + client = tunnel_server.RemoteSocket(fake_protocol) |
1175 | + self.addCleanup(client.stop) |
1176 | + url = urlparse(self.dest_url) |
1177 | + yield self.assertFailure(client.connect(url.netloc), |
1178 | + tunnel_server.ProxyAuthenticationError) |
1179 | + |
1180 | + @defer.inlineCallbacks |
1181 | + def test_proxy_nobody_listens(self): |
1182 | + """The proxy settings point to a proxy that's unreachable.""" |
1183 | + settings = dict(http={ |
1184 | + "host": "127.0.0.1", |
1185 | + "port": 83, # unused port according to /etc/services |
1186 | + }) |
1187 | + proxy = tunnel_server.build_proxy(settings) |
1188 | + tunnel_server.QNetworkProxy.setApplicationProxy(proxy) |
1189 | + fake_protocol = FakeServerTunnelProtocol() |
1190 | + client = tunnel_server.RemoteSocket(fake_protocol) |
1191 | + self.addCleanup(client.stop) |
1192 | + url = urlparse(self.dest_url) |
1193 | + yield self.assertFailure(client.connect(url.netloc), |
1194 | + tunnel_server.ConnectionError) |
1195 | + |
1196 | + def test_use_credentials(self): |
1197 | + """The credentials are used if present.""" |
1198 | + fake_protocol = FakeServerTunnelProtocol() |
1199 | + client = tunnel_server.RemoteSocket(fake_protocol) |
1200 | + proxy = tunnel_server.build_proxy(FAKE_SETTINGS) |
1201 | + authenticator = QAuthenticator() |
1202 | + |
1203 | + client.proxyAuthenticationRequired.emit(proxy, authenticator) |
1204 | + self.assertEqual(proxy.user(), "") |
1205 | + self.assertEqual(proxy.password(), "") |
1206 | + fake_protocol.proxy_credentials = FAKE_CREDS |
1207 | + |
1208 | + client.proxyAuthenticationRequired.emit(proxy, authenticator) |
1209 | + self.assertEqual(authenticator.user(), FAKE_CREDS["username"]) |
1210 | + self.assertEqual(authenticator.password(), FAKE_CREDS["password"]) |
1211 | + |
1212 | + |
1213 | +class FakeNetworkProxyFactoryClass(object): |
1214 | + """A fake QNetworkProxyFactory.""" |
1215 | + last_query = None |
1216 | + |
1217 | + def __init__(self, enabled): |
1218 | + """Initialize this fake instance.""" |
1219 | + if enabled: |
1220 | + self.proxy_type = tunnel_server.QNetworkProxy.HttpProxy |
1221 | + else: |
1222 | + self.proxy_type = tunnel_server.QNetworkProxy.DefaultProxy |
1223 | + |
1224 | + def type(self): |
1225 | + """Return the proxy type configured.""" |
1226 | + return self.proxy_type |
1227 | + |
1228 | + def systemProxyForQuery(self, query): |
1229 | + """A list of proxies, but only type() will be called on the first.""" |
1230 | + return [self] |
1231 | + |
1232 | + |
1233 | +class CheckProxyEnabledTestCase(TestCase): |
1234 | + """Tests for the check_proxy_enabled function.""" |
1235 | + |
1236 | + @defer.inlineCallbacks |
1237 | + def setUp(self): |
1238 | + """Initialize this testcase.""" |
1239 | + yield super(CheckProxyEnabledTestCase, self).setUp() |
1240 | + self.app_proxy = [] |
1241 | + |
1242 | + def _assert_proxy_state(self, platform, state, assertion): |
1243 | + """Assert the proxy is in a given state.""" |
1244 | + self.patch(tunnel_server.QNetworkProxy, "setApplicationProxy", |
1245 | + lambda proxy: self.app_proxy.append(proxy)) |
1246 | + self.patch(tunnel_server.sys, "platform", platform) |
1247 | + ret = tunnel_server.check_proxy_enabled(SAMPLE_HOST, str(SAMPLE_PORT)) |
1248 | + self.assertTrue(ret == state, assertion) |
1249 | + |
1250 | + def _assert_proxy_enabled(self, platform): |
1251 | + """Assert that the proxy is enabled.""" |
1252 | + self._assert_proxy_state(platform, True, "Proxy is enabled.") |
1253 | + |
1254 | + def _assert_proxy_disabled(self, platform): |
1255 | + """Assert that the proxy is disabled.""" |
1256 | + self._assert_proxy_state(platform, False, "Proxy is disabled.") |
1257 | + |
1258 | + def test_platform_linux_enabled(self): |
1259 | + """Tests for the linux platform with proxies enabled.""" |
1260 | + self.patch(tunnel_server.gsettings, "get_proxy_settings", |
1261 | + lambda: FAKE_SETTINGS) |
1262 | + self._assert_proxy_enabled("linux3") |
1263 | + self.assertEqual(len(self.app_proxy), 1) |
1264 | + |
1265 | + def test_platform_linux_disabled(self): |
1266 | + """Tests for the linux platform with proxies disabled.""" |
1267 | + self.patch(tunnel_server.gsettings, "get_proxy_settings", lambda: {}) |
1268 | + self._assert_proxy_disabled("linux3") |
1269 | + self.assertEqual(len(self.app_proxy), 0) |
1270 | + |
1271 | + def test_platform_other_enabled(self): |
1272 | + """Tests for any other platform with proxies enabled.""" |
1273 | + fake_netproxfact = FakeNetworkProxyFactoryClass(True) |
1274 | + self.patch(tunnel_server, "QNetworkProxyFactory", fake_netproxfact) |
1275 | + self._assert_proxy_enabled("windows 1.0") |
1276 | + self.assertEqual(len(self.app_proxy), 0) |
1277 | + |
1278 | + def test_platform_other_disabled(self): |
1279 | + """Tests for any other platform with proxies disabled.""" |
1280 | + fake_netproxfact = FakeNetworkProxyFactoryClass(False) |
1281 | + self.patch(tunnel_server, "QNetworkProxyFactory", fake_netproxfact) |
1282 | + self._assert_proxy_disabled("windows 1.0") |
1283 | + self.assertEqual(len(self.app_proxy), 0) |
1284 | + |
1285 | + |
1286 | +class FakeQCoreApp(object): |
1287 | + """A fake QCoreApplication.""" |
1288 | + |
1289 | + fake_instance = None |
1290 | + |
1291 | + def __init__(self, argv): |
1292 | + """Initialize this fake.""" |
1293 | + self.executed = False |
1294 | + self.argv = argv |
1295 | + FakeQCoreApp.fake_instance = self |
1296 | + |
1297 | + def exec_(self): |
1298 | + """Fake the execution of this app.""" |
1299 | + self.executed = True |
1300 | + |
1301 | + @staticmethod |
1302 | + def instance(): |
1303 | + """But return the real instance.""" |
1304 | + return QCoreApplication.instance() |
1305 | + |
1306 | + |
1307 | +class MainFunctionTestCase(TestCase): |
1308 | + """Tests for the main function of the tunnel server.""" |
1309 | + |
1310 | + @defer.inlineCallbacks |
1311 | + def setUp(self): |
1312 | + """Initialize these testcases.""" |
1313 | + yield super(MainFunctionTestCase, self).setUp() |
1314 | + self.called = [] |
1315 | + self.proxies_enabled = False |
1316 | + |
1317 | + def fake_is_proxy_enabled(*args): |
1318 | + """Store the call, return false.""" |
1319 | + self.called.append(args) |
1320 | + return self.proxies_enabled |
1321 | + |
1322 | + self.patch(tunnel_server, "check_proxy_enabled", fake_is_proxy_enabled) |
1323 | + self.fake_stdout = StringIO() |
1324 | + self.patch(tunnel_server.sys, "stdout", self.fake_stdout) |
1325 | + self.patch(tunnel_server, "QCoreApplication", FakeQCoreApp) |
1326 | + |
1327 | + def test_checks_proxies(self): |
1328 | + """Main checks that the proxies are enabled.""" |
1329 | + tunnel_server.main([]) |
1330 | + self.assertEqual(len(self.called), 1) |
1331 | + |
1332 | + def test_on_proxies_enabled_prints_port_and_cookie(self): |
1333 | + """With proxies enabled print port to stdout and start the mainloop.""" |
1334 | + self.patch(tunnel_server.uuid, "uuid4", lambda: FAKE_COOKIE) |
1335 | + self.proxies_enabled = True |
1336 | + port = 443 |
1337 | + tunnel_server.main(["example.com", str(port)]) |
1338 | + stdout = self.fake_stdout.getvalue() |
1339 | + |
1340 | + self.assertIn(tunnel_server.TUNNEL_PORT_LABEL + ": ", stdout) |
1341 | + cookie_line = tunnel_server.TUNNEL_COOKIE_LABEL + ": " + FAKE_COOKIE |
1342 | + self.assertIn(cookie_line, stdout) |
1343 | + |
1344 | + def test_on_proxies_disabled_exit(self): |
1345 | + """With proxies disabled, print a message and exit gracefully.""" |
1346 | + self.proxies_enabled = False |
1347 | + tunnel_server.main(["example.com", "443"]) |
1348 | + self.assertIn("Proxy not enabled.", self.fake_stdout.getvalue()) |
1349 | + self.assertEqual(FakeQCoreApp.fake_instance, None) |
1350 | |
1351 | === modified file 'tests/syncdaemon/test_action_queue.py' |
1352 | --- tests/syncdaemon/test_action_queue.py 2012-02-18 16:13:04 +0000 |
1353 | +++ tests/syncdaemon/test_action_queue.py 2012-03-20 23:05:30 +0000 |
1354 | @@ -23,7 +23,6 @@ |
1355 | import operator |
1356 | import os |
1357 | import unittest |
1358 | -import urllib2 |
1359 | import uuid |
1360 | |
1361 | from functools import wraps |
1362 | @@ -33,7 +32,7 @@ |
1363 | |
1364 | from mocker import Mocker, MockerTestCase, ANY, expect |
1365 | from oauth import oauth |
1366 | -from twisted.internet import defer, threads, reactor |
1367 | +from twisted.internet import defer, reactor |
1368 | from twisted.internet import error as twisted_error |
1369 | from twisted.python.failure import DefaultException, Failure |
1370 | from twisted.web import server |
1371 | @@ -186,6 +185,35 @@ |
1372 | return FakeRequest() |
1373 | |
1374 | |
1375 | +class FakeTunnelClient(object): |
1376 | + """A fake proxy.tunnel_client.""" |
1377 | + |
1378 | + def __init__(self): |
1379 | + """Fake this proxy tunnel.""" |
1380 | + self.tcp_connected = False |
1381 | + self.ssl_connected = False |
1382 | + |
1383 | + def connectTCP(self, *args, **kwargs): |
1384 | + """Save the connection thru TCP.""" |
1385 | + self.tcp_connected = True |
1386 | + |
1387 | + def connectSSL(self, *args, **kwargs): |
1388 | + """Save the connection thru SSL.""" |
1389 | + self.ssl_connected = True |
1390 | + |
1391 | + |
1392 | +class SavingConnectionTunnelRunner(object): |
1393 | + """A fake proxy.tunnel_client.TunnelRunner.""" |
1394 | + |
1395 | + def __init__(self, *args): |
1396 | + """Fake a proxy tunnel.""" |
1397 | + self.client = FakeTunnelClient() |
1398 | + |
1399 | + def get_client(self): |
1400 | + """Always return the reactor.""" |
1401 | + return defer.succeed(self.client) |
1402 | + |
1403 | + |
1404 | class TestingProtocol(ActionQueue.protocol): |
1405 | """Protocol for testing.""" |
1406 | |
1407 | @@ -370,6 +398,21 @@ |
1408 | self.assertEqual(defined_args[0], 'self') |
1409 | self.assertEqual(set(defined_args[1:]), set(evtargs)) |
1410 | |
1411 | + @defer.inlineCallbacks |
1412 | + def test_get_webclient(self): |
1413 | + """The webclient is created if it does not exist.""" |
1414 | + self.assertEqual(self.action_queue.webclient, None) |
1415 | + webclient = yield self.action_queue.get_webclient() |
1416 | + self.assertNotEqual(webclient, None) |
1417 | + |
1418 | + @defer.inlineCallbacks |
1419 | + def test_get_webclient_existing(self): |
1420 | + """The webclient is not created again if it exists.""" |
1421 | + fake_wc = object() |
1422 | + self.patch(self.action_queue, "webclient", fake_wc) |
1423 | + webclient = yield self.action_queue.get_webclient() |
1424 | + self.assertEqual(webclient, fake_wc) |
1425 | + |
1426 | |
1427 | class TestLoggingStorageClient(TwistedTestCase): |
1428 | """Tests for ensuring magic hash dont show in logs.""" |
1429 | @@ -1295,6 +1338,28 @@ |
1430 | "host 1.2.3.4", "port 4321")) |
1431 | |
1432 | |
1433 | +class TunnelRunnerTestCase(FactoryBaseTestCase): |
1434 | + """Tests for the tunnel runner.""" |
1435 | + |
1436 | + tunnel_runner_class = SavingConnectionTunnelRunner |
1437 | + |
1438 | + @defer.inlineCallbacks |
1439 | + def test_make_connection_uses_tunnelrunner_non_ssl(self): |
1440 | + """Check that _make_connection uses TunnelRunner.""" |
1441 | + self.action_queue.use_ssl = False |
1442 | + yield self.action_queue._make_connection(("127.0.0.1", 1234)) |
1443 | + self.assertTrue(self.action_queue.tunnel_runner.client.tcp_connected, |
1444 | + "connectTCP is called on the client.") |
1445 | + |
1446 | + @defer.inlineCallbacks |
1447 | + def test_make_connection_uses_tunnelrunner_ssl(self): |
1448 | + """Check that _make_connection uses TunnelRunner.""" |
1449 | + self.action_queue.use_ssl = True |
1450 | + yield self.action_queue._make_connection(("127.0.0.1", 1234)) |
1451 | + self.assertTrue(self.action_queue.tunnel_runner.client.ssl_connected, |
1452 | + "connectSSL is called on the client.") |
1453 | + |
1454 | + |
1455 | class ConnectedBaseTestCase(FactoryBaseTestCase): |
1456 | """Base test case generating a connected factory.""" |
1457 | |
1458 | @@ -2468,67 +2533,26 @@ |
1459 | self.assertEqual(NODE, self.command.node_id) |
1460 | self.assertEqual(True, self.command.is_public) |
1461 | |
1462 | - def test_run_defers_work_to_thread(self): |
1463 | - """Test that work is deferred to a thread.""" |
1464 | - original = threads.deferToThread |
1465 | - self.called = False |
1466 | - |
1467 | - def check(function): |
1468 | - self.called = True |
1469 | - self.assertEqual( |
1470 | - self.command._change_public_access_http, function) |
1471 | - return defer.Deferred() |
1472 | - |
1473 | - threads.deferToThread = check |
1474 | - try: |
1475 | - res = self.command._run() |
1476 | - finally: |
1477 | - threads.deferToThread = original |
1478 | - |
1479 | - self.assertIsInstance(res, defer.Deferred) |
1480 | - self.assertTrue(self.called, "deferToThread was called") |
1481 | - |
1482 | + @defer.inlineCallbacks |
1483 | def test_change_public_access_http(self): |
1484 | - """Test the blocking portion of the command.""" |
1485 | - self.called = False |
1486 | - def check(request): |
1487 | - self.called = True |
1488 | - url = 'https://one.ubuntu.com/files/api/set_public/%s:%s' % ( |
1489 | + """Test the command.""" |
1490 | + |
1491 | + def check_webcall(request_iri, method=None, post_content=None): |
1492 | + """Check the webcall made by this command.""" |
1493 | + iri = u'https://one.ubuntu.com/files/api/set_public/%s:%s' % ( |
1494 | base64.urlsafe_b64encode(VOLUME.bytes).strip("="), |
1495 | base64.urlsafe_b64encode(NODE.bytes).strip("=")) |
1496 | - self.assertEqual(url, request.get_full_url()) |
1497 | - self.assertEqual("is_public=True", request.get_data()) |
1498 | - return StringIO( |
1499 | - '{"is_public": true, "public_url": "http://example.com"}') |
1500 | - |
1501 | - from ubuntuone.syncdaemon import action_queue |
1502 | - self.patch(action_queue.timestamp_checker, "get_faithful_time", |
1503 | - lambda: 1) |
1504 | - action_queue.urlopen = check |
1505 | - try: |
1506 | - res = self.command._change_public_access_http() |
1507 | - finally: |
1508 | - action_queue.urlopen = urllib2.urlopen |
1509 | - |
1510 | + self.assertEqual(iri, request_iri) |
1511 | + self.assertEqual("is_public=True", post_content) |
1512 | + content = '{"is_public": true, "public_url": "http://example.com"}' |
1513 | + response = action_queue.txweb.Response(content) |
1514 | + return defer.succeed(response) |
1515 | + |
1516 | + self.patch(self.action_queue, "webcall", check_webcall) |
1517 | + res = yield self.command._run() |
1518 | self.assertEqual( |
1519 | {'is_public': True, 'public_url': 'http://example.com'}, res) |
1520 | |
1521 | - def test_change_public_access_http_uses_timestamp(self): |
1522 | - """The timestamp is used for oauth signing.""" |
1523 | - fake_timestamp = 12345678 |
1524 | - |
1525 | - def fake_urlopen(request): |
1526 | - """A fake urlopen.""" |
1527 | - auth = request.headers["Authorization"] |
1528 | - expected = 'oauth_timestamp="%d"' % fake_timestamp |
1529 | - self.assertIn(expected, auth) |
1530 | - return StringIO("[]") |
1531 | - |
1532 | - self.patch(action_queue.timestamp_checker, "get_faithful_time", |
1533 | - lambda: fake_timestamp) |
1534 | - self.patch(action_queue, "urlopen", fake_urlopen) |
1535 | - self.command._change_public_access_http() |
1536 | - |
1537 | def test_handle_success_push_event(self): |
1538 | """Test AQ_CHANGE_PUBLIC_ACCESS_OK is pushed on success.""" |
1539 | response = {'is_public': True, 'public_url': 'http://example.com'} |
1540 | @@ -2541,8 +2565,7 @@ |
1541 | def test_handle_failure_push_event(self): |
1542 | """Test AQ_CHANGE_PUBLIC_ACCESS_ERROR is pushed on failure.""" |
1543 | msg = 'Something went wrong' |
1544 | - failure = Failure(urllib2.HTTPError( |
1545 | - "http://example.com", 500, "Error", [], StringIO(msg))) |
1546 | + failure = Failure(action_queue.txweb.WebClientError("Misc Error", msg)) |
1547 | self.command.handle_failure(failure=failure) |
1548 | event = ('AQ_CHANGE_PUBLIC_ACCESS_ERROR', |
1549 | {'share_id': VOLUME, 'node_id': NODE, 'error': msg}) |
1550 | @@ -2569,11 +2592,11 @@ |
1551 | default_url = 'https://one.ubuntu.com/files/api/public_files' |
1552 | request_queue = RequestQueue(action_queue=self.action_queue) |
1553 | command = GetPublicFiles(request_queue) |
1554 | - self.assertEqual(command._url, default_url) |
1555 | + self.assertEqual(command._iri, default_url) |
1556 | custom_url = 'http://example.com:1234/files/api/public_files' |
1557 | command_2 = GetPublicFiles(request_queue, |
1558 | - base_url='http://example.com:1234') |
1559 | - self.assertEqual(command_2._url, custom_url) |
1560 | + base_iri=u'http://example.com:1234') |
1561 | + self.assertEqual(command_2._iri, custom_url) |
1562 | |
1563 | def test_change_public_access(self): |
1564 | """Test the get_public_files method..""" |
1565 | @@ -2583,75 +2606,39 @@ |
1566 | """Test proper inheritance.""" |
1567 | self.assertTrue(isinstance(self.command, ActionQueueCommand)) |
1568 | |
1569 | - def test_run_defers_work_to_thread(self): |
1570 | - """Test that work is deferred to a thread.""" |
1571 | - original = threads.deferToThread |
1572 | - self.called = False |
1573 | - |
1574 | - def check(function): |
1575 | - self.called = True |
1576 | - self.assertEqual( |
1577 | - self.command._get_public_files_http, function) |
1578 | - return defer.Deferred() |
1579 | - |
1580 | - threads.deferToThread = check |
1581 | - try: |
1582 | - res = self.command._run() |
1583 | - finally: |
1584 | - threads.deferToThread = original |
1585 | - |
1586 | - self.assertIsInstance(res, defer.Deferred) |
1587 | - self.assertTrue(self.called, "deferToThread was called") |
1588 | - |
1589 | + @defer.inlineCallbacks |
1590 | def test_get_public_files_http(self): |
1591 | - """Test the blocking portion of the command.""" |
1592 | - self.called = False |
1593 | + """Test the _run method of the command.""" |
1594 | node_id = uuid.uuid4() |
1595 | nodekey = '%s' % (base64.urlsafe_b64encode(node_id.bytes).strip("=")) |
1596 | node_id_2 = uuid.uuid4() |
1597 | nodekey_2 = '%s' % (base64.urlsafe_b64encode( |
1598 | node_id_2.bytes).strip("=")) |
1599 | volume_id = uuid.uuid4() |
1600 | - def check(request): |
1601 | - self.called = True |
1602 | - url = 'https://one.ubuntu.com/files/api/public_files' |
1603 | - self.assertEqual(url, request.get_full_url()) |
1604 | - return StringIO( |
1605 | + |
1606 | + def check_webcall(request_iri, method=None): |
1607 | + """Check the webcall made by this command.""" |
1608 | + """Check the webcall made by this command.""" |
1609 | + iri = u'https://one.ubuntu.com/files/api/public_files' |
1610 | + self.assertEqual(method.upper(), "GET") |
1611 | + self.assertEqual(iri, request_iri) |
1612 | + content = ( |
1613 | '[{"nodekey": "%s", "volume_id": null,"public_url": ' |
1614 | '"http://example.com"}, ' |
1615 | '{"nodekey": "%s", "volume_id": "%s", "public_url": ' |
1616 | '"http://example.com"}]' % (nodekey, nodekey_2, volume_id)) |
1617 | - |
1618 | - from ubuntuone.syncdaemon import action_queue |
1619 | - self.patch(action_queue.timestamp_checker, "get_faithful_time", |
1620 | - lambda: 1) |
1621 | - action_queue.urlopen = check |
1622 | - try: |
1623 | - res = self.command._get_public_files_http() |
1624 | - finally: |
1625 | - action_queue.urlopen = urllib2.urlopen |
1626 | + response = action_queue.txweb.Response(content) |
1627 | + return defer.succeed(response) |
1628 | + |
1629 | + self.patch(self.action_queue, "webcall", check_webcall) |
1630 | + res = yield self.command._run() |
1631 | + |
1632 | self.assertEqual([{'node_id': str(node_id), 'volume_id': '', |
1633 | 'public_url': 'http://example.com'}, |
1634 | {'node_id': str(node_id_2), |
1635 | 'volume_id': str(volume_id), |
1636 | 'public_url': 'http://example.com'}], res) |
1637 | |
1638 | - def test_get_public_files_http_uses_timestamp(self): |
1639 | - """The timestamp is used for oauth signing.""" |
1640 | - fake_timestamp = 12345678 |
1641 | - |
1642 | - def fake_urlopen(request): |
1643 | - """A fake urlopen.""" |
1644 | - auth = request.headers["Authorization"] |
1645 | - expected = 'oauth_timestamp="%d"' % fake_timestamp |
1646 | - self.assertIn(expected, auth) |
1647 | - return StringIO("[]") |
1648 | - |
1649 | - self.patch(action_queue.timestamp_checker, "get_faithful_time", |
1650 | - lambda: fake_timestamp) |
1651 | - self.patch(action_queue, "urlopen", fake_urlopen) |
1652 | - self.command._get_public_files_http() |
1653 | - |
1654 | def test_handle_success_push_event(self): |
1655 | """Test AQ_PUBLIC_FILES_LIST_OK is pushed on success.""" |
1656 | response = [{'node_id': uuid.uuid4(), 'volume_id':None, |
1657 | @@ -2663,8 +2650,7 @@ |
1658 | def test_handle_failure_push_event(self): |
1659 | """Test AQ_PUBLIC_FILES_LIST_ERROR is pushed on failure.""" |
1660 | msg = 'Something went wrong' |
1661 | - failure = Failure(urllib2.HTTPError( |
1662 | - "http://example.com", 500, "Error", [], StringIO(msg))) |
1663 | + failure = Failure(action_queue.txweb.WebClientError("Misc Error", msg)) |
1664 | self.command.handle_failure(failure=failure) |
1665 | event = ('AQ_PUBLIC_FILES_LIST_ERROR', {'error': msg}) |
1666 | self.assertIn(event, self.command.action_queue.event_queue.events) |
1667 | @@ -3649,27 +3635,6 @@ |
1668 | self.assertEqual('share_name', name) |
1669 | self.assertTrue(read_only) |
1670 | |
1671 | - @defer.inlineCallbacks |
1672 | - def test_create_share_http_uses_timestamp(self): |
1673 | - """The timestamp is used for oauth signing.""" |
1674 | - fake_timestamp = 12345678 |
1675 | - |
1676 | - def fake_urlopen(request): |
1677 | - """A fake urlopen.""" |
1678 | - auth = request.headers["Authorization"] |
1679 | - expected = 'oauth_timestamp="%d"' % fake_timestamp |
1680 | - self.assertIn(expected, auth) |
1681 | - |
1682 | - self.patch(action_queue.timestamp_checker, "get_faithful_time", |
1683 | - lambda: fake_timestamp) |
1684 | - self.patch(action_queue, "urlopen", fake_urlopen) |
1685 | - self.user_connect() |
1686 | - command = CreateShare(self.request_queue, 'node_id', |
1687 | - 'share_to@example.com', 'share_name', |
1688 | - ACCESS_LEVEL_RO, 'marker', 'path') |
1689 | - self.assertTrue(command.use_http, 'CreateShare should be in http mode') |
1690 | - yield command._run() |
1691 | - |
1692 | def test_possible_markers(self): |
1693 | """Test that it returns the correct values.""" |
1694 | cmd = CreateShare(self.request_queue, 'node_id', 'shareto@example.com', |
1695 | |
1696 | === added file 'tests/syncdaemon/test_tunnel_runner.py' |
1697 | --- tests/syncdaemon/test_tunnel_runner.py 1970-01-01 00:00:00 +0000 |
1698 | +++ tests/syncdaemon/test_tunnel_runner.py 2012-03-20 23:05:30 +0000 |
1699 | @@ -0,0 +1,128 @@ |
1700 | +# -*- coding: utf8 -*- |
1701 | +# |
1702 | +# Copyright 2012 Canonical Ltd. |
1703 | +# |
1704 | +# This program is free software: you can redistribute it and/or modify it |
1705 | +# under the terms of the GNU General Public License version 3, as published |
1706 | +# by the Free Software Foundation. |
1707 | +# |
1708 | +# This program is distributed in the hope that it will be useful, but |
1709 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
1710 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
1711 | +# PURPOSE. See the GNU General Public License for more details. |
1712 | +# |
1713 | +# You should have received a copy of the GNU General Public License along |
1714 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
1715 | +"""Tests for the proxy tunnel runner.""" |
1716 | + |
1717 | +import os |
1718 | +import sys |
1719 | + |
1720 | +from twisted.internet import defer, error, reactor, task |
1721 | +from twisted.trial.unittest import TestCase |
1722 | + |
1723 | +from tests.proxy import FAKE_COOKIE |
1724 | +from ubuntuone.proxy import tunnel_client |
1725 | +from ubuntuone.syncdaemon import tunnel_runner |
1726 | + |
1727 | +FAKE_HOST = "fs-1.two.ubuntu.com" |
1728 | +FAKE_PORT = 443 |
1729 | + |
1730 | + |
1731 | +class TunnelRunnerConstructorTestCase(TestCase): |
1732 | + """Test the tunnel runner constructor.""" |
1733 | + |
1734 | + timeout = 3 |
1735 | + |
1736 | + def raise_import_error(self, *args): |
1737 | + """Raise an import error.""" |
1738 | + raise ImportError |
1739 | + |
1740 | + @defer.inlineCallbacks |
1741 | + def test_proxy_support_not_installed(self): |
1742 | + """The proxy support binary package is not installed.""" |
1743 | + self.patch(tunnel_runner.TunnelRunner, "start_process", |
1744 | + self.raise_import_error) |
1745 | + tr = tunnel_runner.TunnelRunner(FAKE_HOST, FAKE_PORT) |
1746 | + client = yield tr.get_client() |
1747 | + self.assertEqual(client, reactor) |
1748 | + |
1749 | + |
1750 | +class TunnelRunnerTestCase(TestCase): |
1751 | + """Tests for the TunnelRunner.""" |
1752 | + |
1753 | + timeout = 3 |
1754 | + |
1755 | + @defer.inlineCallbacks |
1756 | + def setUp(self): |
1757 | + """Initialize this testcase.""" |
1758 | + yield super(TunnelRunnerTestCase, self).setUp() |
1759 | + self.spawned = [] |
1760 | + self.patch(tunnel_client.reactor, "spawnProcess", |
1761 | + lambda *args, **kwargs: self.spawned.append((args, kwargs))) |
1762 | + self.process_protocol = None |
1763 | + self.process_protocol_class = tunnel_client.TunnelProcessProtocol |
1764 | + self.patch(tunnel_client, "TunnelProcessProtocol", |
1765 | + self.storing_process_protocol_factory) |
1766 | + self.tr = tunnel_runner.TunnelRunner("fs-1.one.ubuntu.com", 443) |
1767 | + |
1768 | + def storing_process_protocol_factory(self, *args, **kwargs): |
1769 | + """Store the process protocol just created.""" |
1770 | + self.process_protocol = self.process_protocol_class(*args, **kwargs) |
1771 | + return self.process_protocol |
1772 | + |
1773 | + def test_tunnel_process_is_started(self): |
1774 | + """The tunnel process is started.""" |
1775 | + self.assertEqual(len(self.spawned), 1, |
1776 | + "The tunnel process is started.") |
1777 | + |
1778 | + @defer.inlineCallbacks |
1779 | + def test_tunnel_process_get_client_yielded_twice(self): |
1780 | + """The get_client method can be yielded twice.""" |
1781 | + self.process_protocol.processExited(error.ProcessTerminated(1)) |
1782 | + client = yield self.tr.get_client() |
1783 | + client = yield self.tr.get_client() |
1784 | + self.assertNotEqual(client, None) |
1785 | + |
1786 | + @defer.inlineCallbacks |
1787 | + def test_tunnel_process_exits_with_error(self): |
1788 | + """The tunnel process exits with an error.""" |
1789 | + self.process_protocol.processExited(error.ProcessTerminated(1)) |
1790 | + client = yield self.tr.get_client() |
1791 | + self.assertEqual(client, reactor) |
1792 | + |
1793 | + @defer.inlineCallbacks |
1794 | + def test_tunnel_process_exits_gracefully(self): |
1795 | + """The tunnel process exits gracefully.""" |
1796 | + self.process_protocol.processExited(error.ProcessDone(0)) |
1797 | + client = yield self.tr.get_client() |
1798 | + self.assertEqual(client, reactor) |
1799 | + |
1800 | + @defer.inlineCallbacks |
1801 | + def test_tunnel_process_prints_random_garbage_and_timeouts(self): |
1802 | + """The tunnel process prints garbage and timeouts.""" |
1803 | + clock = task.Clock() |
1804 | + self.patch(tunnel_client, "reactor", clock) |
1805 | + self.process_protocol.connectionMade() |
1806 | + self.process_protocol.outReceived("Random garbage") |
1807 | + clock.advance(self.process_protocol.timeout) |
1808 | + client = yield self.tr.get_client() |
1809 | + self.assertEqual(client, clock) |
1810 | + |
1811 | + @defer.inlineCallbacks |
1812 | + def test_tunnel_process_prints_port_number_and_cookie(self): |
1813 | + """The tunnel process prints the port number.""" |
1814 | + received = "%s: %d\n%s: %s\n" % ( |
1815 | + tunnel_client.TUNNEL_PORT_LABEL, FAKE_PORT, |
1816 | + tunnel_client.TUNNEL_COOKIE_LABEL, FAKE_COOKIE) |
1817 | + self.process_protocol.outReceived(received) |
1818 | + client = yield self.tr.get_client() |
1819 | + self.assertEqual(client.tunnel_port, FAKE_PORT) |
1820 | + self.assertEqual(client.cookie, FAKE_COOKIE) |
1821 | + |
1822 | + def test_frozen_path(self): |
1823 | + """Ensure the path is correct if sys is frozen.""" |
1824 | + sys.frozen = "Yes" |
1825 | + self.addCleanup(delattr, sys, "frozen") |
1826 | + self.assertEqual(os.path.dirname(self.tr.get_process_path()), |
1827 | + os.path.dirname(sys.executable)) |
1828 | |
1829 | === modified file 'ubuntuone/platform/constants.py' |
1830 | --- ubuntuone/platform/constants.py 2011-12-08 20:56:10 +0000 |
1831 | +++ ubuntuone/platform/constants.py 2012-03-20 23:05:30 +0000 |
1832 | @@ -21,7 +21,12 @@ |
1833 | |
1834 | import sys |
1835 | |
1836 | +TUNNEL_EXECUTABLE = "ubuntuone-proxy-tunnel" |
1837 | + |
1838 | if sys.platform == "win32": |
1839 | HASHQUEUE_DELAY = 3.0 |
1840 | + if getattr(sys, "frozen", None) is not None: |
1841 | + # Only use the .exe suffix in windows+frozen |
1842 | + TUNNEL_EXECUTABLE += ".exe" |
1843 | else: |
1844 | HASHQUEUE_DELAY = 0.5 |
1845 | |
1846 | === modified file 'ubuntuone/platform/linux/notification.py' |
1847 | --- ubuntuone/platform/linux/notification.py 2012-01-25 22:40:02 +0000 |
1848 | +++ ubuntuone/platform/linux/notification.py 2012-03-20 23:05:30 +0000 |
1849 | @@ -68,6 +68,9 @@ |
1850 | self.notification = Notify.Notification(title, message, icon) |
1851 | else: |
1852 | self.notification.update(title, message, icon) |
1853 | + |
1854 | if append: |
1855 | self.notification.set_hint_string('x-canonical-append', '') |
1856 | + |
1857 | + self.notification.set_hint('transient', True) |
1858 | self.notification.show() |
1859 | |
1860 | === added directory 'ubuntuone/proxy' |
1861 | === added file 'ubuntuone/proxy/__init__.py' |
1862 | --- ubuntuone/proxy/__init__.py 1970-01-01 00:00:00 +0000 |
1863 | +++ ubuntuone/proxy/__init__.py 2012-03-20 23:05:30 +0000 |
1864 | @@ -0,0 +1,14 @@ |
1865 | +# Copyright 2012 Canonical Ltd. |
1866 | +# |
1867 | +# This program is free software: you can redistribute it and/or modify it |
1868 | +# under the terms of the GNU General Public License version 3, as published |
1869 | +# by the Free Software Foundation. |
1870 | +# |
1871 | +# This program is distributed in the hope that it will be useful, but |
1872 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
1873 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
1874 | +# PURPOSE. See the GNU General Public License for more details. |
1875 | +# |
1876 | +# You should have received a copy of the GNU General Public License along |
1877 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
1878 | +"""Ubuntu One proxy support.""" |
1879 | |
1880 | === added file 'ubuntuone/proxy/common.py' |
1881 | --- ubuntuone/proxy/common.py 1970-01-01 00:00:00 +0000 |
1882 | +++ ubuntuone/proxy/common.py 2012-03-20 23:05:30 +0000 |
1883 | @@ -0,0 +1,60 @@ |
1884 | +# -*- coding: utf8 -*- |
1885 | +# |
1886 | +# Copyright 2012 Canonical Ltd. |
1887 | +# |
1888 | +# This program is free software: you can redistribute it and/or modify it |
1889 | +# under the terms of the GNU General Public License version 3, as published |
1890 | +# by the Free Software Foundation. |
1891 | +# |
1892 | +# This program is distributed in the hope that it will be useful, but |
1893 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
1894 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
1895 | +# PURPOSE. See the GNU General Public License for more details. |
1896 | +# |
1897 | +# You should have received a copy of the GNU General Public License along |
1898 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
1899 | +"""Common classes to the tunnel client and server.""" |
1900 | + |
1901 | +from twisted.protocols import basic |
1902 | + |
1903 | +CRLF = "\r\n" |
1904 | +TUNNEL_PORT_LABEL = "Tunnel port" |
1905 | +TUNNEL_COOKIE_LABEL = "Tunnel cookie" |
1906 | +TUNNEL_COOKIE_HEADER = "Proxy-Tunnel-Cookie" |
1907 | + |
1908 | + |
1909 | +class BaseTunnelProtocol(basic.LineReceiver): |
1910 | + """CONNECT base protocol for tunnelling connections.""" |
1911 | + |
1912 | + delimiter = CRLF |
1913 | + |
1914 | + def __init__(self): |
1915 | + """Initialize this protocol.""" |
1916 | + self._first_line = True |
1917 | + self.received_headers = [] |
1918 | + |
1919 | + def header_line(self, line): |
1920 | + """Handle each header line received.""" |
1921 | + key, value = line.split(":", 1) |
1922 | + value = value.strip() |
1923 | + self.received_headers.append((key, value)) |
1924 | + |
1925 | + def lineReceived(self, line): |
1926 | + """Process a line in the header.""" |
1927 | + if self._first_line: |
1928 | + self._first_line = False |
1929 | + self.handle_first_line(line) |
1930 | + else: |
1931 | + if line: |
1932 | + self.header_line(line) |
1933 | + else: |
1934 | + self.setRawMode() |
1935 | + self.headers_done() |
1936 | + |
1937 | + def remote_disconnected(self): |
1938 | + """The remote end closed the connection.""" |
1939 | + self.transport.loseConnection() |
1940 | + |
1941 | + def format_headers(self, headers): |
1942 | + """Format some headers as a few response lines.""" |
1943 | + return "".join("%s: %s" % item + CRLF for item in headers.items()) |
1944 | |
1945 | === added file 'ubuntuone/proxy/logger.py' |
1946 | --- ubuntuone/proxy/logger.py 1970-01-01 00:00:00 +0000 |
1947 | +++ ubuntuone/proxy/logger.py 2012-03-20 23:05:30 +0000 |
1948 | @@ -0,0 +1,36 @@ |
1949 | +# ubuntuone.syncdaemon.logger - logging utilities |
1950 | +# |
1951 | +# Copyright 2009-2012 Canonical Ltd. |
1952 | +# |
1953 | +# This program is free software: you can redistribute it and/or modify it |
1954 | +# under the terms of the GNU General Public License version 3, as published |
1955 | +# by the Free Software Foundation. |
1956 | +# |
1957 | +# This program is distributed in the hope that it will be useful, but |
1958 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
1959 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
1960 | +# PURPOSE. See the GNU General Public License for more details. |
1961 | +# |
1962 | +# You should have received a copy of the GNU General Public License along |
1963 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
1964 | + |
1965 | +"""SyncDaemon logging utilities and config.""" |
1966 | + |
1967 | +import logging |
1968 | +import os |
1969 | + |
1970 | +from ubuntuone.logger import ( |
1971 | + basic_formatter, |
1972 | + CustomRotatingFileHandler, |
1973 | +) |
1974 | + |
1975 | +from ubuntuone.platform.xdg_base_directory import ubuntuone_log_dir |
1976 | + |
1977 | + |
1978 | +LOGFILENAME = os.path.join(ubuntuone_log_dir, 'proxy.log') |
1979 | +logger = logging.getLogger("ubuntuone.proxy") |
1980 | +logger.setLevel(logging.DEBUG) |
1981 | +handler = CustomRotatingFileHandler(filename=LOGFILENAME) |
1982 | +handler.setFormatter(basic_formatter) |
1983 | +handler.setLevel(logging.DEBUG) |
1984 | +logger.addHandler(handler) |
1985 | |
1986 | === added file 'ubuntuone/proxy/tunnel_client.py' |
1987 | --- ubuntuone/proxy/tunnel_client.py 1970-01-01 00:00:00 +0000 |
1988 | +++ ubuntuone/proxy/tunnel_client.py 2012-03-20 23:05:30 +0000 |
1989 | @@ -0,0 +1,181 @@ |
1990 | +# -*- coding: utf8 -*- |
1991 | +# |
1992 | +# Copyright 2012 Canonical Ltd. |
1993 | +# |
1994 | +# This program is free software: you can redistribute it and/or modify it |
1995 | +# under the terms of the GNU General Public License version 3, as published |
1996 | +# by the Free Software Foundation. |
1997 | +# |
1998 | +# This program is distributed in the hope that it will be useful, but |
1999 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
2000 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
2001 | +# PURPOSE. See the GNU General Public License for more details. |
2002 | +# |
2003 | +# You should have received a copy of the GNU General Public License along |
2004 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
2005 | +"""Client for the tunnel protocol.""" |
2006 | + |
2007 | +import logging |
2008 | + |
2009 | +from twisted.internet import protocol, reactor |
2010 | + |
2011 | +from ubuntuone.proxy.common import ( |
2012 | + BaseTunnelProtocol, |
2013 | + CRLF, |
2014 | + TUNNEL_COOKIE_LABEL, |
2015 | + TUNNEL_COOKIE_HEADER, |
2016 | + TUNNEL_PORT_LABEL, |
2017 | +) |
2018 | + |
2019 | +METHOD_LINE = "CONNECT %s:%d HTTP/1.0" + CRLF |
2020 | +LOCALHOST = "127.0.0.1" |
2021 | +logger = logging.getLogger("ubuntuone.Proxy.TunnelClient") |
2022 | + |
2023 | + |
2024 | +class TunnelClientProtocol(BaseTunnelProtocol): |
2025 | + """Client protocol for the handshake part of the tunnel.""" |
2026 | + |
2027 | + def connectionMade(self): |
2028 | + """The connection to the tunnel was made so send request.""" |
2029 | + method_line = METHOD_LINE % (self.factory.tunnel_host, |
2030 | + self.factory.tunnel_port) |
2031 | + headers = { |
2032 | + "User-Agent": "Ubuntu One tunnel client", |
2033 | + TUNNEL_COOKIE_HEADER: self.factory.cookie, |
2034 | + } |
2035 | + self.transport.write(method_line + |
2036 | + self.format_headers(headers) + |
2037 | + CRLF) |
2038 | + |
2039 | + def handle_first_line(self, line): |
2040 | + """The first line received is the status line.""" |
2041 | + try: |
2042 | + proto_version, self.status_code, description = line.split(" ", 2) |
2043 | + except ValueError: |
2044 | + self.transport.loseConnection() |
2045 | + |
2046 | + def headers_done(self): |
2047 | + """All the headers have arrived. Time to switch protocols.""" |
2048 | + remaining_data = self.clearLineBuffer() |
2049 | + if self.status_code != "200": |
2050 | + self.transport.loseConnection() |
2051 | + return |
2052 | + addr = self.transport.getPeer() |
2053 | + other_protocol = self.factory.other_factory.buildProtocol(addr) |
2054 | + self.transport.protocol = other_protocol |
2055 | + other_protocol.transport = self.transport |
2056 | + self.transport = None |
2057 | + if self.factory.context_factory: |
2058 | + other_protocol.transport.startTLS(self.factory.context_factory) |
2059 | + other_protocol.connectionMade() |
2060 | + if remaining_data: |
2061 | + other_protocol.dataReceived(remaining_data) |
2062 | + |
2063 | + |
2064 | +class TunnelClientFactory(protocol.ClientFactory): |
2065 | + """A factory for Tunnel Client Protocols.""" |
2066 | + |
2067 | + protocol = TunnelClientProtocol |
2068 | + |
2069 | + def __init__(self, tunnel_host, tunnel_port, other_factory, cookie, |
2070 | + context_factory=None): |
2071 | + """Initialize this factory.""" |
2072 | + self.tunnel_host = tunnel_host |
2073 | + self.tunnel_port = tunnel_port |
2074 | + self.other_factory = other_factory |
2075 | + self.context_factory = context_factory |
2076 | + self.cookie = cookie |
2077 | + |
2078 | + def startedConnecting(self, connector): |
2079 | + """Forward this call to the other factory.""" |
2080 | + self.other_factory.startedConnecting(connector) |
2081 | + |
2082 | + def clientConnectionFailed(self, connector, reason): |
2083 | + """Forward this call to the other factory.""" |
2084 | + self.other_factory.clientConnectionFailed(connector, reason) |
2085 | + |
2086 | + def clientConnectionLost(self, connector, reason): |
2087 | + """Forward this call to the other factory.""" |
2088 | + self.other_factory.clientConnectionLost(connector, reason) |
2089 | + |
2090 | + |
2091 | +class TunnelClient(object): |
2092 | + """A client for the proxy tunnel.""" |
2093 | + |
2094 | + def __init__(self, tunnel_host, tunnel_port, cookie): |
2095 | + """Initialize this client.""" |
2096 | + self.tunnel_host = tunnel_host |
2097 | + self.tunnel_port = tunnel_port |
2098 | + self.cookie = cookie |
2099 | + |
2100 | + def connectTCP(self, host, port, factory, *args, **kwargs): |
2101 | + """A connectTCP going thru the tunnel.""" |
2102 | + logger.info("Connecting (TCP) to %r:%r via tunnel at %r:%r", |
2103 | + host, port, self.tunnel_host, self.tunnel_port) |
2104 | + tunnel_factory = TunnelClientFactory(host, port, factory, self.cookie) |
2105 | + return reactor.connectTCP(self.tunnel_host, self.tunnel_port, |
2106 | + tunnel_factory, *args, **kwargs) |
2107 | + |
2108 | + def connectSSL(self, host, port, factory, |
2109 | + contextFactory, *args, **kwargs): |
2110 | + """A connectSSL going thru the tunnel.""" |
2111 | + logger.info("Connecting (SSL) to %r:%r via tunnel at %r:%r", |
2112 | + host, port, self.tunnel_host, self.tunnel_port) |
2113 | + tunnel_factory = TunnelClientFactory(host, port, factory, self.cookie, |
2114 | + contextFactory) |
2115 | + return reactor.connectTCP(self.tunnel_host, self.tunnel_port, |
2116 | + tunnel_factory, *args, **kwargs) |
2117 | + |
2118 | + |
2119 | +class TunnelProcessProtocol(protocol.ProcessProtocol): |
2120 | + """The dialog thru stdout with the tunnel server.""" |
2121 | + |
2122 | + timeout = 5 |
2123 | + |
2124 | + def __init__(self, client_d): |
2125 | + """Initialize this protocol.""" |
2126 | + self.client_d = client_d |
2127 | + self.timer = None |
2128 | + self.port = None |
2129 | + self.cookie = None |
2130 | + |
2131 | + def connectionMade(self): |
2132 | + """The process has started, start a timer.""" |
2133 | + logger.info("Tunnel process started.") |
2134 | + self.timer = reactor.callLater(self.timeout, self.process_timeouted) |
2135 | + |
2136 | + def process_timeouted(self): |
2137 | + """The process took too long to reply.""" |
2138 | + if not self.client_d.called: |
2139 | + logger.info("Timeout while waiting for tunnel process.") |
2140 | + self.client_d.callback(reactor) |
2141 | + |
2142 | + def finish_timeout(self): |
2143 | + """Stop the timer from firing.""" |
2144 | + if self.timer and self.timer.active(): |
2145 | + self.timer.cancel() |
2146 | + |
2147 | + def processExited(self, status): |
2148 | + """The tunnel process has exited with some error code.""" |
2149 | + self.finish_timeout() |
2150 | + logger.info("Tunnel process exit status %r.", status) |
2151 | + if not self.client_d.called: |
2152 | + self.client_d.callback(reactor) |
2153 | + |
2154 | + def outReceived(self, data): |
2155 | + """Receive the port number.""" |
2156 | + if self.client_d.called: |
2157 | + return |
2158 | + |
2159 | + for line in data.split("\n"): |
2160 | + if line.startswith(TUNNEL_PORT_LABEL): |
2161 | + _header, port = line.split(":", 1) |
2162 | + self.port = int(port.strip()) |
2163 | + if line.startswith(TUNNEL_COOKIE_LABEL): |
2164 | + _header, cookie = line.split(":", 1) |
2165 | + self.cookie = cookie.strip() |
2166 | + |
2167 | + if self.port and self.cookie: |
2168 | + logger.info("Tunnel process listening on port %r.", self.port) |
2169 | + client = TunnelClient(LOCALHOST, self.port, self.cookie) |
2170 | + self.client_d.callback(client) |
2171 | |
2172 | === added file 'ubuntuone/proxy/tunnel_server.py' |
2173 | --- ubuntuone/proxy/tunnel_server.py 1970-01-01 00:00:00 +0000 |
2174 | +++ ubuntuone/proxy/tunnel_server.py 2012-03-20 23:05:30 +0000 |
2175 | @@ -0,0 +1,376 @@ |
2176 | +# -*- coding: utf8 -*- |
2177 | +# |
2178 | +# Copyright 2012 Canonical Ltd. |
2179 | +# |
2180 | +# This program is free software: you can redistribute it and/or modify it |
2181 | +# under the terms of the GNU General Public License version 3, as published |
2182 | +# by the Free Software Foundation. |
2183 | +# |
2184 | +# This program is distributed in the hope that it will be useful, but |
2185 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
2186 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
2187 | +# PURPOSE. See the GNU General Public License for more details. |
2188 | +# |
2189 | +# You should have received a copy of the GNU General Public License along |
2190 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
2191 | +"""A tunnel through proxies. |
2192 | + |
2193 | +The layers in a tunneled proxied connection: |
2194 | + |
2195 | +↓ tunnelclient - initiates tcp to tunnelserver, request outward connection |
2196 | +↕ client protocol - started after the tunneclient gets connected |
2197 | +---process boundary--- |
2198 | +↕ tunnelserver - creates a tunnel instance per incoming connection |
2199 | +↕ tunnel - hold a qtcpsocket to tunnelclient, and srvtunnelproto to the remote |
2200 | +↕ servertunnelprotocol - gets CONNECT from tunnelclient, creates a remotesocket |
2201 | +↕ remotesocket - connects to the destination server via a proxy |
2202 | +↕ proxy server - goes thru firewalls |
2203 | +↑ server - dialogues with the client protocol |
2204 | + |
2205 | +""" |
2206 | + |
2207 | +import sys |
2208 | +import uuid |
2209 | + |
2210 | +from PyQt4.QtCore import QCoreApplication, QTimer |
2211 | +from PyQt4.QtNetwork import ( |
2212 | + QAbstractSocket, |
2213 | + QHostAddress, |
2214 | + QNetworkProxy, |
2215 | + QNetworkProxyQuery, |
2216 | + QNetworkProxyFactory, |
2217 | + QTcpServer, |
2218 | + QTcpSocket, |
2219 | +) |
2220 | +from twisted.internet import defer, interfaces |
2221 | +from zope.interface import implements |
2222 | + |
2223 | +from ubuntu_sso.keyring import Keyring |
2224 | +from ubuntu_sso.utils.webclient import gsettings |
2225 | +from ubuntuone.proxy.common import ( |
2226 | + BaseTunnelProtocol, |
2227 | + CRLF, |
2228 | + TUNNEL_COOKIE_HEADER, |
2229 | + TUNNEL_COOKIE_LABEL, |
2230 | + TUNNEL_PORT_LABEL, |
2231 | +) |
2232 | +from ubuntuone.proxy.logger import logger |
2233 | + |
2234 | +DEFAULT_CODE = 500 |
2235 | +DEFAULT_DESCRIPTION = "Connection error" |
2236 | + |
2237 | + |
2238 | +class ConnectionError(Exception): |
2239 | + """The client failed connecting to the destination.""" |
2240 | + |
2241 | + def __init__(self, code=DEFAULT_CODE, description=DEFAULT_DESCRIPTION): |
2242 | + self.code = code |
2243 | + self.description = description |
2244 | + |
2245 | + |
2246 | +class ProxyAuthenticationError(ConnectionError): |
2247 | + """Credentials mismatch going thru a proxy.""" |
2248 | + |
2249 | + |
2250 | +def build_proxy(settings_groups): |
2251 | + """Create a QNetworkProxy from these settings.""" |
2252 | + proxy_groups = [ |
2253 | + ("socks", QNetworkProxy.Socks5Proxy), |
2254 | + ("https", QNetworkProxy.HttpProxy), |
2255 | + ("http", QNetworkProxy.HttpProxy), |
2256 | + ] |
2257 | + for group, proxy_type in proxy_groups: |
2258 | + if group not in settings_groups: |
2259 | + continue |
2260 | + settings = settings_groups[group] |
2261 | + if "host" in settings and "port" in settings: |
2262 | + return QNetworkProxy(proxy_type, |
2263 | + hostName=settings.get("host", ""), |
2264 | + port=settings.get("port", 0), |
2265 | + user=settings.get("username", ""), |
2266 | + password=settings.get("password", "")) |
2267 | + logger.error("No proxy correctly configured.") |
2268 | + return QNetworkProxy(QNetworkProxy.DefaultProxy) |
2269 | + |
2270 | + |
2271 | +class RemoteSocket(QTcpSocket): |
2272 | + """A dumb connection through a proxy to a destination hostport.""" |
2273 | + |
2274 | + def __init__(self, tunnel_protocol): |
2275 | + """Initialize this object.""" |
2276 | + super(RemoteSocket, self).__init__() |
2277 | + self.protocol = tunnel_protocol |
2278 | + self.connected_d = defer.Deferred() |
2279 | + self.connected.connect(self.handle_connected) |
2280 | + self.proxyAuthenticationRequired.connect(self.handle_auth_required) |
2281 | + self.buffered_data = [] |
2282 | + |
2283 | + def handle_connected(self): |
2284 | + """When connected, send all pending data.""" |
2285 | + self.disconnected.connect(self.handle_disconnected) |
2286 | + self.connected_d.callback(None) |
2287 | + for d in self.buffered_data: |
2288 | + logger.debug("writing remote: %d bytes", len(d)) |
2289 | + super(RemoteSocket, self).write(d) |
2290 | + self.buffered_data = [] |
2291 | + |
2292 | + def handle_disconnected(self): |
2293 | + """Do something with disconnections.""" |
2294 | + logger.debug("Remote socket disconnected") |
2295 | + self.protocol.remote_disconnected() |
2296 | + |
2297 | + def write(self, data): |
2298 | + """Write data to the remote end, buffering if not connected.""" |
2299 | + if self.state() == QAbstractSocket.ConnectedState: |
2300 | + logger.debug("writing remote: %d bytes", len(data)) |
2301 | + super(RemoteSocket, self).write(data) |
2302 | + else: |
2303 | + self.buffered_data.append(data) |
2304 | + |
2305 | + def connect(self, hostport): |
2306 | + """Try to establish the connection to the remote end.""" |
2307 | + host, port = hostport.split(":") |
2308 | + |
2309 | + try: |
2310 | + port = int(port) |
2311 | + except ValueError: |
2312 | + raise ConnectionError(400, "Destination port must be an integer.") |
2313 | + |
2314 | + self.readyRead.connect(self.handle_ready_read) |
2315 | + self.error.connect(self.handle_error) |
2316 | + self.connectToHost(host, port) |
2317 | + |
2318 | + return self.connected_d |
2319 | + |
2320 | + def handle_auth_required(self, proxy, authenticator): |
2321 | + """Handle the proxyAuthenticationRequired signal.""" |
2322 | + self.protocol.proxy_auth_required(proxy, authenticator) |
2323 | + |
2324 | + def handle_error(self, socket_error): |
2325 | + """Some error happened while connecting.""" |
2326 | + error_description = "%s (%d)" % (self.errorString(), socket_error) |
2327 | + logger.error("connection error: %s", error_description) |
2328 | + if self.connected_d.called: |
2329 | + return |
2330 | + |
2331 | + if socket_error == self.ProxyAuthenticationRequiredError: |
2332 | + error = ProxyAuthenticationError(407, error_description) |
2333 | + else: |
2334 | + error = ConnectionError(500, error_description) |
2335 | + |
2336 | + self.connected_d.errback(error) |
2337 | + |
2338 | + def handle_ready_read(self): |
2339 | + """Forward data from the remote end to the parent protocol.""" |
2340 | + data = self.readAll() |
2341 | + self.protocol.response_data_received(data) |
2342 | + |
2343 | + @defer.inlineCallbacks |
2344 | + def stop(self): |
2345 | + """Finish and cleanup.""" |
2346 | + self.disconnectFromHost() |
2347 | + while self.state() != self.UnconnectedState: |
2348 | + d = defer.Deferred() |
2349 | + QTimer.singleShot(100, lambda: d.callback(None)) |
2350 | + yield d |
2351 | + |
2352 | + |
2353 | +class ServerTunnelProtocol(BaseTunnelProtocol): |
2354 | + """CONNECT sever protocol for tunnelling connections.""" |
2355 | + |
2356 | + def __init__(self, client_class): |
2357 | + """Initialize this protocol.""" |
2358 | + BaseTunnelProtocol.__init__(self) |
2359 | + self.hostport = "" |
2360 | + self.client = None |
2361 | + self.client_class = client_class |
2362 | + self.proxy_credentials = None |
2363 | + self.proxy_domain = None |
2364 | + |
2365 | + def error_response(self, code, description): |
2366 | + """Write a response with an error, and disconnect.""" |
2367 | + self.write_transport("HTTP/1.0 %d %s" % (code, description) + CRLF * 2) |
2368 | + self.transport.loseConnection() |
2369 | + if self.client: |
2370 | + self.client.stop() |
2371 | + self.clearLineBuffer() |
2372 | + |
2373 | + def write_transport(self, data): |
2374 | + """Write a response in the transport.""" |
2375 | + self.transport.write(data) |
2376 | + |
2377 | + def proxy_auth_required(self, proxy, authenticator): |
2378 | + """Proxy authentication is required.""" |
2379 | + logger.info("auth_required %r, %r, %r", self.proxy_credentials, |
2380 | + proxy.hostName(), self.proxy_domain) |
2381 | + if self.proxy_credentials and proxy.hostName() == self.proxy_domain: |
2382 | + logger.info("Credentials added to authenticator: %r.", |
2383 | + self.proxy_credentials) |
2384 | + authenticator.setUser(self.proxy_credentials["username"]) |
2385 | + authenticator.setPassword(self.proxy_credentials["password"]) |
2386 | + else: |
2387 | + logger.info("Credentials needed, but none available.") |
2388 | + self.proxy_domain = proxy.hostName() |
2389 | + |
2390 | + def handle_first_line(self, line): |
2391 | + """Special handling for the first line received.""" |
2392 | + try: |
2393 | + method, hostport, proto_version = line.split(" ", 2) |
2394 | + if proto_version != "HTTP/1.0": |
2395 | + self.error_response(505, "HTTP Version Not Supported") |
2396 | + return |
2397 | + if method != "CONNECT": |
2398 | + self.error_response(405, "Only the CONNECT method is allowed") |
2399 | + return |
2400 | + self.hostport = hostport |
2401 | + except ValueError: |
2402 | + self.error_response(400, "Bad request") |
2403 | + |
2404 | + def verify_cookie(self): |
2405 | + """Fail if the cookie is wrong or missing.""" |
2406 | + cookie_received = dict(self.received_headers).get(TUNNEL_COOKIE_HEADER) |
2407 | + if cookie_received != self.transport.cookie: |
2408 | + raise ConnectionError(418, "Please see RFC 2324") |
2409 | + |
2410 | + @defer.inlineCallbacks |
2411 | + def headers_done(self): |
2412 | + """An empty line was received, start connecting and switch mode.""" |
2413 | + try: |
2414 | + self.verify_cookie() |
2415 | + try: |
2416 | + logger.info("Connecting once") |
2417 | + self.client = self.client_class(self) |
2418 | + yield self.client.connect(self.hostport) |
2419 | + except ProxyAuthenticationError: |
2420 | + if not self.proxy_domain: |
2421 | + logger.info("No proxy domain defined") |
2422 | + raise |
2423 | + |
2424 | + credentials = yield Keyring().get_credentials( |
2425 | + str(self.proxy_domain)) |
2426 | + if "username" in credentials: |
2427 | + self.proxy_credentials = credentials |
2428 | + logger.info("Connecting again with keyring credentials") |
2429 | + self.client = self.client_class(self) |
2430 | + yield self.client.connect(self.hostport) |
2431 | + logger.info("Connected with keyring credentials") |
2432 | + |
2433 | + response_headers = { |
2434 | + "Server": "Ubuntu One proxy tunnel", |
2435 | + } |
2436 | + self.write_transport("HTTP/1.0 200 Proxy connection established" + |
2437 | + CRLF + self.format_headers(response_headers) + |
2438 | + CRLF) |
2439 | + except ConnectionError as e: |
2440 | + logger.exception("Connection error") |
2441 | + self.error_response(e.code, e.description) |
2442 | + except Exception: |
2443 | + logger.exception("Unhandled problem while connecting") |
2444 | + |
2445 | + def rawDataReceived(self, data): |
2446 | + """Tunnel all raw data straight to the other side.""" |
2447 | + self.client.write(data) |
2448 | + |
2449 | + def response_data_received(self, data): |
2450 | + """Return data coming from the other side.""" |
2451 | + self.write_transport(data) |
2452 | + |
2453 | + |
2454 | +class Tunnel(object): |
2455 | + """An instance of a running tunnel.""" |
2456 | + |
2457 | + implements(interfaces.ITransport) |
2458 | + |
2459 | + def __init__(self, local_socket, cookie): |
2460 | + """Initialize this Tunnel instance.""" |
2461 | + self.cookie = cookie |
2462 | + self.disconnecting = False |
2463 | + self.local_socket = local_socket |
2464 | + self.protocol = ServerTunnelProtocol(RemoteSocket) |
2465 | + self.protocol.transport = self |
2466 | + local_socket.readyRead.connect(self.server_ready_read) |
2467 | + local_socket.disconnected.connect(self.local_disconnected) |
2468 | + |
2469 | + def server_ready_read(self): |
2470 | + """Data available on the local end. Move it forward.""" |
2471 | + data = bytes(self.local_socket.readAll()) |
2472 | + self.protocol.dataReceived(data) |
2473 | + |
2474 | + def write(self, data): |
2475 | + """Data available on the remote end. Bring it back.""" |
2476 | + logger.debug("writing local: %d bytes", len(data)) |
2477 | + self.local_socket.write(data) |
2478 | + |
2479 | + def loseConnection(self): |
2480 | + """The remote end disconnected.""" |
2481 | + logger.debug("disconnecting local end.") |
2482 | + self.local_socket.close() |
2483 | + |
2484 | + def local_disconnected(self): |
2485 | + """The local end disconnected.""" |
2486 | + logger.debug("The local socket got disconnected.") |
2487 | + # TODO: handle this case in an upcoming branch |
2488 | + |
2489 | + |
2490 | +class TunnelServer(object): |
2491 | + """A server for tunnel instances.""" |
2492 | + |
2493 | + def __init__(self, cookie): |
2494 | + """Initialize this tunnel instance.""" |
2495 | + self.tunnels = [] |
2496 | + self.cookie = cookie |
2497 | + self.server = QTcpServer(QCoreApplication.instance()) |
2498 | + self.server.newConnection.connect(self.new_connection) |
2499 | + self.server.listen(QHostAddress.LocalHost, 0) |
2500 | + logger.info("Starting tunnel server at port %d", self.port) |
2501 | + |
2502 | + def new_connection(self): |
2503 | + """On a new connection create a new tunnel instance.""" |
2504 | + logger.info("New connection made") |
2505 | + local_socket = self.server.nextPendingConnection() |
2506 | + tunnel = Tunnel(local_socket, self.cookie) |
2507 | + self.tunnels.append(tunnel) |
2508 | + |
2509 | + def shutdown(self): |
2510 | + """Terminate every connection.""" |
2511 | + # TODO: handle this gracefully in an upcoming branch |
2512 | + |
2513 | + @property |
2514 | + def port(self): |
2515 | + """The port where this server listens.""" |
2516 | + return self.server.serverPort() |
2517 | + |
2518 | + |
2519 | +def check_proxy_enabled(host, port): |
2520 | + """Check if the proxy is enabled.""" |
2521 | + port = int(port) |
2522 | + if sys.platform.startswith("linux"): |
2523 | + settings = gsettings.get_proxy_settings() |
2524 | + enabled = len(settings) > 0 |
2525 | + if enabled: |
2526 | + proxy = build_proxy(settings) |
2527 | + QNetworkProxy.setApplicationProxy(proxy) |
2528 | + else: |
2529 | + logger.info("Proxy is disabled.") |
2530 | + return enabled |
2531 | + else: |
2532 | + query = QNetworkProxyQuery(host, port) |
2533 | + proxies = QNetworkProxyFactory.systemProxyForQuery(query) |
2534 | + return len(proxies) and proxies[0].type() != QNetworkProxy.DefaultProxy |
2535 | + |
2536 | + |
2537 | +def main(argv): |
2538 | + """The main function for the tunnel server.""" |
2539 | + if not check_proxy_enabled(*argv[1:]): |
2540 | + sys.stdout.write("Proxy not enabled.") |
2541 | + sys.stdout.flush() |
2542 | + else: |
2543 | + from dbus.mainloop.qt import DBusQtMainLoop |
2544 | + DBusQtMainLoop(set_as_default=True) |
2545 | + app = QCoreApplication(argv) |
2546 | + cookie = str(uuid.uuid4()) |
2547 | + tunnel_server = TunnelServer(cookie) |
2548 | + sys.stdout.write("%s: %d\n" % (TUNNEL_PORT_LABEL, tunnel_server.port) + |
2549 | + "%s: %s\n" % (TUNNEL_COOKIE_LABEL, cookie)) |
2550 | + sys.stdout.flush() |
2551 | + app.exec_() |
2552 | |
2553 | === modified file 'ubuntuone/syncdaemon/action_queue.py' |
2554 | --- ubuntuone/syncdaemon/action_queue.py 2012-02-09 21:57:00 +0000 |
2555 | +++ ubuntuone/syncdaemon/action_queue.py 2012-03-20 23:05:30 +0000 |
2556 | @@ -31,19 +31,18 @@ |
2557 | from collections import deque, defaultdict |
2558 | from functools import partial |
2559 | from urllib import urlencode |
2560 | -from urllib2 import urlopen, Request, HTTPError |
2561 | from urlparse import urljoin |
2562 | |
2563 | import OpenSSL.SSL |
2564 | |
2565 | from zope.interface import implements |
2566 | -from twisted.internet import reactor, defer, threads, task |
2567 | +from twisted.internet import reactor, defer, task |
2568 | from twisted.internet import error as twisted_errors |
2569 | from twisted.names import client as dns_client |
2570 | from twisted.python.failure import Failure, DefaultException |
2571 | |
2572 | from oauth import oauth |
2573 | -from ubuntu_sso.utils import timestamp_checker |
2574 | +from ubuntu_sso.utils.webclient import txweb |
2575 | from ubuntuone import clientdefs |
2576 | from ubuntuone.platform import platform, remove_file |
2577 | from ubuntuone.storageprotocol import protocol_pb2, content_hash |
2578 | @@ -57,6 +56,7 @@ |
2579 | from ubuntuone.syncdaemon.logger import mklog, TRACE |
2580 | from ubuntuone.syncdaemon.volume_manager import ACCESS_LEVEL_RW |
2581 | from ubuntuone.syncdaemon import config, offload_queue |
2582 | +from ubuntuone.syncdaemon import tunnel_runner |
2583 | |
2584 | logger = logging.getLogger("ubuntuone.SyncDaemon.ActionQueue") |
2585 | |
2586 | @@ -684,6 +684,8 @@ |
2587 | # credentials |
2588 | self.token = None |
2589 | self.consumer = None |
2590 | + self.credentials = None |
2591 | + self.webclient = None |
2592 | |
2593 | self.client = None # an instance of self.protocol |
2594 | |
2595 | @@ -699,6 +701,7 @@ |
2596 | self.zip_queue = ZipQueue() |
2597 | self.conditions_locker = ConditionsLocker() |
2598 | self.disk_queue = offload_queue.OffloadQueue() |
2599 | + self.tunnel_runner = tunnel_runner.TunnelRunner(self.host, self.port) |
2600 | |
2601 | self.estimated_free_space = {} |
2602 | event_queue.subscribe(self) |
2603 | @@ -728,6 +731,7 @@ |
2604 | |
2605 | def handle_SYS_USER_CONNECT(self, access_token): |
2606 | """Stow the access token away for later use.""" |
2607 | + self.credentials = access_token |
2608 | self.token = oauth.OAuthToken(access_token['token'], |
2609 | access_token['token_secret']) |
2610 | self.consumer = oauth.OAuthConsumer(access_token['consumer_key'], |
2611 | @@ -816,16 +820,37 @@ |
2612 | else: |
2613 | return defer.succeed((self.host, self.port)) |
2614 | |
2615 | + |
2616 | + @defer.inlineCallbacks |
2617 | + def webcall(self, iri, **kwargs): |
2618 | + """Perform a web call to the api servers.""" |
2619 | + webclient = yield self.get_webclient() |
2620 | + response = yield webclient.request(iri, |
2621 | + oauth_credentials=self.credentials, **kwargs) |
2622 | + defer.returnValue(response) |
2623 | + |
2624 | + @defer.inlineCallbacks |
2625 | + def get_webclient(self): |
2626 | + """Get the webclient, creating it if needed.""" |
2627 | + if self.webclient is None: |
2628 | + client = yield self.tunnel_runner.get_client() |
2629 | + self.webclient = txweb.WebClient(connector=client, |
2630 | + appname="Ubuntu One", |
2631 | + oauth_sign_plain=True) |
2632 | + defer.returnValue(self.webclient) |
2633 | + |
2634 | + @defer.inlineCallbacks |
2635 | def _make_connection(self, result): |
2636 | """Do the real connect call.""" |
2637 | host, port = result |
2638 | ssl_context = get_ssl_context(self.disable_ssl_verify) |
2639 | + client = yield self.tunnel_runner.get_client() |
2640 | if self.use_ssl: |
2641 | - self.connector = reactor.connectSSL(host, port, factory=self, |
2642 | + self.connector = client.connectSSL(host, port, factory=self, |
2643 | contextFactory=ssl_context, |
2644 | timeout=self.connection_timeout) |
2645 | else: |
2646 | - self.connector = reactor.connectTCP(host, port, self, |
2647 | + self.connector = client.connectTCP(host, port, self, |
2648 | timeout=self.connection_timeout) |
2649 | |
2650 | def connect(self): |
2651 | @@ -1774,36 +1799,23 @@ |
2652 | if share_to and re.match(EREGEX, share_to): |
2653 | self.use_http = True |
2654 | |
2655 | + @defer.inlineCallbacks |
2656 | def _create_share_http(self, node_id, user, name, read_only): |
2657 | """Create a share using the HTTP Web API method.""" |
2658 | |
2659 | - url = "https://one.ubuntu.com/files/api/offer_share/" |
2660 | - method = oauth.OAuthSignatureMethod_PLAINTEXT() |
2661 | - timestamp = timestamp_checker.get_faithful_time() |
2662 | - parameters = {"oauth_timestamp": timestamp} |
2663 | - request = oauth.OAuthRequest.from_consumer_and_token( |
2664 | - http_url=url, |
2665 | - http_method="POST", |
2666 | - parameters=parameters, |
2667 | - oauth_consumer=self.action_queue.consumer, |
2668 | - token=self.action_queue.token) |
2669 | - request.sign_request(method, self.action_queue.consumer, |
2670 | - self.action_queue.token) |
2671 | + iri = u"https://one.ubuntu.com/files/api/offer_share/" |
2672 | data = dict(offer_to_email=user, |
2673 | read_only=read_only, |
2674 | node_id=node_id, |
2675 | share_name=name) |
2676 | pdata = urlencode(data) |
2677 | - headers = request.to_header() |
2678 | - req = Request(url, pdata, headers) |
2679 | - urlopen(req) |
2680 | + yield self.action_queue.webcall(iri, method="POST", post_content=pdata) |
2681 | |
2682 | def _run(self): |
2683 | """Do the actual running.""" |
2684 | if self.use_http: |
2685 | # External user, do the HTTP REST method |
2686 | - return threads.deferToThread(self._create_share_http, |
2687 | - self.node_id, self.share_to, |
2688 | + return self._create_share_http(self.node_id, self.share_to, |
2689 | self.name, |
2690 | self.access_level != ACCESS_LEVEL_RW) |
2691 | else: |
2692 | @@ -1827,7 +1839,7 @@ |
2693 | """It didn't work! Push the event.""" |
2694 | self.action_queue.event_queue.push('AQ_CREATE_SHARE_ERROR', |
2695 | marker=self.marker, |
2696 | - error=failure.getErrorMessage()) |
2697 | + error=failure.value[1]) |
2698 | |
2699 | def _acquire_pathlock(self): |
2700 | """Acquire pathlock.""" |
2701 | @@ -2109,6 +2121,7 @@ |
2702 | self.node_id = node_id |
2703 | self.is_public = is_public |
2704 | |
2705 | + @defer.inlineCallbacks |
2706 | def _change_public_access_http(self): |
2707 | """Change public access using the HTTP Web API method.""" |
2708 | |
2709 | @@ -2119,28 +2132,16 @@ |
2710 | base64.urlsafe_b64encode(self.share_id.bytes).strip("="), |
2711 | node_key) |
2712 | |
2713 | - url = "https://one.ubuntu.com/files/api/set_public/%s" % (node_key,) |
2714 | - method = oauth.OAuthSignatureMethod_PLAINTEXT() |
2715 | - timestamp = timestamp_checker.get_faithful_time() |
2716 | - parameters = {"oauth_timestamp": timestamp} |
2717 | - request = oauth.OAuthRequest.from_consumer_and_token( |
2718 | - http_url=url, |
2719 | - http_method="POST", |
2720 | - parameters=parameters, |
2721 | - oauth_consumer=self.action_queue.consumer, |
2722 | - token=self.action_queue.token) |
2723 | - request.sign_request(method, self.action_queue.consumer, |
2724 | - self.action_queue.token) |
2725 | + iri = u"https://one.ubuntu.com/files/api/set_public/%s" % (node_key,) |
2726 | data = dict(is_public=bool(self.is_public)) |
2727 | pdata = urlencode(data) |
2728 | - headers = request.to_header() |
2729 | - req = Request(url, pdata, headers) |
2730 | - response = urlopen(req) |
2731 | - return simplejson.load(response) |
2732 | + response = yield self.action_queue.webcall(iri, method="POST", |
2733 | + post_content=pdata) |
2734 | + defer.returnValue(simplejson.loads(response.content)) |
2735 | |
2736 | def _run(self): |
2737 | """See ActionQueueCommand.""" |
2738 | - return threads.deferToThread(self._change_public_access_http) |
2739 | + return self._change_public_access_http() |
2740 | |
2741 | def handle_success(self, success): |
2742 | """See ActionQueueCommand.""" |
2743 | @@ -2152,51 +2153,36 @@ |
2744 | |
2745 | def handle_failure(self, failure): |
2746 | """It didn't work! Push the event.""" |
2747 | - if issubclass(failure.type, HTTPError): |
2748 | - message = failure.value.read() |
2749 | - else: |
2750 | - message = failure.getErrorMessage() |
2751 | self.action_queue.event_queue.push('AQ_CHANGE_PUBLIC_ACCESS_ERROR', |
2752 | share_id=self.share_id, |
2753 | node_id=self.node_id, |
2754 | - error=message) |
2755 | + error=failure.value[1]) |
2756 | |
2757 | |
2758 | class GetPublicFiles(ActionQueueCommand): |
2759 | """Get the list of public files.""" |
2760 | |
2761 | - __slots__ = ('_url',) |
2762 | + __slots__ = ('_iri',) |
2763 | logged_attrs = ActionQueueCommand.logged_attrs + __slots__ |
2764 | |
2765 | - def __init__(self, request_queue, base_url='https://one.ubuntu.com'): |
2766 | + def __init__(self, request_queue, base_iri=u'https://one.ubuntu.com'): |
2767 | super(GetPublicFiles, self).__init__(request_queue) |
2768 | - self._url = urljoin(base_url, 'files/api/public_files') |
2769 | + self._iri = urljoin(base_iri, u'files/api/public_files') |
2770 | |
2771 | + @defer.inlineCallbacks |
2772 | def _get_public_files_http(self): |
2773 | """Get public files list using the HTTP Web API method.""" |
2774 | |
2775 | - method = oauth.OAuthSignatureMethod_PLAINTEXT() |
2776 | - timestamp = timestamp_checker.get_faithful_time() |
2777 | - parameters = {"oauth_timestamp": timestamp} |
2778 | - request = oauth.OAuthRequest.from_consumer_and_token( |
2779 | - http_url=self._url, |
2780 | - http_method="GET", |
2781 | - parameters=parameters, |
2782 | - oauth_consumer=self.action_queue.consumer, |
2783 | - token=self.action_queue.token) |
2784 | - request.sign_request(method, self.action_queue.consumer, |
2785 | - self.action_queue.token) |
2786 | - headers = request.to_header() |
2787 | - req = Request(self._url, headers=headers) |
2788 | - response = urlopen(req) |
2789 | - files = simplejson.load(response) |
2790 | + response = yield self.action_queue.webcall(self._iri, method="GET") |
2791 | + |
2792 | + files = simplejson.loads(response.content) |
2793 | # translate nodekeys to (volume_id, node_id) |
2794 | for pf in files: |
2795 | _, node_id = self.split_nodekey(pf.pop('nodekey')) |
2796 | volume_id = pf['volume_id'] |
2797 | pf['volume_id'] = '' if volume_id is None else volume_id |
2798 | pf['node_id'] = node_id |
2799 | - return files |
2800 | + defer.returnValue(files) |
2801 | |
2802 | @property |
2803 | def uniqueness(self): |
2804 | @@ -2209,7 +2195,7 @@ |
2805 | |
2806 | def _run(self): |
2807 | """See ActionQueueCommand.""" |
2808 | - return threads.deferToThread(self._get_public_files_http) |
2809 | + return self._get_public_files_http() |
2810 | |
2811 | def handle_success(self, success): |
2812 | """See ActionQueueCommand.""" |
2813 | @@ -2218,12 +2204,8 @@ |
2814 | |
2815 | def handle_failure(self, failure): |
2816 | """It didn't work! Push the event.""" |
2817 | - if issubclass(failure.type, HTTPError): |
2818 | - message = failure.value.read() |
2819 | - else: |
2820 | - message = failure.getErrorMessage() |
2821 | self.action_queue.event_queue.push('AQ_PUBLIC_FILES_LIST_ERROR', |
2822 | - error=message) |
2823 | + error=failure.value[1]) |
2824 | |
2825 | def split_nodekey(self, nodekey): |
2826 | """Split a node key into a share_id, node_id.""" |
2827 | |
2828 | === added file 'ubuntuone/syncdaemon/tunnel_runner.py' |
2829 | --- ubuntuone/syncdaemon/tunnel_runner.py 1970-01-01 00:00:00 +0000 |
2830 | +++ ubuntuone/syncdaemon/tunnel_runner.py 2012-03-20 23:05:30 +0000 |
2831 | @@ -0,0 +1,78 @@ |
2832 | +# -*- coding: utf8 -*- |
2833 | +# |
2834 | +# Copyright 2012 Canonical Ltd. |
2835 | +# |
2836 | +# This program is free software: you can redistribute it and/or modify it |
2837 | +# under the terms of the GNU General Public License version 3, as published |
2838 | +# by the Free Software Foundation. |
2839 | +# |
2840 | +# This program is distributed in the hope that it will be useful, but |
2841 | +# WITHOUT ANY WARRANTY; without even the implied warranties of |
2842 | +# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
2843 | +# PURPOSE. See the GNU General Public License for more details. |
2844 | +# |
2845 | +# You should have received a copy of the GNU General Public License along |
2846 | +# with this program. If not, see <http://www.gnu.org/licenses/>. |
2847 | +"""Run the tunnel process and start a client, with a reactor as a fallback.""" |
2848 | + |
2849 | +import logging |
2850 | +import sys |
2851 | + |
2852 | +from os import path |
2853 | + |
2854 | +from twisted.internet import defer, reactor |
2855 | + |
2856 | +from ubuntuone.platform.constants import TUNNEL_EXECUTABLE |
2857 | + |
2858 | +logger = logging.getLogger("ubuntuone.SyncDaemon.TunnelRunner") |
2859 | + |
2860 | + |
2861 | +class TunnelRunner(object): |
2862 | + """Run a tunnel process.""" |
2863 | + |
2864 | + def __init__(self, host, port): |
2865 | + """Start this runner instance.""" |
2866 | + self.client_d = defer.Deferred() |
2867 | + try: |
2868 | + self.start_process(host, port) |
2869 | + except ImportError: |
2870 | + logger.info("Proxy support not installed.") |
2871 | + self.client_d.callback(reactor) |
2872 | + |
2873 | + def start_process(self, host, port): |
2874 | + """Start the tunnel process.""" |
2875 | + from ubuntuone.proxy.tunnel_client import TunnelProcessProtocol |
2876 | + protocol = TunnelProcessProtocol(self.client_d) |
2877 | + process_path = self.get_process_path() |
2878 | + args = [TUNNEL_EXECUTABLE, host, str(port)] |
2879 | + reactor.spawnProcess(protocol, process_path, env=None, args=args) |
2880 | + |
2881 | + def get_process_path(self): |
2882 | + """Get the path to the tunnel process.""" |
2883 | + # In a frozen setting, all binaries are in the same path |
2884 | + if getattr(sys, "frozen", None) is not None: |
2885 | + return path.join(path.dirname( |
2886 | + path.abspath(sys.executable)), TUNNEL_EXECUTABLE) |
2887 | + |
2888 | + # This works when we are running from sources |
2889 | + filename = path.join(path.dirname(__file__), "..", "..", "bin", |
2890 | + TUNNEL_EXECUTABLE) |
2891 | + if path.isfile(filename): |
2892 | + return filename |
2893 | + |
2894 | + # Use the configured LIBEXECDIR |
2895 | + from ubuntuone.clientdefs import LIBEXECDIR |
2896 | + return path.join(LIBEXECDIR, TUNNEL_EXECUTABLE) |
2897 | + |
2898 | + def get_client(self): |
2899 | + """A deferred with the reactor or a tunnel client.""" |
2900 | + |
2901 | + def client_selected(result, d): |
2902 | + """The tunnel_client or the reactor were selected.""" |
2903 | + d.callback(result) |
2904 | + # make sure the result is available for next callback |
2905 | + return result |
2906 | + |
2907 | + d = defer.Deferred() |
2908 | + self.client_d.addCallback(client_selected, d) |
2909 | + return d |
Looks fine.