Merge lp:~nataliabidart/ubuntuone-client/stable-3-0-update-2.99.91 into lp:ubuntuone-client/stable-3-0

Proposed by Natalia Bidart on 2012-03-20
Status: Merged
Approved by: Natalia Bidart on 2012-03-21
Approved revision: 1178
Merged at revision: 1177
Proposed branch: lp:~nataliabidart/ubuntuone-client/stable-3-0-update-2.99.91
Merge into: lp:ubuntuone-client/stable-3-0
Diff against target: 2909 lines (+2139/-257)
25 files modified
Makefile.am (+3/-1)
bin/u1sdtool (+1/-1)
bin/ubuntuone-proxy-tunnel (+24/-0)
contrib/testing/testcase.py (+15/-1)
data/source_ubuntuone-client.py (+2/-29)
docs/Makefile.am (+0/-1)
docs/man/u1sdtool.1 (+1/-1)
docs/man/ubuntuone-preferences.1 (+0/-16)
tests/platform/linux/test_notification.py (+6/-0)
tests/proxy/__init__.py (+150/-0)
tests/proxy/ssl/dummy.cert (+19/-0)
tests/proxy/ssl/dummy.key (+16/-0)
tests/proxy/test_tunnel_client.py (+212/-0)
tests/proxy/test_tunnel_server.py (+655/-0)
tests/syncdaemon/test_action_queue.py (+102/-137)
tests/syncdaemon/test_tunnel_runner.py (+128/-0)
ubuntuone/platform/constants.py (+5/-0)
ubuntuone/platform/linux/notification.py (+3/-0)
ubuntuone/proxy/__init__.py (+14/-0)
ubuntuone/proxy/common.py (+60/-0)
ubuntuone/proxy/logger.py (+36/-0)
ubuntuone/proxy/tunnel_client.py (+181/-0)
ubuntuone/proxy/tunnel_server.py (+376/-0)
ubuntuone/syncdaemon/action_queue.py (+52/-70)
ubuntuone/syncdaemon/tunnel_runner.py (+78/-0)
To merge this branch: bzr merge lp:~nataliabidart/ubuntuone-client/stable-3-0-update-2.99.91
Reviewer Review Type Date Requested Status
Alejandro J. Cura (community) Approve on 2012-03-21
Roberto Alsina (community) 2012-03-20 Approve on 2012-03-21
Review via email: mp+98535@code.launchpad.net

Commit message

- Updating from trunk up to revno 1213:

[ Alejandro J. Cura <email address hidden> ]
  - Only allow connections that provide the right cookie thru
    the tunnel (LP: #929207).
  - Use proxy credentials from the keyring (LP: #929207)
  - QNetwork must use the proxy built; forward disconnections in the
    tunnel client.
  - Use the txweb webclient from sso for webcalls, so they can be
    proxied too (LP: #929207, LP: #929212).
  - Tunnel storage protocol if proxy enabled in system settings
    (LP: #929208).
  - A proxy tunnel process to be started when proxies are enabled
    (LP: #929207).
  - A client for the proxy tunnel, to be used by syncdaemon
    (LP: #929207).
  - The infrastructure for a QtNetwork based process to tunnel
    syncdaemon traffic thru proxies (LP: #929207).

[ Roberto Alsina <email address hidden> ]
  - Fix tunnel spawning code so that it works on windows.

[ Rodney Dawes <email address hidden> ]
  - Don't attach old useless log files, and clean up the report a
    little (LP: #956407).
  - Fix the typo in the u1sdtool --list-shared help text and man page
    (LP: #682954).
  - Don't install the really old preferences man file we don't need.
  - Set the transient hint on the notifications to True
    (LP: #887369).

To post a comment you must log in.
Roberto Alsina (ralsina) :
review: Approve
Alejandro J. Cura (alecu) wrote :

Looks fine.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Makefile.am'
2--- Makefile.am 2012-02-21 20:34:55 +0000
3+++ Makefile.am 2012-03-20 23:05:30 +0000
4@@ -19,6 +19,7 @@
5
6 libexec_SCRIPTS = \
7 bin/ubuntuone-syncdaemon \
8+ bin/ubuntuone-proxy-tunnel \
9 bin/ubuntuone-login
10
11 clientdefsdir = $(pythonpkgdir)/ubuntuone
12@@ -53,7 +54,8 @@
13 test: logging.conf $(clientdefs_DATA) Makefile
14 echo "$(PYTHONPATH)"
15 if test "x$(builddir)" == "x$(srcdir)"; then \
16- PYTHONPATH="$(PYTHONPATH)" u1trial -r $(REACTOR) -p tests/platform/windows tests; \
17+ PYTHONPATH="$(PYTHONPATH)" u1trial -r $(REACTOR) -p tests/platform/windows,tests/proxy tests; \
18+ PYTHONPATH="$(PYTHONPATH)" u1trial -r qt4 -p tests/platform/windows tests/proxy; \
19 fi
20 rm -rf _trial_temp
21
22
23=== modified file 'bin/u1sdtool'
24--- bin/u1sdtool 2012-01-26 00:46:08 +0000
25+++ bin/u1sdtool 2012-03-20 23:05:30 +0000
26@@ -242,7 +242,7 @@
27 help="Share PATH to USER. ")
28 parser.add_option("", "--list-shared", dest="list_shared",
29 action="store_true",
30- help="List the shared path's/shares offered. ")
31+ help="List the shared paths/shares offered. ")
32 parser.add_option("", "--create-folder", dest="create_folder",
33 metavar="PATH",
34 help="Create user defined folder in the specified path")
35
36=== added file 'bin/ubuntuone-proxy-tunnel'
37--- bin/ubuntuone-proxy-tunnel 1970-01-01 00:00:00 +0000
38+++ bin/ubuntuone-proxy-tunnel 2012-03-20 23:05:30 +0000
39@@ -0,0 +1,24 @@
40+#!/usr/bin/python
41+#
42+# Copyright 2012 Canonical Ltd.
43+#
44+# This program is free software: you can redistribute it and/or modify it
45+# under the terms of the GNU General Public License version 3, as published
46+# by the Free Software Foundation.
47+#
48+# This program is distributed in the hope that it will be useful, but
49+# WITHOUT ANY WARRANTY; without even the implied warranties of
50+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
51+# PURPOSE. See the GNU General Public License for more details.
52+#
53+# You should have received a copy of the GNU General Public License along
54+# with this program. If not, see <http://www.gnu.org/licenses/>.
55+
56+"""Ubuntu One tunnel for proxy support."""
57+
58+import sys
59+
60+from ubuntuone.proxy.tunnel_server import main
61+
62+if __name__ == "__main__":
63+ main(sys.argv)
64
65=== modified file 'contrib/testing/testcase.py'
66--- contrib/testing/testcase.py 2012-02-09 21:57:00 +0000
67+++ contrib/testing/testcase.py 2012-03-20 23:05:30 +0000
68@@ -28,7 +28,7 @@
69 from collections import defaultdict
70 from functools import wraps
71
72-from twisted.internet import defer
73+from twisted.internet import defer, reactor
74 from twisted.trial.unittest import TestCase as TwistedTestCase
75 from ubuntuone.devtools.testcases import skipIfOS
76 from zope.interface import implements
77@@ -267,6 +267,17 @@
78 return defer.succeed(True)
79
80
81+class FakeTunnelRunner(object):
82+ """A fake proxy.tunnel_client.TunnelRunner."""
83+
84+ def __init__(self, *args):
85+ """Fake a proxy tunnel."""
86+
87+ def get_client(self):
88+ """Always return the reactor."""
89+ return defer.succeed(reactor)
90+
91+
92 class BaseTwistedTestCase(TwistedTestCase):
93 """Base TestCase with helper methods to handle temp dir.
94
95@@ -276,6 +287,7 @@
96 makedirs(path): support read-only shares
97 """
98 MAX_FILENAME = 32 # some platforms limit lengths of filenames
99+ tunnel_runner_class = FakeTunnelRunner
100
101 def mktemp(self, name='temp'):
102 """ Customized mktemp that accepts an optional name argument. """
103@@ -380,6 +392,8 @@
104 # Patch the user home
105 self.home_dir = self.mktemp('ubuntuonehacker')
106 self.patch(platform, "xdg_home", self.home_dir)
107+ self.patch(action_queue.tunnel_runner, "TunnelRunner",
108+ self.tunnel_runner_class)
109
110
111 class FakeMainTestCase(BaseTwistedTestCase):
112
113=== modified file 'data/source_ubuntuone-client.py'
114--- data/source_ubuntuone-client.py 2011-02-09 14:40:29 +0000
115+++ data/source_ubuntuone-client.py 2012-03-20 23:05:30 +0000
116@@ -1,6 +1,6 @@
117 # Apport integration for Ubuntu One client
118 #
119-# Copyright 2009 Canonical Ltd.
120+# Copyright 2009-2012 Canonical Ltd.
121 #
122 # This program is free software: you can redistribute it and/or modify it
123 # under the terms of the GNU General Public License version 3, as published
124@@ -17,7 +17,7 @@
125 # pylint: disable-msg=F0401,C0103
126 # shut up about apport. We know. We aren't going to backport it for pqm
127 import apport
128-from apport.hookutils import attach_file_if_exists, packaging
129+from apport.hookutils import attach_file_if_exists
130 import os.path
131 from xdg.BaseDirectory import xdg_cache_home, xdg_config_home
132
133@@ -28,11 +28,8 @@
134 u1_client_log = os.path.join(u1_log_path, "syncdaemon.log")
135 u1_except_log = os.path.join(u1_log_path, "syncdaemon-exceptions.log")
136 u1_invalidnames_log = os.path.join(u1_log_path, "syncdaemon-invalid-names.log")
137-u1_oauth_log = os.path.join(u1_log_path, "oauth-login.log")
138-u1_prefs_log = os.path.join(u1_log_path, "u1-prefs.log")
139 u1_sd_conf = os.path.join("etc", "xdg", "ubuntuone", "syncdaemon.conf")
140 u1_usersd_conf = os.path.join(u1_user_config_path, "syncdaemon.conf")
141-u1_user_conf = os.path.join(u1_user_config_path, "ubuntuone-client.conf")
142
143
144 def add_info(report):
145@@ -41,35 +38,11 @@
146 "UbuntuOneSyncdaemonExceptionsLog")
147 attach_file_if_exists(report, u1_invalidnames_log,
148 "UbuntuOneSyncdaemonInvalidNamesLog")
149- attach_file_if_exists(report, u1_oauth_log,
150- "UbuntuOneOAuthLoginLog")
151- attach_file_if_exists(report, u1_prefs_log,
152- "UbuntuOnePreferencesLog")
153 attach_file_if_exists(report, u1_usersd_conf,
154 "UbuntuOneUserSyncdaemonConfig")
155 attach_file_if_exists(report, u1_sd_conf,
156 "UbuntuOneSyncdaemonConfig")
157- attach_file_if_exists(report, u1_user_conf,
158- "UbuntuOneClientConfig")
159
160 if not apport.packaging.is_distro_package(report['Package'].split()[0]):
161 report['ThirdParty'] = 'True'
162 report['CrashDB'] = 'ubuntuone'
163-
164- packages = ['ubuntuone-client',
165- 'python-ubuntuone-client',
166- 'ubuntuone-client-tools',
167- 'ubuntuone-client-gnome',
168- 'python-ubuntuone-storageprotocol',
169- 'ubuntuone-ppa-beta']
170-
171- versions = ''
172- for package in packages:
173- try:
174- version = packaging.get_version(package)
175- except ValueError:
176- version = 'N/A'
177- if version is None:
178- version = 'N/A'
179- versions += '%s %s\n' % (package, version)
180- report['UbuntuOneClientPackages'] = versions
181
182=== modified file 'docs/Makefile.am'
183--- docs/Makefile.am 2011-02-09 14:40:29 +0000
184+++ docs/Makefile.am 2012-03-20 23:05:30 +0000
185@@ -2,7 +2,6 @@
186
187 manfilesdir = $(mandir)/man1
188 manfiles_DATA = \
189- man/ubuntuone-preferences.1 \
190 man/u1sdtool.1
191
192 EXTRA_DIST=$(manfiles_DATA)
193
194=== modified file 'docs/man/u1sdtool.1'
195--- docs/man/u1sdtool.1 2011-07-02 22:37:52 +0000
196+++ docs/man/u1sdtool.1 2012-03-20 23:05:30 +0000
197@@ -131,7 +131,7 @@
198 .RE
199 .TP
200 \fB\-\-list-shared\fR
201-List the shared path's/shares offered.
202+List the shared paths/shares offered.
203 .TP
204 \fB\-\-create-folder\fR=\fIPATH\fR
205 Create user defined folder in the specified path
206
207=== removed file 'docs/man/ubuntuone-preferences.1'
208--- docs/man/ubuntuone-preferences.1 2010-02-04 23:47:35 +0000
209+++ docs/man/ubuntuone-preferences.1 1970-01-01 00:00:00 +0000
210@@ -1,16 +0,0 @@
211-.TH UBUNTUONE-CLIENT-PREFERENCES 1
212-
213-.SH NAME
214-ubuntuone-client-preferences \- A dialog for configuring Ubuntu One
215-
216-.SH SYNOPSYS
217-.B ubutuone-client-preferences
218-
219-.SH DESCRIPTION
220-This manual page briefly documents the
221-.BR ubuntuone-client-preferences
222-process, which provides a configuration dialog for configuring
223-Ubuntu One file sharing.
224-
225-.SH AUTHOR
226-This manual page was written by Rodney Dawes <rodney.dawes@canonical.com>
227
228=== modified file 'tests/platform/linux/test_notification.py'
229--- tests/platform/linux/test_notification.py 2012-01-26 19:28:55 +0000
230+++ tests/platform/linux/test_notification.py 2012-03-20 23:05:30 +0000
231@@ -73,6 +73,7 @@
232 self._set_up_mock_notify(FAKE_TITLE, FAKE_MESSAGE, ICON_NAME)
233 mock_notification = self.mocker.mock()
234 self.mocker.result(mock_notification)
235+ mock_notification.set_hint('transient', True)
236 mock_notification.show()
237 self.mocker.replay()
238 Notification(FAKE_APP_NAME).send_notification(FAKE_TITLE, FAKE_MESSAGE)
239@@ -82,9 +83,11 @@
240 self._set_up_mock_notify(FAKE_TITLE, FAKE_MESSAGE, ICON_NAME)
241 mock_notification = self.mocker.mock()
242 self.mocker.result(mock_notification)
243+ mock_notification.set_hint('transient', True)
244 mock_notification.show()
245 mock_notification.update(
246 FAKE_TITLE + '2', FAKE_MESSAGE + '2', ICON_NAME)
247+ mock_notification.set_hint('transient', True)
248 mock_notification.show()
249 self.mocker.replay()
250 notifier = Notification(FAKE_APP_NAME)
251@@ -96,6 +99,7 @@
252 self._set_up_mock_notify(FAKE_TITLE, FAKE_MESSAGE, FAKE_ICON)
253 mock_notification = self.mocker.mock()
254 self.mocker.result(mock_notification)
255+ mock_notification.set_hint('transient', True)
256 mock_notification.show()
257 self.mocker.replay()
258 Notification(FAKE_APP_NAME).send_notification(
259@@ -107,9 +111,11 @@
260 mock_notification = self.mocker.mock()
261 self.mocker.result(mock_notification)
262 mock_notification.set_hint_string('x-canonical-append', '')
263+ mock_notification.set_hint('transient', True)
264 mock_notification.show()
265 mock_notification.update(FAKE_TITLE, FAKE_APPENDAGE, ICON_NAME)
266 mock_notification.set_hint_string('x-canonical-append', '')
267+ mock_notification.set_hint('transient', True)
268 mock_notification.show()
269 self.mocker.replay()
270 notifier = Notification(FAKE_APP_NAME)
271
272=== added directory 'tests/proxy'
273=== added file 'tests/proxy/__init__.py'
274--- tests/proxy/__init__.py 1970-01-01 00:00:00 +0000
275+++ tests/proxy/__init__.py 2012-03-20 23:05:30 +0000
276@@ -0,0 +1,150 @@
277+# Copyright 2012 Canonical Ltd.
278+#
279+# This program is free software: you can redistribute it and/or modify it
280+# under the terms of the GNU General Public License version 3, as published
281+# by the Free Software Foundation.
282+#
283+# This program is distributed in the hope that it will be useful, but
284+# WITHOUT ANY WARRANTY; without even the implied warranties of
285+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
286+# PURPOSE. See the GNU General Public License for more details.
287+#
288+# You should have received a copy of the GNU General Public License along
289+# with this program. If not, see <http://www.gnu.org/licenses/>.
290+"""Tests for the Ubuntu One proxy support."""
291+
292+from os import path
293+from StringIO import StringIO
294+
295+from twisted.application import internet, service
296+from twisted.internet import defer, ssl
297+from twisted.web import http, resource, server
298+
299+SAMPLE_CONTENT = "hello world!"
300+SIMPLERESOURCE = "simpleresource"
301+DUMMY_KEY_FILENAME = "dummy.key"
302+DUMMY_CERT_FILENAME = "dummy.cert"
303+FAKE_COOKIE = "fa:ke:co:ok:ie"
304+
305+
306+class SaveHTTPChannel(http.HTTPChannel):
307+ """A save protocol to be used in tests."""
308+
309+ protocolInstance = None
310+
311+ # pylint: disable=C0103
312+ def connectionMade(self):
313+ """Keep track of the given protocol."""
314+ SaveHTTPChannel.protocolInstance = self
315+ http.HTTPChannel.connectionMade(self)
316+
317+
318+class SaveSite(server.Site):
319+ """A site that let us know when it's closed."""
320+
321+ protocol = SaveHTTPChannel
322+
323+ def __init__(self, *args, **kwargs):
324+ """Create a new instance."""
325+ server.Site.__init__(self, *args, **kwargs)
326+ # we disable the timeout in the tests, we will deal with it manually.
327+ # pylint: disable=C0103
328+ self.timeOut = None
329+
330+
331+class BaseMockWebServer(object):
332+ """A mock webserver for testing"""
333+
334+ def __init__(self):
335+ """Start up this instance."""
336+ self.root = self.get_root_resource()
337+ self.site = SaveSite(self.root)
338+ application = service.Application('web')
339+ self.service_collection = service.IServiceCollection(application)
340+ #pylint: disable=E1101
341+ self.tcpserver = internet.TCPServer(0, self.site)
342+ self.tcpserver.setServiceParent(self.service_collection)
343+ self.sslserver = internet.SSLServer(0, self.site, self.get_context())
344+ self.sslserver.setServiceParent(self.service_collection)
345+ self.service_collection.startService()
346+
347+ def get_dummy_path(self, filename):
348+ """Path pointing at the dummy certificate files."""
349+ base_path = path.dirname(__file__)
350+ return path.join(base_path, "ssl", filename)
351+
352+ def get_context(self):
353+ """Return an ssl context."""
354+ key_path = self.get_dummy_path(DUMMY_KEY_FILENAME)
355+ cert_path = self.get_dummy_path(DUMMY_CERT_FILENAME)
356+ return ssl.DefaultOpenSSLContextFactory(key_path, cert_path)
357+
358+ def get_root_resource(self):
359+ """Get the root resource with all the children."""
360+ raise NotImplementedError
361+
362+ def get_iri(self):
363+ """Build the iri for this mock server."""
364+ #pylint: disable=W0212
365+ port_num = self.tcpserver._port.getHost().port
366+ return u"http://0.0.0.0:%d/" % port_num
367+
368+ def get_ssl_iri(self):
369+ """Build the iri for the ssl mock server."""
370+ #pylint: disable=W0212
371+ port_num = self.sslserver._port.getHost().port
372+ return u"https://0.0.0.0:%d/" % port_num
373+
374+ def stop(self):
375+ """Shut it down."""
376+ #pylint: disable=E1101
377+ if self.site.protocol.protocolInstance:
378+ self.site.protocol.protocolInstance.timeoutConnection()
379+ return self.service_collection.stopService()
380+
381+
382+class SimpleResource(resource.Resource):
383+ """A simple web resource."""
384+
385+ def __init__(self):
386+ """Initialize this mock resource."""
387+ resource.Resource.__init__(self)
388+ self.rendered = defer.Deferred()
389+
390+ def render_GET(self, request):
391+ """Make a bit of html out of the resource's content."""
392+ if not self.rendered.called:
393+ self.rendered.callback(None)
394+ return SAMPLE_CONTENT
395+
396+
397+class MockWebServer(BaseMockWebServer):
398+ """A mock webserver."""
399+
400+ def __init__(self):
401+ """Initialize this mock server."""
402+ self.simple_resource = SimpleResource()
403+ super(MockWebServer, self).__init__()
404+
405+ def get_root_resource(self):
406+ """Get the root resource with all the children."""
407+ root = resource.Resource()
408+ root.putChild(SIMPLERESOURCE, self.simple_resource)
409+ return root
410+
411+
412+class FakeTransport(StringIO):
413+ """A fake transport that stores everything written to it."""
414+
415+ connected = True
416+ disconnecting = False
417+ cookie = None
418+
419+ def loseConnection(self):
420+ """Mark the connection as lost."""
421+ self.connected = False
422+ self.disconnecting = True
423+
424+ def getPeer(self):
425+ """Return the peer IAddress."""
426+ return None
427
428=== added directory 'tests/proxy/ssl'
429=== added file 'tests/proxy/ssl/dummy.cert'
430--- tests/proxy/ssl/dummy.cert 1970-01-01 00:00:00 +0000
431+++ tests/proxy/ssl/dummy.cert 2012-03-20 23:05:30 +0000
432@@ -0,0 +1,19 @@
433+-----BEGIN CERTIFICATE-----
434+MIIDEDCCAnmgAwIBAgIJAM/bIJ77awBCMA0GCSqGSIb3DQEBBQUAMIGgMQswCQYD
435+VQQGEwJBUjETMBEGA1UECAwKRmFrZSBTdGF0ZTESMBAGA1UEBwwJRmFrZSBDaXR5
436+MRUwEwYDVQQKDAxGYWtlIENvbXBhbnkxFjAUBgNVBAsMDUZha2UgRGl2aXNpb24x
437+EjAQBgNVBAMMCUZha2UgTmFtZTElMCMGCSqGSIb3DQEJARYWZmFrZUBlbWFpbC5h
438+ZGRyZXNzLm5vdDAeFw0xMjAyMjIxOTI0MjBaFw0yMjAyMjMxOTI0MjBaMIGgMQsw
439+CQYDVQQGEwJBUjETMBEGA1UECAwKRmFrZSBTdGF0ZTESMBAGA1UEBwwJRmFrZSBD
440+aXR5MRUwEwYDVQQKDAxGYWtlIENvbXBhbnkxFjAUBgNVBAsMDUZha2UgRGl2aXNp
441+b24xEjAQBgNVBAMMCUZha2UgTmFtZTElMCMGCSqGSIb3DQEJARYWZmFrZUBlbWFp
442+bC5hZGRyZXNzLm5vdDCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA0hbliGty
443+HwfZixU609UHBQdbfO+oObrPIrIawWX5FxD6KhX4ei23idmpyYEcXLK4ivNlT4dW
444+27bvhtpf6/FBbu9e1YdwcdDNoXajr9Ia4NZJyANgo9b5UIsnyTc45NlnpZgRg5zc
445+Oz7Vwwr4qf6r1ljK/I2mAO7rlpH5Ak9J+RkCAwEAAaNQME4wHQYDVR0OBBYEFLwr
446+ps/JLNcfpSuuylMnkvImVvkgMB8GA1UdIwQYMBaAFLwrps/JLNcfpSuuylMnkvIm
447+VvkgMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAWYDBAr0MgpnBxIne
448+WRz8MX0/c7IqrEuZCYMSGnU7PoX3GdNk1Lkif1ufELKSoG8jY16CDgEl26GPxA1k
449+Tho7MWSLikLbuQYJs2saF9by0Y/Mrau0auxEnpHZ7pkybeKFnrIqiNKvTVMnjo5T
450+FMET5qEOKKvp9IOnezCYX1nYXyY=
451+-----END CERTIFICATE-----
452
453=== added file 'tests/proxy/ssl/dummy.key'
454--- tests/proxy/ssl/dummy.key 1970-01-01 00:00:00 +0000
455+++ tests/proxy/ssl/dummy.key 2012-03-20 23:05:30 +0000
456@@ -0,0 +1,16 @@
457+-----BEGIN PRIVATE KEY-----
458+MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBANIW5Yhrch8H2YsV
459+OtPVBwUHW3zvqDm6zyKyGsFl+RcQ+ioV+Hott4nZqcmBHFyyuIrzZU+HVtu274ba
460+X+vxQW7vXtWHcHHQzaF2o6/SGuDWScgDYKPW+VCLJ8k3OOTZZ6WYEYOc3Ds+1cMK
461++Kn+q9ZYyvyNpgDu65aR+QJPSfkZAgMBAAECgYBSxFh7TTExjmsjAyMg700LqyFc
462+8CHLVJBkL9ygkqb2cmbMC8nPgJFNSqY8T5Q35OUVQNyJ31zVxJVLAF9H2c0Xy48K
463+IkbS/hntyqlJYK1yfTbTHkDiweToE3Lm+55Do1TX04AyvBrwA1O/jNGi4xIlUEAy
464+1Bs8MrJ1E/j/XDn9/QJBAOuhPTgG3F7bKuBrQzv98CvC5o2Txf3vLY8nL8V24b3l
465+XgqzkDLhUxReBmmkGxZfKAju3+gXFvGGpbP7V8zShg8CQQDkQGs7kArFq/KR/GCh
466+CAmJaDWy4LJkSqzDHoJbTrS7YuqN6X6mW1xPRnWpYSxae38fJsCpG3Vq8Mv1Zl32
467+VPZXAkEAsAeE9JYri7GwFngLgoXzJr4z/xCmmU5VetyLk7l8a6Eu4E/FKj2rE0wq
468+/kDa+5ubDRFntLuLKGSu5gafUST1gQJABhdmBTfp4a6eEaFPntyNDJq4XCa8/Ao2
469+JBrrVa57Ckkwg0sI8z2a8A6sUzHhsiR7lwQ8vgaakpkMiGcL+Of5jwJAA/qX3PW+
470+9JXbjWxpgh7FHnZJNRZ8xSe47REGA7qS/nIlV9iRuf/9M+k3A5VqitfFxrjPwSyI
471+rvKTYkk13dL4hg==
472+-----END PRIVATE KEY-----
473
474=== added file 'tests/proxy/test_tunnel_client.py'
475--- tests/proxy/test_tunnel_client.py 1970-01-01 00:00:00 +0000
476+++ tests/proxy/test_tunnel_client.py 2012-03-20 23:05:30 +0000
477@@ -0,0 +1,212 @@
478+# -*- coding: utf8 -*-
479+#
480+# Copyright 2012 Canonical Ltd.
481+#
482+# This program is free software: you can redistribute it and/or modify it
483+# under the terms of the GNU General Public License version 3, as published
484+# by the Free Software Foundation.
485+#
486+# This program is distributed in the hope that it will be useful, but
487+# WITHOUT ANY WARRANTY; without even the implied warranties of
488+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
489+# PURPOSE. See the GNU General Public License for more details.
490+#
491+# You should have received a copy of the GNU General Public License along
492+# with this program. If not, see <http://www.gnu.org/licenses/>.
493+"""Tests for the proxy tunnel."""
494+
495+from twisted.internet import defer, protocol, ssl
496+from twisted.trial.unittest import TestCase
497+from twisted.web import client
498+
499+from ubuntuone.devtools.testcases.squid import SquidTestCase
500+
501+from tests.proxy import (
502+ FakeTransport,
503+ FAKE_COOKIE,
504+ MockWebServer,
505+ SAMPLE_CONTENT,
506+ SIMPLERESOURCE,
507+)
508+from ubuntuone.proxy import tunnel_client
509+from ubuntuone.proxy.tunnel_client import CRLF, TunnelClient
510+from ubuntuone.proxy.tunnel_server import TunnelServer
511+
512+
513+FAKE_HEADER = (
514+ "HTTP/1.0 200 Connected!" + CRLF +
515+ "Header1: value1" + CRLF +
516+ "Header2: value2" + CRLF +
517+ CRLF
518+)
519+
520+
521+class SavingProtocol(protocol.Protocol):
522+ """A protocol that saves all that it receives."""
523+
524+ def __init__(self):
525+ """Initialize this protocol."""
526+ self.saved_data = None
527+
528+ def connectionMade(self):
529+ """The connection was made, start saving."""
530+ self.saved_data = []
531+
532+ def dataReceived(self, data):
533+ """Save the data received."""
534+ self.saved_data.append(data)
535+
536+ @property
537+ def content(self):
538+ """All the content so far."""
539+ return "".join(self.saved_data)
540+
541+
542+class TunnelClientProtocolTestCase(TestCase):
543+ """Tests for the client side tunnel protocol."""
544+
545+ timeout = 3
546+
547+ @defer.inlineCallbacks
548+ def setUp(self):
549+ """Initialize this testcase."""
550+ yield super(TunnelClientProtocolTestCase, self).setUp()
551+ self.host, self.port = "9.9.9.9", 8765
552+ fake_addr = object()
553+ self.cookie = FAKE_COOKIE
554+ self.other_proto = SavingProtocol()
555+ other_factory = protocol.ClientFactory()
556+ other_factory.buildProtocol = lambda _addr: self.other_proto
557+ tunnel_client_factory = tunnel_client.TunnelClientFactory(self.host,
558+ self.port, other_factory, self.cookie)
559+ tunnel_client_proto = tunnel_client_factory.buildProtocol(fake_addr)
560+ tunnel_client_proto.transport = FakeTransport()
561+ tunnel_client_proto.connectionMade()
562+ self.tunnel_client_proto = tunnel_client_proto
563+
564+ def test_sends_connect_request(self):
565+ """Sends the expected CONNECT request."""
566+ expected = tunnel_client.METHOD_LINE % (self.host, self.port)
567+ written = self.tunnel_client_proto.transport.getvalue()
568+ first_line = written.split(CRLF)[0]
569+ self.assertEqual(first_line + CRLF, expected)
570+ self.assertTrue(written.endswith(CRLF * 2),
571+ "Ends with a double CRLF")
572+
573+ def test_sends_cookie_header(self):
574+ """Sends the expected cookie header."""
575+ expected = "%s: %s" % (tunnel_client.TUNNEL_COOKIE_HEADER, self.cookie)
576+ written = self.tunnel_client_proto.transport.getvalue()
577+ headers = written.split(CRLF)[1:]
578+ self.assertIn(expected, headers)
579+
580+ def test_handles_successful_connection(self):
581+ """A successful connection is handled."""
582+ self.tunnel_client_proto.dataReceived(FAKE_HEADER)
583+ self.assertEqual(self.tunnel_client_proto.status_code, "200")
584+
585+ def test_protocol_is_switched(self):
586+ """The protocol is switched after the headers are received."""
587+ expected = (SAMPLE_CONTENT + CRLF) * 2
588+ self.tunnel_client_proto.dataReceived(FAKE_HEADER + SAMPLE_CONTENT)
589+ self.other_proto.dataReceived(CRLF + SAMPLE_CONTENT + CRLF)
590+ self.assertEqual(self.other_proto.content, expected)
591+
592+
593+class FakeOtherFactory(object):
594+ """A fake factory."""
595+
596+ def __init__(self):
597+ """Initialize this fake."""
598+ self.started_called = None
599+ self.failed_called = None
600+ self.lost_called = None
601+
602+ def startedConnecting(self, *args):
603+ """Store the call."""
604+ self.started_called = args
605+
606+ def clientConnectionFailed(self, *args):
607+ """Store the call."""
608+ self.failed_called = args
609+
610+ def clientConnectionLost(self, *args):
611+ """Store the call."""
612+ self.lost_called = args
613+
614+
615+class TunnelClientFactoryTestCase(TestCase):
616+ """Tests for the TunnelClientFactory."""
617+
618+ def test_forwards_started(self):
619+ """The factory forwards the startedConnecting call."""
620+ fake_other_factory = FakeOtherFactory()
621+ tcf = tunnel_client.TunnelClientFactory(None, None, fake_other_factory,
622+ FAKE_COOKIE)
623+ fake_connector = object()
624+ tcf.startedConnecting(fake_connector)
625+ self.assertEqual(fake_other_factory.started_called, (fake_connector,))
626+
627+ def test_forwards_failed(self):
628+ """The factory forwards the clientConnectionFailed call."""
629+ fake_reason = object()
630+ fake_other_factory = FakeOtherFactory()
631+ tcf = tunnel_client.TunnelClientFactory(None, None, fake_other_factory,
632+ FAKE_COOKIE)
633+ fake_connector = object()
634+ tcf.clientConnectionFailed(fake_connector, fake_reason)
635+ self.assertEqual(fake_other_factory.failed_called,
636+ (fake_connector, fake_reason))
637+
638+ def test_forwards_lost(self):
639+ """The factory forwards the clientConnectionLost call."""
640+ fake_reason = object()
641+ fake_other_factory = FakeOtherFactory()
642+ tcf = tunnel_client.TunnelClientFactory(None, None, fake_other_factory,
643+ FAKE_COOKIE)
644+ fake_connector = object()
645+ tcf.clientConnectionLost(fake_connector, fake_reason)
646+ self.assertEqual(fake_other_factory.lost_called,
647+ (fake_connector, fake_reason))
648+
649+
650+class TunnelClientTestCase(SquidTestCase):
651+ """Test the client for the tunnel."""
652+
653+ timeout = 3
654+
655+ @defer.inlineCallbacks
656+ def setUp(self):
657+ """Initialize this testcase."""
658+ yield super(TunnelClientTestCase, self).setUp()
659+ self.ws = MockWebServer()
660+ self.addCleanup(self.ws.stop)
661+ self.dest_url = self.ws.get_iri().encode("utf-8") + SIMPLERESOURCE
662+ self.dest_ssl_url = (self.ws.get_ssl_iri().encode("utf-8") +
663+ SIMPLERESOURCE)
664+ self.cookie = FAKE_COOKIE
665+ self.tunnel_server = TunnelServer(self.cookie)
666+ self.addCleanup(self.tunnel_server.shutdown)
667+
668+ @defer.inlineCallbacks
669+ def test_connects_right(self):
670+ """Uses the CONNECT method on the tunnel."""
671+ tunnel_client = TunnelClient("0.0.0.0", self.tunnel_server.port,
672+ self.cookie)
673+ factory = client.HTTPClientFactory(self.dest_url)
674+ scheme, host, port, path = client._parse(self.dest_url)
675+ tunnel_client.connectTCP(host, port, factory)
676+ result = yield factory.deferred
677+ self.assertEqual(result, SAMPLE_CONTENT)
678+
679+ @defer.inlineCallbacks
680+ def test_starts_tls_connection(self):
681+ """TLS is started after connecting; control passed to the client."""
682+ tunnel_client = TunnelClient("0.0.0.0", self.tunnel_server.port,
683+ self.cookie)
684+ factory = client.HTTPClientFactory(self.dest_ssl_url)
685+ scheme, host, port, path = client._parse(self.dest_ssl_url)
686+ context_factory = ssl.ClientContextFactory()
687+ tunnel_client.connectSSL(host, port, factory, context_factory)
688+ result = yield factory.deferred
689+ self.assertEqual(result, SAMPLE_CONTENT)
690
691=== added file 'tests/proxy/test_tunnel_server.py'
692--- tests/proxy/test_tunnel_server.py 1970-01-01 00:00:00 +0000
693+++ tests/proxy/test_tunnel_server.py 2012-03-20 23:05:30 +0000
694@@ -0,0 +1,655 @@
695+# -*- coding: utf8 -*-
696+#
697+# Copyright 2012 Canonical Ltd.
698+#
699+# This program is free software: you can redistribute it and/or modify it
700+# under the terms of the GNU General Public License version 3, as published
701+# by the Free Software Foundation.
702+#
703+# This program is distributed in the hope that it will be useful, but
704+# WITHOUT ANY WARRANTY; without even the implied warranties of
705+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
706+# PURPOSE. See the GNU General Public License for more details.
707+#
708+# You should have received a copy of the GNU General Public License along
709+# with this program. If not, see <http://www.gnu.org/licenses/>.
710+"""Tests for the proxy tunnel."""
711+
712+from StringIO import StringIO
713+from urlparse import urlparse
714+
715+from twisted.internet import defer, protocol, reactor
716+from twisted.trial.unittest import TestCase
717+from PyQt4.QtCore import QCoreApplication
718+from PyQt4.QtNetwork import QAuthenticator
719+from ubuntuone.devtools.testcases.squid import SquidTestCase
720+
721+from tests.proxy import (
722+ FakeTransport,
723+ FAKE_COOKIE,
724+ MockWebServer,
725+ SAMPLE_CONTENT,
726+ SIMPLERESOURCE,
727+)
728+from ubuntuone.proxy import tunnel_server
729+from ubuntuone.proxy.tunnel_server import CRLF
730+
731+
732+FAKE_SESSION_TEMPLATE = (
733+ "CONNECT %s HTTP/1.0" + CRLF +
734+ "Header1: value1" + CRLF +
735+ "Header2: value2" + CRLF +
736+ tunnel_server.TUNNEL_COOKIE_HEADER + ": %s" + CRLF +
737+ CRLF +
738+ "GET %s HTTP/1.0" + CRLF + CRLF
739+)
740+
741+FAKE_SETTINGS = {
742+ "http": {
743+ "host": "myhost",
744+ "port": 8888,
745+ }
746+}
747+
748+SAMPLE_HOST = "samplehost.com"
749+SAMPLE_PORT = 443
750+
751+FAKE_CREDS = {
752+ "username": "rhea",
753+ "password": "caracolcaracola",
754+}
755+
756+
757+class DisconnectingProtocol(protocol.Protocol):
758+ """A protocol that just disconnects."""
759+
760+ def connectionMade(self):
761+ """Upon connecting: just disconnect."""
762+ self.transport.loseConnection()
763+
764+
765+class DisconnectingClientFactory(protocol.ClientFactory):
766+ """A factory that fires a deferred on connection."""
767+
768+ def __init__(self):
769+ """Initialize this instance."""
770+ self.connected = defer.Deferred()
771+
772+ def buildProtocol(self, addr):
773+ """The connection was made."""
774+ proto = DisconnectingProtocol()
775+ if not self.connected.called:
776+ self.connected.callback(proto)
777+ return proto
778+
779+
780+class FakeProtocol(protocol.Protocol):
781+ """A protocol that forwards some data."""
782+
783+ def __init__(self, factory, data):
784+ """Initialize this fake."""
785+ self.factory = factory
786+ self.data = data
787+ self.received_data = []
788+
789+ def connectionMade(self):
790+ """Upon connection: send the stored data."""
791+ self.transport.write(self.data)
792+
793+ def dataReceived(self, data):
794+ """Some data was received."""
795+ self.received_data.append(data)
796+
797+ def connectionLost(self, reason):
798+ """The connection was lost, return the response."""
799+ response = "".join(self.received_data)
800+ if not self.factory.response.called:
801+ self.factory.response.callback(response)
802+
803+
804+class FakeClientFactory(protocol.ClientFactory):
805+ """A factory that forwards some data to the protocol."""
806+
807+ def __init__(self, data):
808+ """Initialize this fake."""
809+ self.data = data
810+ self.response = defer.Deferred()
811+
812+ def buildProtocol(self, addr):
813+ """The connection was made."""
814+ return FakeProtocol(self, self.data)
815+
816+
817+class TunnelIntegrationTestCase(SquidTestCase):
818+ """Basic tunnel integration tests."""
819+
820+ timeout = 3
821+
822+ @defer.inlineCallbacks
823+ def setUp(self):
824+ """Initialize this testcase."""
825+ yield super(TunnelIntegrationTestCase, self).setUp()
826+ self.ws = MockWebServer()
827+ self.addCleanup(self.ws.stop)
828+ self.dest_url = self.ws.get_iri().encode("utf-8") + SIMPLERESOURCE
829+ self.cookie = FAKE_COOKIE
830+ self.tunnel_server = tunnel_server.TunnelServer(self.cookie)
831+ self.addCleanup(self.tunnel_server.shutdown)
832+
833+ def test_init(self):
834+ """The tunnel is started."""
835+ self.assertNotEqual(self.tunnel_server.port, 0)
836+
837+ @defer.inlineCallbacks
838+ def test_accepts_connections(self):
839+ """The tunnel accepts incoming connections."""
840+ ncf = DisconnectingClientFactory()
841+ reactor.connectTCP("0.0.0.0", self.tunnel_server.port, ncf)
842+ yield ncf.connected
843+
844+ @defer.inlineCallbacks
845+ def test_complete_connection(self):
846+ """Test from the tunnel server down."""
847+ url = urlparse(self.dest_url)
848+ fake_session = FAKE_SESSION_TEMPLATE % (
849+ url.netloc, self.cookie, url.path)
850+ client = FakeClientFactory(fake_session)
851+ reactor.connectTCP("0.0.0.0", self.tunnel_server.port, client)
852+ response = yield client.response
853+ self.assertIn(SAMPLE_CONTENT, response)
854+
855+
856+class FakeClient(object):
857+ """A fake destination client."""
858+
859+ protocol = None
860+ connection_result = defer.succeed(True)
861+ credentials = None
862+ check_credentials = False
863+ proxy_domain = None
864+
865+ def connect(self, hostport):
866+ """Establish a connection with the other end."""
867+ if (self.check_credentials and
868+ self.protocol.proxy_credentials != FAKE_CREDS):
869+ self.proxy_domain = "fake domain"
870+ return defer.fail(tunnel_server.ProxyAuthenticationError())
871+ return self.connection_result
872+
873+ def write(self, data):
874+ """Write some data to the other end."""
875+ if data == 'GET /simpleresource HTTP/1.0\r\n\r\n':
876+ self.protocol.transport.write(SAMPLE_CONTENT)
877+
878+ def stop(self):
879+ """Stop this fake client."""
880+
881+ def close(self):
882+ """Reset this client."""
883+
884+
885+
886+class ServerTunnelProtocolTestCase(SquidTestCase):
887+ """Tests for the ServerTunnelProtocol."""
888+
889+ @defer.inlineCallbacks
890+ def setUp(self):
891+ """Initialize this test instance."""
892+ yield super(ServerTunnelProtocolTestCase, self).setUp()
893+ self.ws = MockWebServer()
894+ self.addCleanup(self.ws.stop)
895+ self.dest_url = self.ws.get_iri().encode("utf-8") + SIMPLERESOURCE
896+ self.transport = FakeTransport()
897+ self.transport.cookie = FAKE_COOKIE
898+ self.fake_client = FakeClient()
899+ self.proto = tunnel_server.ServerTunnelProtocol(
900+ lambda _: self.fake_client)
901+ self.fake_client.protocol = self.proto
902+ self.proto.transport = self.transport
903+ self.cookie_line = "%s: %s" % (tunnel_server.TUNNEL_COOKIE_HEADER,
904+ FAKE_COOKIE)
905+
906+ def test_broken_request(self):
907+ """Broken request."""
908+ self.proto.dataReceived("Broken request." + CRLF)
909+ self.assertTrue(self.transport.getvalue().startswith("HTTP/1.0 400 "),
910+ "A broken request must fail.")
911+
912+ def test_wrong_method(self):
913+ """Wrong method."""
914+ self.proto.dataReceived("GET http://slashdot.org HTTP/1.0" + CRLF)
915+ self.assertTrue(self.transport.getvalue().startswith("HTTP/1.0 405 "),
916+ "Using a wrong method fails.")
917+
918+ def test_invalid_http_version(self):
919+ """Invalid HTTP version."""
920+ self.proto.dataReceived("CONNECT 127.0.0.1:9999 HTTP/1.1" + CRLF)
921+ self.assertTrue(self.transport.getvalue().startswith("HTTP/1.0 505 "),
922+ "Invalid http version is not allowed.")
923+
924+ def test_connection_is_established(self):
925+ """The response code is sent."""
926+ expected = "HTTP/1.0 200 Proxy connection established" + CRLF
927+ self.proto.dataReceived("CONNECT 127.0.0.1:9999 HTTP/1.0" + CRLF +
928+ self.cookie_line + CRLF * 2)
929+ self.assertTrue(self.transport.getvalue().startswith(expected),
930+ "First line must be the response status")
931+
932+ def test_connection_fails(self):
933+ """The connection to the other end fails, and it's handled."""
934+ error = tunnel_server.ConnectionError()
935+ self.patch(self.fake_client, "connection_result", defer.fail(error))
936+ expected = "HTTP/1.0 500 Connection error" + CRLF
937+ self.proto.dataReceived("CONNECT 127.0.0.1:9999 HTTP/1.0" + CRLF +
938+ self.cookie_line + CRLF * 2)
939+ self.assertTrue(self.transport.getvalue().startswith(expected),
940+ "The connection should fail at this point.")
941+
942+ def test_headers_stored(self):
943+ """The request headers are stored."""
944+ expected = [
945+ ("Header1", "value1"),
946+ ("Header2", "value2"),
947+ ]
948+ self.proto.dataReceived("CONNECT 127.0.0.1:9999 HTTP/1.0" + CRLF +
949+ "Header1: value1" + CRLF +
950+ "Header2: value2" + CRLF + CRLF)
951+ self.assertEqual(self.proto.received_headers, expected)
952+
953+ def test_cookie_header_present(self):
954+ """The cookie header must be present."""
955+ self.proto.received_headers = [
956+ (tunnel_server.TUNNEL_COOKIE_HEADER, FAKE_COOKIE),
957+ ]
958+ self.proto.verify_cookie()
959+
960+ def test_cookie_header_absent(self):
961+ """The tunnel should refuse connections without the cookie."""
962+ self.proto.received_headers = []
963+ exception = self.assertRaises(tunnel_server.ConnectionError,
964+ self.proto.verify_cookie)
965+ self.assertEqual(exception.code, 418)
966+
967+ def test_successful_connect(self):
968+ """A successful connect thru the tunnel."""
969+ url = urlparse(self.dest_url)
970+ data = FAKE_SESSION_TEMPLATE % (url.netloc, self.transport.cookie,
971+ url.path)
972+ self.proto.dataReceived(data)
973+ lines = self.transport.getvalue().split(CRLF)
974+ self.assertEqual(lines[-1], SAMPLE_CONTENT)
975+
976+ def test_header_split(self):
977+ """Test a header with many colons."""
978+ self.proto.header_line("key: host:port")
979+ self.assertIn("key", dict(self.proto.received_headers))
980+
981+ @defer.inlineCallbacks
982+ def test_keyring_credentials_are_retried(self):
983+ """Wrong credentials are retried with values from keyring."""
984+ self.fake_client.check_credentials = True
985+ self.patch(self.proto, "verify_cookie", lambda: None)
986+ self.patch(self.proto, "error_response",
987+ lambda code, desc: self.fail(desc))
988+ self.proto.proxy_domain = "xxx"
989+ self.patch(tunnel_server.Keyring, "get_credentials",
990+ lambda _, domain: defer.succeed(FAKE_CREDS))
991+ yield self.proto.headers_done()
992+
993+
994+class FakeServerTunnelProtocol(object):
995+ """A fake ServerTunnelProtocol."""
996+
997+ def __init__(self):
998+ """Initialize this fake tunnel."""
999+ self.response_received = defer.Deferred()
1000+ self.proxy_credentials = None
1001+
1002+ def response_data_received(self, data):
1003+ """Fire the response deferred."""
1004+ if not self.response_received.called:
1005+ self.response_received.callback(data)
1006+
1007+ def remote_disconnected(self):
1008+ """The remote server disconnected."""
1009+
1010+ def proxy_auth_required(self, proxy, authenticator):
1011+ """Proxy credentials are needed."""
1012+ if self.proxy_credentials:
1013+ authenticator.setUser(self.proxy_credentials["username"])
1014+ authenticator.setPassword(self.proxy_credentials["password"])
1015+
1016+
1017+class BuildProxyTestCase(TestCase):
1018+ """Tests for the build_proxy function."""
1019+
1020+ def test_socks_is_preferred(self):
1021+ """Socks overrides all protocols."""
1022+ settings = {
1023+ "http": {"host": "httphost", "port": 3128},
1024+ "https": {"host": "httpshost", "port": 3129},
1025+ "socks": {"host": "sockshost", "port": 1080},
1026+ }
1027+ proxy = tunnel_server.build_proxy(settings)
1028+ self.assertEqual(proxy.type(), proxy.Socks5Proxy)
1029+ self.assertEqual(proxy.hostName(), "sockshost")
1030+ self.assertEqual(proxy.port(), 1080)
1031+
1032+ def test_https_beats_http(self):
1033+ """HTTPS wins over HTTP, since all of SD traffic is https."""
1034+ settings = {
1035+ "http": {"host": "httphost", "port": 3128},
1036+ "https": {"host": "httpshost", "port": 3129},
1037+ }
1038+ proxy = tunnel_server.build_proxy(settings)
1039+ self.assertEqual(proxy.type(), proxy.HttpProxy)
1040+ self.assertEqual(proxy.hostName(), "httpshost")
1041+ self.assertEqual(proxy.port(), 3129)
1042+
1043+ def test_http_if_no_other_choice(self):
1044+ """Finally, we use the host configured for HTTP."""
1045+ settings = {
1046+ "http": {"host": "httphost", "port": 3128},
1047+ }
1048+ proxy = tunnel_server.build_proxy(settings)
1049+ self.assertEqual(proxy.type(), proxy.HttpProxy)
1050+ self.assertEqual(proxy.hostName(), "httphost")
1051+ self.assertEqual(proxy.port(), 3128)
1052+
1053+ def test_use_noproxy_as_fallback(self):
1054+ """If nothing useful, revert to no proxy."""
1055+ settings = {}
1056+ proxy = tunnel_server.build_proxy(settings)
1057+ self.assertEqual(proxy.type(), proxy.DefaultProxy)
1058+
1059+
1060+class RemoteSocketTestCase(SquidTestCase):
1061+ """Tests for the client that connects to the other side."""
1062+
1063+ timeout = 3
1064+ get_proxy_settings = lambda _: {}
1065+
1066+ @defer.inlineCallbacks
1067+ def setUp(self):
1068+ """Initialize this testcase."""
1069+ yield super(RemoteSocketTestCase, self).setUp()
1070+ self.ws = MockWebServer()
1071+ self.addCleanup(self.ws.stop)
1072+ self.dest_url = self.ws.get_iri().encode("utf-8") + SIMPLERESOURCE
1073+
1074+ self.addCleanup(tunnel_server.QNetworkProxy.setApplicationProxy,
1075+ tunnel_server.QNetworkProxy.applicationProxy())
1076+ settings = {"http": self.get_proxy_settings()}
1077+ proxy = tunnel_server.build_proxy(settings)
1078+ tunnel_server.QNetworkProxy.setApplicationProxy(proxy)
1079+
1080+ def test_invalid_port(self):
1081+ """A request with an invalid port fails with a 400."""
1082+ protocol = tunnel_server.ServerTunnelProtocol(
1083+ tunnel_server.RemoteSocket)
1084+ protocol.transport = FakeTransport()
1085+ protocol.dataReceived("CONNECT 127.0.0.1:wrong_port HTTP/1.0" +
1086+ CRLF * 2)
1087+
1088+ status_line = protocol.transport.getvalue()
1089+ self.assertTrue(status_line.startswith("HTTP/1.0 400 "),
1090+ "The port must be an integer.")
1091+
1092+ @defer.inlineCallbacks
1093+ def test_connection_is_finished_when_stopping(self):
1094+ """The client disconnects when requested."""
1095+ fake_protocol = FakeServerTunnelProtocol()
1096+ client = tunnel_server.RemoteSocket(fake_protocol)
1097+ url = urlparse(self.dest_url)
1098+ yield client.connect(url.netloc)
1099+ yield client.stop()
1100+
1101+ @defer.inlineCallbacks
1102+ def test_stop_but_never_connected(self):
1103+ """Stop but it was never connected."""
1104+ fake_protocol = FakeServerTunnelProtocol()
1105+ client = tunnel_server.RemoteSocket(fake_protocol)
1106+ yield client.stop()
1107+
1108+ @defer.inlineCallbacks
1109+ def test_client_write(self):
1110+ """Data written to the client is sent to the other side."""
1111+ fake_protocol = FakeServerTunnelProtocol()
1112+ client = tunnel_server.RemoteSocket(fake_protocol)
1113+ self.addCleanup(client.stop)
1114+ url = urlparse(self.dest_url)
1115+ yield client.connect(url.netloc)
1116+ client.write("GET /simpleresource HTTP/1.0" + CRLF * 2)
1117+ yield self.ws.simple_resource.rendered
1118+
1119+ @defer.inlineCallbacks
1120+ def test_client_read(self):
1121+ """Data received by the client is written into the transport."""
1122+ fake_protocol = FakeServerTunnelProtocol()
1123+ client = tunnel_server.RemoteSocket(fake_protocol)
1124+ self.addCleanup(client.stop)
1125+ url = urlparse(self.dest_url)
1126+ yield client.connect(url.netloc)
1127+ client.write("GET /simpleresource HTTP/1.0" + CRLF * 2)
1128+ yield self.ws.simple_resource.rendered
1129+ data = yield fake_protocol.response_received
1130+ _headers, content = str(data).split(CRLF * 2, 1)
1131+ self.assertEqual(content, SAMPLE_CONTENT)
1132+
1133+
1134+class AnonProxyRemoteSocketTestCase(RemoteSocketTestCase):
1135+ """Tests for the client going thru an anonymous proxy."""
1136+
1137+ get_proxy_settings = RemoteSocketTestCase.get_nonauth_proxy_settings
1138+
1139+ def parse_headers(self, raw_headers):
1140+ """Parse the headers."""
1141+ lines = raw_headers.split(CRLF)
1142+ header_lines = lines[1:]
1143+ headers_pairs = (l.split(":", 1) for l in header_lines)
1144+ return dict((k.lower(), v.strip()) for k, v in headers_pairs)
1145+
1146+ @defer.inlineCallbacks
1147+ def test_verify_client_uses_proxy(self):
1148+ """Verify that the client uses the proxy."""
1149+ fake_protocol = FakeServerTunnelProtocol()
1150+ client = tunnel_server.RemoteSocket(fake_protocol)
1151+ self.addCleanup(client.stop)
1152+ url = urlparse(self.dest_url)
1153+ yield client.connect(url.netloc)
1154+ client.write("GET /simpleresource HTTP/1.0" + CRLF * 2)
1155+ yield self.ws.simple_resource.rendered
1156+ data = yield fake_protocol.response_received
1157+ raw_headers, _content = str(data).split(CRLF * 2, 1)
1158+ self.parse_headers(raw_headers)
1159+
1160+
1161+class AuthenticatedProxyRemoteSocketTestCase(AnonProxyRemoteSocketTestCase):
1162+ """Tests for the client going thru an authenticated proxy."""
1163+
1164+ get_proxy_settings = RemoteSocketTestCase.get_auth_proxy_settings
1165+
1166+ @defer.inlineCallbacks
1167+ def test_proxy_authentication_error(self):
1168+ """The proxy credentials were wrong on purpose."""
1169+ settings = {"http": self.get_proxy_settings()}
1170+ settings["http"]["password"] = "wrong password!!!"
1171+ proxy = tunnel_server.build_proxy(settings)
1172+ tunnel_server.QNetworkProxy.setApplicationProxy(proxy)
1173+ fake_protocol = FakeServerTunnelProtocol()
1174+ client = tunnel_server.RemoteSocket(fake_protocol)
1175+ self.addCleanup(client.stop)
1176+ url = urlparse(self.dest_url)
1177+ yield self.assertFailure(client.connect(url.netloc),
1178+ tunnel_server.ProxyAuthenticationError)
1179+
1180+ @defer.inlineCallbacks
1181+ def test_proxy_nobody_listens(self):
1182+ """The proxy settings point to a proxy that's unreachable."""
1183+ settings = dict(http={
1184+ "host": "127.0.0.1",
1185+ "port": 83, # unused port according to /etc/services
1186+ })
1187+ proxy = tunnel_server.build_proxy(settings)
1188+ tunnel_server.QNetworkProxy.setApplicationProxy(proxy)
1189+ fake_protocol = FakeServerTunnelProtocol()
1190+ client = tunnel_server.RemoteSocket(fake_protocol)
1191+ self.addCleanup(client.stop)
1192+ url = urlparse(self.dest_url)
1193+ yield self.assertFailure(client.connect(url.netloc),
1194+ tunnel_server.ConnectionError)
1195+
1196+ def test_use_credentials(self):
1197+ """The credentials are used if present."""
1198+ fake_protocol = FakeServerTunnelProtocol()
1199+ client = tunnel_server.RemoteSocket(fake_protocol)
1200+ proxy = tunnel_server.build_proxy(FAKE_SETTINGS)
1201+ authenticator = QAuthenticator()
1202+
1203+ client.proxyAuthenticationRequired.emit(proxy, authenticator)
1204+ self.assertEqual(proxy.user(), "")
1205+ self.assertEqual(proxy.password(), "")
1206+ fake_protocol.proxy_credentials = FAKE_CREDS
1207+
1208+ client.proxyAuthenticationRequired.emit(proxy, authenticator)
1209+ self.assertEqual(authenticator.user(), FAKE_CREDS["username"])
1210+ self.assertEqual(authenticator.password(), FAKE_CREDS["password"])
1211+
1212+
1213+class FakeNetworkProxyFactoryClass(object):
1214+ """A fake QNetworkProxyFactory."""
1215+ last_query = None
1216+
1217+ def __init__(self, enabled):
1218+ """Initialize this fake instance."""
1219+ if enabled:
1220+ self.proxy_type = tunnel_server.QNetworkProxy.HttpProxy
1221+ else:
1222+ self.proxy_type = tunnel_server.QNetworkProxy.DefaultProxy
1223+
1224+ def type(self):
1225+ """Return the proxy type configured."""
1226+ return self.proxy_type
1227+
1228+ def systemProxyForQuery(self, query):
1229+ """A list of proxies, but only type() will be called on the first."""
1230+ return [self]
1231+
1232+
1233+class CheckProxyEnabledTestCase(TestCase):
1234+ """Tests for the check_proxy_enabled function."""
1235+
1236+ @defer.inlineCallbacks
1237+ def setUp(self):
1238+ """Initialize this testcase."""
1239+ yield super(CheckProxyEnabledTestCase, self).setUp()
1240+ self.app_proxy = []
1241+
1242+ def _assert_proxy_state(self, platform, state, assertion):
1243+ """Assert the proxy is in a given state."""
1244+ self.patch(tunnel_server.QNetworkProxy, "setApplicationProxy",
1245+ lambda proxy: self.app_proxy.append(proxy))
1246+ self.patch(tunnel_server.sys, "platform", platform)
1247+ ret = tunnel_server.check_proxy_enabled(SAMPLE_HOST, str(SAMPLE_PORT))
1248+ self.assertTrue(ret == state, assertion)
1249+
1250+ def _assert_proxy_enabled(self, platform):
1251+ """Assert that the proxy is enabled."""
1252+ self._assert_proxy_state(platform, True, "Proxy is enabled.")
1253+
1254+ def _assert_proxy_disabled(self, platform):
1255+ """Assert that the proxy is disabled."""
1256+ self._assert_proxy_state(platform, False, "Proxy is disabled.")
1257+
1258+ def test_platform_linux_enabled(self):
1259+ """Tests for the linux platform with proxies enabled."""
1260+ self.patch(tunnel_server.gsettings, "get_proxy_settings",
1261+ lambda: FAKE_SETTINGS)
1262+ self._assert_proxy_enabled("linux3")
1263+ self.assertEqual(len(self.app_proxy), 1)
1264+
1265+ def test_platform_linux_disabled(self):
1266+ """Tests for the linux platform with proxies disabled."""
1267+ self.patch(tunnel_server.gsettings, "get_proxy_settings", lambda: {})
1268+ self._assert_proxy_disabled("linux3")
1269+ self.assertEqual(len(self.app_proxy), 0)
1270+
1271+ def test_platform_other_enabled(self):
1272+ """Tests for any other platform with proxies enabled."""
1273+ fake_netproxfact = FakeNetworkProxyFactoryClass(True)
1274+ self.patch(tunnel_server, "QNetworkProxyFactory", fake_netproxfact)
1275+ self._assert_proxy_enabled("windows 1.0")
1276+ self.assertEqual(len(self.app_proxy), 0)
1277+
1278+ def test_platform_other_disabled(self):
1279+ """Tests for any other platform with proxies disabled."""
1280+ fake_netproxfact = FakeNetworkProxyFactoryClass(False)
1281+ self.patch(tunnel_server, "QNetworkProxyFactory", fake_netproxfact)
1282+ self._assert_proxy_disabled("windows 1.0")
1283+ self.assertEqual(len(self.app_proxy), 0)
1284+
1285+
1286+class FakeQCoreApp(object):
1287+ """A fake QCoreApplication."""
1288+
1289+ fake_instance = None
1290+
1291+ def __init__(self, argv):
1292+ """Initialize this fake."""
1293+ self.executed = False
1294+ self.argv = argv
1295+ FakeQCoreApp.fake_instance = self
1296+
1297+ def exec_(self):
1298+ """Fake the execution of this app."""
1299+ self.executed = True
1300+
1301+ @staticmethod
1302+ def instance():
1303+ """But return the real instance."""
1304+ return QCoreApplication.instance()
1305+
1306+
1307+class MainFunctionTestCase(TestCase):
1308+ """Tests for the main function of the tunnel server."""
1309+
1310+ @defer.inlineCallbacks
1311+ def setUp(self):
1312+ """Initialize these testcases."""
1313+ yield super(MainFunctionTestCase, self).setUp()
1314+ self.called = []
1315+ self.proxies_enabled = False
1316+
1317+ def fake_is_proxy_enabled(*args):
1318+ """Store the call, return false."""
1319+ self.called.append(args)
1320+ return self.proxies_enabled
1321+
1322+ self.patch(tunnel_server, "check_proxy_enabled", fake_is_proxy_enabled)
1323+ self.fake_stdout = StringIO()
1324+ self.patch(tunnel_server.sys, "stdout", self.fake_stdout)
1325+ self.patch(tunnel_server, "QCoreApplication", FakeQCoreApp)
1326+
1327+ def test_checks_proxies(self):
1328+ """Main checks that the proxies are enabled."""
1329+ tunnel_server.main([])
1330+ self.assertEqual(len(self.called), 1)
1331+
1332+ def test_on_proxies_enabled_prints_port_and_cookie(self):
1333+ """With proxies enabled print port to stdout and start the mainloop."""
1334+ self.patch(tunnel_server.uuid, "uuid4", lambda: FAKE_COOKIE)
1335+ self.proxies_enabled = True
1336+ port = 443
1337+ tunnel_server.main(["example.com", str(port)])
1338+ stdout = self.fake_stdout.getvalue()
1339+
1340+ self.assertIn(tunnel_server.TUNNEL_PORT_LABEL + ": ", stdout)
1341+ cookie_line = tunnel_server.TUNNEL_COOKIE_LABEL + ": " + FAKE_COOKIE
1342+ self.assertIn(cookie_line, stdout)
1343+
1344+ def test_on_proxies_disabled_exit(self):
1345+ """With proxies disabled, print a message and exit gracefully."""
1346+ self.proxies_enabled = False
1347+ tunnel_server.main(["example.com", "443"])
1348+ self.assertIn("Proxy not enabled.", self.fake_stdout.getvalue())
1349+ self.assertEqual(FakeQCoreApp.fake_instance, None)
1350
1351=== modified file 'tests/syncdaemon/test_action_queue.py'
1352--- tests/syncdaemon/test_action_queue.py 2012-02-18 16:13:04 +0000
1353+++ tests/syncdaemon/test_action_queue.py 2012-03-20 23:05:30 +0000
1354@@ -23,7 +23,6 @@
1355 import operator
1356 import os
1357 import unittest
1358-import urllib2
1359 import uuid
1360
1361 from functools import wraps
1362@@ -33,7 +32,7 @@
1363
1364 from mocker import Mocker, MockerTestCase, ANY, expect
1365 from oauth import oauth
1366-from twisted.internet import defer, threads, reactor
1367+from twisted.internet import defer, reactor
1368 from twisted.internet import error as twisted_error
1369 from twisted.python.failure import DefaultException, Failure
1370 from twisted.web import server
1371@@ -186,6 +185,35 @@
1372 return FakeRequest()
1373
1374
1375+class FakeTunnelClient(object):
1376+ """A fake proxy.tunnel_client."""
1377+
1378+ def __init__(self):
1379+ """Fake this proxy tunnel."""
1380+ self.tcp_connected = False
1381+ self.ssl_connected = False
1382+
1383+ def connectTCP(self, *args, **kwargs):
1384+ """Save the connection thru TCP."""
1385+ self.tcp_connected = True
1386+
1387+ def connectSSL(self, *args, **kwargs):
1388+ """Save the connection thru SSL."""
1389+ self.ssl_connected = True
1390+
1391+
1392+class SavingConnectionTunnelRunner(object):
1393+ """A fake proxy.tunnel_client.TunnelRunner."""
1394+
1395+ def __init__(self, *args):
1396+ """Fake a proxy tunnel."""
1397+ self.client = FakeTunnelClient()
1398+
1399+ def get_client(self):
1400+ """Always return the reactor."""
1401+ return defer.succeed(self.client)
1402+
1403+
1404 class TestingProtocol(ActionQueue.protocol):
1405 """Protocol for testing."""
1406
1407@@ -370,6 +398,21 @@
1408 self.assertEqual(defined_args[0], 'self')
1409 self.assertEqual(set(defined_args[1:]), set(evtargs))
1410
1411+ @defer.inlineCallbacks
1412+ def test_get_webclient(self):
1413+ """The webclient is created if it does not exist."""
1414+ self.assertEqual(self.action_queue.webclient, None)
1415+ webclient = yield self.action_queue.get_webclient()
1416+ self.assertNotEqual(webclient, None)
1417+
1418+ @defer.inlineCallbacks
1419+ def test_get_webclient_existing(self):
1420+ """The webclient is not created again if it exists."""
1421+ fake_wc = object()
1422+ self.patch(self.action_queue, "webclient", fake_wc)
1423+ webclient = yield self.action_queue.get_webclient()
1424+ self.assertEqual(webclient, fake_wc)
1425+
1426
1427 class TestLoggingStorageClient(TwistedTestCase):
1428 """Tests for ensuring magic hash dont show in logs."""
1429@@ -1295,6 +1338,28 @@
1430 "host 1.2.3.4", "port 4321"))
1431
1432
1433+class TunnelRunnerTestCase(FactoryBaseTestCase):
1434+ """Tests for the tunnel runner."""
1435+
1436+ tunnel_runner_class = SavingConnectionTunnelRunner
1437+
1438+ @defer.inlineCallbacks
1439+ def test_make_connection_uses_tunnelrunner_non_ssl(self):
1440+ """Check that _make_connection uses TunnelRunner."""
1441+ self.action_queue.use_ssl = False
1442+ yield self.action_queue._make_connection(("127.0.0.1", 1234))
1443+ self.assertTrue(self.action_queue.tunnel_runner.client.tcp_connected,
1444+ "connectTCP is called on the client.")
1445+
1446+ @defer.inlineCallbacks
1447+ def test_make_connection_uses_tunnelrunner_ssl(self):
1448+ """Check that _make_connection uses TunnelRunner."""
1449+ self.action_queue.use_ssl = True
1450+ yield self.action_queue._make_connection(("127.0.0.1", 1234))
1451+ self.assertTrue(self.action_queue.tunnel_runner.client.ssl_connected,
1452+ "connectSSL is called on the client.")
1453+
1454+
1455 class ConnectedBaseTestCase(FactoryBaseTestCase):
1456 """Base test case generating a connected factory."""
1457
1458@@ -2468,67 +2533,26 @@
1459 self.assertEqual(NODE, self.command.node_id)
1460 self.assertEqual(True, self.command.is_public)
1461
1462- def test_run_defers_work_to_thread(self):
1463- """Test that work is deferred to a thread."""
1464- original = threads.deferToThread
1465- self.called = False
1466-
1467- def check(function):
1468- self.called = True
1469- self.assertEqual(
1470- self.command._change_public_access_http, function)
1471- return defer.Deferred()
1472-
1473- threads.deferToThread = check
1474- try:
1475- res = self.command._run()
1476- finally:
1477- threads.deferToThread = original
1478-
1479- self.assertIsInstance(res, defer.Deferred)
1480- self.assertTrue(self.called, "deferToThread was called")
1481-
1482+ @defer.inlineCallbacks
1483 def test_change_public_access_http(self):
1484- """Test the blocking portion of the command."""
1485- self.called = False
1486- def check(request):
1487- self.called = True
1488- url = 'https://one.ubuntu.com/files/api/set_public/%s:%s' % (
1489+ """Test the command."""
1490+
1491+ def check_webcall(request_iri, method=None, post_content=None):
1492+ """Check the webcall made by this command."""
1493+ iri = u'https://one.ubuntu.com/files/api/set_public/%s:%s' % (
1494 base64.urlsafe_b64encode(VOLUME.bytes).strip("="),
1495 base64.urlsafe_b64encode(NODE.bytes).strip("="))
1496- self.assertEqual(url, request.get_full_url())
1497- self.assertEqual("is_public=True", request.get_data())
1498- return StringIO(
1499- '{"is_public": true, "public_url": "http://example.com"}')
1500-
1501- from ubuntuone.syncdaemon import action_queue
1502- self.patch(action_queue.timestamp_checker, "get_faithful_time",
1503- lambda: 1)
1504- action_queue.urlopen = check
1505- try:
1506- res = self.command._change_public_access_http()
1507- finally:
1508- action_queue.urlopen = urllib2.urlopen
1509-
1510+ self.assertEqual(iri, request_iri)
1511+ self.assertEqual("is_public=True", post_content)
1512+ content = '{"is_public": true, "public_url": "http://example.com"}'
1513+ response = action_queue.txweb.Response(content)
1514+ return defer.succeed(response)
1515+
1516+ self.patch(self.action_queue, "webcall", check_webcall)
1517+ res = yield self.command._run()
1518 self.assertEqual(
1519 {'is_public': True, 'public_url': 'http://example.com'}, res)
1520
1521- def test_change_public_access_http_uses_timestamp(self):
1522- """The timestamp is used for oauth signing."""
1523- fake_timestamp = 12345678
1524-
1525- def fake_urlopen(request):
1526- """A fake urlopen."""
1527- auth = request.headers["Authorization"]
1528- expected = 'oauth_timestamp="%d"' % fake_timestamp
1529- self.assertIn(expected, auth)
1530- return StringIO("[]")
1531-
1532- self.patch(action_queue.timestamp_checker, "get_faithful_time",
1533- lambda: fake_timestamp)
1534- self.patch(action_queue, "urlopen", fake_urlopen)
1535- self.command._change_public_access_http()
1536-
1537 def test_handle_success_push_event(self):
1538 """Test AQ_CHANGE_PUBLIC_ACCESS_OK is pushed on success."""
1539 response = {'is_public': True, 'public_url': 'http://example.com'}
1540@@ -2541,8 +2565,7 @@
1541 def test_handle_failure_push_event(self):
1542 """Test AQ_CHANGE_PUBLIC_ACCESS_ERROR is pushed on failure."""
1543 msg = 'Something went wrong'
1544- failure = Failure(urllib2.HTTPError(
1545- "http://example.com", 500, "Error", [], StringIO(msg)))
1546+ failure = Failure(action_queue.txweb.WebClientError("Misc Error", msg))
1547 self.command.handle_failure(failure=failure)
1548 event = ('AQ_CHANGE_PUBLIC_ACCESS_ERROR',
1549 {'share_id': VOLUME, 'node_id': NODE, 'error': msg})
1550@@ -2569,11 +2592,11 @@
1551 default_url = 'https://one.ubuntu.com/files/api/public_files'
1552 request_queue = RequestQueue(action_queue=self.action_queue)
1553 command = GetPublicFiles(request_queue)
1554- self.assertEqual(command._url, default_url)
1555+ self.assertEqual(command._iri, default_url)
1556 custom_url = 'http://example.com:1234/files/api/public_files'
1557 command_2 = GetPublicFiles(request_queue,
1558- base_url='http://example.com:1234')
1559- self.assertEqual(command_2._url, custom_url)
1560+ base_iri=u'http://example.com:1234')
1561+ self.assertEqual(command_2._iri, custom_url)
1562
1563 def test_change_public_access(self):
1564 """Test the get_public_files method.."""
1565@@ -2583,75 +2606,39 @@
1566 """Test proper inheritance."""
1567 self.assertTrue(isinstance(self.command, ActionQueueCommand))
1568
1569- def test_run_defers_work_to_thread(self):
1570- """Test that work is deferred to a thread."""
1571- original = threads.deferToThread
1572- self.called = False
1573-
1574- def check(function):
1575- self.called = True
1576- self.assertEqual(
1577- self.command._get_public_files_http, function)
1578- return defer.Deferred()
1579-
1580- threads.deferToThread = check
1581- try:
1582- res = self.command._run()
1583- finally:
1584- threads.deferToThread = original
1585-
1586- self.assertIsInstance(res, defer.Deferred)
1587- self.assertTrue(self.called, "deferToThread was called")
1588-
1589+ @defer.inlineCallbacks
1590 def test_get_public_files_http(self):
1591- """Test the blocking portion of the command."""
1592- self.called = False
1593+ """Test the _run method of the command."""
1594 node_id = uuid.uuid4()
1595 nodekey = '%s' % (base64.urlsafe_b64encode(node_id.bytes).strip("="))
1596 node_id_2 = uuid.uuid4()
1597 nodekey_2 = '%s' % (base64.urlsafe_b64encode(
1598 node_id_2.bytes).strip("="))
1599 volume_id = uuid.uuid4()
1600- def check(request):
1601- self.called = True
1602- url = 'https://one.ubuntu.com/files/api/public_files'
1603- self.assertEqual(url, request.get_full_url())
1604- return StringIO(
1605+
1606+ def check_webcall(request_iri, method=None):
1607+ """Check the webcall made by this command."""
1608+ """Check the webcall made by this command."""
1609+ iri = u'https://one.ubuntu.com/files/api/public_files'
1610+ self.assertEqual(method.upper(), "GET")
1611+ self.assertEqual(iri, request_iri)
1612+ content = (
1613 '[{"nodekey": "%s", "volume_id": null,"public_url": '
1614 '"http://example.com"}, '
1615 '{"nodekey": "%s", "volume_id": "%s", "public_url": '
1616 '"http://example.com"}]' % (nodekey, nodekey_2, volume_id))
1617-
1618- from ubuntuone.syncdaemon import action_queue
1619- self.patch(action_queue.timestamp_checker, "get_faithful_time",
1620- lambda: 1)
1621- action_queue.urlopen = check
1622- try:
1623- res = self.command._get_public_files_http()
1624- finally:
1625- action_queue.urlopen = urllib2.urlopen
1626+ response = action_queue.txweb.Response(content)
1627+ return defer.succeed(response)
1628+
1629+ self.patch(self.action_queue, "webcall", check_webcall)
1630+ res = yield self.command._run()
1631+
1632 self.assertEqual([{'node_id': str(node_id), 'volume_id': '',
1633 'public_url': 'http://example.com'},
1634 {'node_id': str(node_id_2),
1635 'volume_id': str(volume_id),
1636 'public_url': 'http://example.com'}], res)
1637
1638- def test_get_public_files_http_uses_timestamp(self):
1639- """The timestamp is used for oauth signing."""
1640- fake_timestamp = 12345678
1641-
1642- def fake_urlopen(request):
1643- """A fake urlopen."""
1644- auth = request.headers["Authorization"]
1645- expected = 'oauth_timestamp="%d"' % fake_timestamp
1646- self.assertIn(expected, auth)
1647- return StringIO("[]")
1648-
1649- self.patch(action_queue.timestamp_checker, "get_faithful_time",
1650- lambda: fake_timestamp)
1651- self.patch(action_queue, "urlopen", fake_urlopen)
1652- self.command._get_public_files_http()
1653-
1654 def test_handle_success_push_event(self):
1655 """Test AQ_PUBLIC_FILES_LIST_OK is pushed on success."""
1656 response = [{'node_id': uuid.uuid4(), 'volume_id':None,
1657@@ -2663,8 +2650,7 @@
1658 def test_handle_failure_push_event(self):
1659 """Test AQ_PUBLIC_FILES_LIST_ERROR is pushed on failure."""
1660 msg = 'Something went wrong'
1661- failure = Failure(urllib2.HTTPError(
1662- "http://example.com", 500, "Error", [], StringIO(msg)))
1663+ failure = Failure(action_queue.txweb.WebClientError("Misc Error", msg))
1664 self.command.handle_failure(failure=failure)
1665 event = ('AQ_PUBLIC_FILES_LIST_ERROR', {'error': msg})
1666 self.assertIn(event, self.command.action_queue.event_queue.events)
1667@@ -3649,27 +3635,6 @@
1668 self.assertEqual('share_name', name)
1669 self.assertTrue(read_only)
1670
1671- @defer.inlineCallbacks
1672- def test_create_share_http_uses_timestamp(self):
1673- """The timestamp is used for oauth signing."""
1674- fake_timestamp = 12345678
1675-
1676- def fake_urlopen(request):
1677- """A fake urlopen."""
1678- auth = request.headers["Authorization"]
1679- expected = 'oauth_timestamp="%d"' % fake_timestamp
1680- self.assertIn(expected, auth)
1681-
1682- self.patch(action_queue.timestamp_checker, "get_faithful_time",
1683- lambda: fake_timestamp)
1684- self.patch(action_queue, "urlopen", fake_urlopen)
1685- self.user_connect()
1686- command = CreateShare(self.request_queue, 'node_id',
1687- 'share_to@example.com', 'share_name',
1688- ACCESS_LEVEL_RO, 'marker', 'path')
1689- self.assertTrue(command.use_http, 'CreateShare should be in http mode')
1690- yield command._run()
1691-
1692 def test_possible_markers(self):
1693 """Test that it returns the correct values."""
1694 cmd = CreateShare(self.request_queue, 'node_id', 'shareto@example.com',
1695
1696=== added file 'tests/syncdaemon/test_tunnel_runner.py'
1697--- tests/syncdaemon/test_tunnel_runner.py 1970-01-01 00:00:00 +0000
1698+++ tests/syncdaemon/test_tunnel_runner.py 2012-03-20 23:05:30 +0000
1699@@ -0,0 +1,128 @@
1700+# -*- coding: utf8 -*-
1701+#
1702+# Copyright 2012 Canonical Ltd.
1703+#
1704+# This program is free software: you can redistribute it and/or modify it
1705+# under the terms of the GNU General Public License version 3, as published
1706+# by the Free Software Foundation.
1707+#
1708+# This program is distributed in the hope that it will be useful, but
1709+# WITHOUT ANY WARRANTY; without even the implied warranties of
1710+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
1711+# PURPOSE. See the GNU General Public License for more details.
1712+#
1713+# You should have received a copy of the GNU General Public License along
1714+# with this program. If not, see <http://www.gnu.org/licenses/>.
1715+"""Tests for the proxy tunnel runner."""
1716+
1717+import os
1718+import sys
1719+
1720+from twisted.internet import defer, error, reactor, task
1721+from twisted.trial.unittest import TestCase
1722+
1723+from tests.proxy import FAKE_COOKIE
1724+from ubuntuone.proxy import tunnel_client
1725+from ubuntuone.syncdaemon import tunnel_runner
1726+
1727+FAKE_HOST = "fs-1.two.ubuntu.com"
1728+FAKE_PORT = 443
1729+
1730+
1731+class TunnelRunnerConstructorTestCase(TestCase):
1732+ """Test the tunnel runner constructor."""
1733+
1734+ timeout = 3
1735+
1736+ def raise_import_error(self, *args):
1737+ """Raise an import error."""
1738+ raise ImportError
1739+
1740+ @defer.inlineCallbacks
1741+ def test_proxy_support_not_installed(self):
1742+ """The proxy support binary package is not installed."""
1743+ self.patch(tunnel_runner.TunnelRunner, "start_process",
1744+ self.raise_import_error)
1745+ tr = tunnel_runner.TunnelRunner(FAKE_HOST, FAKE_PORT)
1746+ client = yield tr.get_client()
1747+ self.assertEqual(client, reactor)
1748+
1749+
1750+class TunnelRunnerTestCase(TestCase):
1751+ """Tests for the TunnelRunner."""
1752+
1753+ timeout = 3
1754+
1755+ @defer.inlineCallbacks
1756+ def setUp(self):
1757+ """Initialize this testcase."""
1758+ yield super(TunnelRunnerTestCase, self).setUp()
1759+ self.spawned = []
1760+ self.patch(tunnel_client.reactor, "spawnProcess",
1761+ lambda *args, **kwargs: self.spawned.append((args, kwargs)))
1762+ self.process_protocol = None
1763+ self.process_protocol_class = tunnel_client.TunnelProcessProtocol
1764+ self.patch(tunnel_client, "TunnelProcessProtocol",
1765+ self.storing_process_protocol_factory)
1766+ self.tr = tunnel_runner.TunnelRunner("fs-1.one.ubuntu.com", 443)
1767+
1768+ def storing_process_protocol_factory(self, *args, **kwargs):
1769+ """Store the process protocol just created."""
1770+ self.process_protocol = self.process_protocol_class(*args, **kwargs)
1771+ return self.process_protocol
1772+
1773+ def test_tunnel_process_is_started(self):
1774+ """The tunnel process is started."""
1775+ self.assertEqual(len(self.spawned), 1,
1776+ "The tunnel process is started.")
1777+
1778+ @defer.inlineCallbacks
1779+ def test_tunnel_process_get_client_yielded_twice(self):
1780+ """The get_client method can be yielded twice."""
1781+ self.process_protocol.processExited(error.ProcessTerminated(1))
1782+ client = yield self.tr.get_client()
1783+ client = yield self.tr.get_client()
1784+ self.assertNotEqual(client, None)
1785+
1786+ @defer.inlineCallbacks
1787+ def test_tunnel_process_exits_with_error(self):
1788+ """The tunnel process exits with an error."""
1789+ self.process_protocol.processExited(error.ProcessTerminated(1))
1790+ client = yield self.tr.get_client()
1791+ self.assertEqual(client, reactor)
1792+
1793+ @defer.inlineCallbacks
1794+ def test_tunnel_process_exits_gracefully(self):
1795+ """The tunnel process exits gracefully."""
1796+ self.process_protocol.processExited(error.ProcessDone(0))
1797+ client = yield self.tr.get_client()
1798+ self.assertEqual(client, reactor)
1799+
1800+ @defer.inlineCallbacks
1801+ def test_tunnel_process_prints_random_garbage_and_timeouts(self):
1802+ """The tunnel process prints garbage and timeouts."""
1803+ clock = task.Clock()
1804+ self.patch(tunnel_client, "reactor", clock)
1805+ self.process_protocol.connectionMade()
1806+ self.process_protocol.outReceived("Random garbage")
1807+ clock.advance(self.process_protocol.timeout)
1808+ client = yield self.tr.get_client()
1809+ self.assertEqual(client, clock)
1810+
1811+ @defer.inlineCallbacks
1812+ def test_tunnel_process_prints_port_number_and_cookie(self):
1813+ """The tunnel process prints the port number."""
1814+ received = "%s: %d\n%s: %s\n" % (
1815+ tunnel_client.TUNNEL_PORT_LABEL, FAKE_PORT,
1816+ tunnel_client.TUNNEL_COOKIE_LABEL, FAKE_COOKIE)
1817+ self.process_protocol.outReceived(received)
1818+ client = yield self.tr.get_client()
1819+ self.assertEqual(client.tunnel_port, FAKE_PORT)
1820+ self.assertEqual(client.cookie, FAKE_COOKIE)
1821+
1822+ def test_frozen_path(self):
1823+ """Ensure the path is correct if sys is frozen."""
1824+ sys.frozen = "Yes"
1825+ self.addCleanup(delattr, sys, "frozen")
1826+ self.assertEqual(os.path.dirname(self.tr.get_process_path()),
1827+ os.path.dirname(sys.executable))
1828
1829=== modified file 'ubuntuone/platform/constants.py'
1830--- ubuntuone/platform/constants.py 2011-12-08 20:56:10 +0000
1831+++ ubuntuone/platform/constants.py 2012-03-20 23:05:30 +0000
1832@@ -21,7 +21,12 @@
1833
1834 import sys
1835
1836+TUNNEL_EXECUTABLE = "ubuntuone-proxy-tunnel"
1837+
1838 if sys.platform == "win32":
1839 HASHQUEUE_DELAY = 3.0
1840+ if getattr(sys, "frozen", None) is not None:
1841+ # Only use the .exe suffix in windows+frozen
1842+ TUNNEL_EXECUTABLE += ".exe"
1843 else:
1844 HASHQUEUE_DELAY = 0.5
1845
1846=== modified file 'ubuntuone/platform/linux/notification.py'
1847--- ubuntuone/platform/linux/notification.py 2012-01-25 22:40:02 +0000
1848+++ ubuntuone/platform/linux/notification.py 2012-03-20 23:05:30 +0000
1849@@ -68,6 +68,9 @@
1850 self.notification = Notify.Notification(title, message, icon)
1851 else:
1852 self.notification.update(title, message, icon)
1853+
1854 if append:
1855 self.notification.set_hint_string('x-canonical-append', '')
1856+
1857+ self.notification.set_hint('transient', True)
1858 self.notification.show()
1859
1860=== added directory 'ubuntuone/proxy'
1861=== added file 'ubuntuone/proxy/__init__.py'
1862--- ubuntuone/proxy/__init__.py 1970-01-01 00:00:00 +0000
1863+++ ubuntuone/proxy/__init__.py 2012-03-20 23:05:30 +0000
1864@@ -0,0 +1,14 @@
1865+# Copyright 2012 Canonical Ltd.
1866+#
1867+# This program is free software: you can redistribute it and/or modify it
1868+# under the terms of the GNU General Public License version 3, as published
1869+# by the Free Software Foundation.
1870+#
1871+# This program is distributed in the hope that it will be useful, but
1872+# WITHOUT ANY WARRANTY; without even the implied warranties of
1873+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
1874+# PURPOSE. See the GNU General Public License for more details.
1875+#
1876+# You should have received a copy of the GNU General Public License along
1877+# with this program. If not, see <http://www.gnu.org/licenses/>.
1878+"""Ubuntu One proxy support."""
1879
1880=== added file 'ubuntuone/proxy/common.py'
1881--- ubuntuone/proxy/common.py 1970-01-01 00:00:00 +0000
1882+++ ubuntuone/proxy/common.py 2012-03-20 23:05:30 +0000
1883@@ -0,0 +1,60 @@
1884+# -*- coding: utf8 -*-
1885+#
1886+# Copyright 2012 Canonical Ltd.
1887+#
1888+# This program is free software: you can redistribute it and/or modify it
1889+# under the terms of the GNU General Public License version 3, as published
1890+# by the Free Software Foundation.
1891+#
1892+# This program is distributed in the hope that it will be useful, but
1893+# WITHOUT ANY WARRANTY; without even the implied warranties of
1894+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
1895+# PURPOSE. See the GNU General Public License for more details.
1896+#
1897+# You should have received a copy of the GNU General Public License along
1898+# with this program. If not, see <http://www.gnu.org/licenses/>.
1899+"""Common classes to the tunnel client and server."""
1900+
1901+from twisted.protocols import basic
1902+
1903+CRLF = "\r\n"
1904+TUNNEL_PORT_LABEL = "Tunnel port"
1905+TUNNEL_COOKIE_LABEL = "Tunnel cookie"
1906+TUNNEL_COOKIE_HEADER = "Proxy-Tunnel-Cookie"
1907+
1908+
1909+class BaseTunnelProtocol(basic.LineReceiver):
1910+ """CONNECT base protocol for tunnelling connections."""
1911+
1912+ delimiter = CRLF
1913+
1914+ def __init__(self):
1915+ """Initialize this protocol."""
1916+ self._first_line = True
1917+ self.received_headers = []
1918+
1919+ def header_line(self, line):
1920+ """Handle each header line received."""
1921+ key, value = line.split(":", 1)
1922+ value = value.strip()
1923+ self.received_headers.append((key, value))
1924+
1925+ def lineReceived(self, line):
1926+ """Process a line in the header."""
1927+ if self._first_line:
1928+ self._first_line = False
1929+ self.handle_first_line(line)
1930+ else:
1931+ if line:
1932+ self.header_line(line)
1933+ else:
1934+ self.setRawMode()
1935+ self.headers_done()
1936+
1937+ def remote_disconnected(self):
1938+ """The remote end closed the connection."""
1939+ self.transport.loseConnection()
1940+
1941+ def format_headers(self, headers):
1942+ """Format some headers as a few response lines."""
1943+ return "".join("%s: %s" % item + CRLF for item in headers.items())
1944
1945=== added file 'ubuntuone/proxy/logger.py'
1946--- ubuntuone/proxy/logger.py 1970-01-01 00:00:00 +0000
1947+++ ubuntuone/proxy/logger.py 2012-03-20 23:05:30 +0000
1948@@ -0,0 +1,36 @@
1949+# ubuntuone.syncdaemon.logger - logging utilities
1950+#
1951+# Copyright 2009-2012 Canonical Ltd.
1952+#
1953+# This program is free software: you can redistribute it and/or modify it
1954+# under the terms of the GNU General Public License version 3, as published
1955+# by the Free Software Foundation.
1956+#
1957+# This program is distributed in the hope that it will be useful, but
1958+# WITHOUT ANY WARRANTY; without even the implied warranties of
1959+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
1960+# PURPOSE. See the GNU General Public License for more details.
1961+#
1962+# You should have received a copy of the GNU General Public License along
1963+# with this program. If not, see <http://www.gnu.org/licenses/>.
1964+
1965+"""SyncDaemon logging utilities and config."""
1966+
1967+import logging
1968+import os
1969+
1970+from ubuntuone.logger import (
1971+ basic_formatter,
1972+ CustomRotatingFileHandler,
1973+)
1974+
1975+from ubuntuone.platform.xdg_base_directory import ubuntuone_log_dir
1976+
1977+
1978+LOGFILENAME = os.path.join(ubuntuone_log_dir, 'proxy.log')
1979+logger = logging.getLogger("ubuntuone.proxy")
1980+logger.setLevel(logging.DEBUG)
1981+handler = CustomRotatingFileHandler(filename=LOGFILENAME)
1982+handler.setFormatter(basic_formatter)
1983+handler.setLevel(logging.DEBUG)
1984+logger.addHandler(handler)
1985
1986=== added file 'ubuntuone/proxy/tunnel_client.py'
1987--- ubuntuone/proxy/tunnel_client.py 1970-01-01 00:00:00 +0000
1988+++ ubuntuone/proxy/tunnel_client.py 2012-03-20 23:05:30 +0000
1989@@ -0,0 +1,181 @@
1990+# -*- coding: utf8 -*-
1991+#
1992+# Copyright 2012 Canonical Ltd.
1993+#
1994+# This program is free software: you can redistribute it and/or modify it
1995+# under the terms of the GNU General Public License version 3, as published
1996+# by the Free Software Foundation.
1997+#
1998+# This program is distributed in the hope that it will be useful, but
1999+# WITHOUT ANY WARRANTY; without even the implied warranties of
2000+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
2001+# PURPOSE. See the GNU General Public License for more details.
2002+#
2003+# You should have received a copy of the GNU General Public License along
2004+# with this program. If not, see <http://www.gnu.org/licenses/>.
2005+"""Client for the tunnel protocol."""
2006+
2007+import logging
2008+
2009+from twisted.internet import protocol, reactor
2010+
2011+from ubuntuone.proxy.common import (
2012+ BaseTunnelProtocol,
2013+ CRLF,
2014+ TUNNEL_COOKIE_LABEL,
2015+ TUNNEL_COOKIE_HEADER,
2016+ TUNNEL_PORT_LABEL,
2017+)
2018+
2019+METHOD_LINE = "CONNECT %s:%d HTTP/1.0" + CRLF
2020+LOCALHOST = "127.0.0.1"
2021+logger = logging.getLogger("ubuntuone.Proxy.TunnelClient")
2022+
2023+
2024+class TunnelClientProtocol(BaseTunnelProtocol):
2025+ """Client protocol for the handshake part of the tunnel."""
2026+
2027+ def connectionMade(self):
2028+ """The connection to the tunnel was made so send request."""
2029+ method_line = METHOD_LINE % (self.factory.tunnel_host,
2030+ self.factory.tunnel_port)
2031+ headers = {
2032+ "User-Agent": "Ubuntu One tunnel client",
2033+ TUNNEL_COOKIE_HEADER: self.factory.cookie,
2034+ }
2035+ self.transport.write(method_line +
2036+ self.format_headers(headers) +
2037+ CRLF)
2038+
2039+ def handle_first_line(self, line):
2040+ """The first line received is the status line."""
2041+ try:
2042+ proto_version, self.status_code, description = line.split(" ", 2)
2043+ except ValueError:
2044+ self.transport.loseConnection()
2045+
2046+ def headers_done(self):
2047+ """All the headers have arrived. Time to switch protocols."""
2048+ remaining_data = self.clearLineBuffer()
2049+ if self.status_code != "200":
2050+ self.transport.loseConnection()
2051+ return
2052+ addr = self.transport.getPeer()
2053+ other_protocol = self.factory.other_factory.buildProtocol(addr)
2054+ self.transport.protocol = other_protocol
2055+ other_protocol.transport = self.transport
2056+ self.transport = None
2057+ if self.factory.context_factory:
2058+ other_protocol.transport.startTLS(self.factory.context_factory)
2059+ other_protocol.connectionMade()
2060+ if remaining_data:
2061+ other_protocol.dataReceived(remaining_data)
2062+
2063+
2064+class TunnelClientFactory(protocol.ClientFactory):
2065+ """A factory for Tunnel Client Protocols."""
2066+
2067+ protocol = TunnelClientProtocol
2068+
2069+ def __init__(self, tunnel_host, tunnel_port, other_factory, cookie,
2070+ context_factory=None):
2071+ """Initialize this factory."""
2072+ self.tunnel_host = tunnel_host
2073+ self.tunnel_port = tunnel_port
2074+ self.other_factory = other_factory
2075+ self.context_factory = context_factory
2076+ self.cookie = cookie
2077+
2078+ def startedConnecting(self, connector):
2079+ """Forward this call to the other factory."""
2080+ self.other_factory.startedConnecting(connector)
2081+
2082+ def clientConnectionFailed(self, connector, reason):
2083+ """Forward this call to the other factory."""
2084+ self.other_factory.clientConnectionFailed(connector, reason)
2085+
2086+ def clientConnectionLost(self, connector, reason):
2087+ """Forward this call to the other factory."""
2088+ self.other_factory.clientConnectionLost(connector, reason)
2089+
2090+
2091+class TunnelClient(object):
2092+ """A client for the proxy tunnel."""
2093+
2094+ def __init__(self, tunnel_host, tunnel_port, cookie):
2095+ """Initialize this client."""
2096+ self.tunnel_host = tunnel_host
2097+ self.tunnel_port = tunnel_port
2098+ self.cookie = cookie
2099+
2100+ def connectTCP(self, host, port, factory, *args, **kwargs):
2101+ """A connectTCP going thru the tunnel."""
2102+ logger.info("Connecting (TCP) to %r:%r via tunnel at %r:%r",
2103+ host, port, self.tunnel_host, self.tunnel_port)
2104+ tunnel_factory = TunnelClientFactory(host, port, factory, self.cookie)
2105+ return reactor.connectTCP(self.tunnel_host, self.tunnel_port,
2106+ tunnel_factory, *args, **kwargs)
2107+
2108+ def connectSSL(self, host, port, factory,
2109+ contextFactory, *args, **kwargs):
2110+ """A connectSSL going thru the tunnel."""
2111+ logger.info("Connecting (SSL) to %r:%r via tunnel at %r:%r",
2112+ host, port, self.tunnel_host, self.tunnel_port)
2113+ tunnel_factory = TunnelClientFactory(host, port, factory, self.cookie,
2114+ contextFactory)
2115+ return reactor.connectTCP(self.tunnel_host, self.tunnel_port,
2116+ tunnel_factory, *args, **kwargs)
2117+
2118+
2119+class TunnelProcessProtocol(protocol.ProcessProtocol):
2120+ """The dialog thru stdout with the tunnel server."""
2121+
2122+ timeout = 5
2123+
2124+ def __init__(self, client_d):
2125+ """Initialize this protocol."""
2126+ self.client_d = client_d
2127+ self.timer = None
2128+ self.port = None
2129+ self.cookie = None
2130+
2131+ def connectionMade(self):
2132+ """The process has started, start a timer."""
2133+ logger.info("Tunnel process started.")
2134+ self.timer = reactor.callLater(self.timeout, self.process_timeouted)
2135+
2136+ def process_timeouted(self):
2137+ """The process took too long to reply."""
2138+ if not self.client_d.called:
2139+ logger.info("Timeout while waiting for tunnel process.")
2140+ self.client_d.callback(reactor)
2141+
2142+ def finish_timeout(self):
2143+ """Stop the timer from firing."""
2144+ if self.timer and self.timer.active():
2145+ self.timer.cancel()
2146+
2147+ def processExited(self, status):
2148+ """The tunnel process has exited with some error code."""
2149+ self.finish_timeout()
2150+ logger.info("Tunnel process exit status %r.", status)
2151+ if not self.client_d.called:
2152+ self.client_d.callback(reactor)
2153+
2154+ def outReceived(self, data):
2155+ """Receive the port number."""
2156+ if self.client_d.called:
2157+ return
2158+
2159+ for line in data.split("\n"):
2160+ if line.startswith(TUNNEL_PORT_LABEL):
2161+ _header, port = line.split(":", 1)
2162+ self.port = int(port.strip())
2163+ if line.startswith(TUNNEL_COOKIE_LABEL):
2164+ _header, cookie = line.split(":", 1)
2165+ self.cookie = cookie.strip()
2166+
2167+ if self.port and self.cookie:
2168+ logger.info("Tunnel process listening on port %r.", self.port)
2169+ client = TunnelClient(LOCALHOST, self.port, self.cookie)
2170+ self.client_d.callback(client)
2171
2172=== added file 'ubuntuone/proxy/tunnel_server.py'
2173--- ubuntuone/proxy/tunnel_server.py 1970-01-01 00:00:00 +0000
2174+++ ubuntuone/proxy/tunnel_server.py 2012-03-20 23:05:30 +0000
2175@@ -0,0 +1,376 @@
2176+# -*- coding: utf8 -*-
2177+#
2178+# Copyright 2012 Canonical Ltd.
2179+#
2180+# This program is free software: you can redistribute it and/or modify it
2181+# under the terms of the GNU General Public License version 3, as published
2182+# by the Free Software Foundation.
2183+#
2184+# This program is distributed in the hope that it will be useful, but
2185+# WITHOUT ANY WARRANTY; without even the implied warranties of
2186+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
2187+# PURPOSE. See the GNU General Public License for more details.
2188+#
2189+# You should have received a copy of the GNU General Public License along
2190+# with this program. If not, see <http://www.gnu.org/licenses/>.
2191+"""A tunnel through proxies.
2192+
2193+The layers in a tunneled proxied connection:
2194+
2195+↓ tunnelclient - initiates tcp to tunnelserver, request outward connection
2196+↕ client protocol - started after the tunneclient gets connected
2197+---process boundary---
2198+↕ tunnelserver - creates a tunnel instance per incoming connection
2199+↕ tunnel - hold a qtcpsocket to tunnelclient, and srvtunnelproto to the remote
2200+↕ servertunnelprotocol - gets CONNECT from tunnelclient, creates a remotesocket
2201+↕ remotesocket - connects to the destination server via a proxy
2202+↕ proxy server - goes thru firewalls
2203+↑ server - dialogues with the client protocol
2204+
2205+"""
2206+
2207+import sys
2208+import uuid
2209+
2210+from PyQt4.QtCore import QCoreApplication, QTimer
2211+from PyQt4.QtNetwork import (
2212+ QAbstractSocket,
2213+ QHostAddress,
2214+ QNetworkProxy,
2215+ QNetworkProxyQuery,
2216+ QNetworkProxyFactory,
2217+ QTcpServer,
2218+ QTcpSocket,
2219+)
2220+from twisted.internet import defer, interfaces
2221+from zope.interface import implements
2222+
2223+from ubuntu_sso.keyring import Keyring
2224+from ubuntu_sso.utils.webclient import gsettings
2225+from ubuntuone.proxy.common import (
2226+ BaseTunnelProtocol,
2227+ CRLF,
2228+ TUNNEL_COOKIE_HEADER,
2229+ TUNNEL_COOKIE_LABEL,
2230+ TUNNEL_PORT_LABEL,
2231+)
2232+from ubuntuone.proxy.logger import logger
2233+
2234+DEFAULT_CODE = 500
2235+DEFAULT_DESCRIPTION = "Connection error"
2236+
2237+
2238+class ConnectionError(Exception):
2239+ """The client failed connecting to the destination."""
2240+
2241+ def __init__(self, code=DEFAULT_CODE, description=DEFAULT_DESCRIPTION):
2242+ self.code = code
2243+ self.description = description
2244+
2245+
2246+class ProxyAuthenticationError(ConnectionError):
2247+ """Credentials mismatch going thru a proxy."""
2248+
2249+
2250+def build_proxy(settings_groups):
2251+ """Create a QNetworkProxy from these settings."""
2252+ proxy_groups = [
2253+ ("socks", QNetworkProxy.Socks5Proxy),
2254+ ("https", QNetworkProxy.HttpProxy),
2255+ ("http", QNetworkProxy.HttpProxy),
2256+ ]
2257+ for group, proxy_type in proxy_groups:
2258+ if group not in settings_groups:
2259+ continue
2260+ settings = settings_groups[group]
2261+ if "host" in settings and "port" in settings:
2262+ return QNetworkProxy(proxy_type,
2263+ hostName=settings.get("host", ""),
2264+ port=settings.get("port", 0),
2265+ user=settings.get("username", ""),
2266+ password=settings.get("password", ""))
2267+ logger.error("No proxy correctly configured.")
2268+ return QNetworkProxy(QNetworkProxy.DefaultProxy)
2269+
2270+
2271+class RemoteSocket(QTcpSocket):
2272+ """A dumb connection through a proxy to a destination hostport."""
2273+
2274+ def __init__(self, tunnel_protocol):
2275+ """Initialize this object."""
2276+ super(RemoteSocket, self).__init__()
2277+ self.protocol = tunnel_protocol
2278+ self.connected_d = defer.Deferred()
2279+ self.connected.connect(self.handle_connected)
2280+ self.proxyAuthenticationRequired.connect(self.handle_auth_required)
2281+ self.buffered_data = []
2282+
2283+ def handle_connected(self):
2284+ """When connected, send all pending data."""
2285+ self.disconnected.connect(self.handle_disconnected)
2286+ self.connected_d.callback(None)
2287+ for d in self.buffered_data:
2288+ logger.debug("writing remote: %d bytes", len(d))
2289+ super(RemoteSocket, self).write(d)
2290+ self.buffered_data = []
2291+
2292+ def handle_disconnected(self):
2293+ """Do something with disconnections."""
2294+ logger.debug("Remote socket disconnected")
2295+ self.protocol.remote_disconnected()
2296+
2297+ def write(self, data):
2298+ """Write data to the remote end, buffering if not connected."""
2299+ if self.state() == QAbstractSocket.ConnectedState:
2300+ logger.debug("writing remote: %d bytes", len(data))
2301+ super(RemoteSocket, self).write(data)
2302+ else:
2303+ self.buffered_data.append(data)
2304+
2305+ def connect(self, hostport):
2306+ """Try to establish the connection to the remote end."""
2307+ host, port = hostport.split(":")
2308+
2309+ try:
2310+ port = int(port)
2311+ except ValueError:
2312+ raise ConnectionError(400, "Destination port must be an integer.")
2313+
2314+ self.readyRead.connect(self.handle_ready_read)
2315+ self.error.connect(self.handle_error)
2316+ self.connectToHost(host, port)
2317+
2318+ return self.connected_d
2319+
2320+ def handle_auth_required(self, proxy, authenticator):
2321+ """Handle the proxyAuthenticationRequired signal."""
2322+ self.protocol.proxy_auth_required(proxy, authenticator)
2323+
2324+ def handle_error(self, socket_error):
2325+ """Some error happened while connecting."""
2326+ error_description = "%s (%d)" % (self.errorString(), socket_error)
2327+ logger.error("connection error: %s", error_description)
2328+ if self.connected_d.called:
2329+ return
2330+
2331+ if socket_error == self.ProxyAuthenticationRequiredError:
2332+ error = ProxyAuthenticationError(407, error_description)
2333+ else:
2334+ error = ConnectionError(500, error_description)
2335+
2336+ self.connected_d.errback(error)
2337+
2338+ def handle_ready_read(self):
2339+ """Forward data from the remote end to the parent protocol."""
2340+ data = self.readAll()
2341+ self.protocol.response_data_received(data)
2342+
2343+ @defer.inlineCallbacks
2344+ def stop(self):
2345+ """Finish and cleanup."""
2346+ self.disconnectFromHost()
2347+ while self.state() != self.UnconnectedState:
2348+ d = defer.Deferred()
2349+ QTimer.singleShot(100, lambda: d.callback(None))
2350+ yield d
2351+
2352+
2353+class ServerTunnelProtocol(BaseTunnelProtocol):
2354+ """CONNECT sever protocol for tunnelling connections."""
2355+
2356+ def __init__(self, client_class):
2357+ """Initialize this protocol."""
2358+ BaseTunnelProtocol.__init__(self)
2359+ self.hostport = ""
2360+ self.client = None
2361+ self.client_class = client_class
2362+ self.proxy_credentials = None
2363+ self.proxy_domain = None
2364+
2365+ def error_response(self, code, description):
2366+ """Write a response with an error, and disconnect."""
2367+ self.write_transport("HTTP/1.0 %d %s" % (code, description) + CRLF * 2)
2368+ self.transport.loseConnection()
2369+ if self.client:
2370+ self.client.stop()
2371+ self.clearLineBuffer()
2372+
2373+ def write_transport(self, data):
2374+ """Write a response in the transport."""
2375+ self.transport.write(data)
2376+
2377+ def proxy_auth_required(self, proxy, authenticator):
2378+ """Proxy authentication is required."""
2379+ logger.info("auth_required %r, %r, %r", self.proxy_credentials,
2380+ proxy.hostName(), self.proxy_domain)
2381+ if self.proxy_credentials and proxy.hostName() == self.proxy_domain:
2382+ logger.info("Credentials added to authenticator: %r.",
2383+ self.proxy_credentials)
2384+ authenticator.setUser(self.proxy_credentials["username"])
2385+ authenticator.setPassword(self.proxy_credentials["password"])
2386+ else:
2387+ logger.info("Credentials needed, but none available.")
2388+ self.proxy_domain = proxy.hostName()
2389+
2390+ def handle_first_line(self, line):
2391+ """Special handling for the first line received."""
2392+ try:
2393+ method, hostport, proto_version = line.split(" ", 2)
2394+ if proto_version != "HTTP/1.0":
2395+ self.error_response(505, "HTTP Version Not Supported")
2396+ return
2397+ if method != "CONNECT":
2398+ self.error_response(405, "Only the CONNECT method is allowed")
2399+ return
2400+ self.hostport = hostport
2401+ except ValueError:
2402+ self.error_response(400, "Bad request")
2403+
2404+ def verify_cookie(self):
2405+ """Fail if the cookie is wrong or missing."""
2406+ cookie_received = dict(self.received_headers).get(TUNNEL_COOKIE_HEADER)
2407+ if cookie_received != self.transport.cookie:
2408+ raise ConnectionError(418, "Please see RFC 2324")
2409+
2410+ @defer.inlineCallbacks
2411+ def headers_done(self):
2412+ """An empty line was received, start connecting and switch mode."""
2413+ try:
2414+ self.verify_cookie()
2415+ try:
2416+ logger.info("Connecting once")
2417+ self.client = self.client_class(self)
2418+ yield self.client.connect(self.hostport)
2419+ except ProxyAuthenticationError:
2420+ if not self.proxy_domain:
2421+ logger.info("No proxy domain defined")
2422+ raise
2423+
2424+ credentials = yield Keyring().get_credentials(
2425+ str(self.proxy_domain))
2426+ if "username" in credentials:
2427+ self.proxy_credentials = credentials
2428+ logger.info("Connecting again with keyring credentials")
2429+ self.client = self.client_class(self)
2430+ yield self.client.connect(self.hostport)
2431+ logger.info("Connected with keyring credentials")
2432+
2433+ response_headers = {
2434+ "Server": "Ubuntu One proxy tunnel",
2435+ }
2436+ self.write_transport("HTTP/1.0 200 Proxy connection established" +
2437+ CRLF + self.format_headers(response_headers) +
2438+ CRLF)
2439+ except ConnectionError as e:
2440+ logger.exception("Connection error")
2441+ self.error_response(e.code, e.description)
2442+ except Exception:
2443+ logger.exception("Unhandled problem while connecting")
2444+
2445+ def rawDataReceived(self, data):
2446+ """Tunnel all raw data straight to the other side."""
2447+ self.client.write(data)
2448+
2449+ def response_data_received(self, data):
2450+ """Return data coming from the other side."""
2451+ self.write_transport(data)
2452+
2453+
2454+class Tunnel(object):
2455+ """An instance of a running tunnel."""
2456+
2457+ implements(interfaces.ITransport)
2458+
2459+ def __init__(self, local_socket, cookie):
2460+ """Initialize this Tunnel instance."""
2461+ self.cookie = cookie
2462+ self.disconnecting = False
2463+ self.local_socket = local_socket
2464+ self.protocol = ServerTunnelProtocol(RemoteSocket)
2465+ self.protocol.transport = self
2466+ local_socket.readyRead.connect(self.server_ready_read)
2467+ local_socket.disconnected.connect(self.local_disconnected)
2468+
2469+ def server_ready_read(self):
2470+ """Data available on the local end. Move it forward."""
2471+ data = bytes(self.local_socket.readAll())
2472+ self.protocol.dataReceived(data)
2473+
2474+ def write(self, data):
2475+ """Data available on the remote end. Bring it back."""
2476+ logger.debug("writing local: %d bytes", len(data))
2477+ self.local_socket.write(data)
2478+
2479+ def loseConnection(self):
2480+ """The remote end disconnected."""
2481+ logger.debug("disconnecting local end.")
2482+ self.local_socket.close()
2483+
2484+ def local_disconnected(self):
2485+ """The local end disconnected."""
2486+ logger.debug("The local socket got disconnected.")
2487+ # TODO: handle this case in an upcoming branch
2488+
2489+
2490+class TunnelServer(object):
2491+ """A server for tunnel instances."""
2492+
2493+ def __init__(self, cookie):
2494+ """Initialize this tunnel instance."""
2495+ self.tunnels = []
2496+ self.cookie = cookie
2497+ self.server = QTcpServer(QCoreApplication.instance())
2498+ self.server.newConnection.connect(self.new_connection)
2499+ self.server.listen(QHostAddress.LocalHost, 0)
2500+ logger.info("Starting tunnel server at port %d", self.port)
2501+
2502+ def new_connection(self):
2503+ """On a new connection create a new tunnel instance."""
2504+ logger.info("New connection made")
2505+ local_socket = self.server.nextPendingConnection()
2506+ tunnel = Tunnel(local_socket, self.cookie)
2507+ self.tunnels.append(tunnel)
2508+
2509+ def shutdown(self):
2510+ """Terminate every connection."""
2511+ # TODO: handle this gracefully in an upcoming branch
2512+
2513+ @property
2514+ def port(self):
2515+ """The port where this server listens."""
2516+ return self.server.serverPort()
2517+
2518+
2519+def check_proxy_enabled(host, port):
2520+ """Check if the proxy is enabled."""
2521+ port = int(port)
2522+ if sys.platform.startswith("linux"):
2523+ settings = gsettings.get_proxy_settings()
2524+ enabled = len(settings) > 0
2525+ if enabled:
2526+ proxy = build_proxy(settings)
2527+ QNetworkProxy.setApplicationProxy(proxy)
2528+ else:
2529+ logger.info("Proxy is disabled.")
2530+ return enabled
2531+ else:
2532+ query = QNetworkProxyQuery(host, port)
2533+ proxies = QNetworkProxyFactory.systemProxyForQuery(query)
2534+ return len(proxies) and proxies[0].type() != QNetworkProxy.DefaultProxy
2535+
2536+
2537+def main(argv):
2538+ """The main function for the tunnel server."""
2539+ if not check_proxy_enabled(*argv[1:]):
2540+ sys.stdout.write("Proxy not enabled.")
2541+ sys.stdout.flush()
2542+ else:
2543+ from dbus.mainloop.qt import DBusQtMainLoop
2544+ DBusQtMainLoop(set_as_default=True)
2545+ app = QCoreApplication(argv)
2546+ cookie = str(uuid.uuid4())
2547+ tunnel_server = TunnelServer(cookie)
2548+ sys.stdout.write("%s: %d\n" % (TUNNEL_PORT_LABEL, tunnel_server.port) +
2549+ "%s: %s\n" % (TUNNEL_COOKIE_LABEL, cookie))
2550+ sys.stdout.flush()
2551+ app.exec_()
2552
2553=== modified file 'ubuntuone/syncdaemon/action_queue.py'
2554--- ubuntuone/syncdaemon/action_queue.py 2012-02-09 21:57:00 +0000
2555+++ ubuntuone/syncdaemon/action_queue.py 2012-03-20 23:05:30 +0000
2556@@ -31,19 +31,18 @@
2557 from collections import deque, defaultdict
2558 from functools import partial
2559 from urllib import urlencode
2560-from urllib2 import urlopen, Request, HTTPError
2561 from urlparse import urljoin
2562
2563 import OpenSSL.SSL
2564
2565 from zope.interface import implements
2566-from twisted.internet import reactor, defer, threads, task
2567+from twisted.internet import reactor, defer, task
2568 from twisted.internet import error as twisted_errors
2569 from twisted.names import client as dns_client
2570 from twisted.python.failure import Failure, DefaultException
2571
2572 from oauth import oauth
2573-from ubuntu_sso.utils import timestamp_checker
2574+from ubuntu_sso.utils.webclient import txweb
2575 from ubuntuone import clientdefs
2576 from ubuntuone.platform import platform, remove_file
2577 from ubuntuone.storageprotocol import protocol_pb2, content_hash
2578@@ -57,6 +56,7 @@
2579 from ubuntuone.syncdaemon.logger import mklog, TRACE
2580 from ubuntuone.syncdaemon.volume_manager import ACCESS_LEVEL_RW
2581 from ubuntuone.syncdaemon import config, offload_queue
2582+from ubuntuone.syncdaemon import tunnel_runner
2583
2584 logger = logging.getLogger("ubuntuone.SyncDaemon.ActionQueue")
2585
2586@@ -684,6 +684,8 @@
2587 # credentials
2588 self.token = None
2589 self.consumer = None
2590+ self.credentials = None
2591+ self.webclient = None
2592
2593 self.client = None # an instance of self.protocol
2594
2595@@ -699,6 +701,7 @@
2596 self.zip_queue = ZipQueue()
2597 self.conditions_locker = ConditionsLocker()
2598 self.disk_queue = offload_queue.OffloadQueue()
2599+ self.tunnel_runner = tunnel_runner.TunnelRunner(self.host, self.port)
2600
2601 self.estimated_free_space = {}
2602 event_queue.subscribe(self)
2603@@ -728,6 +731,7 @@
2604
2605 def handle_SYS_USER_CONNECT(self, access_token):
2606 """Stow the access token away for later use."""
2607+ self.credentials = access_token
2608 self.token = oauth.OAuthToken(access_token['token'],
2609 access_token['token_secret'])
2610 self.consumer = oauth.OAuthConsumer(access_token['consumer_key'],
2611@@ -816,16 +820,37 @@
2612 else:
2613 return defer.succeed((self.host, self.port))
2614
2615+
2616+ @defer.inlineCallbacks
2617+ def webcall(self, iri, **kwargs):
2618+ """Perform a web call to the api servers."""
2619+ webclient = yield self.get_webclient()
2620+ response = yield webclient.request(iri,
2621+ oauth_credentials=self.credentials, **kwargs)
2622+ defer.returnValue(response)
2623+
2624+ @defer.inlineCallbacks
2625+ def get_webclient(self):
2626+ """Get the webclient, creating it if needed."""
2627+ if self.webclient is None:
2628+ client = yield self.tunnel_runner.get_client()
2629+ self.webclient = txweb.WebClient(connector=client,
2630+ appname="Ubuntu One",
2631+ oauth_sign_plain=True)
2632+ defer.returnValue(self.webclient)
2633+
2634+ @defer.inlineCallbacks
2635 def _make_connection(self, result):
2636 """Do the real connect call."""
2637 host, port = result
2638 ssl_context = get_ssl_context(self.disable_ssl_verify)
2639+ client = yield self.tunnel_runner.get_client()
2640 if self.use_ssl:
2641- self.connector = reactor.connectSSL(host, port, factory=self,
2642+ self.connector = client.connectSSL(host, port, factory=self,
2643 contextFactory=ssl_context,
2644 timeout=self.connection_timeout)
2645 else:
2646- self.connector = reactor.connectTCP(host, port, self,
2647+ self.connector = client.connectTCP(host, port, self,
2648 timeout=self.connection_timeout)
2649
2650 def connect(self):
2651@@ -1774,36 +1799,23 @@
2652 if share_to and re.match(EREGEX, share_to):
2653 self.use_http = True
2654
2655+ @defer.inlineCallbacks
2656 def _create_share_http(self, node_id, user, name, read_only):
2657 """Create a share using the HTTP Web API method."""
2658
2659- url = "https://one.ubuntu.com/files/api/offer_share/"
2660- method = oauth.OAuthSignatureMethod_PLAINTEXT()
2661- timestamp = timestamp_checker.get_faithful_time()
2662- parameters = {"oauth_timestamp": timestamp}
2663- request = oauth.OAuthRequest.from_consumer_and_token(
2664- http_url=url,
2665- http_method="POST",
2666- parameters=parameters,
2667- oauth_consumer=self.action_queue.consumer,
2668- token=self.action_queue.token)
2669- request.sign_request(method, self.action_queue.consumer,
2670- self.action_queue.token)
2671+ iri = u"https://one.ubuntu.com/files/api/offer_share/"
2672 data = dict(offer_to_email=user,
2673 read_only=read_only,
2674 node_id=node_id,
2675 share_name=name)
2676 pdata = urlencode(data)
2677- headers = request.to_header()
2678- req = Request(url, pdata, headers)
2679- urlopen(req)
2680+ yield self.action_queue.webcall(iri, method="POST", post_content=pdata)
2681
2682 def _run(self):
2683 """Do the actual running."""
2684 if self.use_http:
2685 # External user, do the HTTP REST method
2686- return threads.deferToThread(self._create_share_http,
2687- self.node_id, self.share_to,
2688+ return self._create_share_http(self.node_id, self.share_to,
2689 self.name,
2690 self.access_level != ACCESS_LEVEL_RW)
2691 else:
2692@@ -1827,7 +1839,7 @@
2693 """It didn't work! Push the event."""
2694 self.action_queue.event_queue.push('AQ_CREATE_SHARE_ERROR',
2695 marker=self.marker,
2696- error=failure.getErrorMessage())
2697+ error=failure.value[1])
2698
2699 def _acquire_pathlock(self):
2700 """Acquire pathlock."""
2701@@ -2109,6 +2121,7 @@
2702 self.node_id = node_id
2703 self.is_public = is_public
2704
2705+ @defer.inlineCallbacks
2706 def _change_public_access_http(self):
2707 """Change public access using the HTTP Web API method."""
2708
2709@@ -2119,28 +2132,16 @@
2710 base64.urlsafe_b64encode(self.share_id.bytes).strip("="),
2711 node_key)
2712
2713- url = "https://one.ubuntu.com/files/api/set_public/%s" % (node_key,)
2714- method = oauth.OAuthSignatureMethod_PLAINTEXT()
2715- timestamp = timestamp_checker.get_faithful_time()
2716- parameters = {"oauth_timestamp": timestamp}
2717- request = oauth.OAuthRequest.from_consumer_and_token(
2718- http_url=url,
2719- http_method="POST",
2720- parameters=parameters,
2721- oauth_consumer=self.action_queue.consumer,
2722- token=self.action_queue.token)
2723- request.sign_request(method, self.action_queue.consumer,
2724- self.action_queue.token)
2725+ iri = u"https://one.ubuntu.com/files/api/set_public/%s" % (node_key,)
2726 data = dict(is_public=bool(self.is_public))
2727 pdata = urlencode(data)
2728- headers = request.to_header()
2729- req = Request(url, pdata, headers)
2730- response = urlopen(req)
2731- return simplejson.load(response)
2732+ response = yield self.action_queue.webcall(iri, method="POST",
2733+ post_content=pdata)
2734+ defer.returnValue(simplejson.loads(response.content))
2735
2736 def _run(self):
2737 """See ActionQueueCommand."""
2738- return threads.deferToThread(self._change_public_access_http)
2739+ return self._change_public_access_http()
2740
2741 def handle_success(self, success):
2742 """See ActionQueueCommand."""
2743@@ -2152,51 +2153,36 @@
2744
2745 def handle_failure(self, failure):
2746 """It didn't work! Push the event."""
2747- if issubclass(failure.type, HTTPError):
2748- message = failure.value.read()
2749- else:
2750- message = failure.getErrorMessage()
2751 self.action_queue.event_queue.push('AQ_CHANGE_PUBLIC_ACCESS_ERROR',
2752 share_id=self.share_id,
2753 node_id=self.node_id,
2754- error=message)
2755+ error=failure.value[1])
2756
2757
2758 class GetPublicFiles(ActionQueueCommand):
2759 """Get the list of public files."""
2760
2761- __slots__ = ('_url',)
2762+ __slots__ = ('_iri',)
2763 logged_attrs = ActionQueueCommand.logged_attrs + __slots__
2764
2765- def __init__(self, request_queue, base_url='https://one.ubuntu.com'):
2766+ def __init__(self, request_queue, base_iri=u'https://one.ubuntu.com'):
2767 super(GetPublicFiles, self).__init__(request_queue)
2768- self._url = urljoin(base_url, 'files/api/public_files')
2769+ self._iri = urljoin(base_iri, u'files/api/public_files')
2770
2771+ @defer.inlineCallbacks
2772 def _get_public_files_http(self):
2773 """Get public files list using the HTTP Web API method."""
2774
2775- method = oauth.OAuthSignatureMethod_PLAINTEXT()
2776- timestamp = timestamp_checker.get_faithful_time()
2777- parameters = {"oauth_timestamp": timestamp}
2778- request = oauth.OAuthRequest.from_consumer_and_token(
2779- http_url=self._url,
2780- http_method="GET",
2781- parameters=parameters,
2782- oauth_consumer=self.action_queue.consumer,
2783- token=self.action_queue.token)
2784- request.sign_request(method, self.action_queue.consumer,
2785- self.action_queue.token)
2786- headers = request.to_header()
2787- req = Request(self._url, headers=headers)
2788- response = urlopen(req)
2789- files = simplejson.load(response)
2790+ response = yield self.action_queue.webcall(self._iri, method="GET")
2791+
2792+ files = simplejson.loads(response.content)
2793 # translate nodekeys to (volume_id, node_id)
2794 for pf in files:
2795 _, node_id = self.split_nodekey(pf.pop('nodekey'))
2796 volume_id = pf['volume_id']
2797 pf['volume_id'] = '' if volume_id is None else volume_id
2798 pf['node_id'] = node_id
2799- return files
2800+ defer.returnValue(files)
2801
2802 @property
2803 def uniqueness(self):
2804@@ -2209,7 +2195,7 @@
2805
2806 def _run(self):
2807 """See ActionQueueCommand."""
2808- return threads.deferToThread(self._get_public_files_http)
2809+ return self._get_public_files_http()
2810
2811 def handle_success(self, success):
2812 """See ActionQueueCommand."""
2813@@ -2218,12 +2204,8 @@
2814
2815 def handle_failure(self, failure):
2816 """It didn't work! Push the event."""
2817- if issubclass(failure.type, HTTPError):
2818- message = failure.value.read()
2819- else:
2820- message = failure.getErrorMessage()
2821 self.action_queue.event_queue.push('AQ_PUBLIC_FILES_LIST_ERROR',
2822- error=message)
2823+ error=failure.value[1])
2824
2825 def split_nodekey(self, nodekey):
2826 """Split a node key into a share_id, node_id."""
2827
2828=== added file 'ubuntuone/syncdaemon/tunnel_runner.py'
2829--- ubuntuone/syncdaemon/tunnel_runner.py 1970-01-01 00:00:00 +0000
2830+++ ubuntuone/syncdaemon/tunnel_runner.py 2012-03-20 23:05:30 +0000
2831@@ -0,0 +1,78 @@
2832+# -*- coding: utf8 -*-
2833+#
2834+# Copyright 2012 Canonical Ltd.
2835+#
2836+# This program is free software: you can redistribute it and/or modify it
2837+# under the terms of the GNU General Public License version 3, as published
2838+# by the Free Software Foundation.
2839+#
2840+# This program is distributed in the hope that it will be useful, but
2841+# WITHOUT ANY WARRANTY; without even the implied warranties of
2842+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
2843+# PURPOSE. See the GNU General Public License for more details.
2844+#
2845+# You should have received a copy of the GNU General Public License along
2846+# with this program. If not, see <http://www.gnu.org/licenses/>.
2847+"""Run the tunnel process and start a client, with a reactor as a fallback."""
2848+
2849+import logging
2850+import sys
2851+
2852+from os import path
2853+
2854+from twisted.internet import defer, reactor
2855+
2856+from ubuntuone.platform.constants import TUNNEL_EXECUTABLE
2857+
2858+logger = logging.getLogger("ubuntuone.SyncDaemon.TunnelRunner")
2859+
2860+
2861+class TunnelRunner(object):
2862+ """Run a tunnel process."""
2863+
2864+ def __init__(self, host, port):
2865+ """Start this runner instance."""
2866+ self.client_d = defer.Deferred()
2867+ try:
2868+ self.start_process(host, port)
2869+ except ImportError:
2870+ logger.info("Proxy support not installed.")
2871+ self.client_d.callback(reactor)
2872+
2873+ def start_process(self, host, port):
2874+ """Start the tunnel process."""
2875+ from ubuntuone.proxy.tunnel_client import TunnelProcessProtocol
2876+ protocol = TunnelProcessProtocol(self.client_d)
2877+ process_path = self.get_process_path()
2878+ args = [TUNNEL_EXECUTABLE, host, str(port)]
2879+ reactor.spawnProcess(protocol, process_path, env=None, args=args)
2880+
2881+ def get_process_path(self):
2882+ """Get the path to the tunnel process."""
2883+ # In a frozen setting, all binaries are in the same path
2884+ if getattr(sys, "frozen", None) is not None:
2885+ return path.join(path.dirname(
2886+ path.abspath(sys.executable)), TUNNEL_EXECUTABLE)
2887+
2888+ # This works when we are running from sources
2889+ filename = path.join(path.dirname(__file__), "..", "..", "bin",
2890+ TUNNEL_EXECUTABLE)
2891+ if path.isfile(filename):
2892+ return filename
2893+
2894+ # Use the configured LIBEXECDIR
2895+ from ubuntuone.clientdefs import LIBEXECDIR
2896+ return path.join(LIBEXECDIR, TUNNEL_EXECUTABLE)
2897+
2898+ def get_client(self):
2899+ """A deferred with the reactor or a tunnel client."""
2900+
2901+ def client_selected(result, d):
2902+ """The tunnel_client or the reactor were selected."""
2903+ d.callback(result)
2904+ # make sure the result is available for next callback
2905+ return result
2906+
2907+ d = defer.Deferred()
2908+ self.client_d.addCallback(client_selected, d)
2909+ return d

Subscribers

People subscribed via source and target branches