Merge lp:~nataliabidart/magicicada-server/clean-deps-for-stable into lp:magicicada-server

Proposed by Natalia Bidart on 2017-04-22
Status: Merged
Approved by: Natalia Bidart on 2017-04-30
Approved revision: 83
Merged at revision: 82
Proposed branch: lp:~nataliabidart/magicicada-server/clean-deps-for-stable
Merge into: lp:magicicada-server
Diff against target: 864 lines (+123/-189)
24 files modified
Makefile (+10/-12)
README.rst (+1/-1)
config-manager.txt (+0/-4)
dependencies-devel.txt (+1/-0)
dependencies.txt (+0/-2)
dev-scripts/cmd_client.py (+1/-1)
dev-scripts/deploy_api_server.py (+1/-1)
dev-scripts/dev_launcher.py (+1/-1)
dev-scripts/supervisor-dev.conf.tpl (+2/-14)
magicicada/rpcdb/tests/test_backend.py (+1/-1)
magicicada/server/content.py (+3/-3)
magicicada/server/integration/run_integtests.py (+1/-6)
magicicada/server/server.py (+14/-15)
magicicada/server/ssl_proxy.py (+9/-9)
magicicada/server/testing/aq_helpers.py (+2/-2)
magicicada/server/testing/testcase.py (+4/-4)
magicicada/server/tests/test_auth.py (+4/-4)
magicicada/server/tests/test_content.py (+13/-13)
magicicada/server/tests/test_generations.py (+3/-3)
magicicada/server/tests/test_server.py (+5/-5)
magicicada/server/tests/test_ssl_proxy.py (+4/-4)
magicicada/settings/__init__.py (+43/-2)
magicicada/settings/api_server.py (+0/-58)
magicicada/settings/ssl_proxy.py (+0/-24)
To merge this branch: bzr merge lp:~nataliabidart/magicicada-server/clean-deps-for-stable
Reviewer Review Type Date Requested Status
Facundo Batista 2017-04-22 Approve on 2017-04-27
Review via email: mp+322999@code.launchpad.net

Commit message

- Unified settings into a single file.
- Removed uneeded dependencies.
- Removed every outdated reference to "oauth".

To post a comment you must log in.
83. By Natalia Bidart on 2017-04-22

Missing replacement.

Facundo Batista (facundo) wrote :

Awesome, thanks!

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Makefile'
2--- Makefile 2016-10-01 12:48:16 +0000
3+++ Makefile 2017-04-22 22:01:02 +0000
4@@ -128,14 +128,12 @@
5 version:
6 bzr version-info --format=python > lib/versioninfo.py || true
7
8-start: build start-base start-filesync-dummy-group publish-api-port
9-
10-start-oauth: build start-base start-filesync-oauth-group publish-api-port
11-
12-resume: start-base start-filesync-oauth-group
13-
14-start-oauth-heapy:
15- USE_HEAPY=1 $(MAKE) start-oauth
16+start: build start-base start-filesync-server-group publish-api-port
17+
18+resume: start-base start-filesync-server-group
19+
20+start-heapy:
21+ USE_HEAPY=1 $(MAKE) start
22
23 start-base:
24 $(MAKE) start-supervisor && $(MAKE) start-dbus || ( $(MAKE) stop ; exit 1 )
25@@ -168,9 +166,9 @@
26 -@dev-scripts/supervisorctl-dev stop $*
27
28 publish-api-port:
29- python -c 'from magicicada import settings; print >> file("tmp/filesyncserver.port", "w"), settings.api_server.TCP_PORT'
30- python -c 'from magicicada import settings; print >> file("tmp/filesyncserver.port.ssl", "w"), settings.ssl_proxy.PORT'
31- python -c 'from magicicada import settings; print >> file("tmp/filesyncserver-status.port", "w"), settings.api_server.STATUS_PORT'
32+ python -c 'from magicicada import settings; print >> file("tmp/filesyncserver.port", "w"), settings.TCP_PORT'
33+ python -c 'from magicicada import settings; print >> file("tmp/filesyncserver.port.ssl", "w"), settings.SSL_PORT'
34+ python -c 'from magicicada import settings; print >> file("tmp/filesyncserver-status.port", "w"), settings.API_STATUS_PORT'
35
36 shell:
37 $(DJANGO_MANAGE) shell
38@@ -184,4 +182,4 @@
39 .PHONY: sourcedeps link-sourcedeps build-sourcedeps build-deploy-sourcedeps \
40 build clean version lint test ci-test build-for-deployment \
41 clean-sourcedeps tarball start stop publish-api-port start-supervisor \
42- stop-supervisor start-dbus stop-dbus start-oauth start-oauth-heapy
43+ stop-supervisor start-dbus stop-dbus start-heapy
44
45=== modified file 'README.rst'
46--- README.rst 2017-02-25 15:29:01 +0000
47+++ README.rst 2017-04-22 22:01:02 +0000
48@@ -139,7 +139,7 @@
49 Start the server via command line this way::
50
51 cd ~/magicicada/magicicada-server
52- make start-oauth
53+ make start
54
55 Note that the server will listen on port 21101, so you need to assure that the
56 client could reach it (open the whole it in your firewall config, etc).
57
58=== modified file 'config-manager.txt'
59--- config-manager.txt 2016-12-27 21:57:00 +0000
60+++ config-manager.txt 2017-04-22 22:01:02 +0000
61@@ -24,10 +24,6 @@
62 ./.sourcecode/configglue ~configglue/configglue/trunk;revno=1
63 ./.sourcecode/dirspec ~ubuntuone-control-tower/dirspec/trunk;revno=14
64 ./.sourcecode/django ~ubuntuone-pqm-team/django/stable;revno=17
65-./.sourcecode/mock ~ubuntuone-pqm-team/python-mock/stable;revno=1
66-./.sourcecode/oauthlib ~ubuntuone-pqm-team/oauthlib/stable;revno=209
67-./.sourcecode/requests ~ubuntuone-pqm-team/requests/stable;revno=816
68-./.sourcecode/requests_oauthlib ~ubuntuone-pqm-team/requests-oauthlib/stable;revno=16
69 ./.sourcecode/u1sync ~facundo/u1sync/opensourcing;revno=10
70 ./.sourcecode/magicicada-client ~chicharreros/magicicada-client/trunk;revno=1433
71 ./.sourcecode/magicicada-protocol ~chicharreros/magicicada-protocol/trunk;revno=169
72
73=== modified file 'dependencies-devel.txt'
74--- dependencies-devel.txt 2016-09-15 13:46:24 +0000
75+++ dependencies-devel.txt 2017-04-22 22:01:02 +0000
76@@ -4,6 +4,7 @@
77 python-dbus
78 python-gobject
79 python-httplib2
80+python-mock
81 python-mocker
82 python-pyinotify
83 python-testtools
84
85=== modified file 'dependencies.txt'
86--- dependencies.txt 2016-02-20 17:42:06 +0000
87+++ dependencies.txt 2017-04-22 22:01:02 +0000
88@@ -15,8 +15,6 @@
89 python-psutil
90 python-psycopg2
91 python-setuptools
92-python-testresources
93-python-transaction
94 python-twisted-web
95 python-tz
96 python-virtualenv
97
98=== modified file 'dev-scripts/cmd_client.py'
99--- dev-scripts/cmd_client.py 2015-09-17 19:30:41 +0000
100+++ dev-scripts/cmd_client.py 2017-04-22 22:01:02 +0000
101@@ -719,7 +719,7 @@
102 @parse_args(str, str)
103 @require_connection
104 @show_exception
105- def do_oauth(self):
106+ def do_auth(self):
107 """Perform authorisation."""
108 self.defer_from_thread(self._auth)
109
110
111=== modified file 'dev-scripts/deploy_api_server.py'
112--- dev-scripts/deploy_api_server.py 2016-05-29 21:20:25 +0000
113+++ dev-scripts/deploy_api_server.py 2017-04-22 22:01:02 +0000
114@@ -55,7 +55,7 @@
115 """Start the server."""
116
117 status_port = int(
118- os.getenv('API_STATUS_PORT', settings.api_server.STATUS_PORT))
119+ os.getenv('API_STATUS_PORT', settings.STATUS_PORT))
120 service = server.create_service(status_port)
121
122 yield service.startService()
123
124=== modified file 'dev-scripts/dev_launcher.py'
125--- dev-scripts/dev_launcher.py 2015-09-17 19:30:41 +0000
126+++ dev-scripts/dev_launcher.py 2017-04-22 22:01:02 +0000
127@@ -37,7 +37,7 @@
128 "runs PROGNAME with args plus:\n"
129 " --port with the local server port\n"
130 " --host localhost\n"
131- " --oauth keys for user USERNAME\n")
132+ " --auth username and password for user USERNAME\n")
133
134
135 def main(args):
136
137=== modified file 'dev-scripts/supervisor-dev.conf.tpl'
138--- dev-scripts/supervisor-dev.conf.tpl 2016-06-03 00:35:53 +0000
139+++ dev-scripts/supervisor-dev.conf.tpl 2017-04-22 22:01:02 +0000
140@@ -24,7 +24,7 @@
141 files=services-supervisor.conf workers-supervisor.conf
142
143 [eventlistener:heartbeat]
144-command=python %(basepath)s/lib/ubuntuone/supervisor/heartbeat_listener.py --interval=10 --timeout=20 --log_level=DEBUG --log_file=%(tmp_dir)s/heartbeat.log --groups=filesync-dummy,filesync-oauth
145+command=python %(basepath)s/lib/ubuntuone/supervisor/heartbeat_listener.py --interval=10 --timeout=20 --log_level=DEBUG --log_file=%(tmp_dir)s/heartbeat.log --groups=filesync-server
146 environment=PYTHONPATH="%(basepath)s:%(basepath)s/lib"
147 events=PROCESS_COMMUNICATION,TICK_5
148 buffer_size=42
149@@ -36,15 +36,6 @@
150 autostart=false
151 stopsignal=INT
152
153-[program:filesync-dummy-auth]
154-command=python %(basepath)s/dev-scripts/deploy_api_server.py
155-environment=PYTHONPATH="%(basepath)s:%(basepath)s/lib",DJANGO_SETTINGS_MODULE="magicicada.settings",PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp
156-stdout_capture_maxbytes=16384
157-autostart=false
158-redirect_stderr=true ; send stderr to the log file
159-stdout_logfile=%(tmp_dir)s/filesync.log
160-stdout_logfile_maxbytes=0
161-
162 [program:ssl-proxy]
163 command=/usr/bin/twistd --pidfile %(tmp_dir)s/ssl-proxy.pid -n -y %(basepath)s/magicicada/server/ssl_proxy.tac --reactor=epoll
164 environment=PYTHONPATH="%(basepath)s:%(basepath)s/lib",DJANGO_SETTINGS_MODULE="magicicada.settings"
165@@ -52,8 +43,5 @@
166 autostart=false
167 stopsignal=INT
168
169-[group:filesync-dummy]
170-programs=filesync-dummy-auth
171-
172-[group:filesync-oauth]
173+[group:filesync-server]
174 programs=filesync,ssl-proxy
175
176=== modified file 'magicicada/rpcdb/tests/test_backend.py'
177--- magicicada/rpcdb/tests/test_backend.py 2016-11-07 01:37:46 +0000
178+++ magicicada/rpcdb/tests/test_backend.py 2017-04-22 22:01:02 +0000
179@@ -57,7 +57,7 @@
180 self.assertEqual(result, dict(user_id=self.usr.id))
181
182 def test_get_user_id_bad_auth(self):
183- """Bad parameters in the oauth request."""
184+ """Bad parameters in the auth request."""
185 bad_parameters = self.auth_parameters.copy()
186 bad_parameters['password'] = "bad"
187 try:
188
189=== modified file 'magicicada/server/content.py'
190--- magicicada/server/content.py 2016-11-07 01:37:46 +0000
191+++ magicicada/server/content.py 2017-04-22 22:01:02 +0000
192@@ -190,8 +190,8 @@
193 def add_part(self, chunk_size):
194 """Add a part to an upload job."""
195 self.unsaved_count += chunk_size
196- if self.unsaved_count >= settings.api_server.STORAGE_CHUNK_SIZE:
197- self.unsaved_count -= settings.api_server.STORAGE_CHUNK_SIZE
198+ if self.unsaved_count >= settings.STORAGE_CHUNK_SIZE:
199+ self.unsaved_count -= settings.STORAGE_CHUNK_SIZE
200 kwargs = dict(user_id=self.user.id, volume_id=self.volume_id,
201 uploadjob_id=self.uploadjob_id,
202 chunk_size=chunk_size)
203@@ -870,7 +870,7 @@
204 if upload is None:
205 # no uploadjob found, create a new one: if the file is small just
206 # get a bo
207- if deflated_size <= settings.api_server.STORAGE_CHUNK_SIZE:
208+ if deflated_size <= settings.STORAGE_CHUNK_SIZE:
209 upload = BogusUploadJob()
210 else:
211 try:
212
213=== modified file 'magicicada/server/integration/run_integtests.py'
214--- magicicada/server/integration/run_integtests.py 2016-08-30 00:50:31 +0000
215+++ magicicada/server/integration/run_integtests.py 2017-04-22 22:01:02 +0000
216@@ -48,11 +48,6 @@
217 dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
218
219 LIB_DIR = os.path.abspath("lib")
220-
221-# this should be done manually before:
222-DEP_STARTOAUTH = """
223-make start-oauth
224-"""
225 CLIENT_DIR = os.path.abspath(os.path.join('.sourcecode', 'magicicada-client'))
226 PROTOCOL_DIR = os.path.abspath(
227 os.path.join('.sourcecode', 'magicicada-protocol'))
228@@ -101,7 +96,7 @@
229 """
230 couchdb_port = os.path.join(TMP_DIR, "couchdb-master0.port")
231 if not os.path.exists(couchdb_port):
232- print "Not ready! Hint: did you do...?:\n" + DEP_STARTOAUTH
233+ print "Not ready! Hint: did you run...?:\nmake start"
234 return True
235
236 log_conf = os.path.join(CLIENT_DIR, "data", "logging.conf")
237
238=== modified file 'magicicada/server/server.py'
239--- magicicada/server/server.py 2016-11-07 01:37:46 +0000
240+++ magicicada/server/server.py 2017-04-22 22:01:02 +0000
241@@ -285,8 +285,7 @@
242 self.pending_requests = collections.deque()
243 self.ping_loop = LoopingPing(StorageServer.PING_INTERVAL,
244 StorageServer.PING_TIMEOUT,
245- settings.api_server.IDLE_TIMEOUT,
246- self)
247+ settings.IDLE_TIMEOUT, self)
248
249 # capabilities that the server is working with
250 self.working_caps = MIN_CAP
251@@ -624,7 +623,7 @@
252
253 def __init__(self, protocol, message):
254 """Create the request response."""
255- self.use_protocol_weakref = settings.api_server.PROTOCOL_WEAKREF
256+ self.use_protocol_weakref = settings.PROTOCOL_WEAKREF
257 self._protocol_ref = None
258 super(BaseRequestResponse, self).__init__(protocol, message)
259
260@@ -1841,7 +1840,7 @@
261 def _get_upload_job(self):
262 """Get the uploadjob."""
263 share_id = self.convert_share_id(self.source_message.put_content.share)
264- if settings.api_server.MAGIC_UPLOAD_ACTIVE:
265+ if settings.MAGIC_UPLOAD_ACTIVE:
266 magic_hash = self.source_message.put_content.magic_hash or None
267 else:
268 magic_hash = None
269@@ -1907,7 +1906,7 @@
270 msg = self.source_message
271 share_id = self.convert_share_id(msg.get_delta.share)
272 from_generation = msg.get_delta.from_generation
273- delta_max_size = settings.api_server.DELTA_MAX_SIZE
274+ delta_max_size = settings.DELTA_MAX_SIZE
275 delta_info = yield self.protocol.user.get_delta(
276 share_id, from_generation, limit=delta_max_size)
277 nodes, vol_generation, free_bytes = delta_info
278@@ -1942,7 +1941,7 @@
279 """Build and send the DELTA_INFO for each node."""
280 count = 0
281 for node in nodes:
282- if count == settings.api_server.MAX_DELTA_INFO:
283+ if count == settings.MAX_DELTA_INFO:
284 count = 0
285 yield
286 message = protocol_pb2.Message()
287@@ -1983,7 +1982,7 @@
288 def _process(self):
289 """Get all the live nodes and send DeltaInfos."""
290 msg = self.source_message
291- limit = settings.api_server.GET_FROM_SCRATCH_LIMIT
292+ limit = settings.GET_FROM_SCRATCH_LIMIT
293 share_id = self.convert_share_id(msg.get_delta.share)
294 # get the first chunk
295 delta_info = yield self.protocol.user.get_from_scratch(
296@@ -2333,7 +2332,7 @@
297 @cvar protocol: The class of the server.
298 """
299 protocol = StorageServer
300- graceful_shutdown = settings.api_server.GRACEFUL_SHUTDOWN
301+ graceful_shutdown = settings.GRACEFUL_SHUTDOWN
302
303 def __init__(self, auth_provider_class=auth.DummyAuthProvider,
304 content_class=content.ContentManager,
305@@ -2341,7 +2340,7 @@
306 """Create a StorageServerFactory."""
307 self.auth_provider = auth_provider_class(self)
308 self.content = content_class(self)
309- self.diskstorage = DiskStorage(settings.api_server.STORAGE_BASEDIR)
310+ self.diskstorage = DiskStorage(settings.STORAGE_BASEDIR)
311
312 self.metrics = metrics.get_meter('root')
313 self.user_metrics = metrics.get_meter('user')
314@@ -2377,7 +2376,7 @@
315
316 self.protocols = []
317 self.reactor = reactor
318- self.trace_users = set(settings.api_server.TRACE_USERS)
319+ self.trace_users = set(settings.TRACE_USERS)
320
321 twisted.python.log.addObserver(self._deferror_handler)
322
323@@ -2631,10 +2630,10 @@
324 OrderedMultiService.__init__(self)
325 self.heartbeat_writer = None
326 if heartbeat_interval is None:
327- heartbeat_interval = float(settings.api_server.HEARTBEAT_INTERVAL)
328+ heartbeat_interval = float(settings.HEARTBEAT_INTERVAL)
329 self.heartbeat_interval = heartbeat_interval
330 self.rpc_dal = None
331- self.servername = settings.api_server.SERVERNAME
332+ self.servername = settings.API_SERVER_NAME
333 logger.info('Starting %s', self.servername)
334 logger.info(
335 'protocol buffers implementation: %s',
336@@ -2726,15 +2725,15 @@
337 logger.debug('activated heapy remote monitor')
338
339 # set GC's debug
340- if settings.api_server.GC_DEBUG:
341+ if settings.GC_DEBUG:
342 import gc
343 gc.set_debug(gc.DEBUG_UNCOLLECTABLE)
344 logger.debug('set gc debug on')
345
346 if status_port is None:
347- status_port = settings.api_server.STATUS_PORT
348+ status_port = settings.API_STATUS_PORT
349
350 # create the service
351 service = StorageServerService(
352- settings.api_server.TCP_PORT, auth_provider_class, status_port)
353+ settings.TCP_PORT, auth_provider_class, status_port)
354 return service
355
356=== modified file 'magicicada/server/ssl_proxy.py'
357--- magicicada/server/ssl_proxy.py 2016-08-08 01:14:41 +0000
358+++ magicicada/server/ssl_proxy.py 2017-04-22 22:01:02 +0000
359@@ -245,7 +245,7 @@
360 self.status_service = create_status_service(self.factory, status_port)
361 self.status_service.setServiceParent(self)
362 # disable ssl compression
363- if settings.ssl_proxy.DISABLE_SSL_COMPRESSION:
364+ if settings.DISABLE_SSL_COMPRESSION:
365 disable_ssl_compression(logger)
366
367 @property
368@@ -265,7 +265,7 @@
369 # setup stats in the factory
370 yield MultiService.startService(self)
371 # only start the HeartbeatWriter if the interval is > 0
372- heartbeat_interval = float(settings.ssl_proxy.HEARTBEAT_INTERVAL)
373+ heartbeat_interval = float(settings.HEARTBEAT_INTERVAL)
374 if heartbeat_interval > 0:
375 self.heartbeat_writer = stdio.StandardIO(
376 supervisor_utils.HeartbeatWriter(heartbeat_interval, logger))
377@@ -283,9 +283,9 @@
378
379 def create_service():
380 """Create the service instance."""
381- server_key = settings.api_server.KEY
382- server_crt = settings.api_server.CRT
383- server_crt_chain = settings.api_server.CRT_CHAIN
384+ server_key = settings.CRT_KEY
385+ server_crt = settings.CRT
386+ server_crt_chain = settings.CRT_CHAIN
387
388 ssl_cert = crypto.load_certificate(crypto.FILETYPE_PEM, server_crt)
389 if server_crt_chain:
390@@ -296,11 +296,11 @@
391 ssl_key = crypto.load_privatekey(crypto.FILETYPE_PEM, server_key)
392
393 ssl_proxy = ProxyService(ssl_cert, ssl_key, ssl_cert_chain,
394- settings.ssl_proxy.PORT,
395+ settings.SSL_PORT,
396 '127.0.0.1',
397- settings.api_server.TCP_PORT,
398- settings.ssl_proxy.SERVER_NAME,
399- settings.ssl_proxy.STATUS_PORT)
400+ settings.TCP_PORT,
401+ settings.SSL_SERVER_NAME,
402+ settings.SSL_STATUS_PORT)
403 return ssl_proxy
404
405
406
407=== modified file 'magicicada/server/testing/aq_helpers.py'
408--- magicicada/server/testing/aq_helpers.py 2016-12-27 21:57:00 +0000
409+++ magicicada/server/testing/aq_helpers.py 2017-04-22 22:01:02 +0000
410@@ -332,8 +332,8 @@
411 0, # port
412 "localhost", self.port,
413 "ssl-proxy-test", 0)
414- self.patch(settings.ssl_proxy, 'HEARTBEAT_INTERVAL',
415- self.ssl_proxy_heartbeat_interval)
416+ self.patch(
417+ settings, 'HEARTBEAT_INTERVAL', self.ssl_proxy_heartbeat_interval)
418 yield self.ssl_service.startService()
419
420 if os.path.exists(self.tmpdir):
421
422=== modified file 'magicicada/server/testing/testcase.py'
423--- magicicada/server/testing/testcase.py 2016-12-27 21:57:00 +0000
424+++ magicicada/server/testing/testcase.py 2017-04-22 22:01:02 +0000
425@@ -43,9 +43,9 @@
426 from magicicada.testing.testcase import BaseTestCase
427
428 logger = logging.getLogger(__name__)
429-server_key = settings.api_server.KEY
430-server_crt = settings.api_server.CRT
431-server_crt_chain = settings.api_server.CRT_CHAIN
432+server_key = settings.CRT_KEY
433+server_crt = settings.CRT
434+server_crt_chain = settings.CRT_CHAIN
435
436
437 class FakeTimestampChecker(object):
438@@ -365,7 +365,7 @@
439 dummy_tokens[password] = user.id
440
441 # tune the config for this tests
442- self.patch(settings.api_server, 'STORAGE_CHUNK_SIZE', 1024 * 64)
443+ self.patch(settings, 'STORAGE_CHUNK_SIZE', 1024 * 64)
444
445 def save_req(self, req, name):
446 """Save a request for later use."""
447
448=== modified file 'magicicada/server/tests/test_auth.py'
449--- magicicada/server/tests/test_auth.py 2016-06-08 21:04:35 +0000
450+++ magicicada/server/tests/test_auth.py 2017-04-22 22:01:02 +0000
451@@ -96,7 +96,7 @@
452
453 @defer.inlineCallbacks
454 def test_authenticate(self):
455- """The OAuth provider succeeds with a valid PLAINTEXT signature."""
456+ """The Auth provider succeeds with a valid password."""
457 user = yield self.provider.authenticate(self.creds, None)
458 self.assertEqual(user.id, self.usr0.id)
459
460@@ -106,7 +106,7 @@
461
462 @defer.inlineCallbacks
463 def test_authenticate_failure(self):
464- """The OAuth provider succeeds with an invalid PLAINTEXT signature."""
465+ """The Auth provider succeeds with an invalid password."""
466 auth_parameters = self.creds.copy()
467 auth_parameters['password'] = 'invalid'
468
469@@ -115,7 +115,7 @@
470
471 @defer.inlineCallbacks
472 def test_authenticate_no_parameters(self):
473- """The OAuth provider fails with no parameters."""
474+ """The Auth provider fails with no parameters."""
475 user = yield self.provider.authenticate({}, None)
476 self.assertEqual(user, None)
477
478@@ -248,7 +248,7 @@
479
480
481 class ClientSimpleAuthTests(ClientDummyAuthTests):
482- """Client authentication tests using the OAuth provider."""
483+ """Client authentication tests using the Auth provider."""
484
485 auth_provider_class = SimpleAuthProvider
486
487
488=== modified file 'magicicada/server/tests/test_content.py'
489--- magicicada/server/tests/test_content.py 2016-11-07 01:37:46 +0000
490+++ magicicada/server/tests/test_content.py 2017-04-22 22:01:02 +0000
491@@ -565,7 +565,7 @@
492
493 def test_putcontent_cancel_middle(self):
494 """Test putting content to a file and cancelling it in the middle."""
495- size = int(settings.api_server.STORAGE_CHUNK_SIZE * 1.5)
496+ size = int(settings.STORAGE_CHUNK_SIZE * 1.5)
497 data = os.urandom(size)
498 deflated_data = zlib.compress(data)
499 hash_object = content_hash_factory()
500@@ -1146,7 +1146,7 @@
501 Test that a PutContent fails and is terminated as soon we get an
502 error, instead of wait until the full upload is done.
503 """
504- chunk_size = settings.api_server.STORAGE_CHUNK_SIZE
505+ chunk_size = settings.STORAGE_CHUNK_SIZE
506 user, content_user = self._get_users(chunk_size ** 2)
507 # create the file
508 a_file = user.root.make_file(u"A new file")
509@@ -1596,7 +1596,7 @@
510 self.handler = self.add_memento_handler(server.logger, level=0)
511 yield super(TestMultipartPutContent, self).setUp()
512 # override defaults set by TestWithDatabase.setUp.
513- self.patch(settings.api_server, 'STORAGE_CHUNK_SIZE', 1024)
514+ self.patch(settings, 'STORAGE_CHUNK_SIZE', 1024)
515
516 def get_data(self, size):
517 """Return random data of the specified size."""
518@@ -1650,7 +1650,7 @@
519 @defer.inlineCallbacks
520 def test_resume_putcontent(self):
521 """Test that the client can resume a putcontent request."""
522- self.patch(settings.api_server, 'STORAGE_CHUNK_SIZE', 1024 * 64)
523+ self.patch(settings, 'STORAGE_CHUNK_SIZE', 1024 * 64)
524 size = 2 * 1024 * 512
525 StorageUser.objects.filter(id=self.usr0.id).update(
526 max_storage_bytes=size * 2)
527@@ -1776,7 +1776,7 @@
528
529 It receives a new upload_id.
530 """
531- self.patch(settings.api_server, 'STORAGE_CHUNK_SIZE', 1024 * 32)
532+ self.patch(settings, 'STORAGE_CHUNK_SIZE', 1024 * 32)
533 size = 2 * 1024 * 128
534 StorageUser.objects.filter(id=self.usr0.id).update(
535 max_storage_bytes=size * 2)
536@@ -1848,7 +1848,7 @@
537
538 def test_putcontent_corrupt(self):
539 """Put content on a file with corrupt data."""
540- self.patch(settings.api_server, 'STORAGE_CHUNK_SIZE', 1024 * 64)
541+ self.patch(settings, 'STORAGE_CHUNK_SIZE', 1024 * 64)
542 size = 2 * 1024 * 512
543 StorageUser.objects.filter(id=self.usr0.id).update(
544 max_storage_bytes=size * 2)
545@@ -1965,7 +1965,7 @@
546 Test that a PutContent fails and is terminated as soon we get an
547 error, instead of wait until the full upload is done.
548 """
549- chunk_size = settings.api_server.STORAGE_CHUNK_SIZE
550+ chunk_size = settings.STORAGE_CHUNK_SIZE
551 user = self.make_user(max_storage_bytes=chunk_size ** 2)
552 content_user = User(
553 self.service.factory.content, user.id, user.root_volume_id,
554@@ -2079,11 +2079,11 @@
555 """Setup the test."""
556 yield super(TestChunkedContent, self).setUp()
557 # tune the config for this tests
558- self.patch(settings.api_server, 'STORAGE_CHUNK_SIZE', 1024 * 1024)
559+ self.patch(settings, 'STORAGE_CHUNK_SIZE', 1024 * 1024)
560
561 def test_putcontent_chunked(self, put_fail=False, get_fail=False):
562 """Checks a chunked putcontent."""
563- size = int(settings.api_server.STORAGE_CHUNK_SIZE * 1.5)
564+ size = int(settings.STORAGE_CHUNK_SIZE * 1.5)
565 data = os.urandom(size)
566 deflated_data = zlib.compress(data)
567 hash_object = content_hash_factory()
568@@ -2162,7 +2162,7 @@
569
570 def test_deferred_add_part_to_uj(self):
571 """Check that parts are added to upload job only after a limit."""
572- size = int(settings.api_server.STORAGE_CHUNK_SIZE * 2.5)
573+ size = int(settings.STORAGE_CHUNK_SIZE * 2.5)
574 data = os.urandom(size)
575 deflated_data = zlib.compress(data)
576 hash_object = content_hash_factory()
577@@ -2344,7 +2344,7 @@
578 def setUp(self):
579 """Setup the test."""
580 yield super(TestUploadJob, self).setUp()
581- self.chunk_size = settings.api_server.STORAGE_CHUNK_SIZE
582+ self.chunk_size = settings.STORAGE_CHUNK_SIZE
583 self.half_size = self.chunk_size / 2
584 self.double_size = self.chunk_size * 2
585 self.user = self.make_user(max_storage_bytes=self.chunk_size ** 2)
586@@ -2673,7 +2673,7 @@
587 def setUp(self):
588 """Setup the test."""
589 yield super(TestNode, self).setUp()
590- self.chunk_size = settings.api_server.STORAGE_CHUNK_SIZE
591+ self.chunk_size = settings.STORAGE_CHUNK_SIZE
592 self.half_size = self.chunk_size / 2
593 self.double_size = self.chunk_size * 2
594 self.user = self.make_user(max_storage_bytes=self.chunk_size ** 2)
595@@ -3073,7 +3073,7 @@
596 def test_add_part(self):
597 """Test add_part method."""
598 dbuj = yield self._make_uj()
599- chunk_size = int(settings.api_server.STORAGE_CHUNK_SIZE) + 1
600+ chunk_size = int(settings.STORAGE_CHUNK_SIZE) + 1
601 yield dbuj.add_part(chunk_size)
602
603 # check it called rpc dal correctly
604
605=== modified file 'magicicada/server/tests/test_generations.py'
606--- magicicada/server/tests/test_generations.py 2016-05-29 20:58:45 +0000
607+++ magicicada/server/tests/test_generations.py 2017-04-22 22:01:02 +0000
608@@ -139,7 +139,7 @@
609 yield client.dummy_authenticate("open sesame")
610 # create some nodes to get a delta
611 limit = 10
612- self.patch(server.settings.api_server, 'DELTA_MAX_SIZE', limit)
613+ self.patch(server.settings, 'DELTA_MAX_SIZE', limit)
614 for i in range(20):
615 self.usr0.root.make_file(u"name%s" % i)
616 req = yield client.get_delta(request.ROOT, 5)
617@@ -182,7 +182,7 @@
618 def auth(client):
619 yield client.dummy_authenticate("open sesame")
620 # create some nodes to get a delta
621- self.patch(server.settings.api_server, 'MAX_DELTA_INFO', 5)
622+ self.patch(server.settings, 'MAX_DELTA_INFO', 5)
623 for i in range(20):
624 self.usr0.root.make_file(u"name%s" % i)
625 req = yield client.get_delta(request.ROOT, 5)
626@@ -248,7 +248,7 @@
627 def auth(client):
628 yield client.dummy_authenticate("open sesame")
629 # create some nodes to get a delta
630- self.patch(server.settings.api_server, 'MAX_DELTA_INFO', 5)
631+ self.patch(server.settings, 'MAX_DELTA_INFO', 5)
632 for i in range(20):
633 self.usr0.root.make_file(u"name%s" % i)
634 req = yield client.get_delta(request.ROOT, 5)
635
636=== modified file 'magicicada/server/tests/test_server.py'
637--- magicicada/server/tests/test_server.py 2016-11-07 01:37:46 +0000
638+++ magicicada/server/tests/test_server.py 2017-04-22 22:01:02 +0000
639@@ -349,7 +349,7 @@
640
641 def test_protocol_ref_enabled(self):
642 """Test that protocol weakref is disabled in tests."""
643- self.patch(settings.api_server, 'PROTOCOL_WEAKREF', True)
644+ self.patch(settings, 'PROTOCOL_WEAKREF', True)
645 _server = StorageServer()
646 response = StorageServerRequestResponse(
647 protocol=_server, message=self.make_protocol_message())
648@@ -358,7 +358,7 @@
649
650 def test_protocol_ref_disabled(self):
651 """Test that protocol weakref is disabled in tests."""
652- self.patch(settings.api_server, 'PROTOCOL_WEAKREF', False)
653+ self.patch(settings, 'PROTOCOL_WEAKREF', False)
654 _server = StorageServer()
655 response = StorageServerRequestResponse(
656 protocol=_server, message=self.make_protocol_message())
657@@ -2340,7 +2340,7 @@
658
659 def test_reset_send_delta_info_counter(self):
660 """Test that the count is reset on each iteration."""
661- self.patch(settings.api_server, 'MAX_DELTA_INFO', 5)
662+ self.patch(settings, 'MAX_DELTA_INFO', 5)
663 # create a few fake nodes
664 nodes = []
665 right_now = now()
666@@ -2415,7 +2415,7 @@
667 @defer.inlineCallbacks
668 def test_chunked_get_from_scratch(self):
669 """Get the nodes list in chunks."""
670- self.patch(settings.api_server, 'GET_FROM_SCRATCH_LIMIT', 5)
671+ self.patch(settings, 'GET_FROM_SCRATCH_LIMIT', 5)
672 # build fake nodes
673 nodes = []
674 right_now = now()
675@@ -2612,7 +2612,7 @@
676 def test_trace_users(self):
677 """Check trace users are correctly set."""
678 # set a specific config to test
679- self.patch(settings.api_server, 'TRACE_USERS', ['foo', 'bar', 'baz'])
680+ self.patch(settings, 'TRACE_USERS', ['foo', 'bar', 'baz'])
681 factory = StorageServerFactory()
682 self.assertEqual(factory.trace_users, set(['foo', 'bar', 'baz']))
683
684
685=== modified file 'magicicada/server/tests/test_ssl_proxy.py'
686--- magicicada/server/tests/test_ssl_proxy.py 2016-08-14 20:40:09 +0000
687+++ magicicada/server/tests/test_ssl_proxy.py 2017-04-22 22:01:02 +0000
688@@ -54,8 +54,8 @@
689 ssl_proxy.logger, level=logging.DEBUG)
690 self.metrics = MetricReceiver()
691 self.patch(metrics, 'get_meter', lambda n: self.metrics)
692- self.patch(settings.ssl_proxy, 'HEARTBEAT_INTERVAL',
693- self.ssl_proxy_heartbeat_interval)
694+ self.patch(
695+ settings, 'HEARTBEAT_INTERVAL', self.ssl_proxy_heartbeat_interval)
696
697 @defer.inlineCallbacks
698 def test_start_stop(self):
699@@ -87,8 +87,8 @@
700 # keep metrics in our MetricReceiver
701 self.metrics = MetricReceiver()
702 self.patch(metrics, 'get_meter', lambda n: self.metrics)
703- self.patch(settings.ssl_proxy, 'HEARTBEAT_INTERVAL',
704- self.ssl_proxy_heartbeat_interval)
705+ self.patch(
706+ settings, 'HEARTBEAT_INTERVAL', self.ssl_proxy_heartbeat_interval)
707 yield self.ssl_service.startService()
708 self.addCleanup(self.ssl_service.stopService)
709
710
711=== modified file 'magicicada/settings/__init__.py'
712--- magicicada/settings/__init__.py 2016-12-27 21:57:00 +0000
713+++ magicicada/settings/__init__.py 2017-04-22 22:01:02 +0000
714@@ -35,6 +35,18 @@
715
716 from psycopg2.extensions import ISOLATION_LEVEL_REPEATABLE_READ
717
718+
719+def get_file_content(folder, filename):
720+ filepath = os.path.join(folder, filename)
721+ if not os.path.exists(filepath):
722+ filepath = os.path.join(folder, 'dev-' + filename)
723+
724+ with open(filepath) as f:
725+ content = f.read()
726+
727+ return content
728+
729+
730 BASE_DIR = os.path.dirname(
731 os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
732
733@@ -216,8 +228,37 @@
734 SYSLOG_FORMAT = (
735 '%(processName)-13s %(levelname)-8s %(name)s[%(process)d]: %(message)s')
736
737-from . import api_server # noqa
738-from . import ssl_proxy # noqa
739+
740+# Server settings
741+
742+API_SERVER_NAME = 'filesync-server'
743+API_STATUS_PORT = 21102
744+CERTS_FOLDER = os.path.join(BASE_DIR, 'certs')
745+# the `crt` key with the content of `cacert.pem` file
746+CRT = get_file_content(CERTS_FOLDER, 'cacert.pem')
747+CRT_CHAIN = None
748+# the `key` key with the content of `privkey.pem` file
749+CRT_KEY = get_file_content(CERTS_FOLDER, 'privkey.pem')
750+DELTA_MAX_SIZE = 1000
751+DISABLE_SSL_COMPRESSION = True
752+GC_DEBUG = False
753+GET_FROM_SCRATCH_LIMIT = 2000
754+GRACEFUL_SHUTDOWN = True
755+HEARTBEAT_INTERVAL = 5
756+IDLE_TIMEOUT = 7200
757+MAGIC_UPLOAD_ACTIVE = True
758+MAX_DELTA_INFO = 20
759+PROTOCOL_WEAKREF = False
760+SSL_LOG_FILENAME = 'ssl-proxy.log'
761+SSL_PORT = 21101
762+SSL_SERVER_NAME = 'ssl-proxy'
763+SSL_STATUS_PORT = 21103
764+STATS_LOG_INTERVAL = 0
765+STORAGE_CHUNK_SIZE = 5242880
766+TCP_PORT = 21100
767+TRACE_USERS = ['test', 'etc']
768+UPLOAD_BUFFER_MAX_SIZE = 10485761
769+STORAGE_BASEDIR = os.path.join(BASE_DIR, 'tmp', 'filestorage')
770
771
772 try:
773
774=== removed file 'magicicada/settings/api_server.py'
775--- magicicada/settings/api_server.py 2016-05-29 23:43:14 +0000
776+++ magicicada/settings/api_server.py 1970-01-01 00:00:00 +0000
777@@ -1,58 +0,0 @@
778-# Copyright 2008-2015 Canonical
779-# Copyright 2015 Chicharreros (https://launchpad.net/~chicharreros)
780-#
781-# This program is free software: you can redistribute it and/or modify
782-# it under the terms of the GNU Affero General Public License as
783-# published by the Free Software Foundation, either version 3 of the
784-# License, or (at your option) any later version.
785-#
786-# This program is distributed in the hope that it will be useful,
787-# but WITHOUT ANY WARRANTY; without even the implied warranty of
788-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
789-# GNU Affero General Public License for more details.
790-#
791-# You should have received a copy of the GNU Affero General Public License
792-# along with this program. If not, see <http://www.gnu.org/licenses/>.
793-#
794-# For further info, check http://launchpad.net/filesync-server
795-
796-import os
797-from django.conf import settings
798-
799-
800-def get_file_content(folder, filename):
801- filepath = os.path.join(folder, filename)
802- if not os.path.exists(filepath):
803- filepath = os.path.join(folder, 'dev-' + filename)
804-
805- with open(filepath) as f:
806- content = f.read()
807-
808- return content
809-
810-
811-CERTS_FOLDER = os.path.join(settings.BASE_DIR, 'certs')
812-# the `crt` key with the content of `cacert.pem` file
813-CRT = get_file_content(CERTS_FOLDER, 'cacert.pem')
814-CRT_CHAIN = None
815-# the `key` key with the content of `privkey.pem` file
816-KEY = get_file_content(CERTS_FOLDER, 'privkey.pem')
817-DELTA_MAX_SIZE = 1000
818-GC_DEBUG = True
819-GET_FROM_SCRATCH_LIMIT = 2000
820-GRACEFUL_SHUTDOWN = True
821-HEARTBEAT_INTERVAL = 5
822-IDLE_TIMEOUT = 7200
823-MAGIC_UPLOAD_ACTIVE = True
824-MAX_DELTA_INFO = 20
825-METRICS_NAMESPACE = 'development.filesync.server'
826-PROTOCOL_WEAKREF = False
827-SERVERNAME = 'filesyncserver-development'
828-SLI_METRIC_NAMESPACE = None
829-STATS_LOG_INTERVAL = 0
830-STATUS_PORT = 21102
831-STORAGE_CHUNK_SIZE = 5242880
832-TCP_PORT = 21100
833-TRACE_USERS = ['test', 'etc']
834-UPLOAD_BUFFER_MAX_SIZE = 10485761
835-STORAGE_BASEDIR = os.path.join(settings.BASE_DIR, 'tmp', 'filestorage')
836
837=== removed file 'magicicada/settings/ssl_proxy.py'
838--- magicicada/settings/ssl_proxy.py 2016-05-31 22:59:18 +0000
839+++ magicicada/settings/ssl_proxy.py 1970-01-01 00:00:00 +0000
840@@ -1,24 +0,0 @@
841-# Copyright 2008-2015 Canonical
842-# Copyright 2015-2016 Chicharreros (https://launchpad.net/~chicharreros)
843-#
844-# This program is free software: you can redistribute it and/or modify
845-# it under the terms of the GNU Affero General Public License as
846-# published by the Free Software Foundation, either version 3 of the
847-# License, or (at your option) any later version.
848-#
849-# This program is distributed in the hope that it will be useful,
850-# but WITHOUT ANY WARRANTY; without even the implied warranty of
851-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
852-# GNU Affero General Public License for more details.
853-#
854-# You should have received a copy of the GNU Affero General Public License
855-# along with this program. If not, see <http://www.gnu.org/licenses/>.
856-#
857-# For further info, check http://launchpad.net/filesync-server
858-
859-DISABLE_SSL_COMPRESSION = True
860-HEARTBEAT_INTERVAL = 5
861-LOG_FILENAME = 'ssl-proxy.log'
862-PORT = 21101
863-SERVER_NAME = 'ssl-proxy-dev'
864-STATUS_PORT = 21103

Subscribers

People subscribed via source and target branches

to all changes: