Merge lp:~diegosarmentero/ubuntuone-client/darwin4-fsevents into lp:ubuntuone-client

Proposed by Diego Sarmentero on 2012-06-25
Status: Superseded
Proposed branch: lp:~diegosarmentero/ubuntuone-client/darwin4-fsevents
Merge into: lp:ubuntuone-client
Prerequisite: lp:~diegosarmentero/ubuntuone-client/darwin3-fsevents
Diff against target: 2739 lines (+1083/-537)
8 files modified
tests/platform/filesystem_notifications/test_darwin.py (+686/-80)
tests/platform/filesystem_notifications/test_filesystem_notifications.py (+5/-0)
tests/platform/filesystem_notifications/test_windows.py (+72/-145)
ubuntuone/platform/filesystem_notifications/__init__.py (+4/-8)
ubuntuone/platform/filesystem_notifications/common.py (+141/-105)
ubuntuone/platform/filesystem_notifications/darwin.py (+74/-82)
ubuntuone/platform/filesystem_notifications/windows.py (+96/-117)
ubuntuone/platform/os_helper/darwin.py (+5/-0)
To merge this branch: bzr merge lp:~diegosarmentero/ubuntuone-client/darwin4-fsevents
Reviewer Review Type Date Requested Status
Alejandro J. Cura (community) 2012-06-25 Needs Information on 2012-07-05
Review via email: mp+111914@code.launchpad.net

This proposal has been superseded by a proposal from 2012-07-11.

Commit Message

- Filesystem Notifications complete for darwin (LP: #1013323).

Description of the Change

Now you can execute:
./run-mac-tests tests/platform/filesystem_notifications/

To post a comment you must log in.
1292. By Diego Sarmentero on 2012-07-03

removing unnecesary line

1293. By Diego Sarmentero on 2012-07-03

reverting

Alejandro J. Cura (alecu) wrote :

The branch looks mostly fine.

A few questions, though:

 * Why is test_file_moved_from_partial skipped? I think this is a critical operation that should be tested.

 * Why is there a test that still uses time.sleep? As mentioned in previous branches we should not be using sleep at all in tests.

And some fixes needed, too:

It looks wrong to import fsevents in /platform/filesystem_notifications/__init__.py, so please, move each platform specific definition of EVENT_CODES to ubuntuone.platform.filesystem_notifications.windows and .darwin

---

Lines 827 and 828 are duplicated.

---

Please move the logic in these few lines to a function, add some unit tests for it, and use it from both places:

837 + index = 0
838 + if not self._path.endswith(os.path.sep):
839 + index = 1
840 + path = path[len(self._path) + index:]

review: Needs Information
1294. By Diego Sarmentero on 2012-07-10

merge

1295. By Diego Sarmentero on 2012-07-10

fixing darwin tests

Manuel de la Peña (mandel) wrote :

The diff is misleading because it should depend on lp:~mandel/ubuntuone-client/clean-fsevents. Diego can you do a resummit with the correct dependency so that we are not scared regading the size :)

1296. By Diego Sarmentero on 2012-07-11

removing time.sleep

1297. By Diego Sarmentero on 2012-07-12

adding join on stop

1298. By Diego Sarmentero on 2012-07-12

moving patch to setUp

1299. By Diego Sarmentero on 2012-07-12

reverting

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'tests/platform/filesystem_notifications/test_darwin.py'
2--- tests/platform/filesystem_notifications/test_darwin.py 2012-06-28 13:20:01 +0000
3+++ tests/platform/filesystem_notifications/test_darwin.py 2012-07-10 21:28:20 +0000
4@@ -33,7 +33,9 @@
5 import tempfile
6 import thread
7 import time
8+import itertools
9
10+import fsevents
11 from twisted.internet import defer
12
13 from contrib.testing.testcase import BaseTwistedTestCase
14@@ -41,20 +43,25 @@
15 from ubuntuone.platform.filesystem_notifications import (
16 darwin as filesystem_notifications,
17 )
18-from ubuntuone.platform.filesystem_notifications.darwin import (
19+from ubuntuone.platform.filesystem_notifications.common import (
20+ NotifyProcessor,
21 Watch,
22 WatchManager,
23 )
24 from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
25 EventsCodes,
26 ProcessEvent,
27+ IN_CLOSE_WRITE,
28 IN_CREATE,
29 IN_DELETE,
30+ IN_OPEN,
31 )
32+from tests.platform.filesystem_notifications import BaseFSMonitorTestCase
33+
34
35 # A reverse mapping for the tests
36 REVERSE_MACOS_ACTIONS = {}
37-for key, value in filesystem_notifications.common.COMMON_ACTIONS.iteritems():
38+for key, value in filesystem_notifications.ACTIONS.iteritems():
39 REVERSE_MACOS_ACTIONS[value] = key
40
41
42@@ -136,6 +143,7 @@
43 yield super(TestWatch, self).setUp()
44 self.basedir = self.mktemp('test_root')
45 self.mask = None
46+ self.stream = None
47 self.memento = MementoHandler()
48 self.memento.setLevel(logging.DEBUG)
49 self.raw_events = []
50@@ -148,16 +156,15 @@
51 self.paths_checked.append((path, result))
52 return result
53
54- self.patch(filesystem_notifications.Watch, '_path_is_dir',
55+ self.patch(Watch, '_path_is_dir',
56 path_is_dir_wrapper)
57
58 @defer.inlineCallbacks
59- def _perform_operations(self, path, mask, auto_add, actions,
60- number_events):
61+ def _perform_operations(self, path, mask, actions, number_events):
62 """Perform the file operations and returns the recorded events."""
63 handler = TestCaseHandler(number_events=number_events)
64 manager = WatchManager(handler)
65- yield manager.add_watch(path, mask, auto_add=auto_add)
66+ yield manager.add_watch(path, mask)
67 # change the logger so that we can check the logs if we wanted
68 manager._wdm[0].log.addHandler(self.memento)
69 # clean logger later
70@@ -169,22 +176,6 @@
71 self.addCleanup(manager.stop)
72 defer.returnValue(ret)
73
74- def _perform_timed_operations(self, path, mask, auto_add, actions,
75- time_out):
76- """Perform the file operations and returns the recorded events."""
77- manager = WatchManager()
78- manager.add_watch(path, mask, auto_add=auto_add)
79- # change the logger so that we can check the logs if we wanted
80- manager._wdm[0].log.addHandler(self.memento)
81- # clean logger later
82- self.addCleanup(manager._wdm[0].log.removeHandler, self.memento)
83- # execution the actions
84- actions()
85- # process the recorded events
86- time.sleep(time_out)
87- events = self.handler.processed_events
88- return events
89-
90 def _assert_logs(self, events):
91 """Assert the debug logs."""
92 logs = []
93@@ -208,7 +199,7 @@
94 os.fsync(fd)
95 fd.close()
96
97- events = yield self._perform_operations(self.basedir, self.mask, False,
98+ events = yield self._perform_operations(self.basedir, self.mask,
99 create_file, 1)
100 event = events[0]
101 self.assertFalse(event.dir)
102@@ -230,7 +221,7 @@
103 """Action for the test."""
104 os.mkdir(dir_name)
105
106- events = yield self._perform_operations(self.basedir, self.mask, False,
107+ events = yield self._perform_operations(self.basedir, self.mask,
108 create_dir, 1)
109 event = events[0]
110 self.assertTrue(event.dir)
111@@ -254,7 +245,7 @@
112 """Action for the test."""
113 os.remove(file_name)
114
115- events = yield self._perform_operations(self.basedir, self.mask, False,
116+ events = yield self._perform_operations(self.basedir, self.mask,
117 remove_file, 1)
118 event = events[0]
119 self.assertFalse(event.dir)
120@@ -278,7 +269,7 @@
121 """Action for the test."""
122 os.rmdir(dir_name)
123
124- events = yield self._perform_operations(self.basedir, self.mask, False,
125+ events = yield self._perform_operations(self.basedir, self.mask,
126 remove_dir, 1)
127 event = events[0]
128 self.assertTrue(event.dir)
129@@ -304,7 +295,7 @@
130 fd.write('test')
131 fd.close()
132
133- events = yield self._perform_operations(self.basedir, self.mask, False,
134+ events = yield self._perform_operations(self.basedir, self.mask,
135 write_file, 1)
136 event = events[0]
137 self.assertFalse(event.dir)
138@@ -332,7 +323,7 @@
139 os.rename(from_file_name, to_file_name)
140
141 events = yield self._perform_operations(self.basedir, self.mask,
142- False, move_file, 2)
143+ move_file, 2)
144 move_from_event = events[0]
145 move_to_event = events[1]
146 # first test the move from
147@@ -375,7 +366,7 @@
148
149 # We need to test that we get a delete operation when moving
150 # a file to an unwatched folder
151- events = yield self._perform_operations(self.basedir, self.mask, False,
152+ events = yield self._perform_operations(self.basedir, self.mask,
153 move_file, 1)
154 event = events[0]
155 self.assertFalse(event.dir)
156@@ -405,7 +396,7 @@
157
158 # We need to test that we get a delete operation when moving
159 # a file from an unwatched folder
160- events = yield self._perform_operations(self.basedir, self.mask, False,
161+ events = yield self._perform_operations(self.basedir, self.mask,
162 move_files, 1)
163 event = events[0]
164 self.assertFalse(event.dir)
165@@ -433,7 +424,7 @@
166 os.rename(from_dir_name, to_dir_name)
167
168 events = yield self._perform_operations(self.basedir,
169- self.mask, False, move_file, 2)
170+ self.mask, move_file, 2)
171 move_from_event = events[0]
172 move_to_event = events[1]
173 # first test the move from
174@@ -476,7 +467,7 @@
175
176 # We need to test that we get a delete operation when moving
177 # a file to an unwatched folder
178- events = yield self._perform_operations(self.basedir, self.mask, False,
179+ events = yield self._perform_operations(self.basedir, self.mask,
180 move_dir, 1)
181 event = events[0]
182 self.assertTrue(event.dir)
183@@ -502,7 +493,7 @@
184 """Action for the test."""
185 os.rename(from_dir_name, to_dir_name)
186
187- events = yield self._perform_operations(self.basedir, self.mask, False,
188+ events = yield self._perform_operations(self.basedir, self.mask,
189 move_dir, 1)
190 event = events[0]
191 self.assertTrue(event.dir)
192@@ -519,8 +510,8 @@
193 handler = TestCaseHandler(number_events=0)
194 manager = WatchManager(handler)
195 # add a watch that will always exclude all actions
196- manager.add_watch(self.basedir, self.mask, auto_add=True,
197- exclude_filter=lambda x: True)
198+ manager.add_watch(self.basedir, self.mask,
199+ exclude_filter=lambda x: True)
200 # execution the actions
201 file_name = os.path.join(self.basedir, 'test_file_create')
202 open(file_name, 'w').close()
203@@ -528,21 +519,6 @@
204 self.assertEqual(0, len(handler.processed_events))
205 test_exclude_filter.skip = "we must rethink this test."
206
207- def test_open_dir_muted(self):
208- """Test that the opening of dirs is ignored."""
209- dir_name = os.path.join(tempfile.mkdtemp(), 'test_dir_open')
210- # create file before we record
211- os.mkdir(dir_name)
212-
213- def open_dir():
214- """Action for the test."""
215- os.startfile(dir_name)
216-
217- events = self._perform_timed_operations(self.basedir, self.mask, False,
218- open_dir, 2)
219- self.assertEqual(0, len(events))
220- test_open_dir_muted.skip = "we must rethink this test."
221-
222 def test_ignore_path(self):
223 """Test that events from a path are ignored."""
224 events = []
225@@ -553,13 +529,13 @@
226
227 path = '/Users/username/folder'
228 child = 'child'
229- watch = Watch(1, path, None, True, fake_processor)
230+ watch = Watch(1, path, fake_processor)
231 watch.ignore_path(os.path.join(path, child))
232 # ensure that the watch is watching
233- watch._watching = True
234+ watch.platform_watch.watching = True
235 for file_name in 'abcdef':
236 event = FakeFileEvent(256, None, os.path.join(child, file_name))
237- watch._process_events(event)
238+ watch.platform_watch._process_events(event)
239 self.assertEqual(0, len(events),
240 'All events should have been ignored.')
241
242@@ -576,7 +552,7 @@
243
244 path = '/Users/username/folder'
245 child = 'child'
246- watch = Watch(1, path, None, True, fake_processor)
247+ watch = Watch(1, path, fake_processor)
248 watch.ignore_path(os.path.join(path, child))
249 paths_not_to_ignore = []
250 for file_name in 'abcdef':
251@@ -584,9 +560,9 @@
252 file_name))
253 paths_not_to_ignore.append(event)
254 # ensure that the watch is watching
255- watch._watching = True
256+ watch.platform_watch.watching = True
257 for event in paths_not_to_ignore:
258- watch._process_events(event)
259+ watch.platform_watch._process_events(event)
260 self.assertEqual(len(paths_not_to_ignore), len(events),
261 'No events should have been ignored.')
262
263@@ -603,7 +579,7 @@
264
265 child = 'child'
266 path = '/Users/username/folder'
267- watch = Watch(1, path, None, True, fake_processor)
268+ watch = Watch(1, path, fake_processor)
269 watch.ignore_path(os.path.join(path, child))
270 paths_not_to_ignore = []
271 paths_to_ignore = []
272@@ -615,9 +591,9 @@
273 paths_not_to_ignore.append(event)
274 expected_events.append(os.path.join(path, valid))
275 # ensure that the watch is watching
276- watch._watching = True
277+ watch.platform_watch.watching = True
278 for event in paths_not_to_ignore:
279- watch._process_events(event)
280+ watch.platform_watch._process_events(event)
281 self.assertEqual(len(paths_not_to_ignore), len(events),
282 'Wrong number of events ignored.')
283 self.assertTrue(all([event in expected_events for event in events]),
284@@ -636,7 +612,7 @@
285
286 path = '/Users/username/folder'
287 child = 'child'
288- watch = Watch(1, path, None, True, fake_processor)
289+ watch = Watch(1, path, fake_processor)
290 watch.ignore_path(os.path.join(path, child))
291 watch.remove_ignored_path(os.path.join(path, child))
292 paths_not_to_ignore = []
293@@ -644,9 +620,9 @@
294 event = FakeFileEvent(256, None, os.path.join(child, file_name))
295 paths_not_to_ignore.append(event)
296 # ensure that the watch is watching
297- watch._watching = True
298+ watch.platform_watch.watching = True
299 for event in paths_not_to_ignore:
300- watch._process_events(event)
301+ watch.platform_watch._process_events(event)
302 self.assertEqual(len(paths_not_to_ignore), len(events),
303 'All events should have been accepted.')
304
305@@ -664,7 +640,7 @@
306 path = '/Users/username/folder'
307 child_a = 'childa'
308 child_b = 'childb'
309- watch = Watch(1, path, None, True, fake_processor)
310+ watch = Watch(1, path, fake_processor)
311 watch.ignore_path(os.path.join(path, child_a))
312 watch.ignore_path(os.path.join(path, child_b))
313 watch.remove_ignored_path(os.path.join(path, child_a))
314@@ -678,9 +654,9 @@
315 paths_not_to_ignore.append(event)
316 expected_events.append(os.path.join(path, valid))
317 # ensure that the watch is watching
318- watch._watching = True
319+ watch.platform_watch.watching = True
320 for event in paths_not_to_ignore:
321- watch._process_events(event)
322+ watch.platform_watch._process_events(event)
323 self.assertEqual(len(paths_not_to_ignore), len(events),
324 'All events should have been accepted.')
325 self.assertTrue(all([event in expected_events for event in events]),
326@@ -691,17 +667,18 @@
327 def fake_call(*args, **kwargs):
328 """Fake call."""
329
330- path = '/Users/username/folder'
331- watch = Watch(1, path, None, True, None)
332- self.assertEqual(watch._process_events, watch.stream.callback)
333- self.assertEqual(watch.stream.paths, [path])
334- self.assertEqual(watch.stream.file_events, True)
335+ path = '/Users/username/folder/'
336+ watch = Watch(1, path, None)
337+ self.assertEqual(watch.platform_watch._process_events,
338+ watch.platform_watch.stream.callback)
339+ self.assertEqual(watch.platform_watch.stream.paths, [path])
340+ self.assertEqual(watch.platform_watch.stream.file_events, True)
341
342 def test_watching_property(self):
343 """Test that the stopped property returns the stopped deferred."""
344 path = '/Users/username/folder'
345- watch = Watch(1, path, None, True, None)
346- self.assertFalse(watch._watching)
347+ watch = Watch(1, path, None)
348+ self.assertFalse(watch.watching)
349
350 def random_error(self, *args):
351 """Throw a fake exception."""
352@@ -712,7 +689,7 @@
353 path = '/Users/username/path/to/not/dir'
354 test_path = self.mktemp("test_directory")
355 self.patch(os.path, 'exists', lambda path: False)
356- watch = Watch(1, test_path, self.mask, True, None)
357+ watch = Watch(1, test_path, None)
358 self.assertFalse(watch._path_is_dir(path))
359
360 def test_is_path_dir_missing_in_subdir(self):
361@@ -720,7 +697,7 @@
362 path = '/Users/username/path/to/not/dir'
363 test_path = self.mktemp("test_directory")
364 self.patch(os.path, 'exists', lambda path: False)
365- watch = Watch(1, test_path, self.mask, True, None)
366+ watch = Watch(1, test_path, None)
367 watch._subdirs.add(path)
368 self.assertTrue(watch._path_is_dir(path))
369
370@@ -730,7 +707,7 @@
371 test_path = self.mktemp("test_directory")
372 self.patch(os.path, 'exists', lambda path: True)
373 self.patch(os.path, 'isdir', lambda path: True)
374- watch = Watch(1, test_path, self.mask, True, None)
375+ watch = Watch(1, test_path, None)
376 watch._subdirs.add(path)
377 self.assertTrue(watch._path_is_dir(path))
378
379@@ -740,7 +717,7 @@
380 test_path = self.mktemp("test_directory")
381 self.patch(os.path, 'exists', lambda path: True)
382 self.patch(os.path, 'isdir', lambda path: False)
383- watch = Watch(1, test_path, self.mask, True, None)
384+ watch = Watch(1, test_path, None)
385 watch._subdirs.add(path)
386 self.assertFalse(watch._path_is_dir(path))
387
388@@ -748,7 +725,7 @@
389 """Test when we update on a create event and not present."""
390 path = '/Users/username/path/to/not/dir'
391 test_path = self.mktemp("test_directory")
392- watch = Watch(1, test_path, self.mask, True, None)
393+ watch = Watch(1, test_path, None)
394 watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_CREATE])
395 self.assertTrue(path in watch._subdirs)
396
397@@ -756,7 +733,7 @@
398 """Test when we update on a create event and is present."""
399 path = '/Users/username/path/to/not/dir'
400 test_path = self.mktemp("test_directory")
401- watch = Watch(1, test_path, self.mask, True, None)
402+ watch = Watch(1, test_path, None)
403 watch._subdirs.add(path)
404 old_length = len(watch._subdirs)
405 watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_CREATE])
406@@ -767,7 +744,7 @@
407 """Test when we delete and is not present."""
408 path = '/Users/username/path/to/not/dir'
409 test_path = self.mktemp("test_directory")
410- watch = Watch(1, test_path, self.mask, True, None)
411+ watch = Watch(1, test_path, None)
412 watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_DELETE])
413 self.assertTrue(path not in watch._subdirs)
414
415@@ -775,7 +752,636 @@
416 """Test when we delete and is present."""
417 path = '/Users/username/path/to/not/dir'
418 test_path = self.mktemp("test_directory")
419- watch = Watch(1, test_path, self.mask, True, None)
420+ watch = Watch(1, test_path, None)
421 watch._subdirs.add(path)
422 watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_DELETE])
423 self.assertTrue(path not in watch._subdirs)
424+
425+
426+class TestWatchManager(BaseTwistedTestCase):
427+ """Test the watch manager."""
428+
429+ @defer.inlineCallbacks
430+ def setUp(self):
431+ """Set each of the tests."""
432+ yield super(TestWatchManager, self).setUp()
433+ self.parent_path = '/Users/username/'
434+ self.path = self.parent_path + 'path'
435+ self.watch = Watch(1, self.path, None)
436+ self.manager = WatchManager(None)
437+ self.manager._wdm = {1: self.watch}
438+
439+ @defer.inlineCallbacks
440+ def test_stop(self):
441+ """Test that the different watches are stopped."""
442+ self.was_called = False
443+
444+ def fake_stop_watching(watch):
445+ """Fake stop watch."""
446+ self.was_called = True
447+ return defer.succeed(True)
448+
449+ self.patch(Watch, "stop_watching", fake_stop_watching)
450+ self.patch(self.manager.manager.observer, "unschedule", lambda x: None)
451+ yield self.manager.stop()
452+ self.assertTrue(self.was_called, 'The watch stop should be called.')
453+
454+ def test_stop_multiple(self):
455+ """The watches should became watching=False and the observer stopped."""
456+ self.patch(self.manager.manager.observer, "unschedule", lambda x: None)
457+ second_path = self.parent_path + "second_path"
458+ second_watch = Watch(2, second_path, None)
459+ second_watch._watching = True
460+ self.manager._wdm[2] = second_watch
461+ self.manager.stop()
462+ self.assertFalse(second_watch.platform_watch.watching)
463+ self.assertEqual(second_watch._subdirs, set())
464+ # Give time to the thread to be finished.
465+ time.sleep(0.5)
466+ self.assertFalse(self.manager.manager.observer.is_alive())
467+
468+ def test_get_present_watch(self):
469+ """Test that we can get a Watch using is wd."""
470+ self.assertEqual(self.watch, self.manager.get_watch(1))
471+
472+ def test_get_missing_watch(self):
473+ """Test that we get an error when trying to get a missing wd."""
474+ self.assertRaises(KeyError, self.manager.get_watch, (1,))
475+
476+ @defer.inlineCallbacks
477+ def test_delete_present_watch(self):
478+ """Test that we can remove a present watch."""
479+ self.was_called = False
480+
481+ def stop_watching():
482+ """Fake stop watch."""
483+ self.was_called = True
484+ return defer.succeed(True)
485+
486+ def fake_unschedule(s):
487+ """Fake function that should receive a Stream object."""
488+ self.stream = s
489+
490+ self.patch(self.manager.manager.observer, "unschedule",
491+ fake_unschedule)
492+
493+ self.watch.stop_watching = stop_watching
494+ yield self.manager.del_watch(1)
495+ self.assertIsInstance(self.stream, fsevents.Stream)
496+ self.assertRaises(KeyError, self.manager.get_watch, (1,))
497+
498+ def test_add_single_watch(self):
499+ """Test the addition of a new single watch."""
500+ self.was_called = False
501+
502+ def fake_start_watching(*args):
503+ """Fake start watch."""
504+ self.was_called = True
505+
506+ self.patch(Watch, "start_watching", fake_start_watching)
507+ self.manager._wdm = {}
508+
509+ mask = 'bit_mask'
510+ self.manager.add_watch(self.path, mask)
511+ self.assertEqual(1, len(self.manager._wdm))
512+ self.assertTrue(self.was_called, 'The watch start was not called.')
513+ self.assertEqual(self.path + os.path.sep, self.manager._wdm[0].path)
514+
515+ def test_get_watch_present_wd(self):
516+ """Test that the correct path is returned."""
517+ self.assertEqual(self.path + os.path.sep, self.manager.get_path(1))
518+
519+ def test_get_watch_missing_wd(self):
520+ """Test that the correct path is returned."""
521+ self.manager._wdm = {}
522+ self.assertEqual(None, self.manager.get_path(1))
523+
524+ def test_get_wd_exact_path(self):
525+ """Test the wd is returned when there is a watch for the path."""
526+ self.assertEqual(1, self.manager.get_wd(self.path))
527+
528+ def test_get_wd_child_path(self):
529+ """Test the wd is returned when we have a child path."""
530+ child = os.path.join(self.path, 'child')
531+ self.assertEqual(1, self.manager.get_wd(child))
532+
533+ def test_get_wd_unwatched(self):
534+ """A watch on an unwatched path returns None."""
535+ self.assertEqual(None, self.manager.get_wd(self.parent_path))
536+
537+ def test_rm_present_wd(self):
538+ """Test the removal of a present watch."""
539+ self.patch(self.watch, "stop_watching", lambda: None)
540+ self.patch(self.manager.manager.observer, "unschedule", lambda x: None)
541+ self.manager.rm_watch(1)
542+ self.assertEqual(None, self.manager._wdm.get(1))
543+
544+ def test_rm_root_path(self):
545+ """Test the removal of a root path."""
546+ events = []
547+
548+ def fake_processor(event):
549+ """Memorize the processed events."""
550+ events.append(event.pathname)
551+
552+ self.watch._processor = fake_processor
553+ self.manager.rm_path(self.path)
554+ self.assertEqual(self.watch, self.manager._wdm.get(1))
555+ self.watch._watching = True
556+ event = FakeFileEvent(256, None, os.path.join(self.path, 'test'))
557+ self.watch.platform_watch._process_events(event)
558+ self.assertEqual(0, len(events))
559+
560+ def test_rm_child_path(self):
561+ """Test the removal of a child path."""
562+ events = []
563+
564+ def fake_processor(event):
565+ """Memorize the processed events."""
566+ events.append(event.pathname)
567+
568+ self.patch(filesystem_notifications.reactor, 'callFromThread',
569+ lambda x, e: x(e))
570+
571+ self.watch._processor = fake_processor
572+ child = os.path.join(self.path, 'child')
573+ self.manager.rm_path(child)
574+ self.assertEqual(self.watch, self.manager._wdm[1])
575+ # assert that the correct event is ignored
576+ self.watch.platform_watch.watching = True
577+ event = FakeFileEvent(256, None, os.path.join('child', 'test'))
578+ self.watch.platform_watch._process_events(event)
579+ self.assertEqual(0, len(events))
580+ # assert that other events are not ignored
581+ event2 = FakeFileEvent(256, None, 'test')
582+ self.watch.platform_watch._process_events(event2)
583+ self.assertEqual(1, len(events))
584+
585+
586+class TestWatchManagerAddWatches(BaseTwistedTestCase):
587+ """Test the watch manager."""
588+ timeout = 5
589+
590+ def test_add_watch_twice(self):
591+ """Adding a watch twice succeeds when the watch is running."""
592+ self.patch(Watch, "start_watching", lambda self: None)
593+ manager = WatchManager(None)
594+ # no need to stop watching because start_watching is fake
595+
596+ path = '/Users/username/path'
597+ mask = 'fake bit mask'
598+ d1 = manager.add_watch(path, mask)
599+ d2 = manager.add_watch(path, mask)
600+
601+ self.assertTrue(d1.result, "Should not be called yet.")
602+ self.assertFalse(d2.result, "Should not be called yet.")
603+
604+
605+class FakeEvent(object):
606+ """Fake event."""
607+
608+ def __init__(self, wd=0, dir=True, name=None, path=None, pathname=None,
609+ cookie=None):
610+ """Create fake event."""
611+ self.dir = dir
612+ self.wd = wd
613+ self.name = name
614+ self.path = path
615+ self.pathname = pathname
616+ self.cookie = cookie
617+
618+
619+class FakeLog(object):
620+ """A fake log that is used by the general processor."""
621+
622+ def __init__(self):
623+ """Create the fake."""
624+ self.called_methods = []
625+
626+ def info(self, *args):
627+ """Fake the info call."""
628+ self.called_methods.append(('info', args))
629+
630+
631+class FakeGeneralProcessor(object):
632+ """Fake implementation of the general processor."""
633+
634+ def __init__(self):
635+ """Create the fake."""
636+ self.called_methods = []
637+ self.paths_to_return = []
638+ self.log = FakeLog()
639+ self.share_id = None
640+ self.ignore = False
641+
642+ def rm_from_mute_filter(self, event, paths):
643+ """Fake rm_from_mute_filter."""
644+ self.called_methods.append(('rm_from_mute_filter', event, paths))
645+
646+ def add_to_mute_filter(self, event, paths):
647+ """Fake add_to_move_filter."""
648+ self.called_methods.append(('add_to_mute_filter', event, paths))
649+
650+ def is_ignored(self, path):
651+ """Fake is_ignored."""
652+ self.called_methods.append(('is_ignored', path))
653+ return self.ignore
654+
655+ def push_event(self, event):
656+ """Fake push event."""
657+ self.called_methods.append(('push_event', event))
658+
659+ def eq_push(self, event, path=None, path_to=None, path_from=None):
660+ """Fake event to push event."""
661+ self.called_methods.append(('eq_push', event, path, path_to,
662+ path_from))
663+
664+ def get_paths_starting_with(self, fullpath, include_base=False):
665+ """Fake get_paths_starting_with."""
666+ self.called_methods.append(('get_paths_starting_with', fullpath,
667+ include_base))
668+ return self.paths_to_return
669+
670+ def get_path_share_id(self, path):
671+ """Fake get_path_share_id."""
672+ self.called_methods.append(('get_path_share_id', path))
673+ return self.share_id
674+
675+ def rm_watch(self, path):
676+ """Fake the remove watch."""
677+ self.called_methods.append(('rm_watch', path))
678+
679+ def freeze_begin(self, path):
680+ """Fake freeze_begin"""
681+ self.called_methods.append(('freeze_begin', path))
682+
683+ def freeze_rollback(self):
684+ """Fake rollback."""
685+ self.called_methods.append(('freeze_rollback',))
686+
687+ def freeze_commit(self, path):
688+ """Fake freeze commit."""
689+ self.called_methods.append(('freeze_commit', path))
690+
691+
692+class TestNotifyProcessor(BaseTwistedTestCase):
693+ """Test the notify processor."""
694+
695+ @defer.inlineCallbacks
696+ def setUp(self):
697+ """set up the diffeent tests."""
698+ yield super(TestNotifyProcessor, self).setUp()
699+ self.processor = NotifyProcessor(None)
700+ self.general = FakeGeneralProcessor()
701+ self.processor.general_processor = self.general
702+
703+ def test_rm_from_mute_filter(self):
704+ """Test that we remove the event from the mute filter."""
705+ event = 'event'
706+ paths = 'paths'
707+ self.processor.rm_from_mute_filter(event, paths)
708+ self.assertEqual(1, len(self.general.called_methods))
709+ self.assertEqual('rm_from_mute_filter',
710+ self.general.called_methods[0][0])
711+ self.assertEqual(event, self.general.called_methods[0][1])
712+ self.assertEqual(paths, self.general.called_methods[0][2])
713+
714+ def test_add_to_mute_filter(self):
715+ """Test that we add the path to the mute filter."""
716+ event = 'event'
717+ paths = 'paths'
718+ self.processor.add_to_mute_filter(event, paths)
719+ self.assertEqual(1, len(self.general.called_methods))
720+ self.assertEqual('add_to_mute_filter',
721+ self.general.called_methods[0][0])
722+ self.assertEqual(event, self.general.called_methods[0][1])
723+ self.assertEqual(paths, self.general.called_methods[0][2])
724+
725+ def test_is_ignored(self):
726+ """Test that we do ensure that the path is ignored."""
727+ path = 'path'
728+ self.processor.is_ignored(path)
729+ self.assertEqual(1, len(self.general.called_methods))
730+ self.assertEqual('is_ignored',
731+ self.general.called_methods[0][0])
732+ self.assertEqual(path, self.general.called_methods[0][1])
733+
734+ def test_release_held_event(self):
735+ """Test that we do release the held event."""
736+ event = 'event'
737+ # set the held event to assert that is pushed
738+ self.processor.held_event = event
739+ self.processor.release_held_event()
740+ self.assertEqual('push_event',
741+ self.general.called_methods[0][0])
742+ self.assertEqual(event, self.general.called_methods[0][1])
743+
744+ def test_process_IN_MODIFY_dir(self):
745+ """Test that the modify works as exepcted with dirs."""
746+ event = FakeEvent(dir=True)
747+ self.processor.process_IN_MODIFY(event)
748+ # no method should be called
749+ self.assertEqual(0, len(self.general.called_methods))
750+
751+ def test_process_IN_MODIFY_file(self):
752+ """Test that the modify works as expected with files."""
753+ event = FakeEvent(dir=False, wd=0, name='name',
754+ path='path', pathname='pathname')
755+ self.processor.process_IN_MODIFY(event)
756+ # we should be getting two different method, and open and a close
757+ self.assertEqual(2, len(self.general.called_methods))
758+ self.assertEqual('push_event',
759+ self.general.called_methods[0][0])
760+ self.assertEqual('push_event',
761+ self.general.called_methods[1][0])
762+ self.assertEqual(event.dir, self.general.called_methods[0][1].dir)
763+ self.assertEqual(event.wd, self.general.called_methods[0][1].wd)
764+ self.assertEqual(event.name, self.general.called_methods[0][1].name)
765+ self.assertEqual(event.path, self.general.called_methods[0][1].path)
766+ self.assertEqual(event.pathname,
767+ self.general.called_methods[0][1].pathname)
768+ self.assertEqual(IN_OPEN,
769+ self.general.called_methods[0][1].mask)
770+ self.assertEqual(event.dir, self.general.called_methods[1][1].dir)
771+ self.assertEqual(event.wd, self.general.called_methods[1][1].wd)
772+ self.assertEqual(event.name, self.general.called_methods[1][1].name)
773+ self.assertEqual(event.path, self.general.called_methods[1][1].path)
774+ self.assertEqual(event.pathname,
775+ self.general.called_methods[1][1].pathname)
776+ self.assertEqual(IN_CLOSE_WRITE,
777+ self.general.called_methods[1][1].mask)
778+
779+ def test_process_IN_MOVED_FROM(self):
780+ """Test that the in moved from works as expected."""
781+ event = FakeEvent(dir=False, wd=0, name='name',
782+ path='path', pathname='pathname')
783+ self.processor.process_IN_MOVED_FROM(event)
784+ self.assertEqual(event, self.processor.held_event)
785+
786+ def test_process_IN_MOVED_TO_dir(self):
787+ """Test that the in moved to works as expected."""
788+ event = FakeEvent(wd=0, dir=True, name='name', path='path',
789+ pathname=os.path.join('test', 'pathname'),
790+ cookie='cookie')
791+ held_event = FakeEvent(wd=0, dir=True, name='hname', path='hpath',
792+ pathname=os.path.join('test', 'hpathname'),
793+ cookie='cookie')
794+ self.general.share_id = 'my_share_id'
795+ self.processor.held_event = held_event
796+ self.processor.process_IN_MOVED_TO(event)
797+ self.assertEqual(5, len(self.general.called_methods))
798+ # assert that the ignores are called
799+ self.assertEqual('is_ignored', self.general.called_methods[0][0])
800+ self.assertEqual(held_event.pathname,
801+ self.general.called_methods[0][1])
802+ self.assertEqual('is_ignored', self.general.called_methods[1][0])
803+ self.assertEqual(event.pathname, self.general.called_methods[1][1])
804+ # assert that we do request the share_id
805+ self.assertEqual('get_path_share_id',
806+ self.general.called_methods[2][0])
807+ self.assertEqual(os.path.split(event.pathname)[0],
808+ self.general.called_methods[2][1],
809+ 'Get the share_id for event')
810+ self.assertEqual('get_path_share_id',
811+ self.general.called_methods[3][0])
812+ self.assertEqual(os.path.split(held_event.pathname)[0],
813+ self.general.called_methods[3][1],
814+ 'Get the share_id for held event.')
815+
816+ self.assertEqual('eq_push', self.general.called_methods[4][0])
817+ self.assertEqual('FS_DIR_MOVE', self.general.called_methods[4][1])
818+ self.assertEqual(event.pathname, self.general.called_methods[4][3])
819+ self.assertEqual(held_event.pathname,
820+ self.general.called_methods[4][4])
821+
822+ def test_process_IN_MOVED_TO_file(self):
823+ """Test that the in moved to works as expected."""
824+ event = FakeEvent(wd=0, dir=False, name='name', path='path',
825+ pathname=os.path.join('test', 'pathname'),
826+ cookie='cookie')
827+ held_event = FakeEvent(wd=0, dir=False, name='hname', path='hpath',
828+ pathname=os.path.join('test', 'hpathname'),
829+ cookie='cookie')
830+ self.general.share_id = 'my_share_id'
831+ self.processor.held_event = held_event
832+ self.processor.process_IN_MOVED_TO(event)
833+ self.assertEqual(5, len(self.general.called_methods))
834+ # assert that the ignores are called
835+ self.assertEqual('is_ignored', self.general.called_methods[0][0])
836+ self.assertEqual(held_event.pathname,
837+ self.general.called_methods[0][1])
838+ self.assertEqual('is_ignored', self.general.called_methods[1][0])
839+ self.assertEqual(event.pathname, self.general.called_methods[1][1])
840+ # assert that we do request the share_id
841+ self.assertEqual('get_path_share_id',
842+ self.general.called_methods[2][0])
843+ self.assertEqual(os.path.split(event.pathname)[0],
844+ self.general.called_methods[2][1],
845+ 'Get the share_id for event')
846+ self.assertEqual('get_path_share_id',
847+ self.general.called_methods[3][0])
848+ self.assertEqual(os.path.split(held_event.pathname)[0],
849+ self.general.called_methods[3][1],
850+ 'Get the share_id for held event.')
851+
852+ self.assertEqual('eq_push', self.general.called_methods[4][0])
853+ self.assertEqual('FS_FILE_MOVE', self.general.called_methods[4][1])
854+ self.assertEqual(event.pathname, self.general.called_methods[4][3])
855+ self.assertEqual(held_event.pathname,
856+ self.general.called_methods[4][4])
857+
858+ def test_fake_create_event_dir(self):
859+ """Test that the in moved to works as expected."""
860+ event = FakeEvent(wd=0, dir=True, name='name', path='path',
861+ pathname='pathname')
862+ self.processor._fake_create_event(event)
863+ self.assertEqual(1, len(self.general.called_methods))
864+ self.assertEqual('eq_push', self.general.called_methods[0][0])
865+ self.assertEqual('FS_DIR_CREATE', self.general.called_methods[0][1])
866+ self.assertEqual(event.pathname, self.general.called_methods[0][2])
867+
868+ def test_fake_create_event_file(self):
869+ """Test that the in moved to works as expected."""
870+ event = FakeEvent(wd=0, dir=False, name='name', path='path',
871+ pathname='pathname')
872+ self.processor._fake_create_event(event)
873+ self.assertEqual(2, len(self.general.called_methods))
874+ self.assertEqual('eq_push', self.general.called_methods[0][0])
875+ self.assertEqual('FS_FILE_CREATE', self.general.called_methods[0][1])
876+ self.assertEqual(event.pathname, self.general.called_methods[0][2])
877+ self.assertEqual('eq_push', self.general.called_methods[1][0])
878+ self.assertEqual('FS_FILE_CLOSE_WRITE',
879+ self.general.called_methods[1][1])
880+ self.assertEqual(event.pathname, self.general.called_methods[1][2])
881+
882+ def test_fake_delete_create_event_dir(self):
883+ """Test that we do fake a delete and a later delete."""
884+ event = FakeEvent(wd=0, dir=True, name='name', path='path',
885+ pathname='pathname')
886+ held_event = FakeEvent(wd=0, dir=True, name='hname', path='hpath',
887+ pathname='hpathname')
888+ self.processor.held_event = held_event
889+ self.processor._fake_delete_create_event(event)
890+ self.assertEqual(2, len(self.general.called_methods))
891+ self.assertEqual('eq_push', self.general.called_methods[0][0])
892+ self.assertEqual('FS_DIR_DELETE', self.general.called_methods[0][1])
893+ self.assertEqual(held_event.pathname,
894+ self.general.called_methods[0][2])
895+ self.assertEqual('eq_push', self.general.called_methods[1][0])
896+ self.assertEqual('FS_DIR_CREATE', self.general.called_methods[1][1])
897+ self.assertEqual(event.pathname, self.general.called_methods[1][2])
898+
899+ def test_fake_delete_create_event_file(self):
900+ """Test that we do fake a delete and a later delete."""
901+ event = FakeEvent(wd=0, dir=False, name='name', path='path',
902+ pathname='pathname')
903+ held_event = FakeEvent(wd=0, dir=False, name='hname', path='hpath',
904+ pathname='hpathname')
905+ self.processor.held_event = held_event
906+ self.processor._fake_delete_create_event(event)
907+ self.assertEqual(3, len(self.general.called_methods))
908+ self.assertEqual('eq_push', self.general.called_methods[0][0])
909+ self.assertEqual('FS_FILE_DELETE', self.general.called_methods[0][1])
910+ self.assertEqual(held_event.pathname,
911+ self.general.called_methods[0][2])
912+ self.assertEqual('eq_push', self.general.called_methods[1][0])
913+ self.assertEqual('FS_FILE_CREATE', self.general.called_methods[1][1])
914+ self.assertEqual(event.pathname, self.general.called_methods[1][2])
915+ self.assertEqual('eq_push', self.general.called_methods[2][0])
916+ self.assertEqual('FS_FILE_CLOSE_WRITE',
917+ self.general.called_methods[2][1])
918+ self.assertEqual(event.pathname, self.general.called_methods[2][2])
919+
920+ def test_process_default_no_held(self):
921+ """Test that the process default works as expected."""
922+ event = 'event'
923+ self.processor.process_default(event)
924+ self.assertEqual(1, len(self.general.called_methods))
925+ self.assertEqual('push_event',
926+ self.general.called_methods[0][0])
927+ self.assertEqual(event,
928+ self.general.called_methods[0][1])
929+
930+ def test_process_default_with_held(self):
931+ """Test that the process default works as expected."""
932+ event = 'event'
933+ held_event = 'held_event'
934+ self.processor.held_event = held_event
935+ self.processor.process_default(event)
936+ self.assertEqual(2, len(self.general.called_methods))
937+ self.assertEqual('push_event',
938+ self.general.called_methods[0][0])
939+ self.assertEqual(held_event,
940+ self.general.called_methods[0][1])
941+ self.assertEqual('push_event',
942+ self.general.called_methods[1][0])
943+ self.assertEqual(event,
944+ self.general.called_methods[1][1])
945+
946+ def test_handle_dir_delete_files(self):
947+ """Test that the handle dir delete works as expected."""
948+ path = 'path'
949+ present_files = 'abcde'
950+ # create files and dirs to be returned from the get paths
951+ for file_name in present_files:
952+ self.general.paths_to_return.append((file_name, False))
953+ self.processor.handle_dir_delete(path)
954+ # there are calls for, rm the watch, get paths and then one per file
955+ self.assertEqual(len(present_files) + 2,
956+ len(self.general.called_methods))
957+ rm_call = self.general.called_methods.pop(0)
958+ self.assertEqual('rm_watch', rm_call[0])
959+ self.assertEqual(path, rm_call[1])
960+ paths_call = self.general.called_methods.pop(0)
961+ self.assertEqual('get_paths_starting_with', paths_call[0])
962+ self.assertEqual(path, paths_call[1])
963+ self.assertFalse(paths_call[2])
964+ # we need to push the delete events in reverse order because we want
965+ # to delete children before we delete parents
966+ present_files = present_files[::-1]
967+ for i, called_method in enumerate(self.general.called_methods):
968+ self.assertEqual('eq_push', called_method[0])
969+ self.assertEqual('FS_FILE_DELETE', called_method[1])
970+ self.assertEqual(present_files[i], called_method[2])
971+
972+ def test_handle_dir_delete_dirs(self):
973+ """Test that the handle dir delete works as expected."""
974+ path = 'path'
975+ present_files = 'abcde'
976+ # create files and dirs to be returned from the get paths
977+ for file_name in present_files:
978+ self.general.paths_to_return.append((file_name, True))
979+ self.processor.handle_dir_delete(path)
980+ # there are calls for, rm the watch, get paths and then one per file
981+ self.assertEqual(2 * len(present_files) + 2,
982+ len(self.general.called_methods))
983+ rm_call = self.general.called_methods.pop(0)
984+ self.assertEqual('rm_watch', rm_call[0])
985+ self.assertEqual(path, rm_call[1])
986+ paths_call = self.general.called_methods.pop(0)
987+ self.assertEqual('get_paths_starting_with', paths_call[0])
988+ self.assertEqual(path, paths_call[1])
989+ self.assertFalse(paths_call[2])
990+ # we need to push the delete events in reverse order because we want
991+ # to delete children before we delete parents
992+ present_files = present_files[::-1]
993+
994+ # from http://docs.python.org/library/itertools.html#recipes
995+ def grouper(n, iterable, fillvalue=None):
996+ "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
997+ args = [iter(iterable)] * n
998+ return itertools.izip_longest(fillvalue=fillvalue, *args)
999+
1000+ for i, called_methods in enumerate(grouper(2,
1001+ self.general.called_methods)):
1002+ rm_call = called_methods[0]
1003+ self.assertEqual('rm_watch', rm_call[0])
1004+ self.assertEqual(present_files[i], rm_call[1])
1005+ push_call = called_methods[1]
1006+ self.assertEqual('eq_push', push_call[0])
1007+ self.assertEqual('FS_DIR_DELETE', push_call[1])
1008+ self.assertEqual(present_files[i], push_call[2])
1009+
1010+ def test_freeze_begin(self):
1011+ """Test that the freeze being works as expected."""
1012+ path = 'path'
1013+ self.processor.freeze_begin(path)
1014+ self.assertEqual(1, len(self.general.called_methods))
1015+ self.assertEqual('freeze_begin',
1016+ self.general.called_methods[0][0])
1017+ self.assertEqual(path, self.general.called_methods[0][1])
1018+
1019+ def test_freeze_rollback(self):
1020+ """Test that the freeze rollback works as expected."""
1021+ self.processor.freeze_rollback()
1022+ self.assertEqual(1, len(self.general.called_methods))
1023+ self.assertEqual('freeze_rollback',
1024+ self.general.called_methods[0][0])
1025+
1026+ def test_freeze_commit(self):
1027+ """Test that the freeze commit works as expected."""
1028+ path = 'path'
1029+ self.processor.freeze_commit(path)
1030+ self.assertEqual(1, len(self.general.called_methods))
1031+ self.assertEqual('freeze_commit',
1032+ self.general.called_methods[0][0])
1033+ self.assertEqual(path, self.general.called_methods[0][1])
1034+
1035+
1036+class FilesystemMonitorTestCase(BaseFSMonitorTestCase):
1037+ """Tests for the FilesystemMonitor."""
1038+
1039+ def test_add_watch_twice(self):
1040+ """Check the deferred returned by a second add_watch."""
1041+ self.patch(Watch, "start_watching", lambda self: None)
1042+ self.patch(Watch, "started", lambda self: True)
1043+ manager = WatchManager(None)
1044+ # no need to stop watching because start_watching is fake
1045+
1046+ path = '/Users/username/path'
1047+ mask = 'fake bit mask'
1048+ d1 = manager.add_watch(path, mask)
1049+ d2 = manager.add_watch(path, mask)
1050+
1051+ self.assertTrue(d1.result, "Should not be called yet.")
1052+ self.assertTrue(d2, "Should not be called yet.")
1053
1054=== modified file 'tests/platform/filesystem_notifications/test_filesystem_notifications.py'
1055--- tests/platform/filesystem_notifications/test_filesystem_notifications.py 2012-06-28 10:26:50 +0000
1056+++ tests/platform/filesystem_notifications/test_filesystem_notifications.py 2012-07-10 21:28:20 +0000
1057@@ -39,6 +39,7 @@
1058 BaseTwistedTestCase,
1059 FakeVolumeManager,
1060 skip_if_win32_missing_fs_event,
1061+ skip_if_darwin_missing_fs_event,
1062 )
1063 from ubuntuone.platform import (
1064 remove_file,
1065@@ -264,6 +265,7 @@
1066 mute_filter = self.monitor._processor.mute_filter
1067 return sum(len(x) for x in mute_filter._cnt.values())
1068
1069+ @skip_if_darwin_missing_fs_event
1070 @skip_if_win32_missing_fs_event
1071 @defer.inlineCallbacks
1072 def test_file_open(self):
1073@@ -281,6 +283,7 @@
1074 test_result = yield self._deferred
1075 defer.returnValue(test_result)
1076
1077+ @skip_if_darwin_missing_fs_event
1078 @skip_if_win32_missing_fs_event
1079 @defer.inlineCallbacks
1080 def test_file_close_nowrite(self):
1081@@ -298,6 +301,7 @@
1082 test_result = yield self._deferred
1083 defer.returnValue(test_result)
1084
1085+ @skip_if_darwin_missing_fs_event
1086 @defer.inlineCallbacks
1087 def test_file_create_close_write(self):
1088 """Test receiving the create and close_write signals on files."""
1089@@ -423,6 +427,7 @@
1090 test_result = yield self._deferred
1091 defer.returnValue(test_result)
1092
1093+ @skip_if_darwin_missing_fs_event
1094 @defer.inlineCallbacks
1095 def test_file_moved_from_partial(self):
1096 """Test the handling of the FILE_MOVE event when source is partial."""
1097
1098=== modified file 'tests/platform/filesystem_notifications/test_windows.py'
1099--- tests/platform/filesystem_notifications/test_windows.py 2012-06-22 14:17:44 +0000
1100+++ tests/platform/filesystem_notifications/test_windows.py 2012-07-10 21:28:20 +0000
1101@@ -51,14 +51,16 @@
1102 IN_OPEN,
1103 )
1104 from ubuntuone.platform.filesystem_notifications import (
1105- common,
1106 windows as filesystem_notifications,
1107 )
1108-from ubuntuone.platform.filesystem_notifications.windows import (
1109+from ubuntuone.platform.filesystem_notifications.common import (
1110 FilesystemMonitor,
1111 NotifyProcessor,
1112 Watch,
1113 WatchManager,
1114+)
1115+from ubuntuone.platform.filesystem_notifications.windows import (
1116+ ACTIONS,
1117 FILE_NOTIFY_CHANGE_FILE_NAME,
1118 FILE_NOTIFY_CHANGE_DIR_NAME,
1119 FILE_NOTIFY_CHANGE_ATTRIBUTES,
1120@@ -70,7 +72,7 @@
1121
1122 #create a rever mapping to use it in the tests.
1123 REVERSE_WINDOWS_ACTIONS = {}
1124-for key, value in common.COMMON_ACTIONS.iteritems():
1125+for key, value in ACTIONS.iteritems():
1126 REVERSE_WINDOWS_ACTIONS[value] = key
1127
1128
1129@@ -147,7 +149,7 @@
1130 # group all events in a single lists which is not what the COM API
1131 # does.
1132 str_events = [
1133- (common.COMMON_ACTIONS_NAMES[action], path) for action, path in
1134+ (filesystem_notifications.ACTIONS_NAMES[action], path) for action, path in
1135 events]
1136 self.raw_events.append(str_events)
1137 return events
1138@@ -160,17 +162,14 @@
1139
1140 self.patch(filesystem_notifications, 'FILE_NOTIFY_INFORMATION',
1141 file_notify_information_wrapper)
1142- self.patch(filesystem_notifications.Watch, '_path_is_dir',
1143- path_is_dir_wrapper)
1144+ self.patch(Watch, '_path_is_dir', path_is_dir_wrapper)
1145
1146 @defer.inlineCallbacks
1147- def _perform_operations(self, path, mask, auto_add, actions,
1148- number_events):
1149+ def _perform_operations(self, path, mask, actions, number_events):
1150 """Perform the file operations and returns the recorded events."""
1151 handler = TestCaseHandler(number_events=number_events)
1152 manager = WatchManager(handler)
1153- yield manager.add_watch(os_helper.get_windows_valid_path(path), mask,
1154- auto_add=auto_add)
1155+ yield manager.add_watch(os_helper.get_windows_valid_path(path), mask)
1156 # change the logger so that we can check the logs if we wanted
1157 manager._wdm[0].log.addHandler(self.memento)
1158 # clean logger later
1159@@ -182,35 +181,6 @@
1160 self.addCleanup(manager.stop)
1161 defer.returnValue(ret)
1162
1163- def _perform_timed_operations(self, path, mask, auto_add, actions,
1164- time_out):
1165- """Perform the file operations and returns the recorded events."""
1166- manager = WatchManager()
1167- manager.add_watch(os_helper.get_windows_valid_path(path), mask,
1168- auto_add=auto_add)
1169- # change the logger so that we can check the logs if we wanted
1170- manager._wdm[0].log.addHandler(self.memento)
1171- # clean logger later
1172- self.addCleanup(manager._wdm[0].log.removeHandler, self.memento)
1173- # execution the actions
1174- actions()
1175- # process the recorded events
1176- time.sleep(time_out)
1177- events = self.handler.processed_events
1178- return events
1179-
1180- def _assert_logs(self, events):
1181- """Assert the debug logs."""
1182- logs = []
1183- msg = 'Is path %r a dir? %s'
1184- logs.extend([msg % data for data in self.paths_checked])
1185- msg = 'Got from ReadDirectoryChangesW %r.'
1186- logs.extend([msg % actions for actions in self.raw_events])
1187- msg = 'Pushing event %r to processor.'
1188- logs.extend([msg % e for e in events])
1189- for msg in logs:
1190- self.assertTrue(self.memento.check_debug(msg))
1191-
1192 @defer.inlineCallbacks
1193 def test_file_create(self):
1194 """Test that the correct event is returned on a file create."""
1195@@ -224,7 +194,7 @@
1196 os.fsync(fd)
1197 fd.close()
1198
1199- events = yield self._perform_operations(self.basedir, self.mask, False,
1200+ events = yield self._perform_operations(self.basedir, self.mask,
1201 create_file, 1)
1202 event = events[0]
1203 self.assertFalse(event.dir)
1204@@ -234,8 +204,6 @@
1205 self.assertEqual('.', event.path)
1206 self.assertEqual(os.path.join(self.basedir, file_name), event.pathname)
1207 self.assertEqual(0, event.wd)
1208- # assert the logging
1209- self._assert_logs(events)
1210
1211 @defer.inlineCallbacks
1212 def test_dir_create(self):
1213@@ -246,7 +214,7 @@
1214 """Action for the test."""
1215 os.mkdir(dir_name)
1216
1217- events = yield self._perform_operations(self.basedir, self.mask, False,
1218+ events = yield self._perform_operations(self.basedir, self.mask,
1219 create_dir, 1)
1220 event = events[0]
1221 self.assertTrue(event.dir)
1222@@ -256,8 +224,6 @@
1223 self.assertEqual('.', event.path)
1224 self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname)
1225 self.assertEqual(0, event.wd)
1226- # assert the logging
1227- self._assert_logs(events)
1228
1229 @defer.inlineCallbacks
1230 def test_file_remove(self):
1231@@ -270,7 +236,7 @@
1232 """Action for the test."""
1233 os.remove(file_name)
1234
1235- events = yield self._perform_operations(self.basedir, self.mask, False,
1236+ events = yield self._perform_operations(self.basedir, self.mask,
1237 remove_file, 1)
1238 event = events[0]
1239 self.assertFalse(event.dir)
1240@@ -280,8 +246,6 @@
1241 self.assertEqual('.', event.path)
1242 self.assertEqual(os.path.join(self.basedir, file_name), event.pathname)
1243 self.assertEqual(0, event.wd)
1244- # assert the logging
1245- self._assert_logs(events)
1246
1247 @defer.inlineCallbacks
1248 def test_dir_remove(self):
1249@@ -294,7 +258,7 @@
1250 """Action for the test."""
1251 os.rmdir(dir_name)
1252
1253- events = yield self._perform_operations(self.basedir, self.mask, False,
1254+ events = yield self._perform_operations(self.basedir, self.mask,
1255 remove_dir, 1)
1256 event = events[0]
1257 self.assertTrue(event.dir)
1258@@ -303,8 +267,6 @@
1259 self.assertEqual('.', event.path)
1260 self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname)
1261 self.assertEqual(0, event.wd)
1262- # assert the logging
1263- self._assert_logs(events)
1264
1265 @defer.inlineCallbacks
1266 def test_file_write(self):
1267@@ -320,7 +282,7 @@
1268 fd.write('test')
1269 fd.close()
1270
1271- events = yield self._perform_operations(self.basedir, self.mask, False,
1272+ events = yield self._perform_operations(self.basedir, self.mask,
1273 write_file, 1)
1274 event = events[0]
1275 self.assertFalse(event.dir)
1276@@ -330,8 +292,6 @@
1277 self.assertEqual('.', event.path)
1278 self.assertEqual(os.path.join(self.basedir, file_name), event.pathname)
1279 self.assertEqual(0, event.wd)
1280- # assert the logging
1281- self._assert_logs(events)
1282
1283 @defer.inlineCallbacks
1284 def test_file_moved_to_watched_dir_same_watcher(self):
1285@@ -348,7 +308,7 @@
1286 os.rename(from_file_name, to_file_name)
1287
1288 events = yield self._perform_operations(self.basedir, self.mask,
1289- False, move_file, 2)
1290+ move_file, 2)
1291 move_from_event = events[0]
1292 move_to_event = events[1]
1293 # first test the move from
1294@@ -374,8 +334,6 @@
1295 self.assertEqual(0, move_to_event.wd)
1296 # assert that both cookies are the same
1297 self.assertEqual(move_from_event.cookie, move_to_event.cookie)
1298- # assert the logging
1299- self._assert_logs(events)
1300
1301 @defer.inlineCallbacks
1302 def test_file_moved_to_not_watched_dir(self):
1303@@ -392,7 +350,7 @@
1304 # while on linux we will have to do some sort of magic like facundo
1305 # did, on windows we will get a deleted event which is much more
1306 # cleaner, first time ever windows is nicer!
1307- events = yield self._perform_operations(self.basedir, self.mask, False,
1308+ events = yield self._perform_operations(self.basedir, self.mask,
1309 move_file, 1)
1310 event = events[0]
1311 self.assertFalse(event.dir)
1312@@ -403,8 +361,6 @@
1313 self.assertEqual(os.path.join(self.basedir, from_file_name),
1314 event.pathname)
1315 self.assertEqual(0, event.wd)
1316- # assert the logging
1317- self._assert_logs(events)
1318
1319 @defer.inlineCallbacks
1320 def test_file_move_from_not_watched_dir(self):
1321@@ -422,7 +378,7 @@
1322
1323 # while on linux we have to do some magic operations like facundo did
1324 # on windows we have a created file, hurray!
1325- events = yield self._perform_operations(self.basedir, self.mask, False,
1326+ events = yield self._perform_operations(self.basedir, self.mask,
1327 move_files, 1)
1328 event = events[0]
1329 self.assertFalse(event.dir)
1330@@ -433,8 +389,6 @@
1331 self.assertEqual(os.path.join(self.basedir, to_file_name),
1332 event.pathname)
1333 self.assertEqual(0, event.wd)
1334- # assert the logging
1335- self._assert_logs(events)
1336
1337 @defer.inlineCallbacks
1338 def test_dir_moved_to_watched_dir_same_watcher(self):
1339@@ -450,7 +404,7 @@
1340 os.rename(from_dir_name, to_dir_name)
1341
1342 events = yield self._perform_operations(self.basedir,
1343- self.mask, False, move_file, 2)
1344+ self.mask, move_file, 2)
1345 move_from_event = events[0]
1346 move_to_event = events[1]
1347 # first test the move from
1348@@ -475,8 +429,6 @@
1349 self.assertEqual(0, move_to_event.wd)
1350 # assert that both cookies are the same
1351 self.assertEqual(move_from_event.cookie, move_to_event.cookie)
1352- # assert the logging
1353- self._assert_logs(events)
1354
1355 @defer.inlineCallbacks
1356 def test_dir_moved_to_not_watched_dir(self):
1357@@ -491,7 +443,7 @@
1358 'test_dir_moved_to_not_watched_dir'))
1359
1360 # on windows a move to outside a watched dir translates to a remove
1361- events = yield self._perform_operations(self.basedir, self.mask, False,
1362+ events = yield self._perform_operations(self.basedir, self.mask,
1363 move_dir, 1)
1364 event = events[0]
1365 self.assertTrue(event.dir)
1366@@ -500,8 +452,6 @@
1367 self.assertEqual('.', event.path)
1368 self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname)
1369 self.assertEqual(0, event.wd)
1370- # assert the logging
1371- self._assert_logs(events)
1372
1373 @defer.inlineCallbacks
1374 def test_dir_move_from_not_watched_dir(self):
1375@@ -517,7 +467,7 @@
1376 """Action for the test."""
1377 os.rename(from_dir_name, to_dir_name)
1378
1379- events = yield self._perform_operations(self.basedir, self.mask, False,
1380+ events = yield self._perform_operations(self.basedir, self.mask,
1381 move_dir, 1)
1382 event = events[0]
1383 self.assertTrue(event.dir)
1384@@ -535,8 +485,7 @@
1385 manager = WatchManager(handler)
1386 # add a watch that will always exclude all actions
1387 manager.add_watch(os_helper.get_windows_valid_path(self.basedir),
1388- self.mask, auto_add=True,
1389- exclude_filter=lambda x: True)
1390+ self.mask, exclude_filter=lambda x: True)
1391 # execution the actions
1392 file_name = os.path.join(self.basedir, 'test_file_create')
1393 open(file_name, 'w').close()
1394@@ -545,21 +494,6 @@
1395 self.assertEqual(0, len(handler.processed_events))
1396 test_exclude_filter.skip = "we must rethink this test."
1397
1398- def test_open_dir_muted(self):
1399- """Test that the opening of dirs is ignored."""
1400- dir_name = os.path.join(tempfile.mkdtemp(), 'test_dir_open')
1401- # create file before we record
1402- os.mkdir(dir_name)
1403-
1404- def open_dir():
1405- """Action for the test."""
1406- os.startfile(dir_name)
1407-
1408- events = self._perform_timed_operations(self.basedir, self.mask, False,
1409- open_dir, 2)
1410- self.assertEqual(0, len(events))
1411- test_open_dir_muted.skip = "we must rethink this test."
1412-
1413 def test_ignore_path(self):
1414 """Test that events from a path are ignored."""
1415 events = []
1416@@ -570,14 +504,14 @@
1417
1418 path = u'\\\\?\\C:\\path' # a valid windows path
1419 child = u'child'
1420- watch = Watch(1, path, None, True, fake_processor)
1421+ watch = Watch(1, path, fake_processor)
1422 watch.ignore_path(os.path.join(path, child))
1423 paths_to_ignore = []
1424 for file_name in 'abcdef':
1425 paths_to_ignore.append((1, os.path.join(child, file_name)))
1426 # ensure that the watch is watching
1427 watch._watching = True
1428- watch._process_events(paths_to_ignore)
1429+ watch.platform_watch._process_events(paths_to_ignore)
1430 self.assertEqual(0, len(events),
1431 'All events should have been ignored.')
1432
1433@@ -591,15 +525,15 @@
1434
1435 path = u'\\\\?\\C:\\path' # a valid windows path
1436 child = u'child'
1437- watch = Watch(1, path, None, True, fake_processor)
1438+ watch = Watch(1, path, fake_processor)
1439 watch.ignore_path(os.path.join(path, child))
1440 paths_not_to_ignore = []
1441 for file_name in 'abcdef':
1442 paths_not_to_ignore.append((1, os.path.join(
1443 child + file_name, file_name)))
1444 # ensure that the watch is watching
1445- watch._watching = True
1446- watch._process_events(paths_not_to_ignore)
1447+ watch.platform_watch.watching = True
1448+ watch.platform_watch._process_events(paths_not_to_ignore)
1449 self.assertEqual(len(paths_not_to_ignore), len(events),
1450 'No events should have been ignored.')
1451
1452@@ -613,7 +547,7 @@
1453
1454 child = u'child'
1455 path = u'\\\\?\\C:\\path\\' # a valid windows path
1456- watch = Watch(1, path, None, True, fake_processor)
1457+ watch = Watch(1, path, fake_processor)
1458 watch.ignore_path(os.path.join(path, child))
1459 paths_not_to_ignore = []
1460 paths_to_ignore = []
1461@@ -624,8 +558,8 @@
1462 paths_not_to_ignore.append((1, valid))
1463 expected_events.append(os.path.join('C:\\path', valid))
1464 # ensure that the watch is watching
1465- watch._watching = True
1466- watch._process_events(paths_not_to_ignore)
1467+ watch.platform_watch.watching = True
1468+ watch.platform_watch._process_events(paths_not_to_ignore)
1469 self.assertEqual(len(paths_not_to_ignore), len(events),
1470 'Wrong number of events ignored.')
1471 self.assertTrue(all([event in expected_events for event in events]),
1472@@ -641,15 +575,15 @@
1473
1474 path = u'\\\\?\\C:\\path' # a valid windows path
1475 child = u'child'
1476- watch = Watch(1, path, None, True, fake_processor)
1477+ watch = Watch(1, path, fake_processor)
1478 watch.ignore_path(os.path.join(path, child))
1479 watch.remove_ignored_path(os.path.join(path, child))
1480 paths_not_to_ignore = []
1481 for file_name in 'abcdef':
1482 paths_not_to_ignore.append((1, os.path.join(child, file_name)))
1483 # ensure that the watch is watching
1484- watch._watching = True
1485- watch._process_events(paths_not_to_ignore)
1486+ watch.platform_watch.watching = True
1487+ watch.platform_watch._process_events(paths_not_to_ignore)
1488 self.assertEqual(len(paths_not_to_ignore), len(events),
1489 'All events should have been accepted.')
1490
1491@@ -664,7 +598,7 @@
1492 path = u'\\\\?\\C:\\path' # a valid windows path
1493 child_a = u'childa'
1494 child_b = u'childb'
1495- watch = Watch(1, path, None, True, fake_processor)
1496+ watch = Watch(1, path, fake_processor)
1497 watch.ignore_path(os.path.join(path, child_a))
1498 watch.ignore_path(os.path.join(path, child_b))
1499 watch.remove_ignored_path(os.path.join(path, child_a))
1500@@ -677,8 +611,8 @@
1501 paths_not_to_ignore.append((1, valid))
1502 expected_events.append(os.path.join('C:\\path', valid))
1503 # ensure that the watch is watching
1504- watch._watching = True
1505- watch._process_events(paths_not_to_ignore)
1506+ watch.platform_watch.watching = True
1507+ watch.platform_watch._process_events(paths_not_to_ignore)
1508 self.assertEqual(len(paths_not_to_ignore), len(events),
1509 'All events should have been accepted.')
1510 self.assertTrue(all([event in expected_events for event in events]),
1511@@ -694,9 +628,9 @@
1512 method_args.append((args, kwargs),)
1513
1514 path = u'\\\\?\\C:\\path' # a valid windows path
1515- watch = Watch(1, path, None, True, None)
1516- yield watch._watch_started_deferred.callback(True)
1517- watch._call_deferred(fake_call, None)
1518+ watch = Watch(1, path, None)
1519+ yield watch.platform_watch._watch_started_deferred.callback(True)
1520+ watch.platform_watch._call_deferred(fake_call, None)
1521 self.assertEqual(0, len(method_args))
1522
1523 def test_call_deferred_not_called(self):
1524@@ -708,21 +642,21 @@
1525 method_args.append((args, kwargs),)
1526
1527 path = u'\\\\?\\C:\\path' # a valid windows path
1528- watch = Watch(1, path, None, True, None)
1529- watch._call_deferred(fake_call, None)
1530+ watch = Watch(1, path, None)
1531+ watch.platform_watch._call_deferred(fake_call, None)
1532 self.assertEqual(1, len(method_args))
1533
1534 def test_started_property(self):
1535 """Test that the started property returns the started deferred."""
1536 path = u'\\\\?\\C:\\path' # a valid windows path
1537- watch = Watch(1, path, None, True, None)
1538- self.assertEqual(watch.started, watch._watch_started_deferred)
1539+ watch = Watch(1, path, None)
1540+ self.assertEqual(watch.started, watch.platform_watch._watch_started_deferred)
1541
1542 def test_stopped_property(self):
1543 """Test that the stopped property returns the stopped deferred."""
1544 path = u'\\\\?\\C:\\path' # a valid windows path
1545- watch = Watch(1, path, None, True, None)
1546- self.assertEqual(watch.stopped, watch._watch_stopped_deferred)
1547+ watch = Watch(1, path, None)
1548+ self.assertEqual(watch.stopped, watch.platform_watch._watch_stopped_deferred)
1549
1550 def random_error(self, *args):
1551 """Throw a fake exception."""
1552@@ -733,7 +667,7 @@
1553 """An early failure inside the thread should errback the deferred."""
1554 test_path = self.mktemp("test_directory")
1555 self.patch(filesystem_notifications, "CreateFileW", self.random_error)
1556- watch = Watch(1, test_path, None, True, None)
1557+ watch = Watch(1, test_path, None)
1558 d = watch.start_watching()
1559 yield self.assertFailure(d, FakeException)
1560
1561@@ -743,7 +677,7 @@
1562 test_path = self.mktemp("test_directory")
1563 self.patch(filesystem_notifications, "ReadDirectoryChangesW",
1564 self.random_error)
1565- watch = Watch(1, test_path, None, True, None)
1566+ watch = Watch(1, test_path, None)
1567 d = watch.start_watching()
1568 yield self.assertFailure(d, FakeException)
1569
1570@@ -757,7 +691,7 @@
1571 close_called.append)
1572 self.patch(filesystem_notifications, "ReadDirectoryChangesW",
1573 self.random_error)
1574- watch = Watch(1, test_path, self.mask, True, None)
1575+ watch = Watch(1, test_path, None)
1576 d = watch.start_watching()
1577 yield self.assertFailure(d, FakeException)
1578 self.assertEqual(len(close_called), 1)
1579@@ -767,18 +701,18 @@
1580 def test_stop_watching_fired_when_watch_thread_finishes(self):
1581 """The deferred returned is fired when the watch thread finishes."""
1582 test_path = self.mktemp("another_test_directory")
1583- watch = Watch(1, test_path, self.mask, True, None)
1584+ watch = Watch(1, test_path, None)
1585 yield watch.start_watching()
1586- self.assertNotEqual(watch._watch_handle, None)
1587+ self.assertNotEqual(watch.platform_watch._watch_handle, None)
1588 yield watch.stop_watching()
1589- self.assertEqual(watch._watch_handle, None)
1590+ self.assertEqual(watch.platform_watch._watch_handle, None)
1591
1592 def test_is_path_dir_missing_no_subdir(self):
1593 """Test when the path does not exist and is no a subdir."""
1594 path = u'\\\\?\\C:\\path\\to\\no\\dir'
1595 test_path = self.mktemp("test_directory")
1596 self.patch(os.path, 'exists', lambda path: False)
1597- watch = Watch(1, test_path, self.mask, True, None)
1598+ watch = Watch(1, test_path, None)
1599 self.assertFalse(watch._path_is_dir(path))
1600
1601 def test_is_path_dir_missing_in_subdir(self):
1602@@ -786,7 +720,7 @@
1603 path = u'\\\\?\\C:\\path\\to\\no\\dir'
1604 test_path = self.mktemp("test_directory")
1605 self.patch(os.path, 'exists', lambda path: False)
1606- watch = Watch(1, test_path, self.mask, True, None)
1607+ watch = Watch(1, test_path, None)
1608 watch._subdirs.add(path)
1609 self.assertTrue(watch._path_is_dir(path))
1610
1611@@ -796,7 +730,7 @@
1612 test_path = self.mktemp("test_directory")
1613 self.patch(os.path, 'exists', lambda path: True)
1614 self.patch(os.path, 'isdir', lambda path: True)
1615- watch = Watch(1, test_path, self.mask, True, None)
1616+ watch = Watch(1, test_path, None)
1617 watch._subdirs.add(path)
1618 self.assertTrue(watch._path_is_dir(path))
1619
1620@@ -806,7 +740,7 @@
1621 test_path = self.mktemp("test_directory")
1622 self.patch(os.path, 'exists', lambda path: True)
1623 self.patch(os.path, 'isdir', lambda path: False)
1624- watch = Watch(1, test_path, self.mask, True, None)
1625+ watch = Watch(1, test_path, None)
1626 watch._subdirs.add(path)
1627 self.assertFalse(watch._path_is_dir(path))
1628
1629@@ -814,7 +748,7 @@
1630 """Test when we update on a create event and not present."""
1631 path = u'\\\\?\\C:\\path\\to\\no\\dir'
1632 test_path = self.mktemp("test_directory")
1633- watch = Watch(1, test_path, self.mask, True, None)
1634+ watch = Watch(1, test_path, None)
1635 watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_CREATE])
1636 self.assertTrue(path in watch._subdirs)
1637
1638@@ -822,7 +756,7 @@
1639 """Test when we update on a create event and is present."""
1640 path = u'\\\\?\\C:\\path\\to\\no\\dir'
1641 test_path = self.mktemp("test_directory")
1642- watch = Watch(1, test_path, self.mask, True, None)
1643+ watch = Watch(1, test_path, None)
1644 watch._subdirs.add(path)
1645 old_length = len(watch._subdirs)
1646 watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_CREATE])
1647@@ -833,7 +767,7 @@
1648 """Test when we delete and is not present."""
1649 path = u'\\\\?\\C:\\path\\to\\no\\dir'
1650 test_path = self.mktemp("test_directory")
1651- watch = Watch(1, test_path, self.mask, True, None)
1652+ watch = Watch(1, test_path, None)
1653 watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_DELETE])
1654 self.assertTrue(path not in watch._subdirs)
1655
1656@@ -841,7 +775,7 @@
1657 """Test when we delete and is present."""
1658 path = u'\\\\?\\C:\\path\\to\\no\\dir'
1659 test_path = self.mktemp("test_directory")
1660- watch = Watch(1, test_path, self.mask, True, None)
1661+ watch = Watch(1, test_path, None)
1662 watch._subdirs.add(path)
1663 watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_DELETE])
1664 self.assertTrue(path not in watch._subdirs)
1665@@ -856,7 +790,7 @@
1666 yield super(TestWatchManager, self).setUp()
1667 self.parent_path = u'\\\\?\\C:\\' # a valid windows path
1668 self.path = self.parent_path + u'path'
1669- self.watch = Watch(1, self.path, None, True, None)
1670+ self.watch = Watch(1, self.path, None)
1671 self.manager = WatchManager(None)
1672 self.manager._wdm = {1: self.watch}
1673
1674@@ -884,7 +818,7 @@
1675
1676 self.patch(Watch, "stop_watching", fake_stop_watching)
1677 second_path = self.parent_path + u"second_path"
1678- second_watch = Watch(2, second_path, None, True, None)
1679+ second_watch = Watch(2, second_path, None)
1680 self.manager._wdm[2] = second_watch
1681 d = self.manager.stop()
1682 self.assertFalse(d.called, "Not fired before all watches end")
1683@@ -928,19 +862,10 @@
1684 self.manager._wdm = {}
1685
1686 mask = 'bit_mask'
1687- auto_add = True
1688- self.manager.add_watch(self.path, mask, auto_add=auto_add)
1689+ self.manager.add_watch(self.path, mask)
1690 self.assertEqual(1, len(self.manager._wdm))
1691 self.assertTrue(self.was_called, 'The watch start was not called.')
1692- self.assertEqual(self.path + os.path.sep, self.manager._wdm[0]._path)
1693- self.assertEqual(mask, self.manager._wdm[0]._mask)
1694- self.assertEqual(auto_add, self.manager._wdm[0]._auto_add)
1695-
1696- def test_update_present_watch(self):
1697- """Test the update of a present watch."""
1698- mask = 'mask'
1699- self.assertRaises(NotImplementedError, self.manager.update_watch,
1700- 1, mask)
1701+ self.assertEqual(self.path + os.path.sep, self.manager._wdm[0].path)
1702
1703 def test_get_watch_present_wd(self):
1704 """Test that the correct path is returned."""
1705@@ -988,7 +913,8 @@
1706 self.manager.rm_path(self.path)
1707 self.assertEqual(self.watch, self.manager._wdm.get(1))
1708 self.watch._watching = True
1709- self.watch._process_events([(1, os.path.join(self.path, 'test'))])
1710+ self.watch.platform_watch._process_events(
1711+ [(1, os.path.join(self.path, 'test'))])
1712 self.assertEqual(0, len(events))
1713
1714 def test_rm_child_path(self):
1715@@ -1004,11 +930,12 @@
1716 self.manager.rm_path(child)
1717 self.assertEqual(self.watch, self.manager._wdm[1])
1718 # assert that the correct event is ignored
1719- self.watch._watching = True
1720- self.watch._process_events([(1, os.path.join('child', 'test'))])
1721+ self.watch.platform_watch.watching = True
1722+ self.watch.platform_watch._process_events(
1723+ [(1, os.path.join('child', 'test'))])
1724 self.assertEqual(0, len(events))
1725 # assert that other events are not ignored
1726- self.watch._process_events([(1, 'test')])
1727+ self.watch.platform_watch._process_events([(1, 'test')])
1728 self.assertEqual(1, len(events))
1729
1730
1731@@ -1160,9 +1087,9 @@
1732 """Test that we do indeed ignore the correct paths."""
1733 not_ignored = 'test'
1734 ignored = not_ignored + '.lnk'
1735- self.assertFalse(self.processor.platform_is_ignored(not_ignored),
1736+ self.assertFalse(filesystem_notifications.path_is_ignored(not_ignored),
1737 'Only links should be ignored.')
1738- self.assertTrue(self.processor.platform_is_ignored(ignored),
1739+ self.assertTrue(filesystem_notifications.path_is_ignored(ignored),
1740 'Links should be ignored.')
1741
1742 def test_is_ignored(self):
1743@@ -1394,7 +1321,7 @@
1744 for file_name in present_files:
1745 self.general.paths_to_return.append((file_name, False))
1746 self.processor.handle_dir_delete(path)
1747- # there are calls for, rm the wathc, get paths and then one per file
1748+ # there are calls for, rm the watch, get paths and then one per file
1749 self.assertEqual(len(present_files) + 2,
1750 len(self.general.called_methods))
1751 rm_call = self.general.called_methods.pop(0)
1752@@ -1420,7 +1347,7 @@
1753 for file_name in present_files:
1754 self.general.paths_to_return.append((file_name, True))
1755 self.processor.handle_dir_delete(path)
1756- # there are calls for, rm the wathc, get paths and then one per file
1757+ # there are calls for, rm the watch, get paths and then one per file
1758 self.assertEqual(2 * len(present_files) + 2,
1759 len(self.general.called_methods))
1760 rm_call = self.general.called_methods.pop(0)
1761
1762=== modified file 'ubuntuone/platform/filesystem_notifications/__init__.py'
1763--- ubuntuone/platform/filesystem_notifications/__init__.py 2012-06-28 16:44:01 +0000
1764+++ ubuntuone/platform/filesystem_notifications/__init__.py 2012-07-10 21:28:20 +0000
1765@@ -31,14 +31,10 @@
1766 import sys
1767
1768
1769-if sys.platform == "win32":
1770- from ubuntuone.platform.filesystem_notifications import windows
1771- FilesystemMonitor = windows.FilesystemMonitor
1772- _GeneralINotifyProcessor = windows.NotifyProcessor
1773-elif sys.platform == "darwin":
1774- from ubuntuone.platform.filesystem_notifications import darwin
1775- FilesystemMonitor = darwin.FilesystemMonitor
1776- _GeneralINotifyProcessor = darwin.NotifyProcessor
1777+if sys.platform in ('darwin', 'win32'):
1778+ from ubuntuone.platform.filesystem_notifications import common
1779+ FilesystemMonitor = common.FilesystemMonitor
1780+ _GeneralINotifyProcessor = common.NotifyProcessor
1781 else:
1782 from ubuntuone.platform.filesystem_notifications import linux
1783 FilesystemMonitor = linux.FilesystemMonitor
1784
1785=== modified file 'ubuntuone/platform/filesystem_notifications/common.py'
1786--- ubuntuone/platform/filesystem_notifications/common.py 2012-06-28 16:44:01 +0000
1787+++ ubuntuone/platform/filesystem_notifications/common.py 2012-07-10 21:28:20 +0000
1788@@ -30,6 +30,7 @@
1789
1790 import logging
1791 import os
1792+import sys
1793
1794 from twisted.internet import defer
1795
1796@@ -44,9 +45,10 @@
1797 IN_IGNORED,
1798 IN_ISDIR,
1799 IN_DELETE,
1800- IN_MODIFY as in_modify,
1801+ IN_MODIFY,
1802 IN_MOVED_FROM,
1803- IN_MOVED_TO)
1804+ IN_MOVED_TO,
1805+)
1806
1807 from ubuntuone import logger
1808
1809@@ -56,16 +58,28 @@
1810 os_path,
1811 )
1812
1813+if sys.platform == 'darwin':
1814+ from ubuntuone.platform.filesystem_notifications import darwin
1815+ source = darwin
1816+elif sys.platform == 'win32':
1817+ from ubuntuone.platform.filesystem_notifications import windows
1818+ source = windows
1819+else:
1820+ raise ImportError('Not supported platform')
1821+
1822 # a map between the few events that we have on common platforms and those
1823 # found in pyinotify
1824-COMMON_ACTIONS = {}
1825+ACTIONS = source.ACTIONS
1826
1827 # a map of the actions to names so that we have better logs.
1828-COMMON_ACTIONS_NAMES = {}
1829-
1830-# We should have this here, because we use if from other modules that
1831-# share this, but we need to declare it this way yo avoid flakes issues.
1832-IN_MODIFY = in_modify
1833+ACTIONS_NAMES = source.ACTIONS_NAMES
1834+
1835+# ignore paths in the platform, mainly links atm
1836+path_is_ignored = source.path_is_ignored
1837+
1838+# the base class to be use for a platform
1839+PlatformWatch = source.Watch
1840+PlatformWatchManager = source.WatchManager
1841
1842 # translates quickly the event and it's is_dir state to our standard events
1843 NAME_TRANSLATIONS = {
1844@@ -89,42 +103,50 @@
1845 class Watch(object):
1846 """Implement the same functions as pyinotify.Watch."""
1847
1848- def __init__(self, watch_descriptor, path, mask, auto_add, processor,
1849- buf_size=8192):
1850- self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.common.' +
1851- 'filesystem_notifications.Watch')
1852- self.log.setLevel(TRACE)
1853+ def __init__(self, watch_descriptor, path, processor):
1854+ """Create a new watch."""
1855+
1856+ # do ensure that we provide a os.path.sep
1857+ if not path.endswith(os.path.sep):
1858+ path += os.path.sep
1859+ self.path = path
1860+ self.ignore_paths = []
1861 self._processor = processor
1862- self._buf_size = buf_size
1863- self._watching = False
1864 self._descriptor = watch_descriptor
1865- self._auto_add = auto_add
1866- self._ignore_paths = []
1867 self._cookie = None
1868 self._source_pathname = None
1869- self._process_thread = None
1870- self._watch_handle = None
1871 # remember the subdirs we have so that when we have a delete we can
1872 # check if it was a remove
1873 self._subdirs = set()
1874- # ensure that we work with an abspath and that we can deal with
1875- # long paths over 260 chars.
1876- if not path.endswith(os.path.sep):
1877- path += os.path.sep
1878- self._path = os.path.abspath(path)
1879- self._mask = mask
1880-
1881- def _process_events_from_filesystem(self, action, file_name, cookie,
1882- syncdaemon_path):
1883+
1884+ # platform watch used to deal with the platform details
1885+ self.platform_watch = PlatformWatch(self.path, self.process_events)
1886+
1887+ self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.common.' +
1888+ 'filesystem_notifications.Watch')
1889+ self.log.setLevel(TRACE)
1890+
1891+ def process_events(self, action, file_name, cookie, syncdaemon_path):
1892 """Process the events from the queue."""
1893+ # do not process events when the watch was stopped
1894+ if not self.platform_watch.watching:
1895+ return
1896+
1897+ # do not process those events that should be ignored
1898+ if any([file_name.startswith(path)
1899+ for path in self.ignore_paths]):
1900+ return
1901+
1902 # map the filesystem events to the pyinotify ones, tis is dirty but
1903 # makes the multiplatform better, linux was first :P
1904- full_dir_path = os.path.join(self._path, file_name)
1905+ full_dir_path = os.path.join(self.path, file_name)
1906 is_dir = self._path_is_dir(full_dir_path)
1907+
1908 if is_dir:
1909 # we need to update the list of subdirs that we have
1910 self._update_subdirs(full_dir_path, action)
1911- mask = COMMON_ACTIONS[action]
1912+
1913+ mask = ACTIONS[action]
1914 head, tail = os.path.split(file_name)
1915 if is_dir:
1916 mask |= IN_ISDIR
1917@@ -134,14 +156,14 @@
1918 'mask': mask,
1919 'name': tail,
1920 'path': '.'}
1921- # by the way in which the win api fires the events we know for
1922+ # by the way in which the api fires the events we know for
1923 # sure that no move events will be added in the wrong order, this
1924 # is kind of hacky, I dont like it too much
1925- if COMMON_ACTIONS[action] == IN_MOVED_FROM:
1926+ if ACTIONS[action] == IN_MOVED_FROM:
1927 self._cookie = cookie
1928 self._source_pathname = tail
1929 event_raw_data['cookie'] = self._cookie
1930- if COMMON_ACTIONS[action] == IN_MOVED_TO:
1931+ if ACTIONS[action] == IN_MOVED_TO:
1932 event_raw_data['src_pathname'] = self._source_pathname
1933 event_raw_data['cookie'] = self._cookie
1934 event = Event(event_raw_data)
1935@@ -161,9 +183,9 @@
1936 The given path is considered to be a path and therefore this
1937 will not be checked.
1938 """
1939- if COMMON_ACTIONS[event] == IN_CREATE:
1940+ if ACTIONS[event] == IN_CREATE:
1941 self._subdirs.add(path)
1942- elif COMMON_ACTIONS[event] == IN_DELETE and path in self._subdirs:
1943+ elif ACTIONS[event] == IN_DELETE and path in self._subdirs:
1944 self._subdirs.remove(path)
1945
1946 @is_valid_os_path(path_indexes=[1])
1947@@ -187,51 +209,53 @@
1948 """Add the path of the events to ignore."""
1949 if not path.endswith(os.path.sep):
1950 path += os.path.sep
1951- if path.startswith(self._path):
1952- path = path[len(self._path):]
1953- self._ignore_paths.append(path)
1954+ if path.startswith(self.path):
1955+ path = path[len(self.path):]
1956+ self.ignore_paths.append(path)
1957
1958 @is_valid_os_path(path_indexes=[1])
1959 def remove_ignored_path(self, path):
1960 """Reaccept path."""
1961 if not path.endswith(os.path.sep):
1962 path += os.path.sep
1963- if path.startswith(self._path):
1964- path = path[len(self._path):]
1965- if path in self._ignore_paths:
1966- self._ignore_paths.remove(path)
1967+ if path.startswith(self.path):
1968+ path = path[len(self.path):]
1969+ if path in self.ignore_paths:
1970+ self.ignore_paths.remove(path)
1971
1972+ @defer.inlineCallbacks
1973 def start_watching(self):
1974 """Tell the watch to start processing events."""
1975- for current_child in os.listdir(self._path):
1976- full_child_path = os.path.join(self._path, current_child)
1977+ for current_child in os.listdir(self.path):
1978+ full_child_path = os.path.join(self.path, current_child)
1979 if os.path.isdir(full_child_path):
1980 self._subdirs.add(full_child_path)
1981 # start to diff threads, one to watch the path, the other to
1982 # process the events.
1983 self.log.debug('Start watching path.')
1984- self._watching = True
1985+ yield self.platform_watch.start_watching()
1986
1987 def stop_watching(self):
1988 """Tell the watch to stop processing events."""
1989- self.log.info('Stop watching %s', self._path)
1990- self._watching = False
1991+ self.log.info('Stop watching %s', self.path)
1992+ self.platform_watch.watching = False
1993 self._subdirs = set()
1994-
1995- def update(self, mask, auto_add=False):
1996- """Update the info used by the watcher."""
1997- self.log.debug('update(%s, %s)', mask, auto_add)
1998- self._mask = mask
1999- self._auto_add = auto_add
2000-
2001- @property
2002- def path(self):
2003- """Return the patch watched."""
2004- return self._path
2005-
2006- @property
2007- def auto_add(self):
2008- return self._auto_add
2009+ return self.platform_watch.stop_watching()
2010+
2011+ @property
2012+ def watching(self):
2013+ """Return if we are watching."""
2014+ return self.platform_watch.watching
2015+
2016+ @property
2017+ def started(self):
2018+ """A deferred that will be called when the watch is running."""
2019+ return self.platform_watch.started
2020+
2021+ @property
2022+ def stopped(self):
2023+ """A deferred fired when the watch thread has finished."""
2024+ return self.platform_watch.stopped
2025
2026
2027 class WatchManager(object):
2028@@ -243,25 +267,48 @@
2029
2030 def __init__(self, processor):
2031 """Init the manager to keep trak of the different watches."""
2032- self._processor = processor
2033 self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.common.'
2034 + 'filesystem_notifications.WatchManager')
2035 self.log.setLevel(TRACE)
2036+ self._processor = processor
2037+ # use the platform manager to perform the actual actions
2038+ self.manager = PlatformWatchManager(self.log)
2039 self._wdm = {}
2040- self._wd_count = 0
2041 self._ignored_paths = []
2042
2043- def add_watch(self, path, mask, auto_add=False, quiet=True):
2044- """Add a new path to be watch.
2045+ @defer.inlineCallbacks
2046+ def _add_single_watch(self, path, mask, quiet=True):
2047+ """A just one watch."""
2048+ if path in self._ignored_paths:
2049+ # simply removed it from the filter
2050+ self._ignored_paths.remove(path)
2051+ return
2052+
2053+ # we need to add a new watch
2054+ self.log.debug('add_single_watch(%s, %s, %s)', path, mask, quiet)
2055+
2056+ # common code that will ensure that we keep track of the watches
2057+ watch = Watch(len(self._wdm), path, self._processor)
2058+ self._wdm[len(self._wdm)] = watch
2059+ yield watch.start_watching()
2060+
2061+ # trust that the platform watch manager to do the rest of the start
2062+ # operations
2063+ defer.returnValue(self.manager.add_watch(watch))
2064+
2065+ @is_valid_os_path(path_indexes=[1])
2066+ def add_watch(self, path, mask, quiet=True):
2067+ """Add a new path to be watched.
2068
2069 The method will ensure that the path is not already present.
2070 """
2071- raise NotImplementedError("Not implemented on this platform.")
2072-
2073- def stop(self):
2074- """Close the manager and stop all watches."""
2075- # Should be implemented for each platform
2076- raise NotImplementedError("Not implemented on this platform.")
2077+ wd = self.get_wd(path)
2078+ if wd is None:
2079+ self.log.debug('Adding single watch on %r', path)
2080+ return self._add_single_watch(path, mask, quiet)
2081+ else:
2082+ self.log.debug('Watch already exists on %r', path)
2083+ return self._wdm[wd].started
2084
2085 def get_watch(self, wd):
2086 """Return the watch with the given descriptor."""
2087@@ -274,30 +321,13 @@
2088 watch = self._wdm[wd]
2089 yield watch.stop_watching()
2090 del self._wdm[wd]
2091+ # trust that the platform watch manager will do the rest of the
2092+ # operations needed to delete a watch
2093+ self.manager.del_watch(watch)
2094 self.log.debug('Watch %s removed.', wd)
2095 except KeyError, e:
2096 logging.error(str(e))
2097
2098- def _add_single_watch(self, path, mask, auto_add=False, quiet=True):
2099- if path in self._ignored_paths:
2100- # simply removed it from the filter
2101- self._ignored_paths.remove(path)
2102- return
2103- # we need to add a new watch
2104- self.log.debug('add_single_watch(%s, %s, %s, %s)', path, mask,
2105- auto_add, quiet)
2106-
2107- return self._adding_watch(path, mask, auto_add)
2108-
2109- def _adding_watch(self, path, mask, auto_add):
2110- """Add the watch to the dict and start it."""
2111- # This should be implemented for each OS
2112- raise NotImplementedError("Not implemented on this platform.")
2113-
2114- def update_watch(self, wd, mask=None, rec=False,
2115- auto_add=False, quiet=True):
2116- raise NotImplementedError("Not implemented on this platform.")
2117-
2118 @is_valid_os_path(path_indexes=[1])
2119 def get_wd(self, path):
2120 """Return the watcher that is used to watch the given path."""
2121@@ -305,8 +335,7 @@
2122 path = path + os.path.sep
2123 for current_wd in self._wdm:
2124 watch_path = self._wdm[current_wd].path
2125- if ((watch_path == path or (
2126- watch_path in path and self._wdm[current_wd].auto_add))
2127+ if ((watch_path == path or watch_path in path)
2128 and path not in self._ignored_paths):
2129 return current_wd
2130
2131@@ -323,6 +352,9 @@
2132 watch = self._wdm[wd]
2133 yield watch.stop_watching()
2134 del self._wdm[wd]
2135+ # trust that the platform watch manager will do the rest of the
2136+ # operations needed to delete a watch
2137+ self.manager.rm_watch(watch)
2138 except KeyError, err:
2139 self.log.error(str(err))
2140 if not quiet:
2141@@ -336,6 +368,16 @@
2142 self.log.debug('Adding exclude filter for %r', path)
2143 self._wdm[wd].ignore_path(path)
2144
2145+ @defer.inlineCallbacks
2146+ def stop(self):
2147+ """Close the manager and stop all watches."""
2148+ self.log.debug('Stopping watches.')
2149+ for current_wd in self._wdm:
2150+ watch = self._wdm[current_wd]
2151+ yield self.manager.stop_watch(watch)
2152+ self.log.debug('Stopping Watch on %r.', watch.path)
2153+ yield self.manager.stop()
2154+
2155
2156 class NotifyProcessor(ProcessEvent):
2157 """Processor that takes care of dealing with the events.
2158@@ -350,7 +392,7 @@
2159 GeneralINotifyProcessor)
2160 self.general_processor = GeneralINotifyProcessor(monitor,
2161 self.handle_dir_delete, NAME_TRANSLATIONS,
2162- self.platform_is_ignored, IN_IGNORED, ignore_config=ignore_config)
2163+ path_is_ignored, IN_IGNORED, ignore_config=ignore_config)
2164 self.held_event = None
2165
2166 def rm_from_mute_filter(self, event, paths):
2167@@ -361,11 +403,6 @@
2168 """Add an event and path(s) to the mute filter."""
2169 self.general_processor.add_to_mute_filter(event, paths)
2170
2171- def platform_is_ignored(self, path):
2172- """Should we ignore this path in the current platform.?"""
2173- # This should be implemented in each platform
2174- raise NotImplementedError("Not implemented on this platform.")
2175-
2176 @is_valid_syncdaemon_path(path_indexes=[1])
2177 def is_ignored(self, path):
2178 """Should we ignore this path?"""
2179@@ -552,8 +589,8 @@
2180 self.log.setLevel(TRACE)
2181 self.fs = fs
2182 self.eq = eq
2183- # You will need to create the NotifyProcessor and WatchManager
2184- # in each OS-specific implementation
2185+ self._processor = NotifyProcessor(self, ignore_config)
2186+ self._watch_manager = WatchManager(self._processor)
2187
2188 def add_to_mute_filter(self, event, **info):
2189 """Add info to mute filter in the processor."""
2190@@ -579,12 +616,11 @@
2191 # the logic to check if the watch is already set
2192 # is all in WatchManager.add_watch
2193 return self._watch_manager.add_watch(dirpath,
2194- self.filesystem_monitor_mask, auto_add=True)
2195+ self.filesystem_monitor_mask)
2196
2197 def add_watches_to_udf_ancestors(self, volume):
2198 """Add a inotify watch to volume's ancestors if it's an UDF."""
2199- # Should be implemented in each OS if necessary
2200- raise NotImplementedError("Not implemented on this platform.")
2201+ return defer.succeed(True)
2202
2203 def is_frozen(self):
2204 """Checks if there's something frozen."""
2205
2206=== modified file 'ubuntuone/platform/filesystem_notifications/darwin.py'
2207--- ubuntuone/platform/filesystem_notifications/darwin.py 2012-06-28 17:21:30 +0000
2208+++ ubuntuone/platform/filesystem_notifications/darwin.py 2012-07-10 21:28:20 +0000
2209@@ -33,37 +33,57 @@
2210 import fsevents
2211 from twisted.internet import defer, reactor
2212
2213-from ubuntuone.platform.filesystem_notifications import common
2214-
2215+from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
2216+ IN_DELETE,
2217+ IN_CREATE,
2218+ IN_MODIFY,
2219+ IN_MOVED_FROM,
2220+ IN_MOVED_TO,
2221+)
2222
2223 # a map between the few events that we have on common platforms and those
2224 # found in pyinotify
2225-common.COMMON_ACTIONS = {
2226- fsevents.IN_CREATE: common.IN_CREATE,
2227- fsevents.IN_DELETE: common.IN_DELETE,
2228- fsevents.IN_MODIFY: common.IN_MODIFY,
2229- fsevents.IN_MOVED_FROM: common.IN_MOVED_FROM,
2230- fsevents.IN_MOVED_TO: common.IN_MOVED_TO,
2231+ACTIONS = {
2232+ fsevents.IN_CREATE: IN_CREATE,
2233+ fsevents.IN_DELETE: IN_DELETE,
2234+ fsevents.IN_MODIFY: IN_MODIFY,
2235+ fsevents.IN_MOVED_FROM: IN_MOVED_FROM,
2236+ fsevents.IN_MOVED_TO: IN_MOVED_TO,
2237 }
2238
2239 # a map of the actions to names so that we have better logs.
2240-common.COMMON_ACTIONS_NAMES = {
2241- fsevents.IN_CREATE: 'IN_CREATE',
2242- fsevents.IN_DELETE: 'IN_DELETE',
2243- fsevents.IN_MODIFY: 'IN_MODIFY',
2244- fsevents.IN_MOVED_FROM: 'IN_MOVED_FROM',
2245- fsevents.IN_MOVED_TO: 'IN_MOVED_TO',
2246+ACTIONS_NAMES = {
2247+ fsevents.IN_CREATE: 'IN_CREATE',
2248+ fsevents.IN_DELETE: 'IN_DELETE',
2249+ fsevents.IN_MODIFY: 'IN_MODIFY',
2250+ fsevents.IN_MOVED_FROM: 'IN_MOVED_FROM',
2251+ fsevents.IN_MOVED_TO: 'IN_MOVED_TO',
2252 }
2253
2254
2255+def path_is_ignored(path):
2256+ """Should we ignore this path in the current platform.?"""
2257+ # don't support links yet
2258+ if os.path.islink(path):
2259+ return True
2260+ return False
2261+
2262+
2263 # The implementation of the code that is provided as the pyinotify substitute
2264-class Watch(common.Watch):
2265+class Watch(object):
2266 """Implement the same functions as pyinotify.Watch."""
2267
2268- def __init__(self, watch_descriptor, path, mask, auto_add, processor,
2269- buf_size=8192):
2270- super(Watch, self).__init__(watch_descriptor, path, mask, auto_add,
2271- processor, buf_size)
2272+ def __init__(self, path, process_events):
2273+ """Create a new instance for the given path.
2274+
2275+ The process_events parameter is a callback to be executed in the main
2276+ reactor thread to convert events in pyinotify events and add them to
2277+ the state machine.
2278+ """
2279+ self.path = os.path.abspath(path)
2280+ self.process_events = process_events
2281+ self.watching = False
2282+ self.ignore_paths = []
2283 # Create stream with folder to watch
2284 self.stream = fsevents.Stream(self._process_events,
2285 path, file_events=True)
2286@@ -74,97 +94,69 @@
2287
2288 def _process_events_in_main_thread(self, event):
2289 """Process the events from the queue."""
2290- # do not do it if we stop watching and the events are empty
2291- if not self._watching:
2292- return
2293-
2294 action, cookie, file_name = (event.mask, event.cookie, event.name)
2295- if any([file_name.startswith(path)
2296- for path in self._ignore_paths]):
2297- return
2298- syncdaemon_path = os.path.join(self._path, file_name)
2299- self._process_events_from_filesystem(action, file_name, cookie,
2300- syncdaemon_path)
2301+
2302+ syncdaemon_path = os.path.join(self.path, file_name)
2303+ self.process_events(action, file_name, cookie,
2304+ syncdaemon_path)
2305+
2306+ def start_watching(self):
2307+ """Start watching."""
2308+ self.watching = True
2309+ return defer.succeed(self.watching)
2310+
2311+ def stop_watching(self):
2312+ """Stop watching."""
2313+ self.watching = False
2314+ return defer.succeed(self.watching)
2315
2316 # For API compatibility
2317 @property
2318 def started(self):
2319 """A deferred that will be called when the watch is running."""
2320- return defer.succeed(self._watching)
2321+ return defer.succeed(self.watching)
2322
2323 @property
2324 def stopped(self):
2325 """A deferred fired when the watch thread has finished."""
2326- return defer.succeed(self._watching)
2327-
2328-
2329-class WatchManager(common.WatchManager):
2330+ return defer.succeed(self.watching)
2331+
2332+
2333+class WatchManager(object):
2334 """Implement the same functions as pyinotify.WatchManager.
2335
2336 All paths passed to methods in this class should be darwin paths.
2337
2338 """
2339
2340- def __init__(self, processor):
2341+ def __init__(self, log):
2342 """Init the manager to keep track of the different watches."""
2343- super(WatchManager, self).__init__(processor)
2344+ self.log = log
2345 self.observer = fsevents.Observer()
2346 self.observer.start()
2347
2348- def add_watch(self, path, mask, auto_add=False, quiet=True):
2349- """Add a new path to be watched.
2350-
2351- The method will ensure that the path is not already present.
2352- """
2353- if not isinstance(path, str):
2354- e = NotImplementedError("No implementation on this platform.")
2355- return defer.fail(e)
2356- wd = self.get_wd(path)
2357- if wd is None:
2358- self.log.debug('Adding single watch on %r', path)
2359- return self._add_single_watch(path, mask, auto_add, quiet)
2360- else:
2361- self.log.debug('Watch already exists on %r', path)
2362- return self._wdm[wd].started
2363-
2364 def __del__(self):
2365 """Stop the observer."""
2366 self.observer.stop()
2367
2368+ def stop_watch(self, watch):
2369+ """Stop a given watch."""
2370+ watch.stop_watching()
2371+ self.observer.unschedule(watch.platform_watch.stream)
2372+ return defer.succeed(True)
2373+
2374 def stop(self):
2375- """Close the manager and stop all watches."""
2376- self.log.debug('Stopping watches.')
2377- for current_wd in self._wdm:
2378- self._wdm[current_wd].stop_watching()
2379- self.observer.unschedule(self._wdm[current_wd].stream)
2380- self.log.debug('Stopping Watch on %r.', self._wdm[current_wd].path)
2381+ """Stop the manager."""
2382 self.observer.stop()
2383
2384- def del_watch(self, wd):
2385- """Delete the watch with the given descriptor."""
2386- watch = self.get_watch(wd)
2387- self.observer.unschedule(watch.stream)
2388- return super(WatchManager, self).del_watch(wd)
2389+ def del_watch(self, watch):
2390+ """Delete the watch and clean resources."""
2391+ self.observer.unschedule(watch.platform_watch.stream)
2392
2393- def _adding_watch(self, path, mask, auto_add):
2394+ def add_watch(self, watch):
2395 """This method perform actually the action of registering the watch."""
2396- watch = Watch(self._wd_count, path, mask, auto_add, self._processor)
2397- self._wdm[self._wd_count] = watch
2398- self._wdm[self._wd_count].start_watching()
2399- self.observer.schedule(self._wdm[self._wd_count].stream)
2400- self._wd_count += 1
2401- return defer.succeed(True)
2402+ self.observer.schedule(watch.platform_watch.stream)
2403+ return True
2404
2405- def rm_watch(self, wd, rec=False, quiet=True):
2406+ def rm_watch(self, watch):
2407 """Remove the the watch with the given wd."""
2408- watch = self.get_watch(wd)
2409- self.observer.unschedule(watch.stream)
2410- super(WatchManager, self).del_watch(wd, rec, quiet)
2411-
2412-
2413-class FilesystemMonitor(object):
2414- """Empty implementation of FilesystemMonitor"""
2415-
2416-
2417-class NotifyProcessor(object):
2418- """Empty implementation of NotifyProcessor"""
2419
2420=== modified file 'ubuntuone/platform/filesystem_notifications/windows.py'
2421--- ubuntuone/platform/filesystem_notifications/windows.py 2012-06-28 16:44:01 +0000
2422+++ ubuntuone/platform/filesystem_notifications/windows.py 2012-07-10 21:28:20 +0000
2423@@ -28,7 +28,9 @@
2424 # files in the program, then also delete it here.
2425 """File notifications on windows."""
2426
2427+import logging
2428 import os
2429+
2430 from uuid import uuid4
2431
2432 from twisted.internet import defer, reactor
2433@@ -66,25 +68,31 @@
2434 get_syncdaemon_valid_path,
2435 )
2436
2437-from ubuntuone.platform.filesystem_notifications import common
2438-
2439-# a map between the few events that we have on common platforms and those
2440-# found in pyinotify
2441-common.COMMON_ACTIONS = {
2442- 1: common.IN_CREATE,
2443- 2: common.IN_DELETE,
2444- 3: common.IN_MODIFY,
2445- 4: common.IN_MOVED_FROM,
2446- 5: common.IN_MOVED_TO,
2447+from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
2448+ IN_CREATE,
2449+ IN_DELETE,
2450+ IN_MODIFY,
2451+ IN_MOVED_FROM,
2452+ IN_MOVED_TO,
2453+)
2454+
2455+
2456+# map between windows events and pyinotify
2457+ACTIONS = {
2458+ 1: IN_CREATE,
2459+ 2: IN_DELETE,
2460+ 3: IN_MODIFY,
2461+ 4: IN_MOVED_FROM,
2462+ 5: IN_MOVED_TO,
2463 }
2464
2465 # a map of the actions to names so that we have better logs.
2466-common.COMMON_ACTIONS_NAMES = {
2467- 1: 'IN_CREATE',
2468- 2: 'IN_DELETE',
2469- 3: 'IN_MODIFY',
2470- 4: 'IN_MOVED_FROM',
2471- 5: 'IN_MOVED_TO',
2472+ACTIONS_NAMES = {
2473+ 1: 'IN_CREATE',
2474+ 2: 'IN_DELETE',
2475+ 3: 'IN_MODIFY',
2476+ 4: 'IN_MOVED_FROM',
2477+ 5: 'IN_MOVED_TO',
2478 }
2479
2480
2481@@ -96,20 +104,43 @@
2482
2483 THREADPOOL_MAX = 20
2484
2485-
2486-class Watch(common.Watch):
2487+FILESYSTEM_MONITOR_MASK = FILE_NOTIFY_CHANGE_FILE_NAME | \
2488+ FILE_NOTIFY_CHANGE_DIR_NAME | \
2489+ FILE_NOTIFY_CHANGE_ATTRIBUTES | \
2490+ FILE_NOTIFY_CHANGE_SIZE | \
2491+ FILE_NOTIFY_CHANGE_LAST_WRITE | \
2492+ FILE_NOTIFY_CHANGE_SECURITY | \
2493+ FILE_NOTIFY_CHANGE_LAST_ACCESS
2494+
2495+
2496+@is_valid_syncdaemon_path()
2497+def path_is_ignored(path):
2498+ """Should we ignore this path in the current platform.?"""
2499+ # don't support links yet
2500+ if path.endswith('.lnk'):
2501+ return True
2502+ return False
2503+
2504+
2505+class Watch(object):
2506 """Implement the same functions as pyinotify.Watch."""
2507
2508- def __init__(self, watch_descriptor, path, mask, auto_add, processor,
2509- buf_size=8192):
2510- super(Watch, self).__init__(watch_descriptor, path, mask, auto_add,
2511- processor, buf_size)
2512+ def __init__(self, path, process_events, mask=FILESYSTEM_MONITOR_MASK,
2513+ buf_size=8192):
2514+ self.path = os.path.abspath(path)
2515+ self.process_events = process_events
2516+ self.watching = False
2517+ self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.windows.' +
2518+ 'filesystem_notifications.Watch')
2519+ self.log.setLevel(logging.DEBUG)
2520+ self._buf_size = buf_size
2521+ self._mask = mask
2522+ self.ignore_paths = []
2523+ self._watch_handle = None
2524+
2525 self._wait_stop = CreateEvent(None, 0, 0, None)
2526 self._overlapped = OVERLAPPED()
2527 self._overlapped.hEvent = CreateEvent(None, 0, 0, None)
2528- # remember the subdirs we have so that when we have a delete we can
2529- # check if it was a remove
2530- self._subdirs = set()
2531 # this deferred is fired when the watch has started monitoring
2532 # a directory from a thread
2533 self._watch_started_deferred = defer.Deferred()
2534@@ -118,22 +149,13 @@
2535
2536 def _process_events(self, events):
2537 """Process the events from the queue."""
2538- # do not do it if we stop watching and the events are empty
2539- if not self._watching:
2540- return
2541-
2542 # we transform the events to be the same as the one in pyinotify
2543 # and then use the proc_fun
2544 for action, file_name in events:
2545- if any([file_name.startswith(path)
2546- for path in self._ignore_paths]):
2547- continue
2548- # map the windows events to the pyinotify ones, tis is dirty but
2549- # makes the multiplatform better, linux was first :P
2550 syncdaemon_path = get_syncdaemon_valid_path(
2551- os.path.join(self._path, file_name))
2552- self._process_events_from_filesystem(action, file_name,
2553- str(uuid4()), syncdaemon_path)
2554+ os.path.join(self.path, file_name))
2555+ self.process_events(action, file_name, str(uuid4()),
2556+ syncdaemon_path)
2557
2558 def _call_deferred(self, f, *args):
2559 """Executes the deferred call avoiding possible race conditions."""
2560@@ -151,12 +173,12 @@
2561 def _watch(self):
2562 """Watch a path that is a directory."""
2563 self.log.debug('Adding watch for %r (exists? %r is dir? %r).',
2564- self._path,
2565- os.path.exists(self._path), os.path.isdir(self._path))
2566+ self.path,
2567+ os.path.exists(self.path), os.path.isdir(self.path))
2568 # we are going to be using the ReadDirectoryChangesW whihc requires
2569 # a directory handle and the mask to be used.
2570 self._watch_handle = CreateFileW(
2571- self._path,
2572+ self.path,
2573 FILE_LIST_DIRECTORY,
2574 FILE_SHARE_READ | FILE_SHARE_WRITE,
2575 None,
2576@@ -184,7 +206,7 @@
2577 ReadDirectoryChangesW(
2578 handle,
2579 buf,
2580- self._auto_add,
2581+ True, # Always watch children
2582 self._mask,
2583 self._overlapped,
2584 )
2585@@ -204,20 +226,20 @@
2586 # lets ead the data and store it in the results
2587 events = FILE_NOTIFY_INFORMATION(buf, data)
2588 self.log.debug('Got from ReadDirectoryChangesW %r.',
2589- [(common.COMMON_ACTIONS_NAMES[action], path) \
2590+ [(ACTIONS_NAMES[action], path) \
2591 for action, path in events])
2592 reactor.callFromThread(self._process_events, events)
2593
2594 def start_watching(self):
2595 """Tell the watch to start processing events."""
2596- super(Watch, self).start_watching()
2597+ self.watching = True
2598 reactor.callInThread(self._watch_wrapper)
2599 return self._watch_started_deferred
2600
2601 def stop_watching(self):
2602 """Tell the watch to stop processing events."""
2603- super(Watch, self).stop_watching()
2604 SetEvent(self._wait_stop)
2605+ self.watching = False
2606 return self.stopped
2607
2608 @property
2609@@ -231,82 +253,39 @@
2610 return self._watch_stopped_deferred
2611
2612
2613-class WatchManager(common.WatchManager):
2614-
2615- @common.is_valid_os_path(path_indexes=[1])
2616- def add_watch(self, path, mask, auto_add=False, quiet=True):
2617- """Add a new path to be watch.
2618-
2619- The method will ensure that the path is not already present.
2620- """
2621- if not isinstance(path, unicode):
2622- e = NotImplementedError("No implementation on this platform.")
2623- return defer.fail(e)
2624- wd = self.get_wd(path)
2625- if wd is None:
2626- self.log.debug('Adding single watch on %r', path)
2627- return self._add_single_watch(path, mask, auto_add, quiet)
2628- else:
2629- self.log.debug('Watch already exists on %r', path)
2630- return self._wdm[wd].started
2631+class WatchManager(object):
2632+ """Implement the same functions as pyinotify.WatchManager.
2633+
2634+ All paths passed to methods in this class should be darwin paths.
2635+
2636+ """
2637+
2638+ def __init__(self, log):
2639+ """Create a new instance."""
2640+ self.log = log
2641+ self._wd_count = 0
2642+
2643+ def del_watch(self, wd):
2644+ """Delete the watch with the given descriptor."""
2645+
2646+ def stop_watch(self, watch):
2647+ """Stop a watch."""
2648+ # decrease the number of watches
2649+ self._wd_count -= 1
2650+ return watch.stop_watching()
2651
2652 def stop(self):
2653 """Close the manager and stop all watches."""
2654 self.log.debug('Stopping watches.')
2655- wait_list = []
2656- for current_wd in self._wdm:
2657- wait_list.append(self._wdm[current_wd].stop_watching())
2658- self.log.debug('Stopping Watch on %r.', self._wdm[current_wd].path)
2659- return defer.DeferredList(wait_list)
2660-
2661- def _adding_watch(self, path, mask, auto_add):
2662+ return defer.succeed(True)
2663+
2664+ def rm_watch(self, watch):
2665+ """Stop the Watch."""
2666+ self._wd_count -= 1
2667+ return defer.succeed(True)
2668+
2669+ def add_watch(self, watch):
2670 # adjust the number of threads based on the UDFs watched
2671- watch = Watch(self._wd_count, path, mask, auto_add, self._processor)
2672+ self._wd_count += 1
2673 reactor.suggestThreadPoolSize(THREADPOOL_MAX + self._wd_count + 1)
2674- self._wdm[self._wd_count] = watch
2675- d = self._wdm[self._wd_count].start_watching()
2676- self._wd_count += 1
2677- return d
2678-
2679-
2680-class NotifyProcessor(common.NotifyProcessor):
2681- """
2682- Processor that takes care of dealing with the events.
2683-
2684- Also, they must not be literal paths, that is the \\?\ prefix should not be
2685- in the path.
2686- """
2687-
2688- @is_valid_syncdaemon_path(path_indexes=[1])
2689- def platform_is_ignored(self, path):
2690- """Should we ignore this path in the current platform.?"""
2691- # don't support links yet
2692- if path.endswith('.lnk'):
2693- return True
2694- return False
2695-
2696-
2697-class FilesystemMonitor(common.FilesystemMonitor):
2698- """Manages the signals from filesystem."""
2699-
2700- def __init__(self, eq, fs, ignore_config=None, timeout=1):
2701- super(FilesystemMonitor, self).__init__(eq, fs, ignore_config, timeout)
2702- self._processor = NotifyProcessor(self, ignore_config)
2703- self._watch_manager = WatchManager(self._processor)
2704- # the default mask to be used in the watches
2705- # added by the FilesystemMonitor class
2706- self.filesystem_monitor_mask = FILE_NOTIFY_CHANGE_FILE_NAME | \
2707- FILE_NOTIFY_CHANGE_DIR_NAME | \
2708- FILE_NOTIFY_CHANGE_ATTRIBUTES | \
2709- FILE_NOTIFY_CHANGE_SIZE | \
2710- FILE_NOTIFY_CHANGE_LAST_WRITE | \
2711- FILE_NOTIFY_CHANGE_SECURITY | \
2712- FILE_NOTIFY_CHANGE_LAST_ACCESS
2713-
2714- def add_watches_to_udf_ancestors(self, volume):
2715- """Add a inotify watch to volume's ancestors if it's an UDF."""
2716- # There is no need to add the watches to the ancestors
2717- # so we will always return true. The reason for this is that the
2718- # handles that we open stop the user from renaming the ancestors of
2719- # the UDF, for a user to do that he has to unsync the udf first
2720- return defer.succeed(True)
2721+ return True
2722
2723=== modified file 'ubuntuone/platform/os_helper/darwin.py'
2724--- ubuntuone/platform/os_helper/darwin.py 2012-06-27 13:51:09 +0000
2725+++ ubuntuone/platform/os_helper/darwin.py 2012-07-10 21:28:20 +0000
2726@@ -127,8 +127,13 @@
2727
2728
2729 def os_path(path_indexes=None):
2730+ if path_indexes is None:
2731+ path_indexes = [0]
2732+
2733 def decorator(func):
2734 def wrapped(*args, **kwargs):
2735+ for i in path_indexes:
2736+ assert isinstance(args[i], str), 'Path %r should be str.'
2737 return func(*args, **kwargs)
2738 return wrapped
2739 return decorator

Subscribers

People subscribed via source and target branches