Merge lp:~mandel/ubuntuone-client/clean-fsevents into lp:ubuntuone-client

Proposed by Manuel de la Peña on 2012-07-10
Status: Merged
Approved by: Manuel de la Peña on 2012-07-11
Approved revision: 1298
Merged at revision: 1273
Proposed branch: lp:~mandel/ubuntuone-client/clean-fsevents
Merge into: lp:ubuntuone-client
Diff against target: 2027 lines (+438/-535)
7 files modified
tests/platform/filesystem_notifications/test_darwin.py (+49/-81)
tests/platform/filesystem_notifications/test_windows.py (+70/-143)
ubuntuone/platform/filesystem_notifications/__init__.py (+4/-8)
ubuntuone/platform/filesystem_notifications/common.py (+140/-105)
ubuntuone/platform/filesystem_notifications/darwin.py (+74/-81)
ubuntuone/platform/filesystem_notifications/windows.py (+96/-117)
ubuntuone/platform/os_helper/darwin.py (+5/-0)
To merge this branch: bzr merge lp:~mandel/ubuntuone-client/clean-fsevents
Reviewer Review Type Date Requested Status
Mike McCracken (community) Approve on 2012-07-11
Diego Sarmentero (community) 2012-07-10 Approve on 2012-07-10
Review via email: mp+114252@code.launchpad.net

Commit Message

Refactor that simplifies the way the diff implementations per platform are done so that in future branches is simpler to add the daemon code.

Description of the Change

Refactor that simplifies the way the diff implementations per platform are done so that in future branches is simpler to add the daemon code.

Please run tests on windows/linux and mac.

To post a comment you must log in.
Diego Sarmentero (diegosarmentero) wrote :

+1

review: Approve
Mike McCracken (mikemc) wrote :

really minor stuff:

- filesystem_notifications/windows.py - the docstring refers to 'darwin paths'.

- filesystem_notifications/darwin.py - WatchManager.add_watch doesn't return a value, but in common.py, it's expecting a return. the version in windows.py returns boolean.
-- I looked around and this return value seems like it's never actually used, just passed around a bit and eventually ignored. Still, it's probably worth being consistent.

review: Needs Fixing
Manuel de la Peña (mandel) wrote :

> really minor stuff:
>
> - filesystem_notifications/windows.py - the docstring refers to 'darwin
> paths'.
>

Left over from moving things around. Fixed!

> - filesystem_notifications/darwin.py - WatchManager.add_watch doesn't return a
> value, but in common.py, it's expecting a return. the version in windows.py
> returns boolean.
> -- I looked around and this return value seems like it's never actually used,
> just passed around a bit and eventually ignored. Still, it's probably worth
> being consistent.

Indeed, I made the changes to make sure both return True.

Mike McCracken (mikemc) :
review: Approve
Ubuntu One Auto Pilot (otto-pilot) wrote :
Download full text (24.9 KiB)

The attempt to merge lp:~mandel/ubuntuone-client/clean-fsevents into lp:ubuntuone-client failed. Below is the output from the failed tests.

/usr/bin/gnome-autogen.sh
checking for autoconf >= 2.53...
  testing autoconf2.50... not found.
  testing autoconf... found 2.69
checking for automake >= 1.10...
  testing automake-1.11... found 1.11.5
checking for libtool >= 1.5...
  testing libtoolize... found 2.4.2
checking for intltool >= 0.30...
  testing intltoolize... found 0.50.2
checking for pkg-config >= 0.14.0...
  testing pkg-config... found 0.26
checking for gtk-doc >= 1.0...
  testing gtkdocize... found 1.18
Checking for required M4 macros...
Checking for forbidden M4 macros...
Processing ./configure.ac
Running libtoolize...
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
Running intltoolize...
Running gtkdocize...
Running aclocal-1.11...
Running autoconf...
Running autoheader...
Running automake-1.11...
Running ./configure --enable-gtk-doc --enable-debug ...
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for library containing strerror... none required
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
che...

Ubuntu One Auto Pilot (otto-pilot) wrote :
Download full text (253.5 KiB)

The attempt to merge lp:~mandel/ubuntuone-client/clean-fsevents into lp:ubuntuone-client failed. Below is the output from the failed tests.

/usr/bin/gnome-autogen.sh
checking for autoconf >= 2.53...
  testing autoconf2.50... not found.
  testing autoconf... found 2.69
checking for automake >= 1.10...
  testing automake-1.11... found 1.11.5
checking for libtool >= 1.5...
  testing libtoolize... found 2.4.2
checking for intltool >= 0.30...
  testing intltoolize... found 0.50.2
checking for pkg-config >= 0.14.0...
  testing pkg-config... found 0.26
checking for gtk-doc >= 1.0...
  testing gtkdocize... found 1.18
Checking for required M4 macros...
Checking for forbidden M4 macros...
Processing ./configure.ac
Running libtoolize...
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
Running intltoolize...
Running gtkdocize...
Running aclocal-1.11...
Running autoconf...
Running autoheader...
Running automake-1.11...
Running ./configure --enable-gtk-doc --enable-debug ...
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for library containing strerror... none required
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
che...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'tests/platform/filesystem_notifications/test_darwin.py'
2--- tests/platform/filesystem_notifications/test_darwin.py 2012-06-28 13:20:01 +0000
3+++ tests/platform/filesystem_notifications/test_darwin.py 2012-07-11 15:52:18 +0000
4@@ -32,7 +32,6 @@
5 import os
6 import tempfile
7 import thread
8-import time
9
10 from twisted.internet import defer
11
12@@ -41,7 +40,7 @@
13 from ubuntuone.platform.filesystem_notifications import (
14 darwin as filesystem_notifications,
15 )
16-from ubuntuone.platform.filesystem_notifications.darwin import (
17+from ubuntuone.platform.filesystem_notifications.common import (
18 Watch,
19 WatchManager,
20 )
21@@ -54,7 +53,7 @@
22
23 # A reverse mapping for the tests
24 REVERSE_MACOS_ACTIONS = {}
25-for key, value in filesystem_notifications.common.COMMON_ACTIONS.iteritems():
26+for key, value in filesystem_notifications.ACTIONS.iteritems():
27 REVERSE_MACOS_ACTIONS[value] = key
28
29
30@@ -148,16 +147,15 @@
31 self.paths_checked.append((path, result))
32 return result
33
34- self.patch(filesystem_notifications.Watch, '_path_is_dir',
35+ self.patch(Watch, '_path_is_dir',
36 path_is_dir_wrapper)
37
38 @defer.inlineCallbacks
39- def _perform_operations(self, path, mask, auto_add, actions,
40- number_events):
41+ def _perform_operations(self, path, mask, actions, number_events):
42 """Perform the file operations and returns the recorded events."""
43 handler = TestCaseHandler(number_events=number_events)
44 manager = WatchManager(handler)
45- yield manager.add_watch(path, mask, auto_add=auto_add)
46+ yield manager.add_watch(path, mask)
47 # change the logger so that we can check the logs if we wanted
48 manager._wdm[0].log.addHandler(self.memento)
49 # clean logger later
50@@ -169,22 +167,6 @@
51 self.addCleanup(manager.stop)
52 defer.returnValue(ret)
53
54- def _perform_timed_operations(self, path, mask, auto_add, actions,
55- time_out):
56- """Perform the file operations and returns the recorded events."""
57- manager = WatchManager()
58- manager.add_watch(path, mask, auto_add=auto_add)
59- # change the logger so that we can check the logs if we wanted
60- manager._wdm[0].log.addHandler(self.memento)
61- # clean logger later
62- self.addCleanup(manager._wdm[0].log.removeHandler, self.memento)
63- # execution the actions
64- actions()
65- # process the recorded events
66- time.sleep(time_out)
67- events = self.handler.processed_events
68- return events
69-
70 def _assert_logs(self, events):
71 """Assert the debug logs."""
72 logs = []
73@@ -208,7 +190,7 @@
74 os.fsync(fd)
75 fd.close()
76
77- events = yield self._perform_operations(self.basedir, self.mask, False,
78+ events = yield self._perform_operations(self.basedir, self.mask,
79 create_file, 1)
80 event = events[0]
81 self.assertFalse(event.dir)
82@@ -230,7 +212,7 @@
83 """Action for the test."""
84 os.mkdir(dir_name)
85
86- events = yield self._perform_operations(self.basedir, self.mask, False,
87+ events = yield self._perform_operations(self.basedir, self.mask,
88 create_dir, 1)
89 event = events[0]
90 self.assertTrue(event.dir)
91@@ -254,7 +236,7 @@
92 """Action for the test."""
93 os.remove(file_name)
94
95- events = yield self._perform_operations(self.basedir, self.mask, False,
96+ events = yield self._perform_operations(self.basedir, self.mask,
97 remove_file, 1)
98 event = events[0]
99 self.assertFalse(event.dir)
100@@ -278,7 +260,7 @@
101 """Action for the test."""
102 os.rmdir(dir_name)
103
104- events = yield self._perform_operations(self.basedir, self.mask, False,
105+ events = yield self._perform_operations(self.basedir, self.mask,
106 remove_dir, 1)
107 event = events[0]
108 self.assertTrue(event.dir)
109@@ -304,7 +286,7 @@
110 fd.write('test')
111 fd.close()
112
113- events = yield self._perform_operations(self.basedir, self.mask, False,
114+ events = yield self._perform_operations(self.basedir, self.mask,
115 write_file, 1)
116 event = events[0]
117 self.assertFalse(event.dir)
118@@ -332,7 +314,7 @@
119 os.rename(from_file_name, to_file_name)
120
121 events = yield self._perform_operations(self.basedir, self.mask,
122- False, move_file, 2)
123+ move_file, 2)
124 move_from_event = events[0]
125 move_to_event = events[1]
126 # first test the move from
127@@ -375,7 +357,7 @@
128
129 # We need to test that we get a delete operation when moving
130 # a file to an unwatched folder
131- events = yield self._perform_operations(self.basedir, self.mask, False,
132+ events = yield self._perform_operations(self.basedir, self.mask,
133 move_file, 1)
134 event = events[0]
135 self.assertFalse(event.dir)
136@@ -405,7 +387,7 @@
137
138 # We need to test that we get a delete operation when moving
139 # a file from an unwatched folder
140- events = yield self._perform_operations(self.basedir, self.mask, False,
141+ events = yield self._perform_operations(self.basedir, self.mask,
142 move_files, 1)
143 event = events[0]
144 self.assertFalse(event.dir)
145@@ -433,7 +415,7 @@
146 os.rename(from_dir_name, to_dir_name)
147
148 events = yield self._perform_operations(self.basedir,
149- self.mask, False, move_file, 2)
150+ self.mask, move_file, 2)
151 move_from_event = events[0]
152 move_to_event = events[1]
153 # first test the move from
154@@ -476,7 +458,7 @@
155
156 # We need to test that we get a delete operation when moving
157 # a file to an unwatched folder
158- events = yield self._perform_operations(self.basedir, self.mask, False,
159+ events = yield self._perform_operations(self.basedir, self.mask,
160 move_dir, 1)
161 event = events[0]
162 self.assertTrue(event.dir)
163@@ -502,7 +484,7 @@
164 """Action for the test."""
165 os.rename(from_dir_name, to_dir_name)
166
167- events = yield self._perform_operations(self.basedir, self.mask, False,
168+ events = yield self._perform_operations(self.basedir, self.mask,
169 move_dir, 1)
170 event = events[0]
171 self.assertTrue(event.dir)
172@@ -519,8 +501,8 @@
173 handler = TestCaseHandler(number_events=0)
174 manager = WatchManager(handler)
175 # add a watch that will always exclude all actions
176- manager.add_watch(self.basedir, self.mask, auto_add=True,
177- exclude_filter=lambda x: True)
178+ manager.add_watch(self.basedir, self.mask,
179+ exclude_filter=lambda x: True)
180 # execution the actions
181 file_name = os.path.join(self.basedir, 'test_file_create')
182 open(file_name, 'w').close()
183@@ -528,21 +510,6 @@
184 self.assertEqual(0, len(handler.processed_events))
185 test_exclude_filter.skip = "we must rethink this test."
186
187- def test_open_dir_muted(self):
188- """Test that the opening of dirs is ignored."""
189- dir_name = os.path.join(tempfile.mkdtemp(), 'test_dir_open')
190- # create file before we record
191- os.mkdir(dir_name)
192-
193- def open_dir():
194- """Action for the test."""
195- os.startfile(dir_name)
196-
197- events = self._perform_timed_operations(self.basedir, self.mask, False,
198- open_dir, 2)
199- self.assertEqual(0, len(events))
200- test_open_dir_muted.skip = "we must rethink this test."
201-
202 def test_ignore_path(self):
203 """Test that events from a path are ignored."""
204 events = []
205@@ -553,13 +520,13 @@
206
207 path = '/Users/username/folder'
208 child = 'child'
209- watch = Watch(1, path, None, True, fake_processor)
210+ watch = Watch(1, path, fake_processor)
211 watch.ignore_path(os.path.join(path, child))
212 # ensure that the watch is watching
213- watch._watching = True
214+ watch.platform_watch.watching = True
215 for file_name in 'abcdef':
216 event = FakeFileEvent(256, None, os.path.join(child, file_name))
217- watch._process_events(event)
218+ watch.platform_watch._process_events(event)
219 self.assertEqual(0, len(events),
220 'All events should have been ignored.')
221
222@@ -576,7 +543,7 @@
223
224 path = '/Users/username/folder'
225 child = 'child'
226- watch = Watch(1, path, None, True, fake_processor)
227+ watch = Watch(1, path, fake_processor)
228 watch.ignore_path(os.path.join(path, child))
229 paths_not_to_ignore = []
230 for file_name in 'abcdef':
231@@ -584,9 +551,9 @@
232 file_name))
233 paths_not_to_ignore.append(event)
234 # ensure that the watch is watching
235- watch._watching = True
236+ watch.platform_watch.watching = True
237 for event in paths_not_to_ignore:
238- watch._process_events(event)
239+ watch.platform_watch._process_events(event)
240 self.assertEqual(len(paths_not_to_ignore), len(events),
241 'No events should have been ignored.')
242
243@@ -603,7 +570,7 @@
244
245 child = 'child'
246 path = '/Users/username/folder'
247- watch = Watch(1, path, None, True, fake_processor)
248+ watch = Watch(1, path, fake_processor)
249 watch.ignore_path(os.path.join(path, child))
250 paths_not_to_ignore = []
251 paths_to_ignore = []
252@@ -615,9 +582,9 @@
253 paths_not_to_ignore.append(event)
254 expected_events.append(os.path.join(path, valid))
255 # ensure that the watch is watching
256- watch._watching = True
257+ watch.platform_watch.watching = True
258 for event in paths_not_to_ignore:
259- watch._process_events(event)
260+ watch.platform_watch._process_events(event)
261 self.assertEqual(len(paths_not_to_ignore), len(events),
262 'Wrong number of events ignored.')
263 self.assertTrue(all([event in expected_events for event in events]),
264@@ -636,7 +603,7 @@
265
266 path = '/Users/username/folder'
267 child = 'child'
268- watch = Watch(1, path, None, True, fake_processor)
269+ watch = Watch(1, path, fake_processor)
270 watch.ignore_path(os.path.join(path, child))
271 watch.remove_ignored_path(os.path.join(path, child))
272 paths_not_to_ignore = []
273@@ -644,9 +611,9 @@
274 event = FakeFileEvent(256, None, os.path.join(child, file_name))
275 paths_not_to_ignore.append(event)
276 # ensure that the watch is watching
277- watch._watching = True
278+ watch.platform_watch.watching = True
279 for event in paths_not_to_ignore:
280- watch._process_events(event)
281+ watch.platform_watch._process_events(event)
282 self.assertEqual(len(paths_not_to_ignore), len(events),
283 'All events should have been accepted.')
284
285@@ -664,7 +631,7 @@
286 path = '/Users/username/folder'
287 child_a = 'childa'
288 child_b = 'childb'
289- watch = Watch(1, path, None, True, fake_processor)
290+ watch = Watch(1, path, fake_processor)
291 watch.ignore_path(os.path.join(path, child_a))
292 watch.ignore_path(os.path.join(path, child_b))
293 watch.remove_ignored_path(os.path.join(path, child_a))
294@@ -678,9 +645,9 @@
295 paths_not_to_ignore.append(event)
296 expected_events.append(os.path.join(path, valid))
297 # ensure that the watch is watching
298- watch._watching = True
299+ watch.platform_watch.watching = True
300 for event in paths_not_to_ignore:
301- watch._process_events(event)
302+ watch.platform_watch._process_events(event)
303 self.assertEqual(len(paths_not_to_ignore), len(events),
304 'All events should have been accepted.')
305 self.assertTrue(all([event in expected_events for event in events]),
306@@ -691,17 +658,18 @@
307 def fake_call(*args, **kwargs):
308 """Fake call."""
309
310- path = '/Users/username/folder'
311- watch = Watch(1, path, None, True, None)
312- self.assertEqual(watch._process_events, watch.stream.callback)
313- self.assertEqual(watch.stream.paths, [path])
314- self.assertEqual(watch.stream.file_events, True)
315+ path = '/Users/username/folder/'
316+ watch = Watch(1, path, None)
317+ self.assertEqual(watch.platform_watch._process_events,
318+ watch.platform_watch.stream.callback)
319+ self.assertEqual(watch.platform_watch.stream.paths, [path])
320+ self.assertEqual(watch.platform_watch.stream.file_events, True)
321
322 def test_watching_property(self):
323 """Test that the stopped property returns the stopped deferred."""
324 path = '/Users/username/folder'
325- watch = Watch(1, path, None, True, None)
326- self.assertFalse(watch._watching)
327+ watch = Watch(1, path, None)
328+ self.assertFalse(watch.watching)
329
330 def random_error(self, *args):
331 """Throw a fake exception."""
332@@ -712,7 +680,7 @@
333 path = '/Users/username/path/to/not/dir'
334 test_path = self.mktemp("test_directory")
335 self.patch(os.path, 'exists', lambda path: False)
336- watch = Watch(1, test_path, self.mask, True, None)
337+ watch = Watch(1, test_path, None)
338 self.assertFalse(watch._path_is_dir(path))
339
340 def test_is_path_dir_missing_in_subdir(self):
341@@ -720,7 +688,7 @@
342 path = '/Users/username/path/to/not/dir'
343 test_path = self.mktemp("test_directory")
344 self.patch(os.path, 'exists', lambda path: False)
345- watch = Watch(1, test_path, self.mask, True, None)
346+ watch = Watch(1, test_path, None)
347 watch._subdirs.add(path)
348 self.assertTrue(watch._path_is_dir(path))
349
350@@ -730,7 +698,7 @@
351 test_path = self.mktemp("test_directory")
352 self.patch(os.path, 'exists', lambda path: True)
353 self.patch(os.path, 'isdir', lambda path: True)
354- watch = Watch(1, test_path, self.mask, True, None)
355+ watch = Watch(1, test_path, None)
356 watch._subdirs.add(path)
357 self.assertTrue(watch._path_is_dir(path))
358
359@@ -740,7 +708,7 @@
360 test_path = self.mktemp("test_directory")
361 self.patch(os.path, 'exists', lambda path: True)
362 self.patch(os.path, 'isdir', lambda path: False)
363- watch = Watch(1, test_path, self.mask, True, None)
364+ watch = Watch(1, test_path, None)
365 watch._subdirs.add(path)
366 self.assertFalse(watch._path_is_dir(path))
367
368@@ -748,7 +716,7 @@
369 """Test when we update on a create event and not present."""
370 path = '/Users/username/path/to/not/dir'
371 test_path = self.mktemp("test_directory")
372- watch = Watch(1, test_path, self.mask, True, None)
373+ watch = Watch(1, test_path, None)
374 watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_CREATE])
375 self.assertTrue(path in watch._subdirs)
376
377@@ -756,7 +724,7 @@
378 """Test when we update on a create event and is present."""
379 path = '/Users/username/path/to/not/dir'
380 test_path = self.mktemp("test_directory")
381- watch = Watch(1, test_path, self.mask, True, None)
382+ watch = Watch(1, test_path, None)
383 watch._subdirs.add(path)
384 old_length = len(watch._subdirs)
385 watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_CREATE])
386@@ -767,7 +735,7 @@
387 """Test when we delete and is not present."""
388 path = '/Users/username/path/to/not/dir'
389 test_path = self.mktemp("test_directory")
390- watch = Watch(1, test_path, self.mask, True, None)
391+ watch = Watch(1, test_path, None)
392 watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_DELETE])
393 self.assertTrue(path not in watch._subdirs)
394
395@@ -775,7 +743,7 @@
396 """Test when we delete and is present."""
397 path = '/Users/username/path/to/not/dir'
398 test_path = self.mktemp("test_directory")
399- watch = Watch(1, test_path, self.mask, True, None)
400+ watch = Watch(1, test_path, None)
401 watch._subdirs.add(path)
402 watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_DELETE])
403 self.assertTrue(path not in watch._subdirs)
404
405=== modified file 'tests/platform/filesystem_notifications/test_windows.py'
406--- tests/platform/filesystem_notifications/test_windows.py 2012-06-22 14:17:44 +0000
407+++ tests/platform/filesystem_notifications/test_windows.py 2012-07-11 15:52:18 +0000
408@@ -51,14 +51,16 @@
409 IN_OPEN,
410 )
411 from ubuntuone.platform.filesystem_notifications import (
412- common,
413 windows as filesystem_notifications,
414 )
415-from ubuntuone.platform.filesystem_notifications.windows import (
416+from ubuntuone.platform.filesystem_notifications.common import (
417 FilesystemMonitor,
418 NotifyProcessor,
419 Watch,
420 WatchManager,
421+)
422+from ubuntuone.platform.filesystem_notifications.windows import (
423+ ACTIONS,
424 FILE_NOTIFY_CHANGE_FILE_NAME,
425 FILE_NOTIFY_CHANGE_DIR_NAME,
426 FILE_NOTIFY_CHANGE_ATTRIBUTES,
427@@ -70,7 +72,7 @@
428
429 #create a rever mapping to use it in the tests.
430 REVERSE_WINDOWS_ACTIONS = {}
431-for key, value in common.COMMON_ACTIONS.iteritems():
432+for key, value in ACTIONS.iteritems():
433 REVERSE_WINDOWS_ACTIONS[value] = key
434
435
436@@ -147,7 +149,7 @@
437 # group all events in a single lists which is not what the COM API
438 # does.
439 str_events = [
440- (common.COMMON_ACTIONS_NAMES[action], path) for action, path in
441+ (filesystem_notifications.ACTIONS_NAMES[action], path) for action, path in
442 events]
443 self.raw_events.append(str_events)
444 return events
445@@ -160,17 +162,14 @@
446
447 self.patch(filesystem_notifications, 'FILE_NOTIFY_INFORMATION',
448 file_notify_information_wrapper)
449- self.patch(filesystem_notifications.Watch, '_path_is_dir',
450- path_is_dir_wrapper)
451+ self.patch(Watch, '_path_is_dir', path_is_dir_wrapper)
452
453 @defer.inlineCallbacks
454- def _perform_operations(self, path, mask, auto_add, actions,
455- number_events):
456+ def _perform_operations(self, path, mask, actions, number_events):
457 """Perform the file operations and returns the recorded events."""
458 handler = TestCaseHandler(number_events=number_events)
459 manager = WatchManager(handler)
460- yield manager.add_watch(os_helper.get_windows_valid_path(path), mask,
461- auto_add=auto_add)
462+ yield manager.add_watch(os_helper.get_windows_valid_path(path), mask)
463 # change the logger so that we can check the logs if we wanted
464 manager._wdm[0].log.addHandler(self.memento)
465 # clean logger later
466@@ -182,35 +181,6 @@
467 self.addCleanup(manager.stop)
468 defer.returnValue(ret)
469
470- def _perform_timed_operations(self, path, mask, auto_add, actions,
471- time_out):
472- """Perform the file operations and returns the recorded events."""
473- manager = WatchManager()
474- manager.add_watch(os_helper.get_windows_valid_path(path), mask,
475- auto_add=auto_add)
476- # change the logger so that we can check the logs if we wanted
477- manager._wdm[0].log.addHandler(self.memento)
478- # clean logger later
479- self.addCleanup(manager._wdm[0].log.removeHandler, self.memento)
480- # execution the actions
481- actions()
482- # process the recorded events
483- time.sleep(time_out)
484- events = self.handler.processed_events
485- return events
486-
487- def _assert_logs(self, events):
488- """Assert the debug logs."""
489- logs = []
490- msg = 'Is path %r a dir? %s'
491- logs.extend([msg % data for data in self.paths_checked])
492- msg = 'Got from ReadDirectoryChangesW %r.'
493- logs.extend([msg % actions for actions in self.raw_events])
494- msg = 'Pushing event %r to processor.'
495- logs.extend([msg % e for e in events])
496- for msg in logs:
497- self.assertTrue(self.memento.check_debug(msg))
498-
499 @defer.inlineCallbacks
500 def test_file_create(self):
501 """Test that the correct event is returned on a file create."""
502@@ -224,7 +194,7 @@
503 os.fsync(fd)
504 fd.close()
505
506- events = yield self._perform_operations(self.basedir, self.mask, False,
507+ events = yield self._perform_operations(self.basedir, self.mask,
508 create_file, 1)
509 event = events[0]
510 self.assertFalse(event.dir)
511@@ -234,8 +204,6 @@
512 self.assertEqual('.', event.path)
513 self.assertEqual(os.path.join(self.basedir, file_name), event.pathname)
514 self.assertEqual(0, event.wd)
515- # assert the logging
516- self._assert_logs(events)
517
518 @defer.inlineCallbacks
519 def test_dir_create(self):
520@@ -246,7 +214,7 @@
521 """Action for the test."""
522 os.mkdir(dir_name)
523
524- events = yield self._perform_operations(self.basedir, self.mask, False,
525+ events = yield self._perform_operations(self.basedir, self.mask,
526 create_dir, 1)
527 event = events[0]
528 self.assertTrue(event.dir)
529@@ -256,8 +224,6 @@
530 self.assertEqual('.', event.path)
531 self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname)
532 self.assertEqual(0, event.wd)
533- # assert the logging
534- self._assert_logs(events)
535
536 @defer.inlineCallbacks
537 def test_file_remove(self):
538@@ -270,7 +236,7 @@
539 """Action for the test."""
540 os.remove(file_name)
541
542- events = yield self._perform_operations(self.basedir, self.mask, False,
543+ events = yield self._perform_operations(self.basedir, self.mask,
544 remove_file, 1)
545 event = events[0]
546 self.assertFalse(event.dir)
547@@ -280,8 +246,6 @@
548 self.assertEqual('.', event.path)
549 self.assertEqual(os.path.join(self.basedir, file_name), event.pathname)
550 self.assertEqual(0, event.wd)
551- # assert the logging
552- self._assert_logs(events)
553
554 @defer.inlineCallbacks
555 def test_dir_remove(self):
556@@ -294,7 +258,7 @@
557 """Action for the test."""
558 os.rmdir(dir_name)
559
560- events = yield self._perform_operations(self.basedir, self.mask, False,
561+ events = yield self._perform_operations(self.basedir, self.mask,
562 remove_dir, 1)
563 event = events[0]
564 self.assertTrue(event.dir)
565@@ -303,8 +267,6 @@
566 self.assertEqual('.', event.path)
567 self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname)
568 self.assertEqual(0, event.wd)
569- # assert the logging
570- self._assert_logs(events)
571
572 @defer.inlineCallbacks
573 def test_file_write(self):
574@@ -320,7 +282,7 @@
575 fd.write('test')
576 fd.close()
577
578- events = yield self._perform_operations(self.basedir, self.mask, False,
579+ events = yield self._perform_operations(self.basedir, self.mask,
580 write_file, 1)
581 event = events[0]
582 self.assertFalse(event.dir)
583@@ -330,8 +292,6 @@
584 self.assertEqual('.', event.path)
585 self.assertEqual(os.path.join(self.basedir, file_name), event.pathname)
586 self.assertEqual(0, event.wd)
587- # assert the logging
588- self._assert_logs(events)
589
590 @defer.inlineCallbacks
591 def test_file_moved_to_watched_dir_same_watcher(self):
592@@ -348,7 +308,7 @@
593 os.rename(from_file_name, to_file_name)
594
595 events = yield self._perform_operations(self.basedir, self.mask,
596- False, move_file, 2)
597+ move_file, 2)
598 move_from_event = events[0]
599 move_to_event = events[1]
600 # first test the move from
601@@ -374,8 +334,6 @@
602 self.assertEqual(0, move_to_event.wd)
603 # assert that both cookies are the same
604 self.assertEqual(move_from_event.cookie, move_to_event.cookie)
605- # assert the logging
606- self._assert_logs(events)
607
608 @defer.inlineCallbacks
609 def test_file_moved_to_not_watched_dir(self):
610@@ -392,7 +350,7 @@
611 # while on linux we will have to do some sort of magic like facundo
612 # did, on windows we will get a deleted event which is much more
613 # cleaner, first time ever windows is nicer!
614- events = yield self._perform_operations(self.basedir, self.mask, False,
615+ events = yield self._perform_operations(self.basedir, self.mask,
616 move_file, 1)
617 event = events[0]
618 self.assertFalse(event.dir)
619@@ -403,8 +361,6 @@
620 self.assertEqual(os.path.join(self.basedir, from_file_name),
621 event.pathname)
622 self.assertEqual(0, event.wd)
623- # assert the logging
624- self._assert_logs(events)
625
626 @defer.inlineCallbacks
627 def test_file_move_from_not_watched_dir(self):
628@@ -422,7 +378,7 @@
629
630 # while on linux we have to do some magic operations like facundo did
631 # on windows we have a created file, hurray!
632- events = yield self._perform_operations(self.basedir, self.mask, False,
633+ events = yield self._perform_operations(self.basedir, self.mask,
634 move_files, 1)
635 event = events[0]
636 self.assertFalse(event.dir)
637@@ -433,8 +389,6 @@
638 self.assertEqual(os.path.join(self.basedir, to_file_name),
639 event.pathname)
640 self.assertEqual(0, event.wd)
641- # assert the logging
642- self._assert_logs(events)
643
644 @defer.inlineCallbacks
645 def test_dir_moved_to_watched_dir_same_watcher(self):
646@@ -450,7 +404,7 @@
647 os.rename(from_dir_name, to_dir_name)
648
649 events = yield self._perform_operations(self.basedir,
650- self.mask, False, move_file, 2)
651+ self.mask, move_file, 2)
652 move_from_event = events[0]
653 move_to_event = events[1]
654 # first test the move from
655@@ -475,8 +429,6 @@
656 self.assertEqual(0, move_to_event.wd)
657 # assert that both cookies are the same
658 self.assertEqual(move_from_event.cookie, move_to_event.cookie)
659- # assert the logging
660- self._assert_logs(events)
661
662 @defer.inlineCallbacks
663 def test_dir_moved_to_not_watched_dir(self):
664@@ -491,7 +443,7 @@
665 'test_dir_moved_to_not_watched_dir'))
666
667 # on windows a move to outside a watched dir translates to a remove
668- events = yield self._perform_operations(self.basedir, self.mask, False,
669+ events = yield self._perform_operations(self.basedir, self.mask,
670 move_dir, 1)
671 event = events[0]
672 self.assertTrue(event.dir)
673@@ -500,8 +452,6 @@
674 self.assertEqual('.', event.path)
675 self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname)
676 self.assertEqual(0, event.wd)
677- # assert the logging
678- self._assert_logs(events)
679
680 @defer.inlineCallbacks
681 def test_dir_move_from_not_watched_dir(self):
682@@ -517,7 +467,7 @@
683 """Action for the test."""
684 os.rename(from_dir_name, to_dir_name)
685
686- events = yield self._perform_operations(self.basedir, self.mask, False,
687+ events = yield self._perform_operations(self.basedir, self.mask,
688 move_dir, 1)
689 event = events[0]
690 self.assertTrue(event.dir)
691@@ -535,8 +485,7 @@
692 manager = WatchManager(handler)
693 # add a watch that will always exclude all actions
694 manager.add_watch(os_helper.get_windows_valid_path(self.basedir),
695- self.mask, auto_add=True,
696- exclude_filter=lambda x: True)
697+ self.mask, exclude_filter=lambda x: True)
698 # execution the actions
699 file_name = os.path.join(self.basedir, 'test_file_create')
700 open(file_name, 'w').close()
701@@ -545,21 +494,6 @@
702 self.assertEqual(0, len(handler.processed_events))
703 test_exclude_filter.skip = "we must rethink this test."
704
705- def test_open_dir_muted(self):
706- """Test that the opening of dirs is ignored."""
707- dir_name = os.path.join(tempfile.mkdtemp(), 'test_dir_open')
708- # create file before we record
709- os.mkdir(dir_name)
710-
711- def open_dir():
712- """Action for the test."""
713- os.startfile(dir_name)
714-
715- events = self._perform_timed_operations(self.basedir, self.mask, False,
716- open_dir, 2)
717- self.assertEqual(0, len(events))
718- test_open_dir_muted.skip = "we must rethink this test."
719-
720 def test_ignore_path(self):
721 """Test that events from a path are ignored."""
722 events = []
723@@ -570,14 +504,14 @@
724
725 path = u'\\\\?\\C:\\path' # a valid windows path
726 child = u'child'
727- watch = Watch(1, path, None, True, fake_processor)
728+ watch = Watch(1, path, fake_processor)
729 watch.ignore_path(os.path.join(path, child))
730 paths_to_ignore = []
731 for file_name in 'abcdef':
732 paths_to_ignore.append((1, os.path.join(child, file_name)))
733 # ensure that the watch is watching
734 watch._watching = True
735- watch._process_events(paths_to_ignore)
736+ watch.platform_watch._process_events(paths_to_ignore)
737 self.assertEqual(0, len(events),
738 'All events should have been ignored.')
739
740@@ -591,15 +525,15 @@
741
742 path = u'\\\\?\\C:\\path' # a valid windows path
743 child = u'child'
744- watch = Watch(1, path, None, True, fake_processor)
745+ watch = Watch(1, path, fake_processor)
746 watch.ignore_path(os.path.join(path, child))
747 paths_not_to_ignore = []
748 for file_name in 'abcdef':
749 paths_not_to_ignore.append((1, os.path.join(
750 child + file_name, file_name)))
751 # ensure that the watch is watching
752- watch._watching = True
753- watch._process_events(paths_not_to_ignore)
754+ watch.platform_watch.watching = True
755+ watch.platform_watch._process_events(paths_not_to_ignore)
756 self.assertEqual(len(paths_not_to_ignore), len(events),
757 'No events should have been ignored.')
758
759@@ -613,7 +547,7 @@
760
761 child = u'child'
762 path = u'\\\\?\\C:\\path\\' # a valid windows path
763- watch = Watch(1, path, None, True, fake_processor)
764+ watch = Watch(1, path, fake_processor)
765 watch.ignore_path(os.path.join(path, child))
766 paths_not_to_ignore = []
767 paths_to_ignore = []
768@@ -624,8 +558,8 @@
769 paths_not_to_ignore.append((1, valid))
770 expected_events.append(os.path.join('C:\\path', valid))
771 # ensure that the watch is watching
772- watch._watching = True
773- watch._process_events(paths_not_to_ignore)
774+ watch.platform_watch.watching = True
775+ watch.platform_watch._process_events(paths_not_to_ignore)
776 self.assertEqual(len(paths_not_to_ignore), len(events),
777 'Wrong number of events ignored.')
778 self.assertTrue(all([event in expected_events for event in events]),
779@@ -641,15 +575,15 @@
780
781 path = u'\\\\?\\C:\\path' # a valid windows path
782 child = u'child'
783- watch = Watch(1, path, None, True, fake_processor)
784+ watch = Watch(1, path, fake_processor)
785 watch.ignore_path(os.path.join(path, child))
786 watch.remove_ignored_path(os.path.join(path, child))
787 paths_not_to_ignore = []
788 for file_name in 'abcdef':
789 paths_not_to_ignore.append((1, os.path.join(child, file_name)))
790 # ensure that the watch is watching
791- watch._watching = True
792- watch._process_events(paths_not_to_ignore)
793+ watch.platform_watch.watching = True
794+ watch.platform_watch._process_events(paths_not_to_ignore)
795 self.assertEqual(len(paths_not_to_ignore), len(events),
796 'All events should have been accepted.')
797
798@@ -664,7 +598,7 @@
799 path = u'\\\\?\\C:\\path' # a valid windows path
800 child_a = u'childa'
801 child_b = u'childb'
802- watch = Watch(1, path, None, True, fake_processor)
803+ watch = Watch(1, path, fake_processor)
804 watch.ignore_path(os.path.join(path, child_a))
805 watch.ignore_path(os.path.join(path, child_b))
806 watch.remove_ignored_path(os.path.join(path, child_a))
807@@ -677,8 +611,8 @@
808 paths_not_to_ignore.append((1, valid))
809 expected_events.append(os.path.join('C:\\path', valid))
810 # ensure that the watch is watching
811- watch._watching = True
812- watch._process_events(paths_not_to_ignore)
813+ watch.platform_watch.watching = True
814+ watch.platform_watch._process_events(paths_not_to_ignore)
815 self.assertEqual(len(paths_not_to_ignore), len(events),
816 'All events should have been accepted.')
817 self.assertTrue(all([event in expected_events for event in events]),
818@@ -694,9 +628,9 @@
819 method_args.append((args, kwargs),)
820
821 path = u'\\\\?\\C:\\path' # a valid windows path
822- watch = Watch(1, path, None, True, None)
823- yield watch._watch_started_deferred.callback(True)
824- watch._call_deferred(fake_call, None)
825+ watch = Watch(1, path, None)
826+ yield watch.platform_watch._watch_started_deferred.callback(True)
827+ watch.platform_watch._call_deferred(fake_call, None)
828 self.assertEqual(0, len(method_args))
829
830 def test_call_deferred_not_called(self):
831@@ -708,21 +642,21 @@
832 method_args.append((args, kwargs),)
833
834 path = u'\\\\?\\C:\\path' # a valid windows path
835- watch = Watch(1, path, None, True, None)
836- watch._call_deferred(fake_call, None)
837+ watch = Watch(1, path, None)
838+ watch.platform_watch._call_deferred(fake_call, None)
839 self.assertEqual(1, len(method_args))
840
841 def test_started_property(self):
842 """Test that the started property returns the started deferred."""
843 path = u'\\\\?\\C:\\path' # a valid windows path
844- watch = Watch(1, path, None, True, None)
845- self.assertEqual(watch.started, watch._watch_started_deferred)
846+ watch = Watch(1, path, None)
847+ self.assertEqual(watch.started, watch.platform_watch._watch_started_deferred)
848
849 def test_stopped_property(self):
850 """Test that the stopped property returns the stopped deferred."""
851 path = u'\\\\?\\C:\\path' # a valid windows path
852- watch = Watch(1, path, None, True, None)
853- self.assertEqual(watch.stopped, watch._watch_stopped_deferred)
854+ watch = Watch(1, path, None)
855+ self.assertEqual(watch.stopped, watch.platform_watch._watch_stopped_deferred)
856
857 def random_error(self, *args):
858 """Throw a fake exception."""
859@@ -733,7 +667,7 @@
860 """An early failure inside the thread should errback the deferred."""
861 test_path = self.mktemp("test_directory")
862 self.patch(filesystem_notifications, "CreateFileW", self.random_error)
863- watch = Watch(1, test_path, None, True, None)
864+ watch = Watch(1, test_path, None)
865 d = watch.start_watching()
866 yield self.assertFailure(d, FakeException)
867
868@@ -743,7 +677,7 @@
869 test_path = self.mktemp("test_directory")
870 self.patch(filesystem_notifications, "ReadDirectoryChangesW",
871 self.random_error)
872- watch = Watch(1, test_path, None, True, None)
873+ watch = Watch(1, test_path, None)
874 d = watch.start_watching()
875 yield self.assertFailure(d, FakeException)
876
877@@ -757,7 +691,7 @@
878 close_called.append)
879 self.patch(filesystem_notifications, "ReadDirectoryChangesW",
880 self.random_error)
881- watch = Watch(1, test_path, self.mask, True, None)
882+ watch = Watch(1, test_path, None)
883 d = watch.start_watching()
884 yield self.assertFailure(d, FakeException)
885 self.assertEqual(len(close_called), 1)
886@@ -767,18 +701,18 @@
887 def test_stop_watching_fired_when_watch_thread_finishes(self):
888 """The deferred returned is fired when the watch thread finishes."""
889 test_path = self.mktemp("another_test_directory")
890- watch = Watch(1, test_path, self.mask, True, None)
891+ watch = Watch(1, test_path, None)
892 yield watch.start_watching()
893- self.assertNotEqual(watch._watch_handle, None)
894+ self.assertNotEqual(watch.platform_watch._watch_handle, None)
895 yield watch.stop_watching()
896- self.assertEqual(watch._watch_handle, None)
897+ self.assertEqual(watch.platform_watch._watch_handle, None)
898
899 def test_is_path_dir_missing_no_subdir(self):
900 """Test when the path does not exist and is no a subdir."""
901 path = u'\\\\?\\C:\\path\\to\\no\\dir'
902 test_path = self.mktemp("test_directory")
903 self.patch(os.path, 'exists', lambda path: False)
904- watch = Watch(1, test_path, self.mask, True, None)
905+ watch = Watch(1, test_path, None)
906 self.assertFalse(watch._path_is_dir(path))
907
908 def test_is_path_dir_missing_in_subdir(self):
909@@ -786,7 +720,7 @@
910 path = u'\\\\?\\C:\\path\\to\\no\\dir'
911 test_path = self.mktemp("test_directory")
912 self.patch(os.path, 'exists', lambda path: False)
913- watch = Watch(1, test_path, self.mask, True, None)
914+ watch = Watch(1, test_path, None)
915 watch._subdirs.add(path)
916 self.assertTrue(watch._path_is_dir(path))
917
918@@ -796,7 +730,7 @@
919 test_path = self.mktemp("test_directory")
920 self.patch(os.path, 'exists', lambda path: True)
921 self.patch(os.path, 'isdir', lambda path: True)
922- watch = Watch(1, test_path, self.mask, True, None)
923+ watch = Watch(1, test_path, None)
924 watch._subdirs.add(path)
925 self.assertTrue(watch._path_is_dir(path))
926
927@@ -806,7 +740,7 @@
928 test_path = self.mktemp("test_directory")
929 self.patch(os.path, 'exists', lambda path: True)
930 self.patch(os.path, 'isdir', lambda path: False)
931- watch = Watch(1, test_path, self.mask, True, None)
932+ watch = Watch(1, test_path, None)
933 watch._subdirs.add(path)
934 self.assertFalse(watch._path_is_dir(path))
935
936@@ -814,7 +748,7 @@
937 """Test when we update on a create event and not present."""
938 path = u'\\\\?\\C:\\path\\to\\no\\dir'
939 test_path = self.mktemp("test_directory")
940- watch = Watch(1, test_path, self.mask, True, None)
941+ watch = Watch(1, test_path, None)
942 watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_CREATE])
943 self.assertTrue(path in watch._subdirs)
944
945@@ -822,7 +756,7 @@
946 """Test when we update on a create event and is present."""
947 path = u'\\\\?\\C:\\path\\to\\no\\dir'
948 test_path = self.mktemp("test_directory")
949- watch = Watch(1, test_path, self.mask, True, None)
950+ watch = Watch(1, test_path, None)
951 watch._subdirs.add(path)
952 old_length = len(watch._subdirs)
953 watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_CREATE])
954@@ -833,7 +767,7 @@
955 """Test when we delete and is not present."""
956 path = u'\\\\?\\C:\\path\\to\\no\\dir'
957 test_path = self.mktemp("test_directory")
958- watch = Watch(1, test_path, self.mask, True, None)
959+ watch = Watch(1, test_path, None)
960 watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_DELETE])
961 self.assertTrue(path not in watch._subdirs)
962
963@@ -841,7 +775,7 @@
964 """Test when we delete and is present."""
965 path = u'\\\\?\\C:\\path\\to\\no\\dir'
966 test_path = self.mktemp("test_directory")
967- watch = Watch(1, test_path, self.mask, True, None)
968+ watch = Watch(1, test_path, None)
969 watch._subdirs.add(path)
970 watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_DELETE])
971 self.assertTrue(path not in watch._subdirs)
972@@ -856,7 +790,7 @@
973 yield super(TestWatchManager, self).setUp()
974 self.parent_path = u'\\\\?\\C:\\' # a valid windows path
975 self.path = self.parent_path + u'path'
976- self.watch = Watch(1, self.path, None, True, None)
977+ self.watch = Watch(1, self.path, None)
978 self.manager = WatchManager(None)
979 self.manager._wdm = {1: self.watch}
980
981@@ -884,7 +818,7 @@
982
983 self.patch(Watch, "stop_watching", fake_stop_watching)
984 second_path = self.parent_path + u"second_path"
985- second_watch = Watch(2, second_path, None, True, None)
986+ second_watch = Watch(2, second_path, None)
987 self.manager._wdm[2] = second_watch
988 d = self.manager.stop()
989 self.assertFalse(d.called, "Not fired before all watches end")
990@@ -928,19 +862,10 @@
991 self.manager._wdm = {}
992
993 mask = 'bit_mask'
994- auto_add = True
995- self.manager.add_watch(self.path, mask, auto_add=auto_add)
996+ self.manager.add_watch(self.path, mask)
997 self.assertEqual(1, len(self.manager._wdm))
998 self.assertTrue(self.was_called, 'The watch start was not called.')
999- self.assertEqual(self.path + os.path.sep, self.manager._wdm[0]._path)
1000- self.assertEqual(mask, self.manager._wdm[0]._mask)
1001- self.assertEqual(auto_add, self.manager._wdm[0]._auto_add)
1002-
1003- def test_update_present_watch(self):
1004- """Test the update of a present watch."""
1005- mask = 'mask'
1006- self.assertRaises(NotImplementedError, self.manager.update_watch,
1007- 1, mask)
1008+ self.assertEqual(self.path + os.path.sep, self.manager._wdm[0].path)
1009
1010 def test_get_watch_present_wd(self):
1011 """Test that the correct path is returned."""
1012@@ -988,7 +913,8 @@
1013 self.manager.rm_path(self.path)
1014 self.assertEqual(self.watch, self.manager._wdm.get(1))
1015 self.watch._watching = True
1016- self.watch._process_events([(1, os.path.join(self.path, 'test'))])
1017+ self.watch.platform_watch._process_events(
1018+ [(1, os.path.join(self.path, 'test'))])
1019 self.assertEqual(0, len(events))
1020
1021 def test_rm_child_path(self):
1022@@ -1004,11 +930,12 @@
1023 self.manager.rm_path(child)
1024 self.assertEqual(self.watch, self.manager._wdm[1])
1025 # assert that the correct event is ignored
1026- self.watch._watching = True
1027- self.watch._process_events([(1, os.path.join('child', 'test'))])
1028+ self.watch.platform_watch.watching = True
1029+ self.watch.platform_watch._process_events(
1030+ [(1, os.path.join('child', 'test'))])
1031 self.assertEqual(0, len(events))
1032 # assert that other events are not ignored
1033- self.watch._process_events([(1, 'test')])
1034+ self.watch.platform_watch._process_events([(1, 'test')])
1035 self.assertEqual(1, len(events))
1036
1037
1038@@ -1160,9 +1087,9 @@
1039 """Test that we do indeed ignore the correct paths."""
1040 not_ignored = 'test'
1041 ignored = not_ignored + '.lnk'
1042- self.assertFalse(self.processor.platform_is_ignored(not_ignored),
1043+ self.assertFalse(filesystem_notifications.path_is_ignored(not_ignored),
1044 'Only links should be ignored.')
1045- self.assertTrue(self.processor.platform_is_ignored(ignored),
1046+ self.assertTrue(filesystem_notifications.path_is_ignored(ignored),
1047 'Links should be ignored.')
1048
1049 def test_is_ignored(self):
1050
1051=== modified file 'ubuntuone/platform/filesystem_notifications/__init__.py'
1052--- ubuntuone/platform/filesystem_notifications/__init__.py 2012-06-28 16:44:01 +0000
1053+++ ubuntuone/platform/filesystem_notifications/__init__.py 2012-07-11 15:52:18 +0000
1054@@ -31,14 +31,10 @@
1055 import sys
1056
1057
1058-if sys.platform == "win32":
1059- from ubuntuone.platform.filesystem_notifications import windows
1060- FilesystemMonitor = windows.FilesystemMonitor
1061- _GeneralINotifyProcessor = windows.NotifyProcessor
1062-elif sys.platform == "darwin":
1063- from ubuntuone.platform.filesystem_notifications import darwin
1064- FilesystemMonitor = darwin.FilesystemMonitor
1065- _GeneralINotifyProcessor = darwin.NotifyProcessor
1066+if sys.platform in ('darwin', 'win32'):
1067+ from ubuntuone.platform.filesystem_notifications import common
1068+ FilesystemMonitor = common.FilesystemMonitor
1069+ _GeneralINotifyProcessor = common.NotifyProcessor
1070 else:
1071 from ubuntuone.platform.filesystem_notifications import linux
1072 FilesystemMonitor = linux.FilesystemMonitor
1073
1074=== modified file 'ubuntuone/platform/filesystem_notifications/common.py'
1075--- ubuntuone/platform/filesystem_notifications/common.py 2012-06-28 16:44:01 +0000
1076+++ ubuntuone/platform/filesystem_notifications/common.py 2012-07-11 15:52:18 +0000
1077@@ -30,6 +30,7 @@
1078
1079 import logging
1080 import os
1081+import sys
1082
1083 from twisted.internet import defer
1084
1085@@ -44,9 +45,9 @@
1086 IN_IGNORED,
1087 IN_ISDIR,
1088 IN_DELETE,
1089- IN_MODIFY as in_modify,
1090 IN_MOVED_FROM,
1091- IN_MOVED_TO)
1092+ IN_MOVED_TO,
1093+)
1094
1095 from ubuntuone import logger
1096
1097@@ -56,16 +57,28 @@
1098 os_path,
1099 )
1100
1101+if sys.platform == 'darwin':
1102+ from ubuntuone.platform.filesystem_notifications import darwin
1103+ source = darwin
1104+elif sys.platform == 'win32':
1105+ from ubuntuone.platform.filesystem_notifications import windows
1106+ source = windows
1107+else:
1108+ raise ImportError('Not supported platform')
1109+
1110 # a map between the few events that we have on common platforms and those
1111 # found in pyinotify
1112-COMMON_ACTIONS = {}
1113+ACTIONS = source.ACTIONS
1114
1115 # a map of the actions to names so that we have better logs.
1116-COMMON_ACTIONS_NAMES = {}
1117-
1118-# We should have this here, because we use if from other modules that
1119-# share this, but we need to declare it this way yo avoid flakes issues.
1120-IN_MODIFY = in_modify
1121+ACTIONS_NAMES = source.ACTIONS_NAMES
1122+
1123+# ignore paths in the platform, mainly links atm
1124+path_is_ignored = source.path_is_ignored
1125+
1126+# the base class to be use for a platform
1127+PlatformWatch = source.Watch
1128+PlatformWatchManager = source.WatchManager
1129
1130 # translates quickly the event and it's is_dir state to our standard events
1131 NAME_TRANSLATIONS = {
1132@@ -89,42 +102,50 @@
1133 class Watch(object):
1134 """Implement the same functions as pyinotify.Watch."""
1135
1136- def __init__(self, watch_descriptor, path, mask, auto_add, processor,
1137- buf_size=8192):
1138- self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.common.' +
1139- 'filesystem_notifications.Watch')
1140- self.log.setLevel(TRACE)
1141+ def __init__(self, watch_descriptor, path, processor):
1142+ """Create a new watch."""
1143+
1144+ # do ensure that we provide a os.path.sep
1145+ if not path.endswith(os.path.sep):
1146+ path += os.path.sep
1147+ self.path = path
1148+ self.ignore_paths = []
1149 self._processor = processor
1150- self._buf_size = buf_size
1151- self._watching = False
1152 self._descriptor = watch_descriptor
1153- self._auto_add = auto_add
1154- self._ignore_paths = []
1155 self._cookie = None
1156 self._source_pathname = None
1157- self._process_thread = None
1158- self._watch_handle = None
1159 # remember the subdirs we have so that when we have a delete we can
1160 # check if it was a remove
1161 self._subdirs = set()
1162- # ensure that we work with an abspath and that we can deal with
1163- # long paths over 260 chars.
1164- if not path.endswith(os.path.sep):
1165- path += os.path.sep
1166- self._path = os.path.abspath(path)
1167- self._mask = mask
1168-
1169- def _process_events_from_filesystem(self, action, file_name, cookie,
1170- syncdaemon_path):
1171+
1172+ # platform watch used to deal with the platform details
1173+ self.platform_watch = PlatformWatch(self.path, self.process_events)
1174+
1175+ self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.common.' +
1176+ 'filesystem_notifications.Watch')
1177+ self.log.setLevel(TRACE)
1178+
1179+ def process_events(self, action, file_name, cookie, syncdaemon_path):
1180 """Process the events from the queue."""
1181+ # do not process events when the watch was stopped
1182+ if not self.platform_watch.watching:
1183+ return
1184+
1185+ # do not process those events that should be ignored
1186+ if any([file_name.startswith(path)
1187+ for path in self.ignore_paths]):
1188+ return
1189+
1190 # map the filesystem events to the pyinotify ones, tis is dirty but
1191 # makes the multiplatform better, linux was first :P
1192- full_dir_path = os.path.join(self._path, file_name)
1193+ full_dir_path = os.path.join(self.path, file_name)
1194 is_dir = self._path_is_dir(full_dir_path)
1195+
1196 if is_dir:
1197 # we need to update the list of subdirs that we have
1198 self._update_subdirs(full_dir_path, action)
1199- mask = COMMON_ACTIONS[action]
1200+
1201+ mask = ACTIONS[action]
1202 head, tail = os.path.split(file_name)
1203 if is_dir:
1204 mask |= IN_ISDIR
1205@@ -134,14 +155,14 @@
1206 'mask': mask,
1207 'name': tail,
1208 'path': '.'}
1209- # by the way in which the win api fires the events we know for
1210+ # by the way in which the api fires the events we know for
1211 # sure that no move events will be added in the wrong order, this
1212 # is kind of hacky, I dont like it too much
1213- if COMMON_ACTIONS[action] == IN_MOVED_FROM:
1214+ if ACTIONS[action] == IN_MOVED_FROM:
1215 self._cookie = cookie
1216 self._source_pathname = tail
1217 event_raw_data['cookie'] = self._cookie
1218- if COMMON_ACTIONS[action] == IN_MOVED_TO:
1219+ if ACTIONS[action] == IN_MOVED_TO:
1220 event_raw_data['src_pathname'] = self._source_pathname
1221 event_raw_data['cookie'] = self._cookie
1222 event = Event(event_raw_data)
1223@@ -161,9 +182,9 @@
1224 The given path is considered to be a path and therefore this
1225 will not be checked.
1226 """
1227- if COMMON_ACTIONS[event] == IN_CREATE:
1228+ if ACTIONS[event] == IN_CREATE:
1229 self._subdirs.add(path)
1230- elif COMMON_ACTIONS[event] == IN_DELETE and path in self._subdirs:
1231+ elif ACTIONS[event] == IN_DELETE and path in self._subdirs:
1232 self._subdirs.remove(path)
1233
1234 @is_valid_os_path(path_indexes=[1])
1235@@ -187,51 +208,53 @@
1236 """Add the path of the events to ignore."""
1237 if not path.endswith(os.path.sep):
1238 path += os.path.sep
1239- if path.startswith(self._path):
1240- path = path[len(self._path):]
1241- self._ignore_paths.append(path)
1242+ if path.startswith(self.path):
1243+ path = path[len(self.path):]
1244+ self.ignore_paths.append(path)
1245
1246 @is_valid_os_path(path_indexes=[1])
1247 def remove_ignored_path(self, path):
1248 """Reaccept path."""
1249 if not path.endswith(os.path.sep):
1250 path += os.path.sep
1251- if path.startswith(self._path):
1252- path = path[len(self._path):]
1253- if path in self._ignore_paths:
1254- self._ignore_paths.remove(path)
1255+ if path.startswith(self.path):
1256+ path = path[len(self.path):]
1257+ if path in self.ignore_paths:
1258+ self.ignore_paths.remove(path)
1259
1260+ @defer.inlineCallbacks
1261 def start_watching(self):
1262 """Tell the watch to start processing events."""
1263- for current_child in os.listdir(self._path):
1264- full_child_path = os.path.join(self._path, current_child)
1265+ for current_child in os.listdir(self.path):
1266+ full_child_path = os.path.join(self.path, current_child)
1267 if os.path.isdir(full_child_path):
1268 self._subdirs.add(full_child_path)
1269 # start to diff threads, one to watch the path, the other to
1270 # process the events.
1271 self.log.debug('Start watching path.')
1272- self._watching = True
1273+ yield self.platform_watch.start_watching()
1274
1275 def stop_watching(self):
1276 """Tell the watch to stop processing events."""
1277- self.log.info('Stop watching %s', self._path)
1278- self._watching = False
1279+ self.log.info('Stop watching %s', self.path)
1280+ self.platform_watch.watching = False
1281 self._subdirs = set()
1282-
1283- def update(self, mask, auto_add=False):
1284- """Update the info used by the watcher."""
1285- self.log.debug('update(%s, %s)', mask, auto_add)
1286- self._mask = mask
1287- self._auto_add = auto_add
1288-
1289- @property
1290- def path(self):
1291- """Return the patch watched."""
1292- return self._path
1293-
1294- @property
1295- def auto_add(self):
1296- return self._auto_add
1297+ return self.platform_watch.stop_watching()
1298+
1299+ @property
1300+ def watching(self):
1301+ """Return if we are watching."""
1302+ return self.platform_watch.watching
1303+
1304+ @property
1305+ def started(self):
1306+ """A deferred that will be called when the watch is running."""
1307+ return self.platform_watch.started
1308+
1309+ @property
1310+ def stopped(self):
1311+ """A deferred fired when the watch thread has finished."""
1312+ return self.platform_watch.stopped
1313
1314
1315 class WatchManager(object):
1316@@ -243,25 +266,48 @@
1317
1318 def __init__(self, processor):
1319 """Init the manager to keep trak of the different watches."""
1320- self._processor = processor
1321 self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.common.'
1322 + 'filesystem_notifications.WatchManager')
1323 self.log.setLevel(TRACE)
1324+ self._processor = processor
1325+ # use the platform manager to perform the actual actions
1326+ self.manager = PlatformWatchManager(self.log)
1327 self._wdm = {}
1328- self._wd_count = 0
1329 self._ignored_paths = []
1330
1331- def add_watch(self, path, mask, auto_add=False, quiet=True):
1332- """Add a new path to be watch.
1333+ @defer.inlineCallbacks
1334+ def _add_single_watch(self, path, mask, quiet=True):
1335+ """A just one watch."""
1336+ if path in self._ignored_paths:
1337+ # simply removed it from the filter
1338+ self._ignored_paths.remove(path)
1339+ return
1340+
1341+ # we need to add a new watch
1342+ self.log.debug('add_single_watch(%s, %s, %s)', path, mask, quiet)
1343+
1344+ # common code that will ensure that we keep track of the watches
1345+ watch = Watch(len(self._wdm), path, self._processor)
1346+ self._wdm[len(self._wdm)] = watch
1347+ yield watch.start_watching()
1348+
1349+ # trust that the platform watch manager to do the rest of the start
1350+ # operations
1351+ defer.returnValue(self.manager.add_watch(watch))
1352+
1353+ @is_valid_os_path(path_indexes=[1])
1354+ def add_watch(self, path, mask, quiet=True):
1355+ """Add a new path to be watched.
1356
1357 The method will ensure that the path is not already present.
1358 """
1359- raise NotImplementedError("Not implemented on this platform.")
1360-
1361- def stop(self):
1362- """Close the manager and stop all watches."""
1363- # Should be implemented for each platform
1364- raise NotImplementedError("Not implemented on this platform.")
1365+ wd = self.get_wd(path)
1366+ if wd is None:
1367+ self.log.debug('Adding single watch on %r', path)
1368+ return self._add_single_watch(path, mask, quiet)
1369+ else:
1370+ self.log.debug('Watch already exists on %r', path)
1371+ return self._wdm[wd].started
1372
1373 def get_watch(self, wd):
1374 """Return the watch with the given descriptor."""
1375@@ -274,30 +320,13 @@
1376 watch = self._wdm[wd]
1377 yield watch.stop_watching()
1378 del self._wdm[wd]
1379+ # trust that the platform watch manager will do the rest of the
1380+ # operations needed to delete a watch
1381+ self.manager.del_watch(watch)
1382 self.log.debug('Watch %s removed.', wd)
1383 except KeyError, e:
1384 logging.error(str(e))
1385
1386- def _add_single_watch(self, path, mask, auto_add=False, quiet=True):
1387- if path in self._ignored_paths:
1388- # simply removed it from the filter
1389- self._ignored_paths.remove(path)
1390- return
1391- # we need to add a new watch
1392- self.log.debug('add_single_watch(%s, %s, %s, %s)', path, mask,
1393- auto_add, quiet)
1394-
1395- return self._adding_watch(path, mask, auto_add)
1396-
1397- def _adding_watch(self, path, mask, auto_add):
1398- """Add the watch to the dict and start it."""
1399- # This should be implemented for each OS
1400- raise NotImplementedError("Not implemented on this platform.")
1401-
1402- def update_watch(self, wd, mask=None, rec=False,
1403- auto_add=False, quiet=True):
1404- raise NotImplementedError("Not implemented on this platform.")
1405-
1406 @is_valid_os_path(path_indexes=[1])
1407 def get_wd(self, path):
1408 """Return the watcher that is used to watch the given path."""
1409@@ -305,8 +334,7 @@
1410 path = path + os.path.sep
1411 for current_wd in self._wdm:
1412 watch_path = self._wdm[current_wd].path
1413- if ((watch_path == path or (
1414- watch_path in path and self._wdm[current_wd].auto_add))
1415+ if ((watch_path == path or watch_path in path)
1416 and path not in self._ignored_paths):
1417 return current_wd
1418
1419@@ -323,6 +351,9 @@
1420 watch = self._wdm[wd]
1421 yield watch.stop_watching()
1422 del self._wdm[wd]
1423+ # trust that the platform watch manager will do the rest of the
1424+ # operations needed to delete a watch
1425+ self.manager.rm_watch(watch)
1426 except KeyError, err:
1427 self.log.error(str(err))
1428 if not quiet:
1429@@ -336,6 +367,16 @@
1430 self.log.debug('Adding exclude filter for %r', path)
1431 self._wdm[wd].ignore_path(path)
1432
1433+ @defer.inlineCallbacks
1434+ def stop(self):
1435+ """Close the manager and stop all watches."""
1436+ self.log.debug('Stopping watches.')
1437+ for current_wd in self._wdm:
1438+ watch = self._wdm[current_wd]
1439+ yield self.manager.stop_watch(watch)
1440+ self.log.debug('Stopping Watch on %r.', watch.path)
1441+ yield self.manager.stop()
1442+
1443
1444 class NotifyProcessor(ProcessEvent):
1445 """Processor that takes care of dealing with the events.
1446@@ -350,7 +391,7 @@
1447 GeneralINotifyProcessor)
1448 self.general_processor = GeneralINotifyProcessor(monitor,
1449 self.handle_dir_delete, NAME_TRANSLATIONS,
1450- self.platform_is_ignored, IN_IGNORED, ignore_config=ignore_config)
1451+ path_is_ignored, IN_IGNORED, ignore_config=ignore_config)
1452 self.held_event = None
1453
1454 def rm_from_mute_filter(self, event, paths):
1455@@ -361,11 +402,6 @@
1456 """Add an event and path(s) to the mute filter."""
1457 self.general_processor.add_to_mute_filter(event, paths)
1458
1459- def platform_is_ignored(self, path):
1460- """Should we ignore this path in the current platform.?"""
1461- # This should be implemented in each platform
1462- raise NotImplementedError("Not implemented on this platform.")
1463-
1464 @is_valid_syncdaemon_path(path_indexes=[1])
1465 def is_ignored(self, path):
1466 """Should we ignore this path?"""
1467@@ -552,8 +588,8 @@
1468 self.log.setLevel(TRACE)
1469 self.fs = fs
1470 self.eq = eq
1471- # You will need to create the NotifyProcessor and WatchManager
1472- # in each OS-specific implementation
1473+ self._processor = NotifyProcessor(self, ignore_config)
1474+ self._watch_manager = WatchManager(self._processor)
1475
1476 def add_to_mute_filter(self, event, **info):
1477 """Add info to mute filter in the processor."""
1478@@ -579,12 +615,11 @@
1479 # the logic to check if the watch is already set
1480 # is all in WatchManager.add_watch
1481 return self._watch_manager.add_watch(dirpath,
1482- self.filesystem_monitor_mask, auto_add=True)
1483+ self.filesystem_monitor_mask)
1484
1485 def add_watches_to_udf_ancestors(self, volume):
1486 """Add a inotify watch to volume's ancestors if it's an UDF."""
1487- # Should be implemented in each OS if necessary
1488- raise NotImplementedError("Not implemented on this platform.")
1489+ return defer.succeed(True)
1490
1491 def is_frozen(self):
1492 """Checks if there's something frozen."""
1493
1494=== modified file 'ubuntuone/platform/filesystem_notifications/darwin.py'
1495--- ubuntuone/platform/filesystem_notifications/darwin.py 2012-06-28 17:21:30 +0000
1496+++ ubuntuone/platform/filesystem_notifications/darwin.py 2012-07-11 15:52:18 +0000
1497@@ -33,37 +33,57 @@
1498 import fsevents
1499 from twisted.internet import defer, reactor
1500
1501-from ubuntuone.platform.filesystem_notifications import common
1502-
1503+from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
1504+ IN_DELETE,
1505+ IN_CREATE,
1506+ IN_MODIFY,
1507+ IN_MOVED_FROM,
1508+ IN_MOVED_TO,
1509+)
1510
1511 # a map between the few events that we have on common platforms and those
1512 # found in pyinotify
1513-common.COMMON_ACTIONS = {
1514- fsevents.IN_CREATE: common.IN_CREATE,
1515- fsevents.IN_DELETE: common.IN_DELETE,
1516- fsevents.IN_MODIFY: common.IN_MODIFY,
1517- fsevents.IN_MOVED_FROM: common.IN_MOVED_FROM,
1518- fsevents.IN_MOVED_TO: common.IN_MOVED_TO,
1519+ACTIONS = {
1520+ fsevents.IN_CREATE: IN_CREATE,
1521+ fsevents.IN_DELETE: IN_DELETE,
1522+ fsevents.IN_MODIFY: IN_MODIFY,
1523+ fsevents.IN_MOVED_FROM: IN_MOVED_FROM,
1524+ fsevents.IN_MOVED_TO: IN_MOVED_TO,
1525 }
1526
1527 # a map of the actions to names so that we have better logs.
1528-common.COMMON_ACTIONS_NAMES = {
1529- fsevents.IN_CREATE: 'IN_CREATE',
1530- fsevents.IN_DELETE: 'IN_DELETE',
1531- fsevents.IN_MODIFY: 'IN_MODIFY',
1532- fsevents.IN_MOVED_FROM: 'IN_MOVED_FROM',
1533- fsevents.IN_MOVED_TO: 'IN_MOVED_TO',
1534+ACTIONS_NAMES = {
1535+ fsevents.IN_CREATE: 'IN_CREATE',
1536+ fsevents.IN_DELETE: 'IN_DELETE',
1537+ fsevents.IN_MODIFY: 'IN_MODIFY',
1538+ fsevents.IN_MOVED_FROM: 'IN_MOVED_FROM',
1539+ fsevents.IN_MOVED_TO: 'IN_MOVED_TO',
1540 }
1541
1542
1543+def path_is_ignored(self, path):
1544+ """Should we ignore this path in the current platform.?"""
1545+ # don't support links yet
1546+ if os.path.islink(path):
1547+ return True
1548+ return False
1549+
1550+
1551 # The implementation of the code that is provided as the pyinotify substitute
1552-class Watch(common.Watch):
1553+class Watch(object):
1554 """Implement the same functions as pyinotify.Watch."""
1555
1556- def __init__(self, watch_descriptor, path, mask, auto_add, processor,
1557- buf_size=8192):
1558- super(Watch, self).__init__(watch_descriptor, path, mask, auto_add,
1559- processor, buf_size)
1560+ def __init__(self, path, process_events):
1561+ """Create a new instance for the given path.
1562+
1563+ The process_events parameter is a callback to be executed in the main
1564+ reactor thread to convert events in pyinotify events and add them to
1565+ the state machine.
1566+ """
1567+ self.path = os.path.abspath(path)
1568+ self.process_events = process_events
1569+ self.watching = False
1570+ self.ignore_paths = []
1571 # Create stream with folder to watch
1572 self.stream = fsevents.Stream(self._process_events,
1573 path, file_events=True)
1574@@ -74,97 +94,70 @@
1575
1576 def _process_events_in_main_thread(self, event):
1577 """Process the events from the queue."""
1578- # do not do it if we stop watching and the events are empty
1579- if not self._watching:
1580- return
1581-
1582 action, cookie, file_name = (event.mask, event.cookie, event.name)
1583- if any([file_name.startswith(path)
1584- for path in self._ignore_paths]):
1585- return
1586- syncdaemon_path = os.path.join(self._path, file_name)
1587- self._process_events_from_filesystem(action, file_name, cookie,
1588- syncdaemon_path)
1589+
1590+ syncdaemon_path = os.path.join(self.path, file_name)
1591+ self.process_events(action, file_name, cookie,
1592+ syncdaemon_path)
1593+
1594+ def start_watching(self):
1595+ """Start watching."""
1596+ self.watching = True
1597+ return defer.succeed(self.watching)
1598+
1599+ def stop_watching(self):
1600+ """Stop watching."""
1601+ self.watching = False
1602+ return defer.succeed(self.watching)
1603
1604 # For API compatibility
1605 @property
1606 def started(self):
1607 """A deferred that will be called when the watch is running."""
1608- return defer.succeed(self._watching)
1609+ return defer.succeed(self.watching)
1610
1611 @property
1612 def stopped(self):
1613 """A deferred fired when the watch thread has finished."""
1614- return defer.succeed(self._watching)
1615-
1616-
1617-class WatchManager(common.WatchManager):
1618+ return defer.succeed(self.watching)
1619+
1620+
1621+class WatchManager(object):
1622 """Implement the same functions as pyinotify.WatchManager.
1623
1624 All paths passed to methods in this class should be darwin paths.
1625
1626 """
1627
1628- def __init__(self, processor):
1629+ def __init__(self, log):
1630 """Init the manager to keep track of the different watches."""
1631- super(WatchManager, self).__init__(processor)
1632+ self.log = log
1633 self.observer = fsevents.Observer()
1634 self.observer.start()
1635
1636- def add_watch(self, path, mask, auto_add=False, quiet=True):
1637- """Add a new path to be watched.
1638-
1639- The method will ensure that the path is not already present.
1640- """
1641- if not isinstance(path, str):
1642- e = NotImplementedError("No implementation on this platform.")
1643- return defer.fail(e)
1644- wd = self.get_wd(path)
1645- if wd is None:
1646- self.log.debug('Adding single watch on %r', path)
1647- return self._add_single_watch(path, mask, auto_add, quiet)
1648- else:
1649- self.log.debug('Watch already exists on %r', path)
1650- return self._wdm[wd].started
1651-
1652 def __del__(self):
1653 """Stop the observer."""
1654 self.observer.stop()
1655
1656+ def stop_watch(self, watch):
1657+ """Stop a given watch."""
1658+ watch.stop_watching()
1659+ self.observer.unschedule(watch.platform_watch.stream)
1660+ return defer.succeed(True)
1661+
1662 def stop(self):
1663- """Close the manager and stop all watches."""
1664- self.log.debug('Stopping watches.')
1665- for current_wd in self._wdm:
1666- self._wdm[current_wd].stop_watching()
1667- self.observer.unschedule(self._wdm[current_wd].stream)
1668- self.log.debug('Stopping Watch on %r.', self._wdm[current_wd].path)
1669+ """Stop the manager."""
1670 self.observer.stop()
1671
1672- def del_watch(self, wd):
1673- """Delete the watch with the given descriptor."""
1674- watch = self.get_watch(wd)
1675+ def del_watch(self, watch):
1676+ """Delete the watch and clean resources."""
1677 self.observer.unschedule(watch.stream)
1678- return super(WatchManager, self).del_watch(wd)
1679
1680- def _adding_watch(self, path, mask, auto_add):
1681+ def add_watch(self, watch):
1682 """This method perform actually the action of registering the watch."""
1683- watch = Watch(self._wd_count, path, mask, auto_add, self._processor)
1684- self._wdm[self._wd_count] = watch
1685- self._wdm[self._wd_count].start_watching()
1686- self.observer.schedule(self._wdm[self._wd_count].stream)
1687- self._wd_count += 1
1688- return defer.succeed(True)
1689+ self.observer.schedule(watch.platform_watch.stream)
1690+ return True
1691
1692- def rm_watch(self, wd, rec=False, quiet=True):
1693+ def rm_watch(self, watch):
1694 """Remove the the watch with the given wd."""
1695- watch = self.get_watch(wd)
1696- self.observer.unschedule(watch.stream)
1697- super(WatchManager, self).del_watch(wd, rec, quiet)
1698-
1699-
1700-class FilesystemMonitor(object):
1701- """Empty implementation of FilesystemMonitor"""
1702-
1703-
1704-class NotifyProcessor(object):
1705- """Empty implementation of NotifyProcessor"""
1706+ self.observer.unschedule(watch.platform_watch.stream)
1707
1708=== modified file 'ubuntuone/platform/filesystem_notifications/windows.py'
1709--- ubuntuone/platform/filesystem_notifications/windows.py 2012-06-28 16:44:01 +0000
1710+++ ubuntuone/platform/filesystem_notifications/windows.py 2012-07-11 15:52:18 +0000
1711@@ -28,7 +28,9 @@
1712 # files in the program, then also delete it here.
1713 """File notifications on windows."""
1714
1715+import logging
1716 import os
1717+
1718 from uuid import uuid4
1719
1720 from twisted.internet import defer, reactor
1721@@ -66,25 +68,31 @@
1722 get_syncdaemon_valid_path,
1723 )
1724
1725-from ubuntuone.platform.filesystem_notifications import common
1726-
1727-# a map between the few events that we have on common platforms and those
1728-# found in pyinotify
1729-common.COMMON_ACTIONS = {
1730- 1: common.IN_CREATE,
1731- 2: common.IN_DELETE,
1732- 3: common.IN_MODIFY,
1733- 4: common.IN_MOVED_FROM,
1734- 5: common.IN_MOVED_TO,
1735+from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
1736+ IN_CREATE,
1737+ IN_DELETE,
1738+ IN_MODIFY,
1739+ IN_MOVED_FROM,
1740+ IN_MOVED_TO,
1741+)
1742+
1743+
1744+# map between windows events and pyinotify
1745+ACTIONS = {
1746+ 1: IN_CREATE,
1747+ 2: IN_DELETE,
1748+ 3: IN_MODIFY,
1749+ 4: IN_MOVED_FROM,
1750+ 5: IN_MOVED_TO,
1751 }
1752
1753 # a map of the actions to names so that we have better logs.
1754-common.COMMON_ACTIONS_NAMES = {
1755- 1: 'IN_CREATE',
1756- 2: 'IN_DELETE',
1757- 3: 'IN_MODIFY',
1758- 4: 'IN_MOVED_FROM',
1759- 5: 'IN_MOVED_TO',
1760+ACTIONS_NAMES = {
1761+ 1: 'IN_CREATE',
1762+ 2: 'IN_DELETE',
1763+ 3: 'IN_MODIFY',
1764+ 4: 'IN_MOVED_FROM',
1765+ 5: 'IN_MOVED_TO',
1766 }
1767
1768
1769@@ -96,20 +104,43 @@
1770
1771 THREADPOOL_MAX = 20
1772
1773-
1774-class Watch(common.Watch):
1775+FILESYSTEM_MONITOR_MASK = FILE_NOTIFY_CHANGE_FILE_NAME | \
1776+ FILE_NOTIFY_CHANGE_DIR_NAME | \
1777+ FILE_NOTIFY_CHANGE_ATTRIBUTES | \
1778+ FILE_NOTIFY_CHANGE_SIZE | \
1779+ FILE_NOTIFY_CHANGE_LAST_WRITE | \
1780+ FILE_NOTIFY_CHANGE_SECURITY | \
1781+ FILE_NOTIFY_CHANGE_LAST_ACCESS
1782+
1783+
1784+@is_valid_syncdaemon_path()
1785+def path_is_ignored(path):
1786+ """Should we ignore this path in the current platform.?"""
1787+ # don't support links yet
1788+ if path.endswith('.lnk'):
1789+ return True
1790+ return False
1791+
1792+
1793+class Watch(object):
1794 """Implement the same functions as pyinotify.Watch."""
1795
1796- def __init__(self, watch_descriptor, path, mask, auto_add, processor,
1797- buf_size=8192):
1798- super(Watch, self).__init__(watch_descriptor, path, mask, auto_add,
1799- processor, buf_size)
1800+ def __init__(self, path, process_events, mask=FILESYSTEM_MONITOR_MASK,
1801+ buf_size=8192):
1802+ self.path = os.path.abspath(path)
1803+ self.process_events = process_events
1804+ self.watching = False
1805+ self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.windows.' +
1806+ 'filesystem_notifications.Watch')
1807+ self.log.setLevel(logging.DEBUG)
1808+ self._buf_size = buf_size
1809+ self._mask = mask
1810+ self.ignore_paths = []
1811+ self._watch_handle = None
1812+
1813 self._wait_stop = CreateEvent(None, 0, 0, None)
1814 self._overlapped = OVERLAPPED()
1815 self._overlapped.hEvent = CreateEvent(None, 0, 0, None)
1816- # remember the subdirs we have so that when we have a delete we can
1817- # check if it was a remove
1818- self._subdirs = set()
1819 # this deferred is fired when the watch has started monitoring
1820 # a directory from a thread
1821 self._watch_started_deferred = defer.Deferred()
1822@@ -118,22 +149,13 @@
1823
1824 def _process_events(self, events):
1825 """Process the events from the queue."""
1826- # do not do it if we stop watching and the events are empty
1827- if not self._watching:
1828- return
1829-
1830 # we transform the events to be the same as the one in pyinotify
1831 # and then use the proc_fun
1832 for action, file_name in events:
1833- if any([file_name.startswith(path)
1834- for path in self._ignore_paths]):
1835- continue
1836- # map the windows events to the pyinotify ones, tis is dirty but
1837- # makes the multiplatform better, linux was first :P
1838 syncdaemon_path = get_syncdaemon_valid_path(
1839- os.path.join(self._path, file_name))
1840- self._process_events_from_filesystem(action, file_name,
1841- str(uuid4()), syncdaemon_path)
1842+ os.path.join(self.path, file_name))
1843+ self.process_events(action, file_name, str(uuid4()),
1844+ syncdaemon_path)
1845
1846 def _call_deferred(self, f, *args):
1847 """Executes the deferred call avoiding possible race conditions."""
1848@@ -151,12 +173,12 @@
1849 def _watch(self):
1850 """Watch a path that is a directory."""
1851 self.log.debug('Adding watch for %r (exists? %r is dir? %r).',
1852- self._path,
1853- os.path.exists(self._path), os.path.isdir(self._path))
1854+ self.path,
1855+ os.path.exists(self.path), os.path.isdir(self.path))
1856 # we are going to be using the ReadDirectoryChangesW whihc requires
1857 # a directory handle and the mask to be used.
1858 self._watch_handle = CreateFileW(
1859- self._path,
1860+ self.path,
1861 FILE_LIST_DIRECTORY,
1862 FILE_SHARE_READ | FILE_SHARE_WRITE,
1863 None,
1864@@ -184,7 +206,7 @@
1865 ReadDirectoryChangesW(
1866 handle,
1867 buf,
1868- self._auto_add,
1869+ True, # Always watch children
1870 self._mask,
1871 self._overlapped,
1872 )
1873@@ -204,20 +226,20 @@
1874 # lets ead the data and store it in the results
1875 events = FILE_NOTIFY_INFORMATION(buf, data)
1876 self.log.debug('Got from ReadDirectoryChangesW %r.',
1877- [(common.COMMON_ACTIONS_NAMES[action], path) \
1878+ [(ACTIONS_NAMES[action], path) \
1879 for action, path in events])
1880 reactor.callFromThread(self._process_events, events)
1881
1882 def start_watching(self):
1883 """Tell the watch to start processing events."""
1884- super(Watch, self).start_watching()
1885+ self.watching = True
1886 reactor.callInThread(self._watch_wrapper)
1887 return self._watch_started_deferred
1888
1889 def stop_watching(self):
1890 """Tell the watch to stop processing events."""
1891- super(Watch, self).stop_watching()
1892 SetEvent(self._wait_stop)
1893+ self.watching = False
1894 return self.stopped
1895
1896 @property
1897@@ -231,82 +253,39 @@
1898 return self._watch_stopped_deferred
1899
1900
1901-class WatchManager(common.WatchManager):
1902-
1903- @common.is_valid_os_path(path_indexes=[1])
1904- def add_watch(self, path, mask, auto_add=False, quiet=True):
1905- """Add a new path to be watch.
1906-
1907- The method will ensure that the path is not already present.
1908- """
1909- if not isinstance(path, unicode):
1910- e = NotImplementedError("No implementation on this platform.")
1911- return defer.fail(e)
1912- wd = self.get_wd(path)
1913- if wd is None:
1914- self.log.debug('Adding single watch on %r', path)
1915- return self._add_single_watch(path, mask, auto_add, quiet)
1916- else:
1917- self.log.debug('Watch already exists on %r', path)
1918- return self._wdm[wd].started
1919+class WatchManager(object):
1920+ """Implement the same functions as pyinotify.WatchManager.
1921+
1922+ All paths passed to methods in this class should be windows paths.
1923+
1924+ """
1925+
1926+ def __init__(self, log):
1927+ """Create a new instance."""
1928+ self.log = log
1929+ self._wd_count = 0
1930+
1931+ def del_watch(self, wd):
1932+ """Delete the watch with the given descriptor."""
1933+
1934+ def stop_watch(self, watch):
1935+ """Stop a watch."""
1936+ # decrease the number of watches
1937+ self._wd_count -= 1
1938+ return watch.stop_watching()
1939
1940 def stop(self):
1941 """Close the manager and stop all watches."""
1942 self.log.debug('Stopping watches.')
1943- wait_list = []
1944- for current_wd in self._wdm:
1945- wait_list.append(self._wdm[current_wd].stop_watching())
1946- self.log.debug('Stopping Watch on %r.', self._wdm[current_wd].path)
1947- return defer.DeferredList(wait_list)
1948-
1949- def _adding_watch(self, path, mask, auto_add):
1950+ return defer.succeed(True)
1951+
1952+ def rm_watch(self, watch):
1953+ """Stop the Watch."""
1954+ self._wd_count -= 1
1955+ return defer.succeed(True)
1956+
1957+ def add_watch(self, watch):
1958 # adjust the number of threads based on the UDFs watched
1959- watch = Watch(self._wd_count, path, mask, auto_add, self._processor)
1960+ self._wd_count += 1
1961 reactor.suggestThreadPoolSize(THREADPOOL_MAX + self._wd_count + 1)
1962- self._wdm[self._wd_count] = watch
1963- d = self._wdm[self._wd_count].start_watching()
1964- self._wd_count += 1
1965- return d
1966-
1967-
1968-class NotifyProcessor(common.NotifyProcessor):
1969- """
1970- Processor that takes care of dealing with the events.
1971-
1972- Also, they must not be literal paths, that is the \\?\ prefix should not be
1973- in the path.
1974- """
1975-
1976- @is_valid_syncdaemon_path(path_indexes=[1])
1977- def platform_is_ignored(self, path):
1978- """Should we ignore this path in the current platform.?"""
1979- # don't support links yet
1980- if path.endswith('.lnk'):
1981- return True
1982- return False
1983-
1984-
1985-class FilesystemMonitor(common.FilesystemMonitor):
1986- """Manages the signals from filesystem."""
1987-
1988- def __init__(self, eq, fs, ignore_config=None, timeout=1):
1989- super(FilesystemMonitor, self).__init__(eq, fs, ignore_config, timeout)
1990- self._processor = NotifyProcessor(self, ignore_config)
1991- self._watch_manager = WatchManager(self._processor)
1992- # the default mask to be used in the watches
1993- # added by the FilesystemMonitor class
1994- self.filesystem_monitor_mask = FILE_NOTIFY_CHANGE_FILE_NAME | \
1995- FILE_NOTIFY_CHANGE_DIR_NAME | \
1996- FILE_NOTIFY_CHANGE_ATTRIBUTES | \
1997- FILE_NOTIFY_CHANGE_SIZE | \
1998- FILE_NOTIFY_CHANGE_LAST_WRITE | \
1999- FILE_NOTIFY_CHANGE_SECURITY | \
2000- FILE_NOTIFY_CHANGE_LAST_ACCESS
2001-
2002- def add_watches_to_udf_ancestors(self, volume):
2003- """Add a inotify watch to volume's ancestors if it's an UDF."""
2004- # There is no need to add the watches to the ancestors
2005- # so we will always return true. The reason for this is that the
2006- # handles that we open stop the user from renaming the ancestors of
2007- # the UDF, for a user to do that he has to unsync the udf first
2008- return defer.succeed(True)
2009+ return True
2010
2011=== modified file 'ubuntuone/platform/os_helper/darwin.py'
2012--- ubuntuone/platform/os_helper/darwin.py 2012-06-27 13:51:09 +0000
2013+++ ubuntuone/platform/os_helper/darwin.py 2012-07-11 15:52:18 +0000
2014@@ -127,8 +127,13 @@
2015
2016
2017 def os_path(path_indexes=None):
2018+ if path_indexes is None:
2019+ path_indexes = [0]
2020+
2021 def decorator(func):
2022 def wrapped(*args, **kwargs):
2023+ for i in path_indexes:
2024+ assert isinstance(args[i], str), 'Path %r should be str.'
2025 return func(*args, **kwargs)
2026 return wrapped
2027 return decorator

Subscribers

People subscribed via source and target branches