Merge lp:~mandel/ubuntuone-client/clean-fsevents into lp:ubuntuone-client
- clean-fsevents
- Merge into trunk
| Status: | Merged |
|---|---|
| Approved by: | Manuel de la Peña on 2012-07-11 |
| Approved revision: | 1298 |
| Merged at revision: | 1273 |
| Proposed branch: | lp:~mandel/ubuntuone-client/clean-fsevents |
| Merge into: | lp:ubuntuone-client |
| Diff against target: |
2027 lines (+438/-535) 7 files modified
tests/platform/filesystem_notifications/test_darwin.py (+49/-81) tests/platform/filesystem_notifications/test_windows.py (+70/-143) ubuntuone/platform/filesystem_notifications/__init__.py (+4/-8) ubuntuone/platform/filesystem_notifications/common.py (+140/-105) ubuntuone/platform/filesystem_notifications/darwin.py (+74/-81) ubuntuone/platform/filesystem_notifications/windows.py (+96/-117) ubuntuone/platform/os_helper/darwin.py (+5/-0) |
| To merge this branch: | bzr merge lp:~mandel/ubuntuone-client/clean-fsevents |
| Related bugs: |
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| Mike McCracken (community) | Approve on 2012-07-11 | ||
| Diego Sarmentero (community) | 2012-07-10 | Approve on 2012-07-10 | |
|
Review via email:
|
|||
Commit Message
Refactor that simplifies the way the diff implementations per platform are done so that in future branches is simpler to add the daemon code.
Description of the Change
Refactor that simplifies the way the diff implementations per platform are done so that in future branches is simpler to add the daemon code.
Please run tests on windows/linux and mac.
| Mike McCracken (mikemc) wrote : | # |
really minor stuff:
- filesystem_
- filesystem_
-- I looked around and this return value seems like it's never actually used, just passed around a bit and eventually ignored. Still, it's probably worth being consistent.
| Manuel de la Peña (mandel) wrote : | # |
> really minor stuff:
>
> - filesystem_
> paths'.
>
Left over from moving things around. Fixed!
> - filesystem_
> value, but in common.py, it's expecting a return. the version in windows.py
> returns boolean.
> -- I looked around and this return value seems like it's never actually used,
> just passed around a bit and eventually ignored. Still, it's probably worth
> being consistent.
Indeed, I made the changes to make sure both return True.
| Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~mandel/ubuntuone-client/clean-fsevents into lp:ubuntuone-client failed. Below is the output from the failed tests.
/usr/bin/
checking for autoconf >= 2.53...
testing autoconf2.50... not found.
testing autoconf... found 2.69
checking for automake >= 1.10...
testing automake-1.11... found 1.11.5
checking for libtool >= 1.5...
testing libtoolize... found 2.4.2
checking for intltool >= 0.30...
testing intltoolize... found 0.50.2
checking for pkg-config >= 0.14.0...
testing pkg-config... found 0.26
checking for gtk-doc >= 1.0...
testing gtkdocize... found 1.18
Checking for required M4 macros...
Checking for forbidden M4 macros...
Processing ./configure.ac
Running libtoolize...
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
Running intltoolize...
Running gtkdocize...
Running aclocal-1.11...
Running autoconf...
Running autoheader...
Running automake-1.11...
Running ./configure --enable-gtk-doc --enable-debug ...
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for library containing strerror... none required
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking build system type... x86_64-
checking host system type... x86_64-
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
che...
| Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~mandel/ubuntuone-client/clean-fsevents into lp:ubuntuone-client failed. Below is the output from the failed tests.
/usr/bin/
checking for autoconf >= 2.53...
testing autoconf2.50... not found.
testing autoconf... found 2.69
checking for automake >= 1.10...
testing automake-1.11... found 1.11.5
checking for libtool >= 1.5...
testing libtoolize... found 2.4.2
checking for intltool >= 0.30...
testing intltoolize... found 0.50.2
checking for pkg-config >= 0.14.0...
testing pkg-config... found 0.26
checking for gtk-doc >= 1.0...
testing gtkdocize... found 1.18
Checking for required M4 macros...
Checking for forbidden M4 macros...
Processing ./configure.ac
Running libtoolize...
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
Running intltoolize...
Running gtkdocize...
Running aclocal-1.11...
Running autoconf...
Running autoheader...
Running automake-1.11...
Running ./configure --enable-gtk-doc --enable-debug ...
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for library containing strerror... none required
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking build system type... x86_64-
checking host system type... x86_64-
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
che...
Preview Diff
| 1 | === modified file 'tests/platform/filesystem_notifications/test_darwin.py' |
| 2 | --- tests/platform/filesystem_notifications/test_darwin.py 2012-06-28 13:20:01 +0000 |
| 3 | +++ tests/platform/filesystem_notifications/test_darwin.py 2012-07-11 15:52:18 +0000 |
| 4 | @@ -32,7 +32,6 @@ |
| 5 | import os |
| 6 | import tempfile |
| 7 | import thread |
| 8 | -import time |
| 9 | |
| 10 | from twisted.internet import defer |
| 11 | |
| 12 | @@ -41,7 +40,7 @@ |
| 13 | from ubuntuone.platform.filesystem_notifications import ( |
| 14 | darwin as filesystem_notifications, |
| 15 | ) |
| 16 | -from ubuntuone.platform.filesystem_notifications.darwin import ( |
| 17 | +from ubuntuone.platform.filesystem_notifications.common import ( |
| 18 | Watch, |
| 19 | WatchManager, |
| 20 | ) |
| 21 | @@ -54,7 +53,7 @@ |
| 22 | |
| 23 | # A reverse mapping for the tests |
| 24 | REVERSE_MACOS_ACTIONS = {} |
| 25 | -for key, value in filesystem_notifications.common.COMMON_ACTIONS.iteritems(): |
| 26 | +for key, value in filesystem_notifications.ACTIONS.iteritems(): |
| 27 | REVERSE_MACOS_ACTIONS[value] = key |
| 28 | |
| 29 | |
| 30 | @@ -148,16 +147,15 @@ |
| 31 | self.paths_checked.append((path, result)) |
| 32 | return result |
| 33 | |
| 34 | - self.patch(filesystem_notifications.Watch, '_path_is_dir', |
| 35 | + self.patch(Watch, '_path_is_dir', |
| 36 | path_is_dir_wrapper) |
| 37 | |
| 38 | @defer.inlineCallbacks |
| 39 | - def _perform_operations(self, path, mask, auto_add, actions, |
| 40 | - number_events): |
| 41 | + def _perform_operations(self, path, mask, actions, number_events): |
| 42 | """Perform the file operations and returns the recorded events.""" |
| 43 | handler = TestCaseHandler(number_events=number_events) |
| 44 | manager = WatchManager(handler) |
| 45 | - yield manager.add_watch(path, mask, auto_add=auto_add) |
| 46 | + yield manager.add_watch(path, mask) |
| 47 | # change the logger so that we can check the logs if we wanted |
| 48 | manager._wdm[0].log.addHandler(self.memento) |
| 49 | # clean logger later |
| 50 | @@ -169,22 +167,6 @@ |
| 51 | self.addCleanup(manager.stop) |
| 52 | defer.returnValue(ret) |
| 53 | |
| 54 | - def _perform_timed_operations(self, path, mask, auto_add, actions, |
| 55 | - time_out): |
| 56 | - """Perform the file operations and returns the recorded events.""" |
| 57 | - manager = WatchManager() |
| 58 | - manager.add_watch(path, mask, auto_add=auto_add) |
| 59 | - # change the logger so that we can check the logs if we wanted |
| 60 | - manager._wdm[0].log.addHandler(self.memento) |
| 61 | - # clean logger later |
| 62 | - self.addCleanup(manager._wdm[0].log.removeHandler, self.memento) |
| 63 | - # execution the actions |
| 64 | - actions() |
| 65 | - # process the recorded events |
| 66 | - time.sleep(time_out) |
| 67 | - events = self.handler.processed_events |
| 68 | - return events |
| 69 | - |
| 70 | def _assert_logs(self, events): |
| 71 | """Assert the debug logs.""" |
| 72 | logs = [] |
| 73 | @@ -208,7 +190,7 @@ |
| 74 | os.fsync(fd) |
| 75 | fd.close() |
| 76 | |
| 77 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 78 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 79 | create_file, 1) |
| 80 | event = events[0] |
| 81 | self.assertFalse(event.dir) |
| 82 | @@ -230,7 +212,7 @@ |
| 83 | """Action for the test.""" |
| 84 | os.mkdir(dir_name) |
| 85 | |
| 86 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 87 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 88 | create_dir, 1) |
| 89 | event = events[0] |
| 90 | self.assertTrue(event.dir) |
| 91 | @@ -254,7 +236,7 @@ |
| 92 | """Action for the test.""" |
| 93 | os.remove(file_name) |
| 94 | |
| 95 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 96 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 97 | remove_file, 1) |
| 98 | event = events[0] |
| 99 | self.assertFalse(event.dir) |
| 100 | @@ -278,7 +260,7 @@ |
| 101 | """Action for the test.""" |
| 102 | os.rmdir(dir_name) |
| 103 | |
| 104 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 105 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 106 | remove_dir, 1) |
| 107 | event = events[0] |
| 108 | self.assertTrue(event.dir) |
| 109 | @@ -304,7 +286,7 @@ |
| 110 | fd.write('test') |
| 111 | fd.close() |
| 112 | |
| 113 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 114 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 115 | write_file, 1) |
| 116 | event = events[0] |
| 117 | self.assertFalse(event.dir) |
| 118 | @@ -332,7 +314,7 @@ |
| 119 | os.rename(from_file_name, to_file_name) |
| 120 | |
| 121 | events = yield self._perform_operations(self.basedir, self.mask, |
| 122 | - False, move_file, 2) |
| 123 | + move_file, 2) |
| 124 | move_from_event = events[0] |
| 125 | move_to_event = events[1] |
| 126 | # first test the move from |
| 127 | @@ -375,7 +357,7 @@ |
| 128 | |
| 129 | # We need to test that we get a delete operation when moving |
| 130 | # a file to an unwatched folder |
| 131 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 132 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 133 | move_file, 1) |
| 134 | event = events[0] |
| 135 | self.assertFalse(event.dir) |
| 136 | @@ -405,7 +387,7 @@ |
| 137 | |
| 138 | # We need to test that we get a delete operation when moving |
| 139 | # a file from an unwatched folder |
| 140 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 141 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 142 | move_files, 1) |
| 143 | event = events[0] |
| 144 | self.assertFalse(event.dir) |
| 145 | @@ -433,7 +415,7 @@ |
| 146 | os.rename(from_dir_name, to_dir_name) |
| 147 | |
| 148 | events = yield self._perform_operations(self.basedir, |
| 149 | - self.mask, False, move_file, 2) |
| 150 | + self.mask, move_file, 2) |
| 151 | move_from_event = events[0] |
| 152 | move_to_event = events[1] |
| 153 | # first test the move from |
| 154 | @@ -476,7 +458,7 @@ |
| 155 | |
| 156 | # We need to test that we get a delete operation when moving |
| 157 | # a file to an unwatched folder |
| 158 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 159 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 160 | move_dir, 1) |
| 161 | event = events[0] |
| 162 | self.assertTrue(event.dir) |
| 163 | @@ -502,7 +484,7 @@ |
| 164 | """Action for the test.""" |
| 165 | os.rename(from_dir_name, to_dir_name) |
| 166 | |
| 167 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 168 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 169 | move_dir, 1) |
| 170 | event = events[0] |
| 171 | self.assertTrue(event.dir) |
| 172 | @@ -519,8 +501,8 @@ |
| 173 | handler = TestCaseHandler(number_events=0) |
| 174 | manager = WatchManager(handler) |
| 175 | # add a watch that will always exclude all actions |
| 176 | - manager.add_watch(self.basedir, self.mask, auto_add=True, |
| 177 | - exclude_filter=lambda x: True) |
| 178 | + manager.add_watch(self.basedir, self.mask, |
| 179 | + exclude_filter=lambda x: True) |
| 180 | # execution the actions |
| 181 | file_name = os.path.join(self.basedir, 'test_file_create') |
| 182 | open(file_name, 'w').close() |
| 183 | @@ -528,21 +510,6 @@ |
| 184 | self.assertEqual(0, len(handler.processed_events)) |
| 185 | test_exclude_filter.skip = "we must rethink this test." |
| 186 | |
| 187 | - def test_open_dir_muted(self): |
| 188 | - """Test that the opening of dirs is ignored.""" |
| 189 | - dir_name = os.path.join(tempfile.mkdtemp(), 'test_dir_open') |
| 190 | - # create file before we record |
| 191 | - os.mkdir(dir_name) |
| 192 | - |
| 193 | - def open_dir(): |
| 194 | - """Action for the test.""" |
| 195 | - os.startfile(dir_name) |
| 196 | - |
| 197 | - events = self._perform_timed_operations(self.basedir, self.mask, False, |
| 198 | - open_dir, 2) |
| 199 | - self.assertEqual(0, len(events)) |
| 200 | - test_open_dir_muted.skip = "we must rethink this test." |
| 201 | - |
| 202 | def test_ignore_path(self): |
| 203 | """Test that events from a path are ignored.""" |
| 204 | events = [] |
| 205 | @@ -553,13 +520,13 @@ |
| 206 | |
| 207 | path = '/Users/username/folder' |
| 208 | child = 'child' |
| 209 | - watch = Watch(1, path, None, True, fake_processor) |
| 210 | + watch = Watch(1, path, fake_processor) |
| 211 | watch.ignore_path(os.path.join(path, child)) |
| 212 | # ensure that the watch is watching |
| 213 | - watch._watching = True |
| 214 | + watch.platform_watch.watching = True |
| 215 | for file_name in 'abcdef': |
| 216 | event = FakeFileEvent(256, None, os.path.join(child, file_name)) |
| 217 | - watch._process_events(event) |
| 218 | + watch.platform_watch._process_events(event) |
| 219 | self.assertEqual(0, len(events), |
| 220 | 'All events should have been ignored.') |
| 221 | |
| 222 | @@ -576,7 +543,7 @@ |
| 223 | |
| 224 | path = '/Users/username/folder' |
| 225 | child = 'child' |
| 226 | - watch = Watch(1, path, None, True, fake_processor) |
| 227 | + watch = Watch(1, path, fake_processor) |
| 228 | watch.ignore_path(os.path.join(path, child)) |
| 229 | paths_not_to_ignore = [] |
| 230 | for file_name in 'abcdef': |
| 231 | @@ -584,9 +551,9 @@ |
| 232 | file_name)) |
| 233 | paths_not_to_ignore.append(event) |
| 234 | # ensure that the watch is watching |
| 235 | - watch._watching = True |
| 236 | + watch.platform_watch.watching = True |
| 237 | for event in paths_not_to_ignore: |
| 238 | - watch._process_events(event) |
| 239 | + watch.platform_watch._process_events(event) |
| 240 | self.assertEqual(len(paths_not_to_ignore), len(events), |
| 241 | 'No events should have been ignored.') |
| 242 | |
| 243 | @@ -603,7 +570,7 @@ |
| 244 | |
| 245 | child = 'child' |
| 246 | path = '/Users/username/folder' |
| 247 | - watch = Watch(1, path, None, True, fake_processor) |
| 248 | + watch = Watch(1, path, fake_processor) |
| 249 | watch.ignore_path(os.path.join(path, child)) |
| 250 | paths_not_to_ignore = [] |
| 251 | paths_to_ignore = [] |
| 252 | @@ -615,9 +582,9 @@ |
| 253 | paths_not_to_ignore.append(event) |
| 254 | expected_events.append(os.path.join(path, valid)) |
| 255 | # ensure that the watch is watching |
| 256 | - watch._watching = True |
| 257 | + watch.platform_watch.watching = True |
| 258 | for event in paths_not_to_ignore: |
| 259 | - watch._process_events(event) |
| 260 | + watch.platform_watch._process_events(event) |
| 261 | self.assertEqual(len(paths_not_to_ignore), len(events), |
| 262 | 'Wrong number of events ignored.') |
| 263 | self.assertTrue(all([event in expected_events for event in events]), |
| 264 | @@ -636,7 +603,7 @@ |
| 265 | |
| 266 | path = '/Users/username/folder' |
| 267 | child = 'child' |
| 268 | - watch = Watch(1, path, None, True, fake_processor) |
| 269 | + watch = Watch(1, path, fake_processor) |
| 270 | watch.ignore_path(os.path.join(path, child)) |
| 271 | watch.remove_ignored_path(os.path.join(path, child)) |
| 272 | paths_not_to_ignore = [] |
| 273 | @@ -644,9 +611,9 @@ |
| 274 | event = FakeFileEvent(256, None, os.path.join(child, file_name)) |
| 275 | paths_not_to_ignore.append(event) |
| 276 | # ensure that the watch is watching |
| 277 | - watch._watching = True |
| 278 | + watch.platform_watch.watching = True |
| 279 | for event in paths_not_to_ignore: |
| 280 | - watch._process_events(event) |
| 281 | + watch.platform_watch._process_events(event) |
| 282 | self.assertEqual(len(paths_not_to_ignore), len(events), |
| 283 | 'All events should have been accepted.') |
| 284 | |
| 285 | @@ -664,7 +631,7 @@ |
| 286 | path = '/Users/username/folder' |
| 287 | child_a = 'childa' |
| 288 | child_b = 'childb' |
| 289 | - watch = Watch(1, path, None, True, fake_processor) |
| 290 | + watch = Watch(1, path, fake_processor) |
| 291 | watch.ignore_path(os.path.join(path, child_a)) |
| 292 | watch.ignore_path(os.path.join(path, child_b)) |
| 293 | watch.remove_ignored_path(os.path.join(path, child_a)) |
| 294 | @@ -678,9 +645,9 @@ |
| 295 | paths_not_to_ignore.append(event) |
| 296 | expected_events.append(os.path.join(path, valid)) |
| 297 | # ensure that the watch is watching |
| 298 | - watch._watching = True |
| 299 | + watch.platform_watch.watching = True |
| 300 | for event in paths_not_to_ignore: |
| 301 | - watch._process_events(event) |
| 302 | + watch.platform_watch._process_events(event) |
| 303 | self.assertEqual(len(paths_not_to_ignore), len(events), |
| 304 | 'All events should have been accepted.') |
| 305 | self.assertTrue(all([event in expected_events for event in events]), |
| 306 | @@ -691,17 +658,18 @@ |
| 307 | def fake_call(*args, **kwargs): |
| 308 | """Fake call.""" |
| 309 | |
| 310 | - path = '/Users/username/folder' |
| 311 | - watch = Watch(1, path, None, True, None) |
| 312 | - self.assertEqual(watch._process_events, watch.stream.callback) |
| 313 | - self.assertEqual(watch.stream.paths, [path]) |
| 314 | - self.assertEqual(watch.stream.file_events, True) |
| 315 | + path = '/Users/username/folder/' |
| 316 | + watch = Watch(1, path, None) |
| 317 | + self.assertEqual(watch.platform_watch._process_events, |
| 318 | + watch.platform_watch.stream.callback) |
| 319 | + self.assertEqual(watch.platform_watch.stream.paths, [path]) |
| 320 | + self.assertEqual(watch.platform_watch.stream.file_events, True) |
| 321 | |
| 322 | def test_watching_property(self): |
| 323 | """Test that the stopped property returns the stopped deferred.""" |
| 324 | path = '/Users/username/folder' |
| 325 | - watch = Watch(1, path, None, True, None) |
| 326 | - self.assertFalse(watch._watching) |
| 327 | + watch = Watch(1, path, None) |
| 328 | + self.assertFalse(watch.watching) |
| 329 | |
| 330 | def random_error(self, *args): |
| 331 | """Throw a fake exception.""" |
| 332 | @@ -712,7 +680,7 @@ |
| 333 | path = '/Users/username/path/to/not/dir' |
| 334 | test_path = self.mktemp("test_directory") |
| 335 | self.patch(os.path, 'exists', lambda path: False) |
| 336 | - watch = Watch(1, test_path, self.mask, True, None) |
| 337 | + watch = Watch(1, test_path, None) |
| 338 | self.assertFalse(watch._path_is_dir(path)) |
| 339 | |
| 340 | def test_is_path_dir_missing_in_subdir(self): |
| 341 | @@ -720,7 +688,7 @@ |
| 342 | path = '/Users/username/path/to/not/dir' |
| 343 | test_path = self.mktemp("test_directory") |
| 344 | self.patch(os.path, 'exists', lambda path: False) |
| 345 | - watch = Watch(1, test_path, self.mask, True, None) |
| 346 | + watch = Watch(1, test_path, None) |
| 347 | watch._subdirs.add(path) |
| 348 | self.assertTrue(watch._path_is_dir(path)) |
| 349 | |
| 350 | @@ -730,7 +698,7 @@ |
| 351 | test_path = self.mktemp("test_directory") |
| 352 | self.patch(os.path, 'exists', lambda path: True) |
| 353 | self.patch(os.path, 'isdir', lambda path: True) |
| 354 | - watch = Watch(1, test_path, self.mask, True, None) |
| 355 | + watch = Watch(1, test_path, None) |
| 356 | watch._subdirs.add(path) |
| 357 | self.assertTrue(watch._path_is_dir(path)) |
| 358 | |
| 359 | @@ -740,7 +708,7 @@ |
| 360 | test_path = self.mktemp("test_directory") |
| 361 | self.patch(os.path, 'exists', lambda path: True) |
| 362 | self.patch(os.path, 'isdir', lambda path: False) |
| 363 | - watch = Watch(1, test_path, self.mask, True, None) |
| 364 | + watch = Watch(1, test_path, None) |
| 365 | watch._subdirs.add(path) |
| 366 | self.assertFalse(watch._path_is_dir(path)) |
| 367 | |
| 368 | @@ -748,7 +716,7 @@ |
| 369 | """Test when we update on a create event and not present.""" |
| 370 | path = '/Users/username/path/to/not/dir' |
| 371 | test_path = self.mktemp("test_directory") |
| 372 | - watch = Watch(1, test_path, self.mask, True, None) |
| 373 | + watch = Watch(1, test_path, None) |
| 374 | watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_CREATE]) |
| 375 | self.assertTrue(path in watch._subdirs) |
| 376 | |
| 377 | @@ -756,7 +724,7 @@ |
| 378 | """Test when we update on a create event and is present.""" |
| 379 | path = '/Users/username/path/to/not/dir' |
| 380 | test_path = self.mktemp("test_directory") |
| 381 | - watch = Watch(1, test_path, self.mask, True, None) |
| 382 | + watch = Watch(1, test_path, None) |
| 383 | watch._subdirs.add(path) |
| 384 | old_length = len(watch._subdirs) |
| 385 | watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_CREATE]) |
| 386 | @@ -767,7 +735,7 @@ |
| 387 | """Test when we delete and is not present.""" |
| 388 | path = '/Users/username/path/to/not/dir' |
| 389 | test_path = self.mktemp("test_directory") |
| 390 | - watch = Watch(1, test_path, self.mask, True, None) |
| 391 | + watch = Watch(1, test_path, None) |
| 392 | watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_DELETE]) |
| 393 | self.assertTrue(path not in watch._subdirs) |
| 394 | |
| 395 | @@ -775,7 +743,7 @@ |
| 396 | """Test when we delete and is present.""" |
| 397 | path = '/Users/username/path/to/not/dir' |
| 398 | test_path = self.mktemp("test_directory") |
| 399 | - watch = Watch(1, test_path, self.mask, True, None) |
| 400 | + watch = Watch(1, test_path, None) |
| 401 | watch._subdirs.add(path) |
| 402 | watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_DELETE]) |
| 403 | self.assertTrue(path not in watch._subdirs) |
| 404 | |
| 405 | === modified file 'tests/platform/filesystem_notifications/test_windows.py' |
| 406 | --- tests/platform/filesystem_notifications/test_windows.py 2012-06-22 14:17:44 +0000 |
| 407 | +++ tests/platform/filesystem_notifications/test_windows.py 2012-07-11 15:52:18 +0000 |
| 408 | @@ -51,14 +51,16 @@ |
| 409 | IN_OPEN, |
| 410 | ) |
| 411 | from ubuntuone.platform.filesystem_notifications import ( |
| 412 | - common, |
| 413 | windows as filesystem_notifications, |
| 414 | ) |
| 415 | -from ubuntuone.platform.filesystem_notifications.windows import ( |
| 416 | +from ubuntuone.platform.filesystem_notifications.common import ( |
| 417 | FilesystemMonitor, |
| 418 | NotifyProcessor, |
| 419 | Watch, |
| 420 | WatchManager, |
| 421 | +) |
| 422 | +from ubuntuone.platform.filesystem_notifications.windows import ( |
| 423 | + ACTIONS, |
| 424 | FILE_NOTIFY_CHANGE_FILE_NAME, |
| 425 | FILE_NOTIFY_CHANGE_DIR_NAME, |
| 426 | FILE_NOTIFY_CHANGE_ATTRIBUTES, |
| 427 | @@ -70,7 +72,7 @@ |
| 428 | |
| 429 | #create a rever mapping to use it in the tests. |
| 430 | REVERSE_WINDOWS_ACTIONS = {} |
| 431 | -for key, value in common.COMMON_ACTIONS.iteritems(): |
| 432 | +for key, value in ACTIONS.iteritems(): |
| 433 | REVERSE_WINDOWS_ACTIONS[value] = key |
| 434 | |
| 435 | |
| 436 | @@ -147,7 +149,7 @@ |
| 437 | # group all events in a single lists which is not what the COM API |
| 438 | # does. |
| 439 | str_events = [ |
| 440 | - (common.COMMON_ACTIONS_NAMES[action], path) for action, path in |
| 441 | + (filesystem_notifications.ACTIONS_NAMES[action], path) for action, path in |
| 442 | events] |
| 443 | self.raw_events.append(str_events) |
| 444 | return events |
| 445 | @@ -160,17 +162,14 @@ |
| 446 | |
| 447 | self.patch(filesystem_notifications, 'FILE_NOTIFY_INFORMATION', |
| 448 | file_notify_information_wrapper) |
| 449 | - self.patch(filesystem_notifications.Watch, '_path_is_dir', |
| 450 | - path_is_dir_wrapper) |
| 451 | + self.patch(Watch, '_path_is_dir', path_is_dir_wrapper) |
| 452 | |
| 453 | @defer.inlineCallbacks |
| 454 | - def _perform_operations(self, path, mask, auto_add, actions, |
| 455 | - number_events): |
| 456 | + def _perform_operations(self, path, mask, actions, number_events): |
| 457 | """Perform the file operations and returns the recorded events.""" |
| 458 | handler = TestCaseHandler(number_events=number_events) |
| 459 | manager = WatchManager(handler) |
| 460 | - yield manager.add_watch(os_helper.get_windows_valid_path(path), mask, |
| 461 | - auto_add=auto_add) |
| 462 | + yield manager.add_watch(os_helper.get_windows_valid_path(path), mask) |
| 463 | # change the logger so that we can check the logs if we wanted |
| 464 | manager._wdm[0].log.addHandler(self.memento) |
| 465 | # clean logger later |
| 466 | @@ -182,35 +181,6 @@ |
| 467 | self.addCleanup(manager.stop) |
| 468 | defer.returnValue(ret) |
| 469 | |
| 470 | - def _perform_timed_operations(self, path, mask, auto_add, actions, |
| 471 | - time_out): |
| 472 | - """Perform the file operations and returns the recorded events.""" |
| 473 | - manager = WatchManager() |
| 474 | - manager.add_watch(os_helper.get_windows_valid_path(path), mask, |
| 475 | - auto_add=auto_add) |
| 476 | - # change the logger so that we can check the logs if we wanted |
| 477 | - manager._wdm[0].log.addHandler(self.memento) |
| 478 | - # clean logger later |
| 479 | - self.addCleanup(manager._wdm[0].log.removeHandler, self.memento) |
| 480 | - # execution the actions |
| 481 | - actions() |
| 482 | - # process the recorded events |
| 483 | - time.sleep(time_out) |
| 484 | - events = self.handler.processed_events |
| 485 | - return events |
| 486 | - |
| 487 | - def _assert_logs(self, events): |
| 488 | - """Assert the debug logs.""" |
| 489 | - logs = [] |
| 490 | - msg = 'Is path %r a dir? %s' |
| 491 | - logs.extend([msg % data for data in self.paths_checked]) |
| 492 | - msg = 'Got from ReadDirectoryChangesW %r.' |
| 493 | - logs.extend([msg % actions for actions in self.raw_events]) |
| 494 | - msg = 'Pushing event %r to processor.' |
| 495 | - logs.extend([msg % e for e in events]) |
| 496 | - for msg in logs: |
| 497 | - self.assertTrue(self.memento.check_debug(msg)) |
| 498 | - |
| 499 | @defer.inlineCallbacks |
| 500 | def test_file_create(self): |
| 501 | """Test that the correct event is returned on a file create.""" |
| 502 | @@ -224,7 +194,7 @@ |
| 503 | os.fsync(fd) |
| 504 | fd.close() |
| 505 | |
| 506 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 507 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 508 | create_file, 1) |
| 509 | event = events[0] |
| 510 | self.assertFalse(event.dir) |
| 511 | @@ -234,8 +204,6 @@ |
| 512 | self.assertEqual('.', event.path) |
| 513 | self.assertEqual(os.path.join(self.basedir, file_name), event.pathname) |
| 514 | self.assertEqual(0, event.wd) |
| 515 | - # assert the logging |
| 516 | - self._assert_logs(events) |
| 517 | |
| 518 | @defer.inlineCallbacks |
| 519 | def test_dir_create(self): |
| 520 | @@ -246,7 +214,7 @@ |
| 521 | """Action for the test.""" |
| 522 | os.mkdir(dir_name) |
| 523 | |
| 524 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 525 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 526 | create_dir, 1) |
| 527 | event = events[0] |
| 528 | self.assertTrue(event.dir) |
| 529 | @@ -256,8 +224,6 @@ |
| 530 | self.assertEqual('.', event.path) |
| 531 | self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname) |
| 532 | self.assertEqual(0, event.wd) |
| 533 | - # assert the logging |
| 534 | - self._assert_logs(events) |
| 535 | |
| 536 | @defer.inlineCallbacks |
| 537 | def test_file_remove(self): |
| 538 | @@ -270,7 +236,7 @@ |
| 539 | """Action for the test.""" |
| 540 | os.remove(file_name) |
| 541 | |
| 542 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 543 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 544 | remove_file, 1) |
| 545 | event = events[0] |
| 546 | self.assertFalse(event.dir) |
| 547 | @@ -280,8 +246,6 @@ |
| 548 | self.assertEqual('.', event.path) |
| 549 | self.assertEqual(os.path.join(self.basedir, file_name), event.pathname) |
| 550 | self.assertEqual(0, event.wd) |
| 551 | - # assert the logging |
| 552 | - self._assert_logs(events) |
| 553 | |
| 554 | @defer.inlineCallbacks |
| 555 | def test_dir_remove(self): |
| 556 | @@ -294,7 +258,7 @@ |
| 557 | """Action for the test.""" |
| 558 | os.rmdir(dir_name) |
| 559 | |
| 560 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 561 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 562 | remove_dir, 1) |
| 563 | event = events[0] |
| 564 | self.assertTrue(event.dir) |
| 565 | @@ -303,8 +267,6 @@ |
| 566 | self.assertEqual('.', event.path) |
| 567 | self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname) |
| 568 | self.assertEqual(0, event.wd) |
| 569 | - # assert the logging |
| 570 | - self._assert_logs(events) |
| 571 | |
| 572 | @defer.inlineCallbacks |
| 573 | def test_file_write(self): |
| 574 | @@ -320,7 +282,7 @@ |
| 575 | fd.write('test') |
| 576 | fd.close() |
| 577 | |
| 578 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 579 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 580 | write_file, 1) |
| 581 | event = events[0] |
| 582 | self.assertFalse(event.dir) |
| 583 | @@ -330,8 +292,6 @@ |
| 584 | self.assertEqual('.', event.path) |
| 585 | self.assertEqual(os.path.join(self.basedir, file_name), event.pathname) |
| 586 | self.assertEqual(0, event.wd) |
| 587 | - # assert the logging |
| 588 | - self._assert_logs(events) |
| 589 | |
| 590 | @defer.inlineCallbacks |
| 591 | def test_file_moved_to_watched_dir_same_watcher(self): |
| 592 | @@ -348,7 +308,7 @@ |
| 593 | os.rename(from_file_name, to_file_name) |
| 594 | |
| 595 | events = yield self._perform_operations(self.basedir, self.mask, |
| 596 | - False, move_file, 2) |
| 597 | + move_file, 2) |
| 598 | move_from_event = events[0] |
| 599 | move_to_event = events[1] |
| 600 | # first test the move from |
| 601 | @@ -374,8 +334,6 @@ |
| 602 | self.assertEqual(0, move_to_event.wd) |
| 603 | # assert that both cookies are the same |
| 604 | self.assertEqual(move_from_event.cookie, move_to_event.cookie) |
| 605 | - # assert the logging |
| 606 | - self._assert_logs(events) |
| 607 | |
| 608 | @defer.inlineCallbacks |
| 609 | def test_file_moved_to_not_watched_dir(self): |
| 610 | @@ -392,7 +350,7 @@ |
| 611 | # while on linux we will have to do some sort of magic like facundo |
| 612 | # did, on windows we will get a deleted event which is much more |
| 613 | # cleaner, first time ever windows is nicer! |
| 614 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 615 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 616 | move_file, 1) |
| 617 | event = events[0] |
| 618 | self.assertFalse(event.dir) |
| 619 | @@ -403,8 +361,6 @@ |
| 620 | self.assertEqual(os.path.join(self.basedir, from_file_name), |
| 621 | event.pathname) |
| 622 | self.assertEqual(0, event.wd) |
| 623 | - # assert the logging |
| 624 | - self._assert_logs(events) |
| 625 | |
| 626 | @defer.inlineCallbacks |
| 627 | def test_file_move_from_not_watched_dir(self): |
| 628 | @@ -422,7 +378,7 @@ |
| 629 | |
| 630 | # while on linux we have to do some magic operations like facundo did |
| 631 | # on windows we have a created file, hurray! |
| 632 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 633 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 634 | move_files, 1) |
| 635 | event = events[0] |
| 636 | self.assertFalse(event.dir) |
| 637 | @@ -433,8 +389,6 @@ |
| 638 | self.assertEqual(os.path.join(self.basedir, to_file_name), |
| 639 | event.pathname) |
| 640 | self.assertEqual(0, event.wd) |
| 641 | - # assert the logging |
| 642 | - self._assert_logs(events) |
| 643 | |
| 644 | @defer.inlineCallbacks |
| 645 | def test_dir_moved_to_watched_dir_same_watcher(self): |
| 646 | @@ -450,7 +404,7 @@ |
| 647 | os.rename(from_dir_name, to_dir_name) |
| 648 | |
| 649 | events = yield self._perform_operations(self.basedir, |
| 650 | - self.mask, False, move_file, 2) |
| 651 | + self.mask, move_file, 2) |
| 652 | move_from_event = events[0] |
| 653 | move_to_event = events[1] |
| 654 | # first test the move from |
| 655 | @@ -475,8 +429,6 @@ |
| 656 | self.assertEqual(0, move_to_event.wd) |
| 657 | # assert that both cookies are the same |
| 658 | self.assertEqual(move_from_event.cookie, move_to_event.cookie) |
| 659 | - # assert the logging |
| 660 | - self._assert_logs(events) |
| 661 | |
| 662 | @defer.inlineCallbacks |
| 663 | def test_dir_moved_to_not_watched_dir(self): |
| 664 | @@ -491,7 +443,7 @@ |
| 665 | 'test_dir_moved_to_not_watched_dir')) |
| 666 | |
| 667 | # on windows a move to outside a watched dir translates to a remove |
| 668 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 669 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 670 | move_dir, 1) |
| 671 | event = events[0] |
| 672 | self.assertTrue(event.dir) |
| 673 | @@ -500,8 +452,6 @@ |
| 674 | self.assertEqual('.', event.path) |
| 675 | self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname) |
| 676 | self.assertEqual(0, event.wd) |
| 677 | - # assert the logging |
| 678 | - self._assert_logs(events) |
| 679 | |
| 680 | @defer.inlineCallbacks |
| 681 | def test_dir_move_from_not_watched_dir(self): |
| 682 | @@ -517,7 +467,7 @@ |
| 683 | """Action for the test.""" |
| 684 | os.rename(from_dir_name, to_dir_name) |
| 685 | |
| 686 | - events = yield self._perform_operations(self.basedir, self.mask, False, |
| 687 | + events = yield self._perform_operations(self.basedir, self.mask, |
| 688 | move_dir, 1) |
| 689 | event = events[0] |
| 690 | self.assertTrue(event.dir) |
| 691 | @@ -535,8 +485,7 @@ |
| 692 | manager = WatchManager(handler) |
| 693 | # add a watch that will always exclude all actions |
| 694 | manager.add_watch(os_helper.get_windows_valid_path(self.basedir), |
| 695 | - self.mask, auto_add=True, |
| 696 | - exclude_filter=lambda x: True) |
| 697 | + self.mask, exclude_filter=lambda x: True) |
| 698 | # execution the actions |
| 699 | file_name = os.path.join(self.basedir, 'test_file_create') |
| 700 | open(file_name, 'w').close() |
| 701 | @@ -545,21 +494,6 @@ |
| 702 | self.assertEqual(0, len(handler.processed_events)) |
| 703 | test_exclude_filter.skip = "we must rethink this test." |
| 704 | |
| 705 | - def test_open_dir_muted(self): |
| 706 | - """Test that the opening of dirs is ignored.""" |
| 707 | - dir_name = os.path.join(tempfile.mkdtemp(), 'test_dir_open') |
| 708 | - # create file before we record |
| 709 | - os.mkdir(dir_name) |
| 710 | - |
| 711 | - def open_dir(): |
| 712 | - """Action for the test.""" |
| 713 | - os.startfile(dir_name) |
| 714 | - |
| 715 | - events = self._perform_timed_operations(self.basedir, self.mask, False, |
| 716 | - open_dir, 2) |
| 717 | - self.assertEqual(0, len(events)) |
| 718 | - test_open_dir_muted.skip = "we must rethink this test." |
| 719 | - |
| 720 | def test_ignore_path(self): |
| 721 | """Test that events from a path are ignored.""" |
| 722 | events = [] |
| 723 | @@ -570,14 +504,14 @@ |
| 724 | |
| 725 | path = u'\\\\?\\C:\\path' # a valid windows path |
| 726 | child = u'child' |
| 727 | - watch = Watch(1, path, None, True, fake_processor) |
| 728 | + watch = Watch(1, path, fake_processor) |
| 729 | watch.ignore_path(os.path.join(path, child)) |
| 730 | paths_to_ignore = [] |
| 731 | for file_name in 'abcdef': |
| 732 | paths_to_ignore.append((1, os.path.join(child, file_name))) |
| 733 | # ensure that the watch is watching |
| 734 | watch._watching = True |
| 735 | - watch._process_events(paths_to_ignore) |
| 736 | + watch.platform_watch._process_events(paths_to_ignore) |
| 737 | self.assertEqual(0, len(events), |
| 738 | 'All events should have been ignored.') |
| 739 | |
| 740 | @@ -591,15 +525,15 @@ |
| 741 | |
| 742 | path = u'\\\\?\\C:\\path' # a valid windows path |
| 743 | child = u'child' |
| 744 | - watch = Watch(1, path, None, True, fake_processor) |
| 745 | + watch = Watch(1, path, fake_processor) |
| 746 | watch.ignore_path(os.path.join(path, child)) |
| 747 | paths_not_to_ignore = [] |
| 748 | for file_name in 'abcdef': |
| 749 | paths_not_to_ignore.append((1, os.path.join( |
| 750 | child + file_name, file_name))) |
| 751 | # ensure that the watch is watching |
| 752 | - watch._watching = True |
| 753 | - watch._process_events(paths_not_to_ignore) |
| 754 | + watch.platform_watch.watching = True |
| 755 | + watch.platform_watch._process_events(paths_not_to_ignore) |
| 756 | self.assertEqual(len(paths_not_to_ignore), len(events), |
| 757 | 'No events should have been ignored.') |
| 758 | |
| 759 | @@ -613,7 +547,7 @@ |
| 760 | |
| 761 | child = u'child' |
| 762 | path = u'\\\\?\\C:\\path\\' # a valid windows path |
| 763 | - watch = Watch(1, path, None, True, fake_processor) |
| 764 | + watch = Watch(1, path, fake_processor) |
| 765 | watch.ignore_path(os.path.join(path, child)) |
| 766 | paths_not_to_ignore = [] |
| 767 | paths_to_ignore = [] |
| 768 | @@ -624,8 +558,8 @@ |
| 769 | paths_not_to_ignore.append((1, valid)) |
| 770 | expected_events.append(os.path.join('C:\\path', valid)) |
| 771 | # ensure that the watch is watching |
| 772 | - watch._watching = True |
| 773 | - watch._process_events(paths_not_to_ignore) |
| 774 | + watch.platform_watch.watching = True |
| 775 | + watch.platform_watch._process_events(paths_not_to_ignore) |
| 776 | self.assertEqual(len(paths_not_to_ignore), len(events), |
| 777 | 'Wrong number of events ignored.') |
| 778 | self.assertTrue(all([event in expected_events for event in events]), |
| 779 | @@ -641,15 +575,15 @@ |
| 780 | |
| 781 | path = u'\\\\?\\C:\\path' # a valid windows path |
| 782 | child = u'child' |
| 783 | - watch = Watch(1, path, None, True, fake_processor) |
| 784 | + watch = Watch(1, path, fake_processor) |
| 785 | watch.ignore_path(os.path.join(path, child)) |
| 786 | watch.remove_ignored_path(os.path.join(path, child)) |
| 787 | paths_not_to_ignore = [] |
| 788 | for file_name in 'abcdef': |
| 789 | paths_not_to_ignore.append((1, os.path.join(child, file_name))) |
| 790 | # ensure that the watch is watching |
| 791 | - watch._watching = True |
| 792 | - watch._process_events(paths_not_to_ignore) |
| 793 | + watch.platform_watch.watching = True |
| 794 | + watch.platform_watch._process_events(paths_not_to_ignore) |
| 795 | self.assertEqual(len(paths_not_to_ignore), len(events), |
| 796 | 'All events should have been accepted.') |
| 797 | |
| 798 | @@ -664,7 +598,7 @@ |
| 799 | path = u'\\\\?\\C:\\path' # a valid windows path |
| 800 | child_a = u'childa' |
| 801 | child_b = u'childb' |
| 802 | - watch = Watch(1, path, None, True, fake_processor) |
| 803 | + watch = Watch(1, path, fake_processor) |
| 804 | watch.ignore_path(os.path.join(path, child_a)) |
| 805 | watch.ignore_path(os.path.join(path, child_b)) |
| 806 | watch.remove_ignored_path(os.path.join(path, child_a)) |
| 807 | @@ -677,8 +611,8 @@ |
| 808 | paths_not_to_ignore.append((1, valid)) |
| 809 | expected_events.append(os.path.join('C:\\path', valid)) |
| 810 | # ensure that the watch is watching |
| 811 | - watch._watching = True |
| 812 | - watch._process_events(paths_not_to_ignore) |
| 813 | + watch.platform_watch.watching = True |
| 814 | + watch.platform_watch._process_events(paths_not_to_ignore) |
| 815 | self.assertEqual(len(paths_not_to_ignore), len(events), |
| 816 | 'All events should have been accepted.') |
| 817 | self.assertTrue(all([event in expected_events for event in events]), |
| 818 | @@ -694,9 +628,9 @@ |
| 819 | method_args.append((args, kwargs),) |
| 820 | |
| 821 | path = u'\\\\?\\C:\\path' # a valid windows path |
| 822 | - watch = Watch(1, path, None, True, None) |
| 823 | - yield watch._watch_started_deferred.callback(True) |
| 824 | - watch._call_deferred(fake_call, None) |
| 825 | + watch = Watch(1, path, None) |
| 826 | + yield watch.platform_watch._watch_started_deferred.callback(True) |
| 827 | + watch.platform_watch._call_deferred(fake_call, None) |
| 828 | self.assertEqual(0, len(method_args)) |
| 829 | |
| 830 | def test_call_deferred_not_called(self): |
| 831 | @@ -708,21 +642,21 @@ |
| 832 | method_args.append((args, kwargs),) |
| 833 | |
| 834 | path = u'\\\\?\\C:\\path' # a valid windows path |
| 835 | - watch = Watch(1, path, None, True, None) |
| 836 | - watch._call_deferred(fake_call, None) |
| 837 | + watch = Watch(1, path, None) |
| 838 | + watch.platform_watch._call_deferred(fake_call, None) |
| 839 | self.assertEqual(1, len(method_args)) |
| 840 | |
| 841 | def test_started_property(self): |
| 842 | """Test that the started property returns the started deferred.""" |
| 843 | path = u'\\\\?\\C:\\path' # a valid windows path |
| 844 | - watch = Watch(1, path, None, True, None) |
| 845 | - self.assertEqual(watch.started, watch._watch_started_deferred) |
| 846 | + watch = Watch(1, path, None) |
| 847 | + self.assertEqual(watch.started, watch.platform_watch._watch_started_deferred) |
| 848 | |
| 849 | def test_stopped_property(self): |
| 850 | """Test that the stopped property returns the stopped deferred.""" |
| 851 | path = u'\\\\?\\C:\\path' # a valid windows path |
| 852 | - watch = Watch(1, path, None, True, None) |
| 853 | - self.assertEqual(watch.stopped, watch._watch_stopped_deferred) |
| 854 | + watch = Watch(1, path, None) |
| 855 | + self.assertEqual(watch.stopped, watch.platform_watch._watch_stopped_deferred) |
| 856 | |
| 857 | def random_error(self, *args): |
| 858 | """Throw a fake exception.""" |
| 859 | @@ -733,7 +667,7 @@ |
| 860 | """An early failure inside the thread should errback the deferred.""" |
| 861 | test_path = self.mktemp("test_directory") |
| 862 | self.patch(filesystem_notifications, "CreateFileW", self.random_error) |
| 863 | - watch = Watch(1, test_path, None, True, None) |
| 864 | + watch = Watch(1, test_path, None) |
| 865 | d = watch.start_watching() |
| 866 | yield self.assertFailure(d, FakeException) |
| 867 | |
| 868 | @@ -743,7 +677,7 @@ |
| 869 | test_path = self.mktemp("test_directory") |
| 870 | self.patch(filesystem_notifications, "ReadDirectoryChangesW", |
| 871 | self.random_error) |
| 872 | - watch = Watch(1, test_path, None, True, None) |
| 873 | + watch = Watch(1, test_path, None) |
| 874 | d = watch.start_watching() |
| 875 | yield self.assertFailure(d, FakeException) |
| 876 | |
| 877 | @@ -757,7 +691,7 @@ |
| 878 | close_called.append) |
| 879 | self.patch(filesystem_notifications, "ReadDirectoryChangesW", |
| 880 | self.random_error) |
| 881 | - watch = Watch(1, test_path, self.mask, True, None) |
| 882 | + watch = Watch(1, test_path, None) |
| 883 | d = watch.start_watching() |
| 884 | yield self.assertFailure(d, FakeException) |
| 885 | self.assertEqual(len(close_called), 1) |
| 886 | @@ -767,18 +701,18 @@ |
| 887 | def test_stop_watching_fired_when_watch_thread_finishes(self): |
| 888 | """The deferred returned is fired when the watch thread finishes.""" |
| 889 | test_path = self.mktemp("another_test_directory") |
| 890 | - watch = Watch(1, test_path, self.mask, True, None) |
| 891 | + watch = Watch(1, test_path, None) |
| 892 | yield watch.start_watching() |
| 893 | - self.assertNotEqual(watch._watch_handle, None) |
| 894 | + self.assertNotEqual(watch.platform_watch._watch_handle, None) |
| 895 | yield watch.stop_watching() |
| 896 | - self.assertEqual(watch._watch_handle, None) |
| 897 | + self.assertEqual(watch.platform_watch._watch_handle, None) |
| 898 | |
| 899 | def test_is_path_dir_missing_no_subdir(self): |
| 900 | """Test when the path does not exist and is no a subdir.""" |
| 901 | path = u'\\\\?\\C:\\path\\to\\no\\dir' |
| 902 | test_path = self.mktemp("test_directory") |
| 903 | self.patch(os.path, 'exists', lambda path: False) |
| 904 | - watch = Watch(1, test_path, self.mask, True, None) |
| 905 | + watch = Watch(1, test_path, None) |
| 906 | self.assertFalse(watch._path_is_dir(path)) |
| 907 | |
| 908 | def test_is_path_dir_missing_in_subdir(self): |
| 909 | @@ -786,7 +720,7 @@ |
| 910 | path = u'\\\\?\\C:\\path\\to\\no\\dir' |
| 911 | test_path = self.mktemp("test_directory") |
| 912 | self.patch(os.path, 'exists', lambda path: False) |
| 913 | - watch = Watch(1, test_path, self.mask, True, None) |
| 914 | + watch = Watch(1, test_path, None) |
| 915 | watch._subdirs.add(path) |
| 916 | self.assertTrue(watch._path_is_dir(path)) |
| 917 | |
| 918 | @@ -796,7 +730,7 @@ |
| 919 | test_path = self.mktemp("test_directory") |
| 920 | self.patch(os.path, 'exists', lambda path: True) |
| 921 | self.patch(os.path, 'isdir', lambda path: True) |
| 922 | - watch = Watch(1, test_path, self.mask, True, None) |
| 923 | + watch = Watch(1, test_path, None) |
| 924 | watch._subdirs.add(path) |
| 925 | self.assertTrue(watch._path_is_dir(path)) |
| 926 | |
| 927 | @@ -806,7 +740,7 @@ |
| 928 | test_path = self.mktemp("test_directory") |
| 929 | self.patch(os.path, 'exists', lambda path: True) |
| 930 | self.patch(os.path, 'isdir', lambda path: False) |
| 931 | - watch = Watch(1, test_path, self.mask, True, None) |
| 932 | + watch = Watch(1, test_path, None) |
| 933 | watch._subdirs.add(path) |
| 934 | self.assertFalse(watch._path_is_dir(path)) |
| 935 | |
| 936 | @@ -814,7 +748,7 @@ |
| 937 | """Test when we update on a create event and not present.""" |
| 938 | path = u'\\\\?\\C:\\path\\to\\no\\dir' |
| 939 | test_path = self.mktemp("test_directory") |
| 940 | - watch = Watch(1, test_path, self.mask, True, None) |
| 941 | + watch = Watch(1, test_path, None) |
| 942 | watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_CREATE]) |
| 943 | self.assertTrue(path in watch._subdirs) |
| 944 | |
| 945 | @@ -822,7 +756,7 @@ |
| 946 | """Test when we update on a create event and is present.""" |
| 947 | path = u'\\\\?\\C:\\path\\to\\no\\dir' |
| 948 | test_path = self.mktemp("test_directory") |
| 949 | - watch = Watch(1, test_path, self.mask, True, None) |
| 950 | + watch = Watch(1, test_path, None) |
| 951 | watch._subdirs.add(path) |
| 952 | old_length = len(watch._subdirs) |
| 953 | watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_CREATE]) |
| 954 | @@ -833,7 +767,7 @@ |
| 955 | """Test when we delete and is not present.""" |
| 956 | path = u'\\\\?\\C:\\path\\to\\no\\dir' |
| 957 | test_path = self.mktemp("test_directory") |
| 958 | - watch = Watch(1, test_path, self.mask, True, None) |
| 959 | + watch = Watch(1, test_path, None) |
| 960 | watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_DELETE]) |
| 961 | self.assertTrue(path not in watch._subdirs) |
| 962 | |
| 963 | @@ -841,7 +775,7 @@ |
| 964 | """Test when we delete and is present.""" |
| 965 | path = u'\\\\?\\C:\\path\\to\\no\\dir' |
| 966 | test_path = self.mktemp("test_directory") |
| 967 | - watch = Watch(1, test_path, self.mask, True, None) |
| 968 | + watch = Watch(1, test_path, None) |
| 969 | watch._subdirs.add(path) |
| 970 | watch._update_subdirs(path, REVERSE_WINDOWS_ACTIONS[IN_DELETE]) |
| 971 | self.assertTrue(path not in watch._subdirs) |
| 972 | @@ -856,7 +790,7 @@ |
| 973 | yield super(TestWatchManager, self).setUp() |
| 974 | self.parent_path = u'\\\\?\\C:\\' # a valid windows path |
| 975 | self.path = self.parent_path + u'path' |
| 976 | - self.watch = Watch(1, self.path, None, True, None) |
| 977 | + self.watch = Watch(1, self.path, None) |
| 978 | self.manager = WatchManager(None) |
| 979 | self.manager._wdm = {1: self.watch} |
| 980 | |
| 981 | @@ -884,7 +818,7 @@ |
| 982 | |
| 983 | self.patch(Watch, "stop_watching", fake_stop_watching) |
| 984 | second_path = self.parent_path + u"second_path" |
| 985 | - second_watch = Watch(2, second_path, None, True, None) |
| 986 | + second_watch = Watch(2, second_path, None) |
| 987 | self.manager._wdm[2] = second_watch |
| 988 | d = self.manager.stop() |
| 989 | self.assertFalse(d.called, "Not fired before all watches end") |
| 990 | @@ -928,19 +862,10 @@ |
| 991 | self.manager._wdm = {} |
| 992 | |
| 993 | mask = 'bit_mask' |
| 994 | - auto_add = True |
| 995 | - self.manager.add_watch(self.path, mask, auto_add=auto_add) |
| 996 | + self.manager.add_watch(self.path, mask) |
| 997 | self.assertEqual(1, len(self.manager._wdm)) |
| 998 | self.assertTrue(self.was_called, 'The watch start was not called.') |
| 999 | - self.assertEqual(self.path + os.path.sep, self.manager._wdm[0]._path) |
| 1000 | - self.assertEqual(mask, self.manager._wdm[0]._mask) |
| 1001 | - self.assertEqual(auto_add, self.manager._wdm[0]._auto_add) |
| 1002 | - |
| 1003 | - def test_update_present_watch(self): |
| 1004 | - """Test the update of a present watch.""" |
| 1005 | - mask = 'mask' |
| 1006 | - self.assertRaises(NotImplementedError, self.manager.update_watch, |
| 1007 | - 1, mask) |
| 1008 | + self.assertEqual(self.path + os.path.sep, self.manager._wdm[0].path) |
| 1009 | |
| 1010 | def test_get_watch_present_wd(self): |
| 1011 | """Test that the correct path is returned.""" |
| 1012 | @@ -988,7 +913,8 @@ |
| 1013 | self.manager.rm_path(self.path) |
| 1014 | self.assertEqual(self.watch, self.manager._wdm.get(1)) |
| 1015 | self.watch._watching = True |
| 1016 | - self.watch._process_events([(1, os.path.join(self.path, 'test'))]) |
| 1017 | + self.watch.platform_watch._process_events( |
| 1018 | + [(1, os.path.join(self.path, 'test'))]) |
| 1019 | self.assertEqual(0, len(events)) |
| 1020 | |
| 1021 | def test_rm_child_path(self): |
| 1022 | @@ -1004,11 +930,12 @@ |
| 1023 | self.manager.rm_path(child) |
| 1024 | self.assertEqual(self.watch, self.manager._wdm[1]) |
| 1025 | # assert that the correct event is ignored |
| 1026 | - self.watch._watching = True |
| 1027 | - self.watch._process_events([(1, os.path.join('child', 'test'))]) |
| 1028 | + self.watch.platform_watch.watching = True |
| 1029 | + self.watch.platform_watch._process_events( |
| 1030 | + [(1, os.path.join('child', 'test'))]) |
| 1031 | self.assertEqual(0, len(events)) |
| 1032 | # assert that other events are not ignored |
| 1033 | - self.watch._process_events([(1, 'test')]) |
| 1034 | + self.watch.platform_watch._process_events([(1, 'test')]) |
| 1035 | self.assertEqual(1, len(events)) |
| 1036 | |
| 1037 | |
| 1038 | @@ -1160,9 +1087,9 @@ |
| 1039 | """Test that we do indeed ignore the correct paths.""" |
| 1040 | not_ignored = 'test' |
| 1041 | ignored = not_ignored + '.lnk' |
| 1042 | - self.assertFalse(self.processor.platform_is_ignored(not_ignored), |
| 1043 | + self.assertFalse(filesystem_notifications.path_is_ignored(not_ignored), |
| 1044 | 'Only links should be ignored.') |
| 1045 | - self.assertTrue(self.processor.platform_is_ignored(ignored), |
| 1046 | + self.assertTrue(filesystem_notifications.path_is_ignored(ignored), |
| 1047 | 'Links should be ignored.') |
| 1048 | |
| 1049 | def test_is_ignored(self): |
| 1050 | |
| 1051 | === modified file 'ubuntuone/platform/filesystem_notifications/__init__.py' |
| 1052 | --- ubuntuone/platform/filesystem_notifications/__init__.py 2012-06-28 16:44:01 +0000 |
| 1053 | +++ ubuntuone/platform/filesystem_notifications/__init__.py 2012-07-11 15:52:18 +0000 |
| 1054 | @@ -31,14 +31,10 @@ |
| 1055 | import sys |
| 1056 | |
| 1057 | |
| 1058 | -if sys.platform == "win32": |
| 1059 | - from ubuntuone.platform.filesystem_notifications import windows |
| 1060 | - FilesystemMonitor = windows.FilesystemMonitor |
| 1061 | - _GeneralINotifyProcessor = windows.NotifyProcessor |
| 1062 | -elif sys.platform == "darwin": |
| 1063 | - from ubuntuone.platform.filesystem_notifications import darwin |
| 1064 | - FilesystemMonitor = darwin.FilesystemMonitor |
| 1065 | - _GeneralINotifyProcessor = darwin.NotifyProcessor |
| 1066 | +if sys.platform in ('darwin', 'win32'): |
| 1067 | + from ubuntuone.platform.filesystem_notifications import common |
| 1068 | + FilesystemMonitor = common.FilesystemMonitor |
| 1069 | + _GeneralINotifyProcessor = common.NotifyProcessor |
| 1070 | else: |
| 1071 | from ubuntuone.platform.filesystem_notifications import linux |
| 1072 | FilesystemMonitor = linux.FilesystemMonitor |
| 1073 | |
| 1074 | === modified file 'ubuntuone/platform/filesystem_notifications/common.py' |
| 1075 | --- ubuntuone/platform/filesystem_notifications/common.py 2012-06-28 16:44:01 +0000 |
| 1076 | +++ ubuntuone/platform/filesystem_notifications/common.py 2012-07-11 15:52:18 +0000 |
| 1077 | @@ -30,6 +30,7 @@ |
| 1078 | |
| 1079 | import logging |
| 1080 | import os |
| 1081 | +import sys |
| 1082 | |
| 1083 | from twisted.internet import defer |
| 1084 | |
| 1085 | @@ -44,9 +45,9 @@ |
| 1086 | IN_IGNORED, |
| 1087 | IN_ISDIR, |
| 1088 | IN_DELETE, |
| 1089 | - IN_MODIFY as in_modify, |
| 1090 | IN_MOVED_FROM, |
| 1091 | - IN_MOVED_TO) |
| 1092 | + IN_MOVED_TO, |
| 1093 | +) |
| 1094 | |
| 1095 | from ubuntuone import logger |
| 1096 | |
| 1097 | @@ -56,16 +57,28 @@ |
| 1098 | os_path, |
| 1099 | ) |
| 1100 | |
| 1101 | +if sys.platform == 'darwin': |
| 1102 | + from ubuntuone.platform.filesystem_notifications import darwin |
| 1103 | + source = darwin |
| 1104 | +elif sys.platform == 'win32': |
| 1105 | + from ubuntuone.platform.filesystem_notifications import windows |
| 1106 | + source = windows |
| 1107 | +else: |
| 1108 | + raise ImportError('Not supported platform') |
| 1109 | + |
| 1110 | # a map between the few events that we have on common platforms and those |
| 1111 | # found in pyinotify |
| 1112 | -COMMON_ACTIONS = {} |
| 1113 | +ACTIONS = source.ACTIONS |
| 1114 | |
| 1115 | # a map of the actions to names so that we have better logs. |
| 1116 | -COMMON_ACTIONS_NAMES = {} |
| 1117 | - |
| 1118 | -# We should have this here, because we use if from other modules that |
| 1119 | -# share this, but we need to declare it this way yo avoid flakes issues. |
| 1120 | -IN_MODIFY = in_modify |
| 1121 | +ACTIONS_NAMES = source.ACTIONS_NAMES |
| 1122 | + |
| 1123 | +# ignore paths in the platform, mainly links atm |
| 1124 | +path_is_ignored = source.path_is_ignored |
| 1125 | + |
| 1126 | +# the base class to be use for a platform |
| 1127 | +PlatformWatch = source.Watch |
| 1128 | +PlatformWatchManager = source.WatchManager |
| 1129 | |
| 1130 | # translates quickly the event and it's is_dir state to our standard events |
| 1131 | NAME_TRANSLATIONS = { |
| 1132 | @@ -89,42 +102,50 @@ |
| 1133 | class Watch(object): |
| 1134 | """Implement the same functions as pyinotify.Watch.""" |
| 1135 | |
| 1136 | - def __init__(self, watch_descriptor, path, mask, auto_add, processor, |
| 1137 | - buf_size=8192): |
| 1138 | - self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.common.' + |
| 1139 | - 'filesystem_notifications.Watch') |
| 1140 | - self.log.setLevel(TRACE) |
| 1141 | + def __init__(self, watch_descriptor, path, processor): |
| 1142 | + """Create a new watch.""" |
| 1143 | + |
| 1144 | + # do ensure that we provide a os.path.sep |
| 1145 | + if not path.endswith(os.path.sep): |
| 1146 | + path += os.path.sep |
| 1147 | + self.path = path |
| 1148 | + self.ignore_paths = [] |
| 1149 | self._processor = processor |
| 1150 | - self._buf_size = buf_size |
| 1151 | - self._watching = False |
| 1152 | self._descriptor = watch_descriptor |
| 1153 | - self._auto_add = auto_add |
| 1154 | - self._ignore_paths = [] |
| 1155 | self._cookie = None |
| 1156 | self._source_pathname = None |
| 1157 | - self._process_thread = None |
| 1158 | - self._watch_handle = None |
| 1159 | # remember the subdirs we have so that when we have a delete we can |
| 1160 | # check if it was a remove |
| 1161 | self._subdirs = set() |
| 1162 | - # ensure that we work with an abspath and that we can deal with |
| 1163 | - # long paths over 260 chars. |
| 1164 | - if not path.endswith(os.path.sep): |
| 1165 | - path += os.path.sep |
| 1166 | - self._path = os.path.abspath(path) |
| 1167 | - self._mask = mask |
| 1168 | - |
| 1169 | - def _process_events_from_filesystem(self, action, file_name, cookie, |
| 1170 | - syncdaemon_path): |
| 1171 | + |
| 1172 | + # platform watch used to deal with the platform details |
| 1173 | + self.platform_watch = PlatformWatch(self.path, self.process_events) |
| 1174 | + |
| 1175 | + self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.common.' + |
| 1176 | + 'filesystem_notifications.Watch') |
| 1177 | + self.log.setLevel(TRACE) |
| 1178 | + |
| 1179 | + def process_events(self, action, file_name, cookie, syncdaemon_path): |
| 1180 | """Process the events from the queue.""" |
| 1181 | + # do not process events when the watch was stopped |
| 1182 | + if not self.platform_watch.watching: |
| 1183 | + return |
| 1184 | + |
| 1185 | + # do not process those events that should be ignored |
| 1186 | + if any([file_name.startswith(path) |
| 1187 | + for path in self.ignore_paths]): |
| 1188 | + return |
| 1189 | + |
| 1190 | # map the filesystem events to the pyinotify ones, tis is dirty but |
| 1191 | # makes the multiplatform better, linux was first :P |
| 1192 | - full_dir_path = os.path.join(self._path, file_name) |
| 1193 | + full_dir_path = os.path.join(self.path, file_name) |
| 1194 | is_dir = self._path_is_dir(full_dir_path) |
| 1195 | + |
| 1196 | if is_dir: |
| 1197 | # we need to update the list of subdirs that we have |
| 1198 | self._update_subdirs(full_dir_path, action) |
| 1199 | - mask = COMMON_ACTIONS[action] |
| 1200 | + |
| 1201 | + mask = ACTIONS[action] |
| 1202 | head, tail = os.path.split(file_name) |
| 1203 | if is_dir: |
| 1204 | mask |= IN_ISDIR |
| 1205 | @@ -134,14 +155,14 @@ |
| 1206 | 'mask': mask, |
| 1207 | 'name': tail, |
| 1208 | 'path': '.'} |
| 1209 | - # by the way in which the win api fires the events we know for |
| 1210 | + # by the way in which the api fires the events we know for |
| 1211 | # sure that no move events will be added in the wrong order, this |
| 1212 | # is kind of hacky, I dont like it too much |
| 1213 | - if COMMON_ACTIONS[action] == IN_MOVED_FROM: |
| 1214 | + if ACTIONS[action] == IN_MOVED_FROM: |
| 1215 | self._cookie = cookie |
| 1216 | self._source_pathname = tail |
| 1217 | event_raw_data['cookie'] = self._cookie |
| 1218 | - if COMMON_ACTIONS[action] == IN_MOVED_TO: |
| 1219 | + if ACTIONS[action] == IN_MOVED_TO: |
| 1220 | event_raw_data['src_pathname'] = self._source_pathname |
| 1221 | event_raw_data['cookie'] = self._cookie |
| 1222 | event = Event(event_raw_data) |
| 1223 | @@ -161,9 +182,9 @@ |
| 1224 | The given path is considered to be a path and therefore this |
| 1225 | will not be checked. |
| 1226 | """ |
| 1227 | - if COMMON_ACTIONS[event] == IN_CREATE: |
| 1228 | + if ACTIONS[event] == IN_CREATE: |
| 1229 | self._subdirs.add(path) |
| 1230 | - elif COMMON_ACTIONS[event] == IN_DELETE and path in self._subdirs: |
| 1231 | + elif ACTIONS[event] == IN_DELETE and path in self._subdirs: |
| 1232 | self._subdirs.remove(path) |
| 1233 | |
| 1234 | @is_valid_os_path(path_indexes=[1]) |
| 1235 | @@ -187,51 +208,53 @@ |
| 1236 | """Add the path of the events to ignore.""" |
| 1237 | if not path.endswith(os.path.sep): |
| 1238 | path += os.path.sep |
| 1239 | - if path.startswith(self._path): |
| 1240 | - path = path[len(self._path):] |
| 1241 | - self._ignore_paths.append(path) |
| 1242 | + if path.startswith(self.path): |
| 1243 | + path = path[len(self.path):] |
| 1244 | + self.ignore_paths.append(path) |
| 1245 | |
| 1246 | @is_valid_os_path(path_indexes=[1]) |
| 1247 | def remove_ignored_path(self, path): |
| 1248 | """Reaccept path.""" |
| 1249 | if not path.endswith(os.path.sep): |
| 1250 | path += os.path.sep |
| 1251 | - if path.startswith(self._path): |
| 1252 | - path = path[len(self._path):] |
| 1253 | - if path in self._ignore_paths: |
| 1254 | - self._ignore_paths.remove(path) |
| 1255 | + if path.startswith(self.path): |
| 1256 | + path = path[len(self.path):] |
| 1257 | + if path in self.ignore_paths: |
| 1258 | + self.ignore_paths.remove(path) |
| 1259 | |
| 1260 | + @defer.inlineCallbacks |
| 1261 | def start_watching(self): |
| 1262 | """Tell the watch to start processing events.""" |
| 1263 | - for current_child in os.listdir(self._path): |
| 1264 | - full_child_path = os.path.join(self._path, current_child) |
| 1265 | + for current_child in os.listdir(self.path): |
| 1266 | + full_child_path = os.path.join(self.path, current_child) |
| 1267 | if os.path.isdir(full_child_path): |
| 1268 | self._subdirs.add(full_child_path) |
| 1269 | # start to diff threads, one to watch the path, the other to |
| 1270 | # process the events. |
| 1271 | self.log.debug('Start watching path.') |
| 1272 | - self._watching = True |
| 1273 | + yield self.platform_watch.start_watching() |
| 1274 | |
| 1275 | def stop_watching(self): |
| 1276 | """Tell the watch to stop processing events.""" |
| 1277 | - self.log.info('Stop watching %s', self._path) |
| 1278 | - self._watching = False |
| 1279 | + self.log.info('Stop watching %s', self.path) |
| 1280 | + self.platform_watch.watching = False |
| 1281 | self._subdirs = set() |
| 1282 | - |
| 1283 | - def update(self, mask, auto_add=False): |
| 1284 | - """Update the info used by the watcher.""" |
| 1285 | - self.log.debug('update(%s, %s)', mask, auto_add) |
| 1286 | - self._mask = mask |
| 1287 | - self._auto_add = auto_add |
| 1288 | - |
| 1289 | - @property |
| 1290 | - def path(self): |
| 1291 | - """Return the patch watched.""" |
| 1292 | - return self._path |
| 1293 | - |
| 1294 | - @property |
| 1295 | - def auto_add(self): |
| 1296 | - return self._auto_add |
| 1297 | + return self.platform_watch.stop_watching() |
| 1298 | + |
| 1299 | + @property |
| 1300 | + def watching(self): |
| 1301 | + """Return if we are watching.""" |
| 1302 | + return self.platform_watch.watching |
| 1303 | + |
| 1304 | + @property |
| 1305 | + def started(self): |
| 1306 | + """A deferred that will be called when the watch is running.""" |
| 1307 | + return self.platform_watch.started |
| 1308 | + |
| 1309 | + @property |
| 1310 | + def stopped(self): |
| 1311 | + """A deferred fired when the watch thread has finished.""" |
| 1312 | + return self.platform_watch.stopped |
| 1313 | |
| 1314 | |
| 1315 | class WatchManager(object): |
| 1316 | @@ -243,25 +266,48 @@ |
| 1317 | |
| 1318 | def __init__(self, processor): |
| 1319 | """Init the manager to keep trak of the different watches.""" |
| 1320 | - self._processor = processor |
| 1321 | self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.common.' |
| 1322 | + 'filesystem_notifications.WatchManager') |
| 1323 | self.log.setLevel(TRACE) |
| 1324 | + self._processor = processor |
| 1325 | + # use the platform manager to perform the actual actions |
| 1326 | + self.manager = PlatformWatchManager(self.log) |
| 1327 | self._wdm = {} |
| 1328 | - self._wd_count = 0 |
| 1329 | self._ignored_paths = [] |
| 1330 | |
| 1331 | - def add_watch(self, path, mask, auto_add=False, quiet=True): |
| 1332 | - """Add a new path to be watch. |
| 1333 | + @defer.inlineCallbacks |
| 1334 | + def _add_single_watch(self, path, mask, quiet=True): |
| 1335 | + """A just one watch.""" |
| 1336 | + if path in self._ignored_paths: |
| 1337 | + # simply removed it from the filter |
| 1338 | + self._ignored_paths.remove(path) |
| 1339 | + return |
| 1340 | + |
| 1341 | + # we need to add a new watch |
| 1342 | + self.log.debug('add_single_watch(%s, %s, %s)', path, mask, quiet) |
| 1343 | + |
| 1344 | + # common code that will ensure that we keep track of the watches |
| 1345 | + watch = Watch(len(self._wdm), path, self._processor) |
| 1346 | + self._wdm[len(self._wdm)] = watch |
| 1347 | + yield watch.start_watching() |
| 1348 | + |
| 1349 | + # trust that the platform watch manager to do the rest of the start |
| 1350 | + # operations |
| 1351 | + defer.returnValue(self.manager.add_watch(watch)) |
| 1352 | + |
| 1353 | + @is_valid_os_path(path_indexes=[1]) |
| 1354 | + def add_watch(self, path, mask, quiet=True): |
| 1355 | + """Add a new path to be watched. |
| 1356 | |
| 1357 | The method will ensure that the path is not already present. |
| 1358 | """ |
| 1359 | - raise NotImplementedError("Not implemented on this platform.") |
| 1360 | - |
| 1361 | - def stop(self): |
| 1362 | - """Close the manager and stop all watches.""" |
| 1363 | - # Should be implemented for each platform |
| 1364 | - raise NotImplementedError("Not implemented on this platform.") |
| 1365 | + wd = self.get_wd(path) |
| 1366 | + if wd is None: |
| 1367 | + self.log.debug('Adding single watch on %r', path) |
| 1368 | + return self._add_single_watch(path, mask, quiet) |
| 1369 | + else: |
| 1370 | + self.log.debug('Watch already exists on %r', path) |
| 1371 | + return self._wdm[wd].started |
| 1372 | |
| 1373 | def get_watch(self, wd): |
| 1374 | """Return the watch with the given descriptor.""" |
| 1375 | @@ -274,30 +320,13 @@ |
| 1376 | watch = self._wdm[wd] |
| 1377 | yield watch.stop_watching() |
| 1378 | del self._wdm[wd] |
| 1379 | + # trust that the platform watch manager will do the rest of the |
| 1380 | + # operations needed to delete a watch |
| 1381 | + self.manager.del_watch(watch) |
| 1382 | self.log.debug('Watch %s removed.', wd) |
| 1383 | except KeyError, e: |
| 1384 | logging.error(str(e)) |
| 1385 | |
| 1386 | - def _add_single_watch(self, path, mask, auto_add=False, quiet=True): |
| 1387 | - if path in self._ignored_paths: |
| 1388 | - # simply removed it from the filter |
| 1389 | - self._ignored_paths.remove(path) |
| 1390 | - return |
| 1391 | - # we need to add a new watch |
| 1392 | - self.log.debug('add_single_watch(%s, %s, %s, %s)', path, mask, |
| 1393 | - auto_add, quiet) |
| 1394 | - |
| 1395 | - return self._adding_watch(path, mask, auto_add) |
| 1396 | - |
| 1397 | - def _adding_watch(self, path, mask, auto_add): |
| 1398 | - """Add the watch to the dict and start it.""" |
| 1399 | - # This should be implemented for each OS |
| 1400 | - raise NotImplementedError("Not implemented on this platform.") |
| 1401 | - |
| 1402 | - def update_watch(self, wd, mask=None, rec=False, |
| 1403 | - auto_add=False, quiet=True): |
| 1404 | - raise NotImplementedError("Not implemented on this platform.") |
| 1405 | - |
| 1406 | @is_valid_os_path(path_indexes=[1]) |
| 1407 | def get_wd(self, path): |
| 1408 | """Return the watcher that is used to watch the given path.""" |
| 1409 | @@ -305,8 +334,7 @@ |
| 1410 | path = path + os.path.sep |
| 1411 | for current_wd in self._wdm: |
| 1412 | watch_path = self._wdm[current_wd].path |
| 1413 | - if ((watch_path == path or ( |
| 1414 | - watch_path in path and self._wdm[current_wd].auto_add)) |
| 1415 | + if ((watch_path == path or watch_path in path) |
| 1416 | and path not in self._ignored_paths): |
| 1417 | return current_wd |
| 1418 | |
| 1419 | @@ -323,6 +351,9 @@ |
| 1420 | watch = self._wdm[wd] |
| 1421 | yield watch.stop_watching() |
| 1422 | del self._wdm[wd] |
| 1423 | + # trust that the platform watch manager will do the rest of the |
| 1424 | + # operations needed to delete a watch |
| 1425 | + self.manager.rm_watch(watch) |
| 1426 | except KeyError, err: |
| 1427 | self.log.error(str(err)) |
| 1428 | if not quiet: |
| 1429 | @@ -336,6 +367,16 @@ |
| 1430 | self.log.debug('Adding exclude filter for %r', path) |
| 1431 | self._wdm[wd].ignore_path(path) |
| 1432 | |
| 1433 | + @defer.inlineCallbacks |
| 1434 | + def stop(self): |
| 1435 | + """Close the manager and stop all watches.""" |
| 1436 | + self.log.debug('Stopping watches.') |
| 1437 | + for current_wd in self._wdm: |
| 1438 | + watch = self._wdm[current_wd] |
| 1439 | + yield self.manager.stop_watch(watch) |
| 1440 | + self.log.debug('Stopping Watch on %r.', watch.path) |
| 1441 | + yield self.manager.stop() |
| 1442 | + |
| 1443 | |
| 1444 | class NotifyProcessor(ProcessEvent): |
| 1445 | """Processor that takes care of dealing with the events. |
| 1446 | @@ -350,7 +391,7 @@ |
| 1447 | GeneralINotifyProcessor) |
| 1448 | self.general_processor = GeneralINotifyProcessor(monitor, |
| 1449 | self.handle_dir_delete, NAME_TRANSLATIONS, |
| 1450 | - self.platform_is_ignored, IN_IGNORED, ignore_config=ignore_config) |
| 1451 | + path_is_ignored, IN_IGNORED, ignore_config=ignore_config) |
| 1452 | self.held_event = None |
| 1453 | |
| 1454 | def rm_from_mute_filter(self, event, paths): |
| 1455 | @@ -361,11 +402,6 @@ |
| 1456 | """Add an event and path(s) to the mute filter.""" |
| 1457 | self.general_processor.add_to_mute_filter(event, paths) |
| 1458 | |
| 1459 | - def platform_is_ignored(self, path): |
| 1460 | - """Should we ignore this path in the current platform.?""" |
| 1461 | - # This should be implemented in each platform |
| 1462 | - raise NotImplementedError("Not implemented on this platform.") |
| 1463 | - |
| 1464 | @is_valid_syncdaemon_path(path_indexes=[1]) |
| 1465 | def is_ignored(self, path): |
| 1466 | """Should we ignore this path?""" |
| 1467 | @@ -552,8 +588,8 @@ |
| 1468 | self.log.setLevel(TRACE) |
| 1469 | self.fs = fs |
| 1470 | self.eq = eq |
| 1471 | - # You will need to create the NotifyProcessor and WatchManager |
| 1472 | - # in each OS-specific implementation |
| 1473 | + self._processor = NotifyProcessor(self, ignore_config) |
| 1474 | + self._watch_manager = WatchManager(self._processor) |
| 1475 | |
| 1476 | def add_to_mute_filter(self, event, **info): |
| 1477 | """Add info to mute filter in the processor.""" |
| 1478 | @@ -579,12 +615,11 @@ |
| 1479 | # the logic to check if the watch is already set |
| 1480 | # is all in WatchManager.add_watch |
| 1481 | return self._watch_manager.add_watch(dirpath, |
| 1482 | - self.filesystem_monitor_mask, auto_add=True) |
| 1483 | + self.filesystem_monitor_mask) |
| 1484 | |
| 1485 | def add_watches_to_udf_ancestors(self, volume): |
| 1486 | """Add a inotify watch to volume's ancestors if it's an UDF.""" |
| 1487 | - # Should be implemented in each OS if necessary |
| 1488 | - raise NotImplementedError("Not implemented on this platform.") |
| 1489 | + return defer.succeed(True) |
| 1490 | |
| 1491 | def is_frozen(self): |
| 1492 | """Checks if there's something frozen.""" |
| 1493 | |
| 1494 | === modified file 'ubuntuone/platform/filesystem_notifications/darwin.py' |
| 1495 | --- ubuntuone/platform/filesystem_notifications/darwin.py 2012-06-28 17:21:30 +0000 |
| 1496 | +++ ubuntuone/platform/filesystem_notifications/darwin.py 2012-07-11 15:52:18 +0000 |
| 1497 | @@ -33,37 +33,57 @@ |
| 1498 | import fsevents |
| 1499 | from twisted.internet import defer, reactor |
| 1500 | |
| 1501 | -from ubuntuone.platform.filesystem_notifications import common |
| 1502 | - |
| 1503 | +from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import ( |
| 1504 | + IN_DELETE, |
| 1505 | + IN_CREATE, |
| 1506 | + IN_MODIFY, |
| 1507 | + IN_MOVED_FROM, |
| 1508 | + IN_MOVED_TO, |
| 1509 | +) |
| 1510 | |
| 1511 | # a map between the few events that we have on common platforms and those |
| 1512 | # found in pyinotify |
| 1513 | -common.COMMON_ACTIONS = { |
| 1514 | - fsevents.IN_CREATE: common.IN_CREATE, |
| 1515 | - fsevents.IN_DELETE: common.IN_DELETE, |
| 1516 | - fsevents.IN_MODIFY: common.IN_MODIFY, |
| 1517 | - fsevents.IN_MOVED_FROM: common.IN_MOVED_FROM, |
| 1518 | - fsevents.IN_MOVED_TO: common.IN_MOVED_TO, |
| 1519 | +ACTIONS = { |
| 1520 | + fsevents.IN_CREATE: IN_CREATE, |
| 1521 | + fsevents.IN_DELETE: IN_DELETE, |
| 1522 | + fsevents.IN_MODIFY: IN_MODIFY, |
| 1523 | + fsevents.IN_MOVED_FROM: IN_MOVED_FROM, |
| 1524 | + fsevents.IN_MOVED_TO: IN_MOVED_TO, |
| 1525 | } |
| 1526 | |
| 1527 | # a map of the actions to names so that we have better logs. |
| 1528 | -common.COMMON_ACTIONS_NAMES = { |
| 1529 | - fsevents.IN_CREATE: 'IN_CREATE', |
| 1530 | - fsevents.IN_DELETE: 'IN_DELETE', |
| 1531 | - fsevents.IN_MODIFY: 'IN_MODIFY', |
| 1532 | - fsevents.IN_MOVED_FROM: 'IN_MOVED_FROM', |
| 1533 | - fsevents.IN_MOVED_TO: 'IN_MOVED_TO', |
| 1534 | +ACTIONS_NAMES = { |
| 1535 | + fsevents.IN_CREATE: 'IN_CREATE', |
| 1536 | + fsevents.IN_DELETE: 'IN_DELETE', |
| 1537 | + fsevents.IN_MODIFY: 'IN_MODIFY', |
| 1538 | + fsevents.IN_MOVED_FROM: 'IN_MOVED_FROM', |
| 1539 | + fsevents.IN_MOVED_TO: 'IN_MOVED_TO', |
| 1540 | } |
| 1541 | |
| 1542 | |
| 1543 | +def path_is_ignored(self, path): |
| 1544 | + """Should we ignore this path in the current platform.?""" |
| 1545 | + # don't support links yet |
| 1546 | + if os.path.islink(path): |
| 1547 | + return True |
| 1548 | + return False |
| 1549 | + |
| 1550 | + |
| 1551 | # The implementation of the code that is provided as the pyinotify substitute |
| 1552 | -class Watch(common.Watch): |
| 1553 | +class Watch(object): |
| 1554 | """Implement the same functions as pyinotify.Watch.""" |
| 1555 | |
| 1556 | - def __init__(self, watch_descriptor, path, mask, auto_add, processor, |
| 1557 | - buf_size=8192): |
| 1558 | - super(Watch, self).__init__(watch_descriptor, path, mask, auto_add, |
| 1559 | - processor, buf_size) |
| 1560 | + def __init__(self, path, process_events): |
| 1561 | + """Create a new instance for the given path. |
| 1562 | + |
| 1563 | + The process_events parameter is a callback to be executed in the main |
| 1564 | + reactor thread to convert events in pyinotify events and add them to |
| 1565 | + the state machine. |
| 1566 | + """ |
| 1567 | + self.path = os.path.abspath(path) |
| 1568 | + self.process_events = process_events |
| 1569 | + self.watching = False |
| 1570 | + self.ignore_paths = [] |
| 1571 | # Create stream with folder to watch |
| 1572 | self.stream = fsevents.Stream(self._process_events, |
| 1573 | path, file_events=True) |
| 1574 | @@ -74,97 +94,70 @@ |
| 1575 | |
| 1576 | def _process_events_in_main_thread(self, event): |
| 1577 | """Process the events from the queue.""" |
| 1578 | - # do not do it if we stop watching and the events are empty |
| 1579 | - if not self._watching: |
| 1580 | - return |
| 1581 | - |
| 1582 | action, cookie, file_name = (event.mask, event.cookie, event.name) |
| 1583 | - if any([file_name.startswith(path) |
| 1584 | - for path in self._ignore_paths]): |
| 1585 | - return |
| 1586 | - syncdaemon_path = os.path.join(self._path, file_name) |
| 1587 | - self._process_events_from_filesystem(action, file_name, cookie, |
| 1588 | - syncdaemon_path) |
| 1589 | + |
| 1590 | + syncdaemon_path = os.path.join(self.path, file_name) |
| 1591 | + self.process_events(action, file_name, cookie, |
| 1592 | + syncdaemon_path) |
| 1593 | + |
| 1594 | + def start_watching(self): |
| 1595 | + """Start watching.""" |
| 1596 | + self.watching = True |
| 1597 | + return defer.succeed(self.watching) |
| 1598 | + |
| 1599 | + def stop_watching(self): |
| 1600 | + """Stop watching.""" |
| 1601 | + self.watching = False |
| 1602 | + return defer.succeed(self.watching) |
| 1603 | |
| 1604 | # For API compatibility |
| 1605 | @property |
| 1606 | def started(self): |
| 1607 | """A deferred that will be called when the watch is running.""" |
| 1608 | - return defer.succeed(self._watching) |
| 1609 | + return defer.succeed(self.watching) |
| 1610 | |
| 1611 | @property |
| 1612 | def stopped(self): |
| 1613 | """A deferred fired when the watch thread has finished.""" |
| 1614 | - return defer.succeed(self._watching) |
| 1615 | - |
| 1616 | - |
| 1617 | -class WatchManager(common.WatchManager): |
| 1618 | + return defer.succeed(self.watching) |
| 1619 | + |
| 1620 | + |
| 1621 | +class WatchManager(object): |
| 1622 | """Implement the same functions as pyinotify.WatchManager. |
| 1623 | |
| 1624 | All paths passed to methods in this class should be darwin paths. |
| 1625 | |
| 1626 | """ |
| 1627 | |
| 1628 | - def __init__(self, processor): |
| 1629 | + def __init__(self, log): |
| 1630 | """Init the manager to keep track of the different watches.""" |
| 1631 | - super(WatchManager, self).__init__(processor) |
| 1632 | + self.log = log |
| 1633 | self.observer = fsevents.Observer() |
| 1634 | self.observer.start() |
| 1635 | |
| 1636 | - def add_watch(self, path, mask, auto_add=False, quiet=True): |
| 1637 | - """Add a new path to be watched. |
| 1638 | - |
| 1639 | - The method will ensure that the path is not already present. |
| 1640 | - """ |
| 1641 | - if not isinstance(path, str): |
| 1642 | - e = NotImplementedError("No implementation on this platform.") |
| 1643 | - return defer.fail(e) |
| 1644 | - wd = self.get_wd(path) |
| 1645 | - if wd is None: |
| 1646 | - self.log.debug('Adding single watch on %r', path) |
| 1647 | - return self._add_single_watch(path, mask, auto_add, quiet) |
| 1648 | - else: |
| 1649 | - self.log.debug('Watch already exists on %r', path) |
| 1650 | - return self._wdm[wd].started |
| 1651 | - |
| 1652 | def __del__(self): |
| 1653 | """Stop the observer.""" |
| 1654 | self.observer.stop() |
| 1655 | |
| 1656 | + def stop_watch(self, watch): |
| 1657 | + """Stop a given watch.""" |
| 1658 | + watch.stop_watching() |
| 1659 | + self.observer.unschedule(watch.platform_watch.stream) |
| 1660 | + return defer.succeed(True) |
| 1661 | + |
| 1662 | def stop(self): |
| 1663 | - """Close the manager and stop all watches.""" |
| 1664 | - self.log.debug('Stopping watches.') |
| 1665 | - for current_wd in self._wdm: |
| 1666 | - self._wdm[current_wd].stop_watching() |
| 1667 | - self.observer.unschedule(self._wdm[current_wd].stream) |
| 1668 | - self.log.debug('Stopping Watch on %r.', self._wdm[current_wd].path) |
| 1669 | + """Stop the manager.""" |
| 1670 | self.observer.stop() |
| 1671 | |
| 1672 | - def del_watch(self, wd): |
| 1673 | - """Delete the watch with the given descriptor.""" |
| 1674 | - watch = self.get_watch(wd) |
| 1675 | + def del_watch(self, watch): |
| 1676 | + """Delete the watch and clean resources.""" |
| 1677 | self.observer.unschedule(watch.stream) |
| 1678 | - return super(WatchManager, self).del_watch(wd) |
| 1679 | |
| 1680 | - def _adding_watch(self, path, mask, auto_add): |
| 1681 | + def add_watch(self, watch): |
| 1682 | """This method perform actually the action of registering the watch.""" |
| 1683 | - watch = Watch(self._wd_count, path, mask, auto_add, self._processor) |
| 1684 | - self._wdm[self._wd_count] = watch |
| 1685 | - self._wdm[self._wd_count].start_watching() |
| 1686 | - self.observer.schedule(self._wdm[self._wd_count].stream) |
| 1687 | - self._wd_count += 1 |
| 1688 | - return defer.succeed(True) |
| 1689 | + self.observer.schedule(watch.platform_watch.stream) |
| 1690 | + return True |
| 1691 | |
| 1692 | - def rm_watch(self, wd, rec=False, quiet=True): |
| 1693 | + def rm_watch(self, watch): |
| 1694 | """Remove the the watch with the given wd.""" |
| 1695 | - watch = self.get_watch(wd) |
| 1696 | - self.observer.unschedule(watch.stream) |
| 1697 | - super(WatchManager, self).del_watch(wd, rec, quiet) |
| 1698 | - |
| 1699 | - |
| 1700 | -class FilesystemMonitor(object): |
| 1701 | - """Empty implementation of FilesystemMonitor""" |
| 1702 | - |
| 1703 | - |
| 1704 | -class NotifyProcessor(object): |
| 1705 | - """Empty implementation of NotifyProcessor""" |
| 1706 | + self.observer.unschedule(watch.platform_watch.stream) |
| 1707 | |
| 1708 | === modified file 'ubuntuone/platform/filesystem_notifications/windows.py' |
| 1709 | --- ubuntuone/platform/filesystem_notifications/windows.py 2012-06-28 16:44:01 +0000 |
| 1710 | +++ ubuntuone/platform/filesystem_notifications/windows.py 2012-07-11 15:52:18 +0000 |
| 1711 | @@ -28,7 +28,9 @@ |
| 1712 | # files in the program, then also delete it here. |
| 1713 | """File notifications on windows.""" |
| 1714 | |
| 1715 | +import logging |
| 1716 | import os |
| 1717 | + |
| 1718 | from uuid import uuid4 |
| 1719 | |
| 1720 | from twisted.internet import defer, reactor |
| 1721 | @@ -66,25 +68,31 @@ |
| 1722 | get_syncdaemon_valid_path, |
| 1723 | ) |
| 1724 | |
| 1725 | -from ubuntuone.platform.filesystem_notifications import common |
| 1726 | - |
| 1727 | -# a map between the few events that we have on common platforms and those |
| 1728 | -# found in pyinotify |
| 1729 | -common.COMMON_ACTIONS = { |
| 1730 | - 1: common.IN_CREATE, |
| 1731 | - 2: common.IN_DELETE, |
| 1732 | - 3: common.IN_MODIFY, |
| 1733 | - 4: common.IN_MOVED_FROM, |
| 1734 | - 5: common.IN_MOVED_TO, |
| 1735 | +from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import ( |
| 1736 | + IN_CREATE, |
| 1737 | + IN_DELETE, |
| 1738 | + IN_MODIFY, |
| 1739 | + IN_MOVED_FROM, |
| 1740 | + IN_MOVED_TO, |
| 1741 | +) |
| 1742 | + |
| 1743 | + |
| 1744 | +# map between windows events and pyinotify |
| 1745 | +ACTIONS = { |
| 1746 | + 1: IN_CREATE, |
| 1747 | + 2: IN_DELETE, |
| 1748 | + 3: IN_MODIFY, |
| 1749 | + 4: IN_MOVED_FROM, |
| 1750 | + 5: IN_MOVED_TO, |
| 1751 | } |
| 1752 | |
| 1753 | # a map of the actions to names so that we have better logs. |
| 1754 | -common.COMMON_ACTIONS_NAMES = { |
| 1755 | - 1: 'IN_CREATE', |
| 1756 | - 2: 'IN_DELETE', |
| 1757 | - 3: 'IN_MODIFY', |
| 1758 | - 4: 'IN_MOVED_FROM', |
| 1759 | - 5: 'IN_MOVED_TO', |
| 1760 | +ACTIONS_NAMES = { |
| 1761 | + 1: 'IN_CREATE', |
| 1762 | + 2: 'IN_DELETE', |
| 1763 | + 3: 'IN_MODIFY', |
| 1764 | + 4: 'IN_MOVED_FROM', |
| 1765 | + 5: 'IN_MOVED_TO', |
| 1766 | } |
| 1767 | |
| 1768 | |
| 1769 | @@ -96,20 +104,43 @@ |
| 1770 | |
| 1771 | THREADPOOL_MAX = 20 |
| 1772 | |
| 1773 | - |
| 1774 | -class Watch(common.Watch): |
| 1775 | +FILESYSTEM_MONITOR_MASK = FILE_NOTIFY_CHANGE_FILE_NAME | \ |
| 1776 | + FILE_NOTIFY_CHANGE_DIR_NAME | \ |
| 1777 | + FILE_NOTIFY_CHANGE_ATTRIBUTES | \ |
| 1778 | + FILE_NOTIFY_CHANGE_SIZE | \ |
| 1779 | + FILE_NOTIFY_CHANGE_LAST_WRITE | \ |
| 1780 | + FILE_NOTIFY_CHANGE_SECURITY | \ |
| 1781 | + FILE_NOTIFY_CHANGE_LAST_ACCESS |
| 1782 | + |
| 1783 | + |
| 1784 | +@is_valid_syncdaemon_path() |
| 1785 | +def path_is_ignored(path): |
| 1786 | + """Should we ignore this path in the current platform.?""" |
| 1787 | + # don't support links yet |
| 1788 | + if path.endswith('.lnk'): |
| 1789 | + return True |
| 1790 | + return False |
| 1791 | + |
| 1792 | + |
| 1793 | +class Watch(object): |
| 1794 | """Implement the same functions as pyinotify.Watch.""" |
| 1795 | |
| 1796 | - def __init__(self, watch_descriptor, path, mask, auto_add, processor, |
| 1797 | - buf_size=8192): |
| 1798 | - super(Watch, self).__init__(watch_descriptor, path, mask, auto_add, |
| 1799 | - processor, buf_size) |
| 1800 | + def __init__(self, path, process_events, mask=FILESYSTEM_MONITOR_MASK, |
| 1801 | + buf_size=8192): |
| 1802 | + self.path = os.path.abspath(path) |
| 1803 | + self.process_events = process_events |
| 1804 | + self.watching = False |
| 1805 | + self.log = logging.getLogger('ubuntuone.SyncDaemon.platform.windows.' + |
| 1806 | + 'filesystem_notifications.Watch') |
| 1807 | + self.log.setLevel(logging.DEBUG) |
| 1808 | + self._buf_size = buf_size |
| 1809 | + self._mask = mask |
| 1810 | + self.ignore_paths = [] |
| 1811 | + self._watch_handle = None |
| 1812 | + |
| 1813 | self._wait_stop = CreateEvent(None, 0, 0, None) |
| 1814 | self._overlapped = OVERLAPPED() |
| 1815 | self._overlapped.hEvent = CreateEvent(None, 0, 0, None) |
| 1816 | - # remember the subdirs we have so that when we have a delete we can |
| 1817 | - # check if it was a remove |
| 1818 | - self._subdirs = set() |
| 1819 | # this deferred is fired when the watch has started monitoring |
| 1820 | # a directory from a thread |
| 1821 | self._watch_started_deferred = defer.Deferred() |
| 1822 | @@ -118,22 +149,13 @@ |
| 1823 | |
| 1824 | def _process_events(self, events): |
| 1825 | """Process the events from the queue.""" |
| 1826 | - # do not do it if we stop watching and the events are empty |
| 1827 | - if not self._watching: |
| 1828 | - return |
| 1829 | - |
| 1830 | # we transform the events to be the same as the one in pyinotify |
| 1831 | # and then use the proc_fun |
| 1832 | for action, file_name in events: |
| 1833 | - if any([file_name.startswith(path) |
| 1834 | - for path in self._ignore_paths]): |
| 1835 | - continue |
| 1836 | - # map the windows events to the pyinotify ones, tis is dirty but |
| 1837 | - # makes the multiplatform better, linux was first :P |
| 1838 | syncdaemon_path = get_syncdaemon_valid_path( |
| 1839 | - os.path.join(self._path, file_name)) |
| 1840 | - self._process_events_from_filesystem(action, file_name, |
| 1841 | - str(uuid4()), syncdaemon_path) |
| 1842 | + os.path.join(self.path, file_name)) |
| 1843 | + self.process_events(action, file_name, str(uuid4()), |
| 1844 | + syncdaemon_path) |
| 1845 | |
| 1846 | def _call_deferred(self, f, *args): |
| 1847 | """Executes the deferred call avoiding possible race conditions.""" |
| 1848 | @@ -151,12 +173,12 @@ |
| 1849 | def _watch(self): |
| 1850 | """Watch a path that is a directory.""" |
| 1851 | self.log.debug('Adding watch for %r (exists? %r is dir? %r).', |
| 1852 | - self._path, |
| 1853 | - os.path.exists(self._path), os.path.isdir(self._path)) |
| 1854 | + self.path, |
| 1855 | + os.path.exists(self.path), os.path.isdir(self.path)) |
| 1856 | # we are going to be using the ReadDirectoryChangesW whihc requires |
| 1857 | # a directory handle and the mask to be used. |
| 1858 | self._watch_handle = CreateFileW( |
| 1859 | - self._path, |
| 1860 | + self.path, |
| 1861 | FILE_LIST_DIRECTORY, |
| 1862 | FILE_SHARE_READ | FILE_SHARE_WRITE, |
| 1863 | None, |
| 1864 | @@ -184,7 +206,7 @@ |
| 1865 | ReadDirectoryChangesW( |
| 1866 | handle, |
| 1867 | buf, |
| 1868 | - self._auto_add, |
| 1869 | + True, # Always watch children |
| 1870 | self._mask, |
| 1871 | self._overlapped, |
| 1872 | ) |
| 1873 | @@ -204,20 +226,20 @@ |
| 1874 | # lets ead the data and store it in the results |
| 1875 | events = FILE_NOTIFY_INFORMATION(buf, data) |
| 1876 | self.log.debug('Got from ReadDirectoryChangesW %r.', |
| 1877 | - [(common.COMMON_ACTIONS_NAMES[action], path) \ |
| 1878 | + [(ACTIONS_NAMES[action], path) \ |
| 1879 | for action, path in events]) |
| 1880 | reactor.callFromThread(self._process_events, events) |
| 1881 | |
| 1882 | def start_watching(self): |
| 1883 | """Tell the watch to start processing events.""" |
| 1884 | - super(Watch, self).start_watching() |
| 1885 | + self.watching = True |
| 1886 | reactor.callInThread(self._watch_wrapper) |
| 1887 | return self._watch_started_deferred |
| 1888 | |
| 1889 | def stop_watching(self): |
| 1890 | """Tell the watch to stop processing events.""" |
| 1891 | - super(Watch, self).stop_watching() |
| 1892 | SetEvent(self._wait_stop) |
| 1893 | + self.watching = False |
| 1894 | return self.stopped |
| 1895 | |
| 1896 | @property |
| 1897 | @@ -231,82 +253,39 @@ |
| 1898 | return self._watch_stopped_deferred |
| 1899 | |
| 1900 | |
| 1901 | -class WatchManager(common.WatchManager): |
| 1902 | - |
| 1903 | - @common.is_valid_os_path(path_indexes=[1]) |
| 1904 | - def add_watch(self, path, mask, auto_add=False, quiet=True): |
| 1905 | - """Add a new path to be watch. |
| 1906 | - |
| 1907 | - The method will ensure that the path is not already present. |
| 1908 | - """ |
| 1909 | - if not isinstance(path, unicode): |
| 1910 | - e = NotImplementedError("No implementation on this platform.") |
| 1911 | - return defer.fail(e) |
| 1912 | - wd = self.get_wd(path) |
| 1913 | - if wd is None: |
| 1914 | - self.log.debug('Adding single watch on %r', path) |
| 1915 | - return self._add_single_watch(path, mask, auto_add, quiet) |
| 1916 | - else: |
| 1917 | - self.log.debug('Watch already exists on %r', path) |
| 1918 | - return self._wdm[wd].started |
| 1919 | +class WatchManager(object): |
| 1920 | + """Implement the same functions as pyinotify.WatchManager. |
| 1921 | + |
| 1922 | + All paths passed to methods in this class should be windows paths. |
| 1923 | + |
| 1924 | + """ |
| 1925 | + |
| 1926 | + def __init__(self, log): |
| 1927 | + """Create a new instance.""" |
| 1928 | + self.log = log |
| 1929 | + self._wd_count = 0 |
| 1930 | + |
| 1931 | + def del_watch(self, wd): |
| 1932 | + """Delete the watch with the given descriptor.""" |
| 1933 | + |
| 1934 | + def stop_watch(self, watch): |
| 1935 | + """Stop a watch.""" |
| 1936 | + # decrease the number of watches |
| 1937 | + self._wd_count -= 1 |
| 1938 | + return watch.stop_watching() |
| 1939 | |
| 1940 | def stop(self): |
| 1941 | """Close the manager and stop all watches.""" |
| 1942 | self.log.debug('Stopping watches.') |
| 1943 | - wait_list = [] |
| 1944 | - for current_wd in self._wdm: |
| 1945 | - wait_list.append(self._wdm[current_wd].stop_watching()) |
| 1946 | - self.log.debug('Stopping Watch on %r.', self._wdm[current_wd].path) |
| 1947 | - return defer.DeferredList(wait_list) |
| 1948 | - |
| 1949 | - def _adding_watch(self, path, mask, auto_add): |
| 1950 | + return defer.succeed(True) |
| 1951 | + |
| 1952 | + def rm_watch(self, watch): |
| 1953 | + """Stop the Watch.""" |
| 1954 | + self._wd_count -= 1 |
| 1955 | + return defer.succeed(True) |
| 1956 | + |
| 1957 | + def add_watch(self, watch): |
| 1958 | # adjust the number of threads based on the UDFs watched |
| 1959 | - watch = Watch(self._wd_count, path, mask, auto_add, self._processor) |
| 1960 | + self._wd_count += 1 |
| 1961 | reactor.suggestThreadPoolSize(THREADPOOL_MAX + self._wd_count + 1) |
| 1962 | - self._wdm[self._wd_count] = watch |
| 1963 | - d = self._wdm[self._wd_count].start_watching() |
| 1964 | - self._wd_count += 1 |
| 1965 | - return d |
| 1966 | - |
| 1967 | - |
| 1968 | -class NotifyProcessor(common.NotifyProcessor): |
| 1969 | - """ |
| 1970 | - Processor that takes care of dealing with the events. |
| 1971 | - |
| 1972 | - Also, they must not be literal paths, that is the \\?\ prefix should not be |
| 1973 | - in the path. |
| 1974 | - """ |
| 1975 | - |
| 1976 | - @is_valid_syncdaemon_path(path_indexes=[1]) |
| 1977 | - def platform_is_ignored(self, path): |
| 1978 | - """Should we ignore this path in the current platform.?""" |
| 1979 | - # don't support links yet |
| 1980 | - if path.endswith('.lnk'): |
| 1981 | - return True |
| 1982 | - return False |
| 1983 | - |
| 1984 | - |
| 1985 | -class FilesystemMonitor(common.FilesystemMonitor): |
| 1986 | - """Manages the signals from filesystem.""" |
| 1987 | - |
| 1988 | - def __init__(self, eq, fs, ignore_config=None, timeout=1): |
| 1989 | - super(FilesystemMonitor, self).__init__(eq, fs, ignore_config, timeout) |
| 1990 | - self._processor = NotifyProcessor(self, ignore_config) |
| 1991 | - self._watch_manager = WatchManager(self._processor) |
| 1992 | - # the default mask to be used in the watches |
| 1993 | - # added by the FilesystemMonitor class |
| 1994 | - self.filesystem_monitor_mask = FILE_NOTIFY_CHANGE_FILE_NAME | \ |
| 1995 | - FILE_NOTIFY_CHANGE_DIR_NAME | \ |
| 1996 | - FILE_NOTIFY_CHANGE_ATTRIBUTES | \ |
| 1997 | - FILE_NOTIFY_CHANGE_SIZE | \ |
| 1998 | - FILE_NOTIFY_CHANGE_LAST_WRITE | \ |
| 1999 | - FILE_NOTIFY_CHANGE_SECURITY | \ |
| 2000 | - FILE_NOTIFY_CHANGE_LAST_ACCESS |
| 2001 | - |
| 2002 | - def add_watches_to_udf_ancestors(self, volume): |
| 2003 | - """Add a inotify watch to volume's ancestors if it's an UDF.""" |
| 2004 | - # There is no need to add the watches to the ancestors |
| 2005 | - # so we will always return true. The reason for this is that the |
| 2006 | - # handles that we open stop the user from renaming the ancestors of |
| 2007 | - # the UDF, for a user to do that he has to unsync the udf first |
| 2008 | - return defer.succeed(True) |
| 2009 | + return True |
| 2010 | |
| 2011 | === modified file 'ubuntuone/platform/os_helper/darwin.py' |
| 2012 | --- ubuntuone/platform/os_helper/darwin.py 2012-06-27 13:51:09 +0000 |
| 2013 | +++ ubuntuone/platform/os_helper/darwin.py 2012-07-11 15:52:18 +0000 |
| 2014 | @@ -127,8 +127,13 @@ |
| 2015 | |
| 2016 | |
| 2017 | def os_path(path_indexes=None): |
| 2018 | + if path_indexes is None: |
| 2019 | + path_indexes = [0] |
| 2020 | + |
| 2021 | def decorator(func): |
| 2022 | def wrapped(*args, **kwargs): |
| 2023 | + for i in path_indexes: |
| 2024 | + assert isinstance(args[i], str), 'Path %r should be str.' |
| 2025 | return func(*args, **kwargs) |
| 2026 | return wrapped |
| 2027 | return decorator |


+1