Merge lp:~diegosarmentero/ubuntuone-client/darwin3-fsevents into lp:ubuntuone-client

[Diego Sarmentero]

- Adding Watch and WatchManager to darwin (LP: #1013323).

Now you can execute:
./run-mac-tests tests/platform/filesystem_notifications/test_darwin.py

[lp:~diegosarmentero/ubuntuone-client/darwin3-fsevents]

1269. By Diego Sarmentero on 2012-06-22

tests fixed for darwin

[Manuel de la Peña (mandel)]

There seems to be a merge issue:

838 +<<<<<<< TREE
839 + For Windows:
840 + Also, they must not be literal paths, that is the \\?\ prefix should not be
841 + in the path.
842 +
843 +=======
844 +>>>>>>> MERGE-SOURCE
845 """

review: Needs Fixing

[Diego Sarmentero (diegosarmentero)]

> There seems to be a merge issue:
>
> 838 +<<<<<<< TREE
> 839 + For Windows:
> 840 + Also, they must not be literal paths, that is the \\?\ prefix should
> not be
> 841 + in the path.
> 842 +
> 843 +=======
> 844 +>>>>>>> MERGE-SOURCE
> 845 """

Yes, i realize in the next branch, i miss it here, and i didn't notice because it's inside a string :P
Fixed

[Manuel de la Peña (mandel)]

I noticed that in the tests we have asserts like:

self.assertEqual(0x100, event.mask)
self.assertEqual(0x40000100, event.mask)
self.assertEqual(0x200, event.mask)
...

Can we use constants that point to those value, there are two reasons for that:

1. if the constant value change we catch the error.
2. Is more readable for the possible maintainer.

I guess there was some inspiration from windows on test_darwin:

478 + # on windows a move to outside a watched dir translates to a remove

and

377 + # while on linux we will have to do some sort of magic like facundo
378 + # did, on windows we will get a deleted event which is much more
379 + # cleaner, first time ever windows is nicer!

Are we doing the exact same tests, can't we merge them?

Does watch take unicode of bytes, the following tests use diff values:

690 + def test_stream_created(self):
691 + """Test that the stream is created."""
692 + def fake_call(*args, **kwargs):
693 + """Fake call."""
694 +
695 + path = '/Users/username/folder'
696 + watch = Watch(1, path, None, True, None)
697 + self.assertEqual(watch._process_events, watch.stream.callback)
698 + self.assertEqual(watch.stream.paths, [path])
699 + self.assertEqual(watch.stream.file_events, True)
700 +
701 + def test_watching_property(self):
702 + """Test that the stopped property returns the stopped deferred."""
703 + path = u'/Users/username/folder'
704 + watch = Watch(1, path, None, True, None)
705 + self.assertFalse(watch._watching)

We should agree on one, either always bytes or always unicode. I'm guessing it should be unicode due to:

1028 + if not isinstance(path, unicode):
1029 + e = NotImplementedError("No implementation on this platform.")

which brings me to point out that we are doing:

873 + def __init__(self, watch_descriptor, path, mask, auto_add, processor,
874 + buf_size=8192):
875 + super(Watch, self).__init__(watch_descriptor, path, mask, auto_add,
876 + processor, buf_size)
877 + # Create stream with folder to watch
878 + try:
879 + path = self._path.encode('utf-8')
880 + except (UnicodeDecodeError, UnicodeEncodeError):
881 + path = self._path
882 + self.stream = fsevents.Stream(self._process_events,
883 + path, file_events=True)

We certainly need to make it more clear what we want to avoid mixing bytes and unicode. I prefer to have sd fail because we passed the wrong type to be honest.

If I understand correctly we get from the fsevents api:

896 + action, cookie, file_name = (event.mask, event.cookie, event.name)

which probably means that cookie is unique, right? If so we do not need to do:

self._cookie = str(uuid4())

when creating the cookie for the fake IN_MOVED_FROM and IN_MOVED_TO events because we should be able to reuse the cookie passed by fsevents, is the assumption correct?

AFAIK we don't need the [] in:

897 + if any([file_name.startswith(path)
898 + for path in self._ignore_paths]):
899 + return

Looks like a copy paste from the windows one (me to blame) we should fix it in both, what is more, the method has a number of steps that are very similar do we really need to copy the entire func?

The following comment looks outdated and from the windows code:

932 + # add the event only if we ...

review: Needs Fixing

[Manuel de la Peña (mandel)]

One more issue due to the copy&paste:

902 + syncdaemon_path = os.path.join(self._path, file_name)
903 + full_dir_path = os.path.join(self._path, file_name)

Those two are the same on darwin and there is no reason to have both.

review: Needs Fixing

[lp:~diegosarmentero/ubuntuone-client/darwin3-fsevents]

1270. By Diego Sarmentero on 2012-06-26

improving branch

1271. By Diego Sarmentero on 2012-06-26

merge

[Diego Sarmentero (diegosarmentero)]

> I noticed that in the tests we have asserts like:
>
> self.assertEqual(0x100, event.mask)
> self.assertEqual(0x40000100, event.mask)
> self.assertEqual(0x200, event.mask)
> ...
>
> Can we use constants that point to those value, there are two reasons for
> that:
>
> 1. if the constant value change we catch the error.
> 2. Is more readable for the possible maintainer.
>
> I guess there was some inspiration from windows on test_darwin:
>
> 478 + # on windows a move to outside a watched dir translates to a remove
>
> and
>
> 377 + # while on linux we will have to do some sort of magic like facundo
> 378 + # did, on windows we will get a deleted event which is much more
> 379 + # cleaner, first time ever windows is nicer!
>
> Are we doing the exact same tests, can't we merge them?
>
> Does watch take unicode of bytes, the following tests use diff values:
>
> 690 + def test_stream_created(self):
> 691 + """Test that the stream is created."""
> 692 + def fake_call(*args, **kwargs):
> 693 + """Fake call."""
> 694 +
> 695 + path = '/Users/username/folder'
> 696 + watch = Watch(1, path, None, True, None)
> 697 + self.assertEqual(watch._process_events, watch.stream.callback)
> 698 + self.assertEqual(watch.stream.paths, [path])
> 699 + self.assertEqual(watch.stream.file_events, True)
> 700 +
> 701 + def test_watching_property(self):
> 702 + """Test that the stopped property returns the stopped deferred."""
> 703 + path = u'/Users/username/folder'
> 704 + watch = Watch(1, path, None, True, None)
> 705 + self.assertFalse(watch._watching)
>
> We should agree on one, either always bytes or always unicode. I'm guessing it
> should be unicode due to:
>
> 1028 + if not isinstance(path, unicode):
> 1029 + e = NotImplementedError("No implementation on this platform.")
>
> which brings me to point out that we are doing:
>
> 873 + def __init__(self, watch_descriptor, path, mask, auto_add,
> processor,
> 874 + buf_size=8192):
> 875 + super(Watch, self).__init__(watch_descriptor, path, mask, auto_add,
> 876 + processor, buf_size)
> 877 + # Create stream with folder to watch
> 878 + try:
> 879 + path = self._path.encode('utf-8')
> 880 + except (UnicodeDecodeError, UnicodeEncodeError):
> 881 + path = self._path
> 882 + self.stream = fsevents.Stream(self._process_events,
> 883 + path, file_events=True)
>
> We certainly need to make it more clear what we want to avoid mixing bytes and
> unicode. I prefer to have sd fail because we passed the wrong type to be
> honest.
>
> If I understand correctly we get from the fsevents api:
>
> 896 + action, cookie, file_name = (event.mask, event.cookie, event.name)
>
> which probably means that cookie is unique, right? If so we do not need to do:
>
> self._cookie = str(uuid4())
>
> when creating the cookie for the fake IN_MOVED_FROM and IN_MOVED_TO events
> because we should be able to reuse the cookie passed by fsevents, is the
> assumption correct?
>
> AFAIK we don't need the [] in:
>
> 897 + if any([file_name.startswith(path)
> 898 + for path in self._ignore_paths]):...

[lp:~diegosarmentero/ubuntuone-client/darwin3-fsevents]

1272. By Diego Sarmentero on 2012-06-26

merge

[Manuel de la Peña (mandel)]

Minor things:

Typo:

997 + """Add a new path tp be watch.
1103 + """Add a new path tp be watch.

Seems to happen a number of times, a :%s/tp/to in vim or a search and replace will do the trick :)

This are in the unix.py should they be in the darwin one:

1169 +# TODO: Implement this decorators to fix some encoding issues in darwin
1170 +
1171 +def is_valid_syncdaemon_path(path_indexes=None):
1172 + def decorator(func):
1173 + def wrapped(*args, **kwargs):
1174 + return func(*args, **kwargs)
1175 + return wrapped
1176 + return decorator
1177 +
1178 +
1179 +def is_valid_os_path(path_indexes=None):
1180 + def decorator(func):
1181 + def wrapped(*args, **kwargs):
1182 + return func(*args, **kwargs)
1183 + return wrapped
1184 + return decorator
1185 +
1186 +
1187 +def os_path(path_indexes=None):
1188 + def decorator(func):
1189 + def wrapped(*args, **kwargs):
1190 + return func(*args, **kwargs)
1191 + return wrapped
1192 + return decorator

There is not yield, so there most me something wrong, or the decorator is not needed or the yield to super is missing:

1025 + @defer.inlineCallbacks
1026 + def del_watch(self, wd):
1027 + """Delete the watch with the given descriptor."""
1028 + watch = self.get_watch(wd)
1029 + self.observer.unschedule(watch.stream)
1030 + super(WatchManager, self).del_watch(wd)

This have to be sorted in alphabetical order:

45 +from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
46 + EventsCodes,
47 + ProcessEvent,
48 + IN_CREATE,
49 + IN_DELETE,
50 +)
51 +from ubuntuone.platform.filesystem_notifications import (
52 + darwin as filesystem_notifications,
53 +)
54 +from ubuntuone.platform.filesystem_notifications.darwin import (
55 + Watch,
56 + WatchManager,
57 +)

review: Needs Fixing

[lp:~diegosarmentero/ubuntuone-client/darwin3-fsevents]

1273. By Diego Sarmentero on 2012-06-27

Fixing merge proposal

1274. By Diego Sarmentero on 2012-06-27

resolving conflict

1275. By Diego Sarmentero on 2012-06-27

removing unnecessary functions from unix

1276. By Diego Sarmentero on 2012-06-27

changing decorator+yield for return

[Manuel de la Peña (mandel)]

When running the tests on windows I get the following error:

tests.syncdaemon.test_sync.TestSync.test_deleting_open_files_is_no_cause_for_des
pair
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "C:\Users\Mandel\Projects\Canonical\ubuntuone-client\darwin3-fsevents\tes
ts\syncdaemon\test_config.py", line 519, in test_get_config_files_path_encoding
    self.assertIn(branch_config, config_files)
twisted.trial.unittest.FailTest: u'tests.syncdaemon.test_config\\UnicodePathsTes
tCase\\test_get_config_files_path_encod\\tijav6\\temp\\\xd1and\xfa\\syncdaemon.c
onf' not in ['C:\\Users\\Mandel\\Projects\\Canonical\\ubuntuone-client\\darwin3-
fsevents\\ubuntuone\\syncdaemon\\..\\..\\data\\syncdaemon.conf']

tests.syncdaemon.test_config.UnicodePathsTestCase.test_get_config_files_path_enc
oding

Can you reproduce it?

review: Needs Fixing

[Manuel de la Peña (mandel)]

The error is in trunk to, so +1

review: Approve

[Alejandro J. Cura (alecu)]

I think that EVENT_CODES should be defined in darwin.py and windows.py and imported from __init__.py

----

"Add a new path to be watch" -> "Add a new path to be watched"
"oberserver" -> "observer"
"All paths passed to methods in this class should be __windows__ paths" in darwin.py
"to keep trak" -> "to keep track"

----

"_adding_watch" has no docstring, and the comment it has is wrong.

"_process_events", "append_event" have no docstring.

----

Please rework this comment:

        # we transform the events to be the same as the one in pyinotify
        # and then use the proc_fun

----

"#create a rever mapping to use it in the tests." ->
"# A reverse mapping for the tests"

----

Why is this in test_darwin.py???

        msg = 'Got from ReadDirectoryChangesW %r.'

----

[Diego Sarmentero (diegosarmentero)]

> I think that EVENT_CODES should be defined in darwin.py and windows.py and
> imported from __init__.py
>
> ----
>
> "Add a new path to be watch" -> "Add a new path to be watched"
> "oberserver" -> "observer"
> "All paths passed to methods in this class should be __windows__ paths" in
> darwin.py
> "to keep trak" -> "to keep track"
>
> ----
>
> "_adding_watch" has no docstring, and the comment it has is wrong.
>
> "_process_events", "append_event" have no docstring.
>
> ----
>
> Please rework this comment:
>
> # we transform the events to be the same as the one in pyinotify
> # and then use the proc_fun
>
> ----
>
> "#create a rever mapping to use it in the tests." ->
> "# A reverse mapping for the tests"
>
> ----
>
> Why is this in test_darwin.py???
>
> msg = 'Got from ReadDirectoryChangesW %r.'
>
> ----

Everything has been fixed!
Except for the EVENT_CODES thing, the problem is that we are using the event codes at a module level in common.py
So the __init__ import-> darwin/windows which imports-> common, so when common is loaded we can't have access to EVENT_CODES in __init__ because if we define it after importing darwin/windows, it won't exist yet.

[lp:~diegosarmentero/ubuntuone-client/darwin3-fsevents]

1277. By Diego Sarmentero on 2012-06-28

changing event codes

1278. By Diego Sarmentero on 2012-06-28

merge

[Alejandro J. Cura (alecu)]

+1

review: Approve

=== added file 'tests/platform/filesystem_notifications/test_darwin.py'
--- tests/platform/filesystem_notifications/test_darwin.py	1970-01-01 00:00:00 +0000
+++ tests/platform/filesystem_notifications/test_darwin.py	2012-06-28 17:23:38 +0000
@@ -0,0 +1,781 @@
+# -*- coding: utf-8 *-*
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# In addition, as a special exception, the copyright holders give
+# permission to link the code of portions of this program with the
+# OpenSSL library under certain conditions as described in each
+# individual source file, and distribute linked combinations
+# including the two.
+# You must obey the GNU General Public License in all respects
+# for all of the code used other than OpenSSL.  If you modify
+# file(s) with this exception, you may extend this exception to your
+# version of the file(s), but you are not obligated to do so.  If you
+# do not wish to do so, delete this exception statement from your
+# version.  If you delete this exception statement from all source
+# files in the program, then also delete it here.
+"""Test the filesystem notifications on MAC OS."""
+
+import logging
+import os
+import tempfile
+import thread
+import time
+
+from twisted.internet import defer
+
+from contrib.testing.testcase import BaseTwistedTestCase
+from ubuntuone.devtools.handlers import MementoHandler
+from ubuntuone.platform.filesystem_notifications import (
+    darwin as filesystem_notifications,
+)
+from ubuntuone.platform.filesystem_notifications.darwin import (
+    Watch,
+    WatchManager,
+)
+from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
+    EventsCodes,
+    ProcessEvent,
+    IN_CREATE,
+    IN_DELETE,
+)
+
+# A reverse mapping for the tests
+REVERSE_MACOS_ACTIONS = {}
+for key, value in filesystem_notifications.common.COMMON_ACTIONS.iteritems():
+    REVERSE_MACOS_ACTIONS[value] = key
+
+
+OP_FLAGS = EventsCodes.FLAG_COLLECTIONS['OP_FLAGS']
+IS_DIR = EventsCodes.FLAG_COLLECTIONS['SPECIAL_FLAGS']['IN_ISDIR']
+
+
+class FakeException(Exception):
+    """A fake Exception used in tests."""
+
+
+class FakeVolume(object):
+    """A fake volume."""
+
+    def __init__(self, path, ancestors):
+        """Create a new instance."""
+        super(FakeVolume, self).__init__()
+        self.volume_id = path
+        self.path = path
+        self.ancestors = ancestors
+
+
+class FakeFileEvent(object):
+    """A Fake FileEvent from macfsevents"""
+
+    def __init__(self, mask, cookie, name):
+        self.mask = mask
+        self.cookie = cookie
+        self.name = name
+
+
+class TestCaseHandler(ProcessEvent):
+    """ProcessEvent used for test cases."""
+
+    thread_id = None
+
+    def my_init(self, main_thread_id=None, number_events=None, **kwargs):
+        """Init the event notifier."""
+        self.processed_events = []
+        self.main_thread_id = main_thread_id
+        self.deferred = defer.Deferred()
+        assert number_events is not None
+        self.number_events = number_events
+
+    def append_event(self, event):
+        """Control that we received the number of events that we want."""
+        self.processed_events.append(event)
+        if len(self.processed_events) == self.number_events:
+            self.deferred.callback(self.processed_events)
+
+    def process_IN_CREATE(self, event):
+        """Process the event and add it to the list."""
+        self.append_event(event)
+        self._verify_thread_id()
+
+    def process_IN_DELETE(self, event):
+        """Process the event and add it to the list."""
+        self.append_event(event)
+        self._verify_thread_id()
+
+    def process_default(self, event):
+        """Process the event and add it to the list."""
+        self.append_event(event)
+        self._verify_thread_id()
+
+    def _verify_thread_id(self):
+        """Verify that the event was processed in the correct thread."""
+        if self.main_thread_id:
+            assert self.main_thread_id == thread.get_ident()
+
+
+class TestWatch(BaseTwistedTestCase):
+    """Test the watch so that it returns the same events as pyinotify."""
+
+    timeout = 5
+
+    @defer.inlineCallbacks
+    def setUp(self):
+        yield super(TestWatch, self).setUp()
+        self.basedir = self.mktemp('test_root')
+        self.mask = None
+        self.memento = MementoHandler()
+        self.memento.setLevel(logging.DEBUG)
+        self.raw_events = []
+        self.paths_checked = []
+        old_is_dir = Watch._path_is_dir
+
+        def path_is_dir_wrapper(watch, path):
+            """Wrapper that gets the checked paths."""
+            result = old_is_dir(watch, path)
+            self.paths_checked.append((path, result))
+            return result
+
+        self.patch(filesystem_notifications.Watch, '_path_is_dir',
+                   path_is_dir_wrapper)
+
+    @defer.inlineCallbacks
+    def _perform_operations(self, path, mask, auto_add, actions,
+                            number_events):
+        """Perform the file operations and returns the recorded events."""
+        handler = TestCaseHandler(number_events=number_events)
+        manager = WatchManager(handler)
+        yield manager.add_watch(path, mask, auto_add=auto_add)
+        # change the logger so that we can check the logs if we wanted
+        manager._wdm[0].log.addHandler(self.memento)
+        # clean logger later
+        self.addCleanup(manager._wdm[0].log.removeHandler, self.memento)
+        # execution the actions
+        actions()
+        # process the recorded events
+        ret = yield handler.deferred
+        self.addCleanup(manager.stop)
+        defer.returnValue(ret)
+
+    def _perform_timed_operations(self, path, mask, auto_add, actions,
+                                  time_out):
+        """Perform the file operations and returns the recorded events."""
+        manager = WatchManager()
+        manager.add_watch(path, mask, auto_add=auto_add)
+        # change the logger so that we can check the logs if we wanted
+        manager._wdm[0].log.addHandler(self.memento)
+        # clean logger later
+        self.addCleanup(manager._wdm[0].log.removeHandler, self.memento)
+        # execution the actions
+        actions()
+        # process the recorded events
+        time.sleep(time_out)
+        events = self.handler.processed_events
+        return events
+
+    def _assert_logs(self, events):
+        """Assert the debug logs."""
+        logs = []
+        msg = 'Is path %r a dir? %s'
+        logs.extend([msg % data for data in self.paths_checked])
+        msg = 'Pushing event %r to processor.'
+        logs.extend([msg % e for e in events])
+        for msg in logs:
+            self.assertTrue(self.memento.check_debug(msg))
+
+    @defer.inlineCallbacks
+    def test_file_create(self):
+        """Test that the correct event is returned on a file create."""
+        file_name = os.path.join(self.basedir, 'test_file_create')
+
+        def create_file():
+            """Action used for the test."""
+            # simply create a new file
+            fd = open(file_name, 'w')
+            fd.flush()
+            os.fsync(fd)
+            fd.close()
+
+        events = yield self._perform_operations(self.basedir, self.mask, False,
+            create_file, 1)
+        event = events[0]
+        self.assertFalse(event.dir)
+        self.assertEqual(OP_FLAGS['IN_CREATE'], event.mask)
+        self.assertEqual('IN_CREATE', event.maskname)
+        self.assertEqual(os.path.split(file_name)[1], event.name)
+        self.assertEqual('.', event.path)
+        self.assertEqual(os.path.join(self.basedir, file_name), event.pathname)
+        self.assertEqual(0, event.wd)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_dir_create(self):
+        """Test that the correct event is returned on a dir creation."""
+        dir_name = os.path.join(self.basedir, 'test_dir_create')
+
+        def create_dir():
+            """Action for the test."""
+            os.mkdir(dir_name)
+
+        events = yield self._perform_operations(self.basedir, self.mask, False,
+            create_dir, 1)
+        event = events[0]
+        self.assertTrue(event.dir)
+        self.assertEqual(OP_FLAGS['IN_CREATE'] | IS_DIR, event.mask)
+        self.assertEqual('IN_CREATE|IN_ISDIR', event.maskname)
+        self.assertEqual(os.path.split(dir_name)[1], event.name)
+        self.assertEqual('.', event.path)
+        self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname)
+        self.assertEqual(0, event.wd)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_file_remove(self):
+        """Test that the correct event is raised when a file is removed."""
+        file_name = os.path.join(self.basedir, 'test_file_remove')
+        # create the file before recording
+        open(file_name, 'w').close()
+
+        def remove_file():
+            """Action for the test."""
+            os.remove(file_name)
+
+        events = yield self._perform_operations(self.basedir, self.mask, False,
+                                                remove_file, 1)
+        event = events[0]
+        self.assertFalse(event.dir)
+        self.assertEqual(OP_FLAGS['IN_DELETE'], event.mask)
+        self.assertEqual('IN_DELETE', event.maskname)
+        self.assertEqual(os.path.split(file_name)[1], event.name)
+        self.assertEqual('.', event.path)
+        self.assertEqual(os.path.join(self.basedir, file_name), event.pathname)
+        self.assertEqual(0, event.wd)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_dir_remove(self):
+        """Test that the correct event is raised when a dir is removed."""
+        dir_name = os.path.join(self.basedir, 'test_dir_remove')
+        # create the dir before recording
+        os.mkdir(dir_name)
+
+        def remove_dir():
+            """Action for the test."""
+            os.rmdir(dir_name)
+
+        events = yield self._perform_operations(self.basedir, self.mask, False,
+                                                remove_dir, 1)
+        event = events[0]
+        self.assertTrue(event.dir)
+        self.assertEqual(OP_FLAGS['IN_DELETE'] | IS_DIR, event.mask)
+        self.assertEqual('IN_DELETE|IN_ISDIR', event.maskname)
+        self.assertEqual('.', event.path)
+        self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname)
+        self.assertEqual(0, event.wd)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_file_write(self):
+        """Test that the correct event is raised when a file is written."""
+        file_name = os.path.join(self.basedir, 'test_file_write')
+        # clean behind us by removing the file
+        self.addCleanup(os.remove, file_name)
+
+        def write_file():
+            """Action for the test."""
+            # create the file before recording
+            fd = open(file_name, 'w')
+            fd.write('test')
+            fd.close()
+
+        events = yield self._perform_operations(self.basedir, self.mask, False,
+                                                write_file, 1)
+        event = events[0]
+        self.assertFalse(event.dir)
+        self.assertEqual(OP_FLAGS['IN_CREATE'], event.mask)
+        self.assertEqual('IN_CREATE', event.maskname)
+        self.assertEqual(os.path.split(file_name)[1], event.name)
+        self.assertEqual('.', event.path)
+        self.assertEqual(os.path.join(self.basedir, file_name), event.pathname)
+        self.assertEqual(0, event.wd)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_file_moved_to_watched_dir_same_watcher(self):
+        """Test that the correct event is raised when a file is moved."""
+        from_file_name = os.path.join(self.basedir,
+            'test_file_moved_to_watched_dir_same_watcher')
+        to_file_name = os.path.join(self.basedir,
+            'test_file_moved_to_watched_dir_same_watcher_2')
+        open(from_file_name, 'w').close()
+        #create file before recording
+
+        def move_file():
+            """Action for the test."""
+            os.rename(from_file_name, to_file_name)
+
+        events = yield self._perform_operations(self.basedir, self.mask,
+                                                False, move_file, 2)
+        move_from_event = events[0]
+        move_to_event = events[1]
+        # first test the move from
+        self.assertFalse(move_from_event.dir)
+        self.assertEqual(OP_FLAGS['IN_MOVED_FROM'], move_from_event.mask)
+        self.assertEqual('IN_MOVED_FROM', move_from_event.maskname)
+        self.assertEqual(os.path.split(from_file_name)[1],
+                         move_from_event.name)
+        self.assertEqual('.', move_from_event.path)
+        self.assertEqual(os.path.join(self.basedir, from_file_name),
+            move_from_event.pathname)
+        self.assertEqual(0, move_from_event.wd)
+        # test the move to
+        self.assertFalse(move_to_event.dir)
+        self.assertEqual(OP_FLAGS['IN_MOVED_TO'], move_to_event.mask)
+        self.assertEqual('IN_MOVED_TO', move_to_event.maskname)
+        self.assertEqual(os.path.split(to_file_name)[1], move_to_event.name)
+        self.assertEqual('.', move_to_event.path)
+        self.assertEqual(os.path.join(self.basedir, to_file_name),
+            move_to_event.pathname)
+        self.assertEqual(os.path.split(from_file_name)[1],
+            move_to_event.src_pathname)
+        self.assertEqual(0, move_to_event.wd)
+        # assert that both cookies are the same
+        self.assertEqual(move_from_event.cookie, move_to_event.cookie)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_file_moved_to_not_watched_dir(self):
+        """Test that the correct event is raised when a file is moved."""
+        from_file_name = os.path.join(self.basedir,
+            'test_file_moved_to_not_watched_dir')
+        open(from_file_name, 'w').close()
+
+        def move_file():
+            """Action for the test."""
+            os.rename(from_file_name, os.path.join(tempfile.mkdtemp(),
+                'test_file_moved_to_not_watched_dir'))
+
+        # We need to test that we get a delete operation when moving
+        # a file to an unwatched folder
+        events = yield self._perform_operations(self.basedir, self.mask, False,
+                                                move_file, 1)
+        event = events[0]
+        self.assertFalse(event.dir)
+        self.assertEqual(OP_FLAGS['IN_DELETE'], event.mask)
+        self.assertEqual('IN_DELETE', event.maskname)
+        self.assertEqual(os.path.split(from_file_name)[1], event.name)
+        self.assertEqual('.', event.path)
+        self.assertEqual(os.path.join(self.basedir, from_file_name),
+                         event.pathname)
+        self.assertEqual(0, event.wd)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_file_move_from_not_watched_dir(self):
+        """Test that the correct event is raised when a file is moved."""
+        from_file_name = os.path.join(tempfile.mkdtemp(),
+            'test_file_move_from_not_watched_dir')
+        to_file_name = os.path.join(self.basedir,
+            'test_file_move_from_not_watched_dir')
+        # create file before we record
+        open(from_file_name, 'w').close()
+
+        def move_files():
+            """Action for the test."""
+            os.rename(from_file_name, to_file_name)
+
+        # We need to test that we get a delete operation when moving
+        # a file from an unwatched folder
+        events = yield self._perform_operations(self.basedir, self.mask, False,
+                                                move_files, 1)
+        event = events[0]
+        self.assertFalse(event.dir)
+        self.assertEqual(OP_FLAGS['IN_CREATE'], event.mask)
+        self.assertEqual('IN_CREATE', event.maskname)
+        self.assertEqual(os.path.split(to_file_name)[1], event.name)
+        self.assertEqual('.', event.path)
+        self.assertEqual(os.path.join(self.basedir, to_file_name),
+            event.pathname)
+        self.assertEqual(0, event.wd)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_dir_moved_to_watched_dir_same_watcher(self):
+        """Test that the correct event is raised when a dir is moved."""
+        from_dir_name = os.path.join(self.basedir,
+            'test_dir_moved_to_watched_dir_same_watcher')
+        to_dir_name = os.path.join(self.basedir,
+            'test_dir_moved_to_watched_dir_same_watcher_2')
+        os.mkdir(from_dir_name)
+
+        def move_file():
+            """Action for the test."""
+            os.rename(from_dir_name, to_dir_name)
+
+        events = yield self._perform_operations(self.basedir,
+            self.mask, False, move_file, 2)
+        move_from_event = events[0]
+        move_to_event = events[1]
+        # first test the move from
+        self.assertTrue(move_from_event.dir)
+        self.assertEqual(OP_FLAGS['IN_MOVED_FROM'] | IS_DIR,
+            move_from_event.mask)
+        self.assertEqual('IN_MOVED_FROM|IN_ISDIR', move_from_event.maskname)
+        self.assertEqual(os.path.split(from_dir_name)[1], move_from_event.name)
+        self.assertEqual('.', move_from_event.path)
+        self.assertEqual(os.path.join(self.basedir, from_dir_name),
+            move_from_event.pathname)
+        self.assertEqual(0, move_from_event.wd)
+        # test the move to
+        self.assertTrue(move_to_event.dir)
+        self.assertEqual(OP_FLAGS['IN_MOVED_TO'] | IS_DIR, move_to_event.mask)
+        self.assertEqual('IN_MOVED_TO|IN_ISDIR', move_to_event.maskname)
+        self.assertEqual(os.path.split(to_dir_name)[1], move_to_event.name)
+        self.assertEqual('.', move_to_event.path)
+        self.assertEqual(os.path.join(self.basedir, to_dir_name),
+            move_to_event.pathname)
+        self.assertEqual(os.path.split(from_dir_name)[1],
+                         move_to_event.src_pathname)
+        self.assertEqual(0, move_to_event.wd)
+        # assert that both cookies are the same
+        self.assertEqual(move_from_event.cookie, move_to_event.cookie)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_dir_moved_to_not_watched_dir(self):
+        """Test that the correct event is raised when a file is moved."""
+        dir_name = os.path.join(self.basedir,
+            'test_dir_moved_to_not_watched_dir')
+        os.mkdir(dir_name)
+
+        def move_dir():
+            """Action for the test."""
+            os.rename(dir_name, os.path.join(tempfile.mkdtemp(),
+                'test_dir_moved_to_not_watched_dir'))
+
+        # We need to test that we get a delete operation when moving
+        # a file to an unwatched folder
+        events = yield self._perform_operations(self.basedir, self.mask, False,
+                                                move_dir, 1)
+        event = events[0]
+        self.assertTrue(event.dir)
+        self.assertEqual(OP_FLAGS['IN_DELETE'] | IS_DIR, event.mask)
+        self.assertEqual('IN_DELETE|IN_ISDIR', event.maskname)
+        self.assertEqual('.', event.path)
+        self.assertEqual(os.path.join(self.basedir, dir_name), event.pathname)
+        self.assertEqual(0, event.wd)
+        # assert the logging
+        self._assert_logs(events)
+
+    @defer.inlineCallbacks
+    def test_dir_move_from_not_watched_dir(self):
+        """Test that the correct event is raised when a file is moved."""
+        from_dir_name = os.path.join(tempfile.mkdtemp(),
+            'test_dir_move_from_not_watched_dir')
+        to_dir_name = os.path.join(self.basedir,
+            'test_dir_move_from_not_watched_dir')
+        # create file before we record
+        os.mkdir(from_dir_name)
+
+        def move_dir():
+            """Action for the test."""
+            os.rename(from_dir_name, to_dir_name)
+
+        events = yield self._perform_operations(self.basedir, self.mask, False,
+                                                move_dir, 1)
+        event = events[0]
+        self.assertTrue(event.dir)
+        self.assertEqual(OP_FLAGS['IN_CREATE'] | IS_DIR, event.mask)
+        self.assertEqual('IN_CREATE|IN_ISDIR', event.maskname)
+        self.assertEqual(os.path.split(from_dir_name)[1], event.name)
+        self.assertEqual('.', event.path)
+        self.assertEqual(os.path.join(self.basedir, to_dir_name),
+                         event.pathname)
+        self.assertEqual(0, event.wd)
+
+    def test_exclude_filter(self):
+        """Test that the exclude filter works as expectd."""
+        handler = TestCaseHandler(number_events=0)
+        manager = WatchManager(handler)
+        # add a watch that will always exclude all actions
+        manager.add_watch(self.basedir, self.mask, auto_add=True,
+                          exclude_filter=lambda x: True)
+        # execution the actions
+        file_name = os.path.join(self.basedir, 'test_file_create')
+        open(file_name, 'w').close()
+        # give some time for the system to get the events
+        self.assertEqual(0, len(handler.processed_events))
+    test_exclude_filter.skip = "we must rethink this test."
+
+    def test_open_dir_muted(self):
+        """Test that the opening of dirs is ignored."""
+        dir_name = os.path.join(tempfile.mkdtemp(), 'test_dir_open')
+        # create file before we record
+        os.mkdir(dir_name)
+
+        def open_dir():
+            """Action for the test."""
+            os.startfile(dir_name)
+
+        events = self._perform_timed_operations(self.basedir, self.mask, False,
+            open_dir, 2)
+        self.assertEqual(0, len(events))
+    test_open_dir_muted.skip = "we must rethink this test."
+
+    def test_ignore_path(self):
+        """Test that events from a path are ignored."""
+        events = []
+
+        def fake_processor(event):
+            """Memorize the processed events."""
+            events.append(event)
+
+        path = '/Users/username/folder'
+        child = 'child'
+        watch = Watch(1, path, None, True, fake_processor)
+        watch.ignore_path(os.path.join(path, child))
+        # ensure that the watch is watching
+        watch._watching = True
+        for file_name in 'abcdef':
+            event = FakeFileEvent(256, None, os.path.join(child, file_name))
+            watch._process_events(event)
+        self.assertEqual(0, len(events),
+                         'All events should have been ignored.')
+
+    def test_not_ignore_path(self):
+        """Test that we do get the events when they do not match."""
+        events = []
+
+        def fake_processor(event):
+            """Memorize the processed events."""
+            events.append(event)
+
+        self.patch(filesystem_notifications.reactor, 'callFromThread',
+            lambda x, e: x(e))
+
+        path = '/Users/username/folder'
+        child = 'child'
+        watch = Watch(1, path, None, True, fake_processor)
+        watch.ignore_path(os.path.join(path, child))
+        paths_not_to_ignore = []
+        for file_name in 'abcdef':
+            event = FakeFileEvent(256, None, os.path.join(child + file_name,
+                                  file_name))
+            paths_not_to_ignore.append(event)
+        # ensure that the watch is watching
+        watch._watching = True
+        for event in paths_not_to_ignore:
+            watch._process_events(event)
+        self.assertEqual(len(paths_not_to_ignore), len(events),
+                         'No events should have been ignored.')
+
+    def test_mixed_ignore_path(self):
+        """Test that we do get the correct events."""
+        events = []
+
+        def fake_processor(event):
+            """Memorize the processed events."""
+            events.append(event.pathname)
+
+        self.patch(filesystem_notifications.reactor, 'callFromThread',
+            lambda x, e: x(e))
+
+        child = 'child'
+        path = '/Users/username/folder'
+        watch = Watch(1, path, None, True, fake_processor)
+        watch.ignore_path(os.path.join(path, child))
+        paths_not_to_ignore = []
+        paths_to_ignore = []
+        expected_events = []
+        for file_name in 'abcdef':
+            valid = os.path.join(child + file_name, file_name)
+            paths_to_ignore.append((1, os.path.join(child, file_name)))
+            event = FakeFileEvent(256, None, valid)
+            paths_not_to_ignore.append(event)
+            expected_events.append(os.path.join(path, valid))
+        # ensure that the watch is watching
+        watch._watching = True
+        for event in paths_not_to_ignore:
+            watch._process_events(event)
+        self.assertEqual(len(paths_not_to_ignore), len(events),
+                         'Wrong number of events ignored.')
+        self.assertTrue(all([event in expected_events for event in events]),
+                        'Paths ignored that should have not been ignored.')
+
+    def test_undo_ignore_path_ignored(self):
+        """Test that we do deal with events from and old ignored path."""
+        events = []
+
+        def fake_processor(event):
+            """Memorize the processed events."""
+            events.append(event)
+
+        self.patch(filesystem_notifications.reactor, 'callFromThread',
+            lambda x, e: x(e))
+
+        path = '/Users/username/folder'
+        child = 'child'
+        watch = Watch(1, path, None, True, fake_processor)
+        watch.ignore_path(os.path.join(path, child))
+        watch.remove_ignored_path(os.path.join(path, child))
+        paths_not_to_ignore = []
+        for file_name in 'abcdef':
+            event = FakeFileEvent(256, None, os.path.join(child, file_name))
+            paths_not_to_ignore.append(event)
+        # ensure that the watch is watching
+        watch._watching = True
+        for event in paths_not_to_ignore:
+            watch._process_events(event)
+        self.assertEqual(len(paths_not_to_ignore), len(events),
+                         'All events should have been accepted.')
+
+    def test_undo_ignore_path_other_ignored(self):
+        """Test that we can undo and the other path is ignored."""
+        events = []
+
+        def fake_processor(event):
+            """Memorize the processed events."""
+            events.append(event.pathname)
+
+        self.patch(filesystem_notifications.reactor, 'callFromThread',
+            lambda x, e: x(e))
+
+        path = '/Users/username/folder'
+        child_a = 'childa'
+        child_b = 'childb'
+        watch = Watch(1, path, None, True, fake_processor)
+        watch.ignore_path(os.path.join(path, child_a))
+        watch.ignore_path(os.path.join(path, child_b))
+        watch.remove_ignored_path(os.path.join(path, child_a))
+        paths_to_ignore = []
+        paths_not_to_ignore = []
+        expected_events = []
+        for file_name in 'abcdef':
+            paths_to_ignore.append((1, os.path.join(child_b, file_name)))
+            valid = os.path.join(child_a, file_name)
+            event = FakeFileEvent(256, None, valid)
+            paths_not_to_ignore.append(event)
+            expected_events.append(os.path.join(path, valid))
+        # ensure that the watch is watching
+        watch._watching = True
+        for event in paths_not_to_ignore:
+            watch._process_events(event)
+        self.assertEqual(len(paths_not_to_ignore), len(events),
+                         'All events should have been accepted.')
+        self.assertTrue(all([event in expected_events for event in events]),
+                        'Paths ignored that should have not been ignored.')
+
+    def test_stream_created(self):
+        """Test that the stream is created."""
+        def fake_call(*args, **kwargs):
+            """Fake call."""
+
+        path = '/Users/username/folder'
+        watch = Watch(1, path, None, True, None)
+        self.assertEqual(watch._process_events, watch.stream.callback)
+        self.assertEqual(watch.stream.paths, [path])
+        self.assertEqual(watch.stream.file_events, True)
+
+    def test_watching_property(self):
+        """Test that the stopped property returns the stopped deferred."""
+        path = '/Users/username/folder'
+        watch = Watch(1, path, None, True, None)
+        self.assertFalse(watch._watching)
+
+    def random_error(self, *args):
+        """Throw a fake exception."""
+        raise FakeException()
+
+    def test_is_path_dir_missing_no_subdir(self):
+        """Test when the path does not exist and is no a subdir."""
+        path = '/Users/username/path/to/not/dir'
+        test_path = self.mktemp("test_directory")
+        self.patch(os.path, 'exists', lambda path: False)
+        watch = Watch(1, test_path, self.mask, True, None)
+        self.assertFalse(watch._path_is_dir(path))
+
+    def test_is_path_dir_missing_in_subdir(self):
+        """Test when the path does not exist and is a subdir."""
+        path = '/Users/username/path/to/not/dir'
+        test_path = self.mktemp("test_directory")
+        self.patch(os.path, 'exists', lambda path: False)
+        watch = Watch(1, test_path, self.mask, True, None)
+        watch._subdirs.add(path)
+        self.assertTrue(watch._path_is_dir(path))
+
+    def test_is_path_dir_present_is_dir(self):
+        """Test when the path is present and is dir."""
+        path = '/Users/username/path/to/not/dir'
+        test_path = self.mktemp("test_directory")
+        self.patch(os.path, 'exists', lambda path: True)
+        self.patch(os.path, 'isdir', lambda path: True)
+        watch = Watch(1, test_path, self.mask, True, None)
+        watch._subdirs.add(path)
+        self.assertTrue(watch._path_is_dir(path))
+
+    def test_is_path_dir_present_no_dir(self):
+        """Test when the path is present but not a dir."""
+        path = '/Users/username/path/to/not/dir'
+        test_path = self.mktemp("test_directory")
+        self.patch(os.path, 'exists', lambda path: True)
+        self.patch(os.path, 'isdir', lambda path: False)
+        watch = Watch(1, test_path, self.mask, True, None)
+        watch._subdirs.add(path)
+        self.assertFalse(watch._path_is_dir(path))
+
+    def test_update_subdirs_create_not_present(self):
+        """Test when we update on a create event and not present."""
+        path = '/Users/username/path/to/not/dir'
+        test_path = self.mktemp("test_directory")
+        watch = Watch(1, test_path, self.mask, True, None)
+        watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_CREATE])
+        self.assertTrue(path in watch._subdirs)
+
+    def test_update_subdirs_create_present(self):
+        """Test when we update on a create event and is present."""
+        path = '/Users/username/path/to/not/dir'
+        test_path = self.mktemp("test_directory")
+        watch = Watch(1, test_path, self.mask, True, None)
+        watch._subdirs.add(path)
+        old_length = len(watch._subdirs)
+        watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_CREATE])
+        self.assertTrue(path in watch._subdirs)
+        self.assertEqual(old_length, len(watch._subdirs))
+
+    def test_update_subdirs_delete_not_present(self):
+        """Test when we delete and is not present."""
+        path = '/Users/username/path/to/not/dir'
+        test_path = self.mktemp("test_directory")
+        watch = Watch(1, test_path, self.mask, True, None)
+        watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_DELETE])
+        self.assertTrue(path not in watch._subdirs)
+
+    def test_update_subdirs_delete_present(self):
+        """Test when we delete and is present."""
+        path = '/Users/username/path/to/not/dir'
+        test_path = self.mktemp("test_directory")
+        watch = Watch(1, test_path, self.mask, True, None)
+        watch._subdirs.add(path)
+        watch._update_subdirs(path, REVERSE_MACOS_ACTIONS[IN_DELETE])
+        self.assertTrue(path not in watch._subdirs)

=== modified file 'ubuntuone/platform/filesystem_notifications/__init__.py'
--- ubuntuone/platform/filesystem_notifications/__init__.py	2012-06-22 19:07:11 +0000
+++ ubuntuone/platform/filesystem_notifications/__init__.py	2012-06-28 17:23:38 +0000
@@ -32,7 +32,6 @@
 
 
 if sys.platform == "win32":
-    EVENT_CODES = [1, 2, 3, 4, 5]
     from ubuntuone.platform.filesystem_notifications import windows
     FilesystemMonitor = windows.FilesystemMonitor
     _GeneralINotifyProcessor = windows.NotifyProcessor

=== modified file 'ubuntuone/platform/filesystem_notifications/common.py'
--- ubuntuone/platform/filesystem_notifications/common.py	2012-06-26 19:20:38 +0000
+++ ubuntuone/platform/filesystem_notifications/common.py	2012-06-28 17:23:38 +0000
@@ -33,7 +33,6 @@
 
 from twisted.internet import defer
 
-from ubuntuone.platform.filesystem_notifications import EVENT_CODES
 from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
     Event,
     WatchManagerError,
@@ -45,36 +44,28 @@
     IN_IGNORED,
     IN_ISDIR,
     IN_DELETE,
+    IN_MODIFY as in_modify,
     IN_MOVED_FROM,
-    IN_MOVED_TO,
-    IN_MODIFY)
+    IN_MOVED_TO)
 
 from ubuntuone import logger
 
-from ubuntuone.platform.os_helper.windows import (
+from ubuntuone.platform.os_helper import (
     is_valid_syncdaemon_path,
-    is_valid_windows_path as is_valid_os_path,
-    windowspath as os_path,
+    is_valid_os_path,
+    os_path,
 )
 
 # a map between the few events that we have on common platforms and those
 # found in pyinotify
-COMMON_ACTIONS = {
-  EVENT_CODES[0]: IN_CREATE,
-  EVENT_CODES[1]: IN_DELETE,
-  EVENT_CODES[2]: IN_MODIFY,
-  EVENT_CODES[3]: IN_MOVED_FROM,
-  EVENT_CODES[4]: IN_MOVED_TO,
-}
+COMMON_ACTIONS = {}
 
 # a map of the actions to names so that we have better logs.
-COMMON_ACTIONS_NAMES = {
-  EVENT_CODES[0]: 'IN_CREATE',
-  EVENT_CODES[1]: 'IN_DELETE',
-  EVENT_CODES[2]: 'IN_MODIFY',
-  EVENT_CODES[3]: 'IN_MOVED_FROM',
-  EVENT_CODES[4]: 'IN_MOVED_TO',
-}
+COMMON_ACTIONS_NAMES = {}
+
+# We should have this here, because we use if from other modules that
+# share this, but we need to declare it this way yo avoid flakes issues.
+IN_MODIFY = in_modify
 
 # translates quickly the event and it's is_dir state to our standard events
 NAME_TRANSLATIONS = {
@@ -123,6 +114,46 @@
         self._path = os.path.abspath(path)
         self._mask = mask
 
+    def _process_events_from_filesystem(self, action, file_name, cookie,
+        syncdaemon_path):
+        """Process the events from the queue."""
+        # map the filesystem events to the pyinotify ones, tis is dirty but
+        # makes the multiplatform better, linux was first :P
+        full_dir_path = os.path.join(self._path, file_name)
+        is_dir = self._path_is_dir(full_dir_path)
+        if is_dir:
+            # we need to update the list of subdirs that we have
+            self._update_subdirs(full_dir_path, action)
+        mask = COMMON_ACTIONS[action]
+        head, tail = os.path.split(file_name)
+        if is_dir:
+            mask |= IN_ISDIR
+        event_raw_data = {
+            'wd': self._descriptor,
+            'dir': is_dir,
+            'mask': mask,
+            'name': tail,
+            'path': '.'}
+        # by the way in which the win api fires the events we know for
+        # sure that no move events will be added in the wrong order, this
+        # is kind of hacky, I dont like it too much
+        if COMMON_ACTIONS[action] == IN_MOVED_FROM:
+            self._cookie = cookie
+            self._source_pathname = tail
+            event_raw_data['cookie'] = self._cookie
+        if COMMON_ACTIONS[action] == IN_MOVED_TO:
+            event_raw_data['src_pathname'] = self._source_pathname
+            event_raw_data['cookie'] = self._cookie
+        event = Event(event_raw_data)
+        # FIXME: event deduces the pathname wrong and we need to manually
+        # set it
+        event.pathname = syncdaemon_path
+        # add the event only if we do not have an exclude filter or
+        # the exclude filter returns False, that is, the event will not
+        # be excluded
+        self.log.debug('Pushing event %r to processor.', event)
+        self._processor(event)
+
     @is_valid_os_path(path_indexes=[1])
     def _update_subdirs(self, path, event):
         """Adds the path to the internal subdirs.
@@ -220,6 +251,13 @@
         self._wd_count = 0
         self._ignored_paths = []
 
+    def add_watch(self, path, mask, auto_add=False, quiet=True):
+        """Add a new path to be watch.
+
+        The method will ensure that the path is not already present.
+        """
+        raise NotImplementedError("Not implemented on this platform.")
+
     def stop(self):
         """Close the manager and stop all watches."""
         # Should be implemented for each platform
@@ -256,23 +294,6 @@
         # This should be implemented for each OS
         raise NotImplementedError("Not implemented on this platform.")
 
-    @is_valid_os_path(path_indexes=[1])
-    def add_watch(self, path, mask, auto_add=False, quiet=True):
-        """Add a new path tp be watch.
-
-        The method will ensure that the path is not already present.
-        """
-        if not isinstance(path, unicode):
-            e = NotImplementedError("No implementation on this platform.")
-            return defer.fail(e)
-        wd = self.get_wd(path)
-        if wd is None:
-            self.log.debug('Adding single watch on %r', path)
-            return self._add_single_watch(path, mask, auto_add, quiet)
-        else:
-            self.log.debug('Watch already exists on %r', path)
-            return self._wdm[wd].started
-
     def update_watch(self, wd, mask=None, rec=False,
                      auto_add=False, quiet=True):
         raise NotImplementedError("Not implemented on this platform.")
@@ -321,7 +342,6 @@
 
     This interface will be exposed to syncdaemon, ergo all passed
     and returned paths must be a sequence of BYTES encoded with utf8.
-
     """
 
     def __init__(self, monitor, ignore_config=None):
@@ -563,8 +583,8 @@
 
     def add_watches_to_udf_ancestors(self, volume):
         """Add a inotify watch to volume's ancestors if it's an UDF."""
-
         # Should be implemented in each OS if necessary
+        raise NotImplementedError("Not implemented on this platform.")
 
     def is_frozen(self):
         """Checks if there's something frozen."""

=== modified file 'ubuntuone/platform/filesystem_notifications/darwin.py'
--- ubuntuone/platform/filesystem_notifications/darwin.py	2012-05-15 17:10:02 +0000
+++ ubuntuone/platform/filesystem_notifications/darwin.py	2012-06-28 17:23:38 +0000
@@ -28,6 +28,139 @@
 # files in the program, then also delete it here.
 """Filesystem Notifications module for MAC OS."""
 
+import os
+
+import fsevents
+from twisted.internet import defer, reactor
+
+from ubuntuone.platform.filesystem_notifications import common
+
+
+# a map between the few events that we have on common platforms and those
+# found in pyinotify
+common.COMMON_ACTIONS = {
+  fsevents.IN_CREATE: common.IN_CREATE,
+  fsevents.IN_DELETE: common.IN_DELETE,
+  fsevents.IN_MODIFY: common.IN_MODIFY,
+  fsevents.IN_MOVED_FROM: common.IN_MOVED_FROM,
+  fsevents.IN_MOVED_TO: common.IN_MOVED_TO,
+}
+
+# a map of the actions to names so that we have better logs.
+common.COMMON_ACTIONS_NAMES = {
+  fsevents.IN_CREATE: 'IN_CREATE',
+  fsevents.IN_DELETE: 'IN_DELETE',
+  fsevents.IN_MODIFY: 'IN_MODIFY',
+  fsevents.IN_MOVED_FROM: 'IN_MOVED_FROM',
+  fsevents.IN_MOVED_TO: 'IN_MOVED_TO',
+}
+
+
+# The implementation of the code that is provided as the pyinotify substitute
+class Watch(common.Watch):
+    """Implement the same functions as pyinotify.Watch."""
+
+    def __init__(self, watch_descriptor, path, mask, auto_add, processor,
+        buf_size=8192):
+        super(Watch, self).__init__(watch_descriptor, path, mask, auto_add,
+                                    processor, buf_size)
+        # Create stream with folder to watch
+        self.stream = fsevents.Stream(self._process_events,
+            path, file_events=True)
+
+    def _process_events(self, event):
+        """Receive the filesystem event and move it to the main thread."""
+        reactor.callFromThread(self._process_events_in_main_thread, event)
+
+    def _process_events_in_main_thread(self, event):
+        """Process the events from the queue."""
+        # do not do it if we stop watching and the events are empty
+        if not self._watching:
+            return
+
+        action, cookie, file_name = (event.mask, event.cookie, event.name)
+        if any([file_name.startswith(path)
+                    for path in self._ignore_paths]):
+            return
+        syncdaemon_path = os.path.join(self._path, file_name)
+        self._process_events_from_filesystem(action, file_name, cookie,
+                                             syncdaemon_path)
+
+    # For API compatibility
+    @property
+    def started(self):
+        """A deferred that will be called when the watch is running."""
+        return defer.succeed(self._watching)
+
+    @property
+    def stopped(self):
+        """A deferred fired when the watch thread has finished."""
+        return defer.succeed(self._watching)
+
+
+class WatchManager(common.WatchManager):
+    """Implement the same functions as pyinotify.WatchManager.
+
+    All paths passed to methods in this class should be darwin paths.
+
+    """
+
+    def __init__(self, processor):
+        """Init the manager to keep track of the different watches."""
+        super(WatchManager, self).__init__(processor)
+        self.observer = fsevents.Observer()
+        self.observer.start()
+
+    def add_watch(self, path, mask, auto_add=False, quiet=True):
+        """Add a new path to be watched.
+
+        The method will ensure that the path is not already present.
+        """
+        if not isinstance(path, str):
+            e = NotImplementedError("No implementation on this platform.")
+            return defer.fail(e)
+        wd = self.get_wd(path)
+        if wd is None:
+            self.log.debug('Adding single watch on %r', path)
+            return self._add_single_watch(path, mask, auto_add, quiet)
+        else:
+            self.log.debug('Watch already exists on %r', path)
+            return self._wdm[wd].started
+
+    def __del__(self):
+        """Stop the observer."""
+        self.observer.stop()
+
+    def stop(self):
+        """Close the manager and stop all watches."""
+        self.log.debug('Stopping watches.')
+        for current_wd in self._wdm:
+            self._wdm[current_wd].stop_watching()
+            self.observer.unschedule(self._wdm[current_wd].stream)
+            self.log.debug('Stopping Watch on %r.', self._wdm[current_wd].path)
+        self.observer.stop()
+
+    def del_watch(self, wd):
+        """Delete the watch with the given descriptor."""
+        watch = self.get_watch(wd)
+        self.observer.unschedule(watch.stream)
+        return super(WatchManager, self).del_watch(wd)
+
+    def _adding_watch(self, path, mask, auto_add):
+        """This method perform actually the action of registering the watch."""
+        watch = Watch(self._wd_count, path, mask, auto_add, self._processor)
+        self._wdm[self._wd_count] = watch
+        self._wdm[self._wd_count].start_watching()
+        self.observer.schedule(self._wdm[self._wd_count].stream)
+        self._wd_count += 1
+        return defer.succeed(True)
+
+    def rm_watch(self, wd, rec=False, quiet=True):
+        """Remove the the watch with the given wd."""
+        watch = self.get_watch(wd)
+        self.observer.unschedule(watch.stream)
+        super(WatchManager, self).del_watch(wd, rec, quiet)
+
 
 class FilesystemMonitor(object):
     """Empty implementation of FilesystemMonitor"""

=== modified file 'ubuntuone/platform/filesystem_notifications/windows.py'
--- ubuntuone/platform/filesystem_notifications/windows.py	2012-06-22 14:17:44 +0000
+++ ubuntuone/platform/filesystem_notifications/windows.py	2012-06-28 17:23:38 +0000
@@ -68,6 +68,25 @@
 
 from ubuntuone.platform.filesystem_notifications import common
 
+# a map between the few events that we have on common platforms and those
+# found in pyinotify
+common.COMMON_ACTIONS = {
+  1: common.IN_CREATE,
+  2: common.IN_DELETE,
+  3: common.IN_MODIFY,
+  4: common.IN_MOVED_FROM,
+  5: common.IN_MOVED_TO,
+}
+
+# a map of the actions to names so that we have better logs.
+common.COMMON_ACTIONS_NAMES = {
+  1: 'IN_CREATE',
+  2: 'IN_DELETE',
+  3: 'IN_MODIFY',
+  4: 'IN_MOVED_FROM',
+  5: 'IN_MOVED_TO',
+}
+
 
 # constant found in the msdn documentation:
 # http://msdn.microsoft.com/en-us/library/ff538834(v=vs.85).aspx
@@ -113,40 +132,8 @@
             # makes the multiplatform better, linux was first :P
             syncdaemon_path = get_syncdaemon_valid_path(
                                         os.path.join(self._path, file_name))
-            full_dir_path = os.path.join(self._path, file_name)
-            is_dir = self._path_is_dir(full_dir_path)
-            if is_dir:
-                # we need to update the list of subdirs that we have
-                self._update_subdirs(full_dir_path, action)
-            mask = common.COMMON_ACTIONS[action]
-            head, tail = os.path.split(file_name)
-            if is_dir:
-                mask |= common.IN_ISDIR
-            event_raw_data = {
-                'wd': self._descriptor,
-                'dir': is_dir,
-                'mask': mask,
-                'name': tail,
-                'path': '.'}
-            # by the way in which the win api fires the events we know for
-            # sure that no move events will be added in the wrong order, this
-            # is kind of hacky, I dont like it too much
-            if common.COMMON_ACTIONS[action] == common.IN_MOVED_FROM:
-                self._cookie = str(uuid4())
-                self._source_pathname = tail
-                event_raw_data['cookie'] = self._cookie
-            if common.COMMON_ACTIONS[action] == common.IN_MOVED_TO:
-                event_raw_data['src_pathname'] = self._source_pathname
-                event_raw_data['cookie'] = self._cookie
-            event = common.Event(event_raw_data)
-            # FIXME: event deduces the pathname wrong and we need to manually
-            # set it
-            event.pathname = syncdaemon_path
-            # add the event only if we do not have an exclude filter or
-            # the exclude filter returns False, that is, the event will not
-            # be excluded
-            self.log.debug('Pushing event %r to processor.', event)
-            self._processor(event)
+            self._process_events_from_filesystem(action, file_name,
+                str(uuid4()), syncdaemon_path)
 
     def _call_deferred(self, f, *args):
         """Executes the deferred call avoiding possible race conditions."""
@@ -246,6 +233,23 @@
 
 class WatchManager(common.WatchManager):
 
+    @common.is_valid_os_path(path_indexes=[1])
+    def add_watch(self, path, mask, auto_add=False, quiet=True):
+        """Add a new path to be watch.
+
+        The method will ensure that the path is not already present.
+        """
+        if not isinstance(path, unicode):
+            e = NotImplementedError("No implementation on this platform.")
+            return defer.fail(e)
+        wd = self.get_wd(path)
+        if wd is None:
+            self.log.debug('Adding single watch on %r', path)
+            return self._add_single_watch(path, mask, auto_add, quiet)
+        else:
+            self.log.debug('Watch already exists on %r', path)
+            return self._wdm[wd].started
+
     def stop(self):
         """Close the manager and stop all watches."""
         self.log.debug('Stopping watches.')

=== modified file 'ubuntuone/platform/os_helper/__init__.py'
--- ubuntuone/platform/os_helper/__init__.py	2012-05-15 13:25:37 +0000
+++ ubuntuone/platform/os_helper/__init__.py	2012-06-28 17:23:38 +0000
@@ -70,3 +70,9 @@
 is_root = source.is_root
 get_path_list = source.get_path_list
 normpath = source.normpath
+
+# Decorators
+
+is_valid_syncdaemon_path = source.is_valid_syncdaemon_path
+is_valid_os_path = source.is_valid_os_path
+os_path = source.os_path

=== modified file 'ubuntuone/platform/os_helper/darwin.py'
--- ubuntuone/platform/os_helper/darwin.py	2012-06-22 13:54:58 +0000
+++ ubuntuone/platform/os_helper/darwin.py	2012-06-28 17:23:38 +0000
@@ -107,6 +107,33 @@
     """Set the name of the application."""
     # nothing to be done let the plist take care of it
 
+
+# TODO: Implement this decorators to fix some encoding issues in darwin
+
+def is_valid_syncdaemon_path(path_indexes=None):
+    def decorator(func):
+        def wrapped(*args, **kwargs):
+            return func(*args, **kwargs)
+        return wrapped
+    return decorator
+
+
+def is_valid_os_path(path_indexes=None):
+    def decorator(func):
+        def wrapped(*args, **kwargs):
+            return func(*args, **kwargs)
+        return wrapped
+    return decorator
+
+
+def os_path(path_indexes=None):
+    def decorator(func):
+        def wrapped(*args, **kwargs):
+            return func(*args, **kwargs)
+        return wrapped
+    return decorator
+
+
 def get_syncdaemon_valid_path(path):
     """Get a 'darwin' path and modify it so that it can be used in sd."""
     return unicodedata.normalize('NFC', path.decode('utf-8')).encode('utf-8')

=== modified file 'ubuntuone/platform/os_helper/linux.py'
--- ubuntuone/platform/os_helper/linux.py	2012-05-17 11:57:49 +0000
+++ ubuntuone/platform/os_helper/linux.py	2012-06-28 17:23:38 +0000
@@ -123,3 +123,6 @@
 is_root = unix.is_root
 get_path_list = unix.get_path_list
 normpath = unix.normpath
+is_valid_syncdaemon_path = None
+is_valid_os_path = None
+os_path = None

=== modified file 'ubuntuone/platform/os_helper/windows.py'
--- ubuntuone/platform/os_helper/windows.py	2012-06-28 10:26:50 +0000
+++ ubuntuone/platform/os_helper/windows.py	2012-06-28 17:23:38 +0000
@@ -326,6 +326,8 @@
     """
     return _is_valid_path(assert_windows_path, path_indexes)
 
+is_valid_os_path = is_valid_windows_path
+
 
 def is_valid_syncdaemon_path(path_indexes=None):
     """Decorator to validate the parameters using assert_syncdaemon_path.
@@ -410,6 +412,8 @@
     """
     return _transform_path(get_windows_valid_path, path_indexes)
 
+os_path = windowspath
+
 
 def syncdamonpath(path_indexes=None):
     """Decorator to validate and transform path parameters.
@@ -885,7 +889,7 @@
 
 def is_root():
     """Return if the user is running as root."""
-    # Always return False. Trying to be smart about OS versions and 
+    # Always return False. Trying to be smart about OS versions and
     # only calling Windows APIs under certain conditions has still
     # proven not to work in some cases. Overall it should not matter
     # if we know whether we are Administrator or not on Windows.