Merge lp:~mandel/ubuntuone-client/fsevents-daemon into lp:ubuntuone-client

[Manuel de la Peña]

Added initial inclusion of the file system monitor that will allow to use the fsevents daemon in darwin.

Added initial inclusion of the file system monitor that will allow to use the fsevents daemon in darwin.

[Diego Sarmentero (diegosarmentero)]

+1

review: Approve

[Alejandro J. Cura (alecu)]

A few typos that I told mandel about; other than that, this is an awesome branch. Congrats mandel!

review: Approve

[Ubuntu One Auto Pilot (otto-pilot)]

The attempt to merge lp:~mandel/ubuntuone-client/fsevents-daemon into lp:ubuntuone-client failed. Below is the output from the failed tests.

/usr/bin/gnome-autogen.sh
checking for autoconf >= 2.53...
  testing autoconf2.50... not found.
  testing autoconf... found 2.69
checking for automake >= 1.10...
  testing automake-1.11... found 1.11.5
checking for libtool >= 1.5...
  testing libtoolize... found 2.4.2
checking for intltool >= 0.30...
  testing intltoolize... found 0.50.2
checking for pkg-config >= 0.14.0...
  testing pkg-config... found 0.26
checking for gtk-doc >= 1.0...
  testing gtkdocize... found 1.18
Checking for required M4 macros...
Checking for forbidden M4 macros...
Processing ./configure.ac
Running libtoolize...
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
Running intltoolize...
Running gtkdocize...
Running aclocal-1.11...
Running autoconf...
Running autoheader...
Running automake-1.11...
Running ./configure --enable-gtk-doc --enable-debug ...
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for library containing strerror... none required
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
ch...

[Manuel de la Peña (mandel)]

I'll fix the typos etc.. before it lands :)

[Ubuntu One Auto Pilot (otto-pilot)]

The attempt to merge lp:~mandel/ubuntuone-client/fsevents-daemon into lp:ubuntuone-client failed. Below is the output from the failed tests.

/usr/bin/gnome-autogen.sh
checking for autoconf >= 2.53...
  testing autoconf2.50... not found.
  testing autoconf... found 2.69
checking for automake >= 1.10...
  testing automake-1.11... found 1.11.5
checking for libtool >= 1.5...
  testing libtoolize... found 2.4.2
checking for intltool >= 0.30...
  testing intltoolize... found 0.50.2
checking for pkg-config >= 0.14.0...
  testing pkg-config... found 0.26
checking for gtk-doc >= 1.0...
  testing gtkdocize... found 1.18
Checking for required M4 macros...
Checking for forbidden M4 macros...
Processing ./configure.ac
Running libtoolize...
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
Running intltoolize...
Running gtkdocize...
Running aclocal-1.11...
Running autoconf...
Running autoheader...
Running automake-1.11...
Running ./configure --enable-gtk-doc --enable-debug ...
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for library containing strerror... none required
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
ch...

=== modified file 'Makefile.am'
--- Makefile.am	2012-05-15 13:25:37 +0000
+++ Makefile.am	2012-07-19 14:21:30 +0000
@@ -54,7 +54,7 @@
 test: logging.conf $(clientdefs_DATA) Makefile
 	echo "$(PYTHONPATH)"
 	if test "x$(builddir)" == "x$(srcdir)"; then \
-		PYTHONPATH="$(PYTHONPATH)" u1trial -r $(REACTOR) -p tests/platform/windows,tests/proxy -i "test_windows.py,test_darwin.py" tests || exit 1; \
+		PYTHONPATH="$(PYTHONPATH)" u1trial -r $(REACTOR) -p tests/platform/windows,tests/proxy -i "test_windows.py,test_darwin.py,test_fsevents_daemon.py" tests || exit 1; \
 		PYTHONPATH="$(PYTHONPATH)" u1trial -r qt4 -p tests/platform/windows -i "test_windows.py,test_darwin.py" tests/proxy || exit 1; \
 	fi
 	rm -rf _trial_temp

=== modified file 'run-tests.bat'
--- run-tests.bat	2012-05-16 16:56:41 +0000
+++ run-tests.bat	2012-07-19 14:21:30 +0000
@@ -74,7 +74,7 @@
 COPY windows\clientdefs.py ubuntuone\clientdefs.py
 COPY windows\logging.conf data\logging.conf
 :: execute the tests with a number of ignored linux and mac os only modules
-"%PYTHONEXEPATH%" "%TRIALPATH%" --reactor=twisted -c -p tests\platform\linux -i "test_linux.py,test_darwin.py" %PARAMS% tests
+"%PYTHONEXEPATH%" "%TRIALPATH%" --reactor=twisted -c -p tests\platform\linux -i "test_linux.py,test_darwin.py,test_fsevents_daemon.py" %PARAMS% tests
 
 IF %SKIPLINT% == 1 (
     ECHO Skipping style checks

=== added file 'tests/platform/filesystem_notifications/test_fsevents_daemon.py'
--- tests/platform/filesystem_notifications/test_fsevents_daemon.py	1970-01-01 00:00:00 +0000
+++ tests/platform/filesystem_notifications/test_fsevents_daemon.py	2012-07-19 14:21:30 +0000
@@ -0,0 +1,437 @@
+# -*- coding: utf-8 *-*
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# In addition, as a special exception, the copyright holders give
+# permission to link the code of portions of this program with the
+# OpenSSL library under certain conditions as described in each
+# individual source file, and distribute linked combinations
+# including the two.
+# You must obey the GNU General Public License in all respects
+# for all of the code used other than OpenSSL.  If you modify
+# file(s) with this exception, you may extend this exception to your
+# version of the file(s), but you are not obligated to do so.  If you
+# do not wish to do so, delete this exception statement from your
+# version.  If you delete this exception statement from all source
+# files in the program, then also delete it here.
+"""Tests for the fsevents daemon integration."""
+
+import os
+
+from twisted.internet import defer
+
+from contrib.testing.testcase import BaseTwistedTestCase
+from ubuntuone.darwin import fsevents
+from ubuntuone.platform.filesystem_notifications.darwin import fsevents_daemon
+from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
+    IN_CREATE,
+    IN_DELETE,
+    IN_MOVED_FROM,
+    IN_MOVED_TO,
+)
+
+class FakeDaemonEvent(object):
+    """A fake daemon event."""
+
+    def __init__(self):
+        self.event_paths = []
+        self.is_directory = False
+        self.event_type = None
+
+
+class FakeProcessor(object):
+    """A fake processor."""
+
+    def __init__(self):
+        """Create a new instance."""
+        self.processed_events = []
+
+    def __call__(self, event):
+        """Process and event."""
+        self.processed_events.append(event)
+
+
+class FakePyInotifyEventsFactory(object):
+    """Fake factory."""
+
+    def __init__(self, processor):
+        """Create a new instance."""
+        self.processor = processor
+        self.called = []
+        self.watched_paths = []
+        self.ignored_paths = []
+
+
+class FakeTransport(object):
+    """A fake transport for the protocol."""
+
+    def __init__(self):
+        """Create a new instance."""
+        self.called = []
+
+    def loseConnection(self):
+        """"Lost the connection."""
+        self.called.append('loseConnection')
+
+
+class FakeProtocol(object):
+    """A fake protocol object to interact with the daemon."""
+
+    def __init__(self):
+        """Create a new instance."""
+        self.called = []
+        self.transport = FakeTransport()
+
+    def remove_user(self):
+        """Remove the user."""
+        self.called.append('remove_user')
+        return defer.succeed(None)
+
+    def remove_path(self, path):
+        """Remove a path."""
+        self.called.extend(['remove_path', path])
+        return defer.succeed(True)
+
+    def add_path(self, path):
+        """Add a path."""
+        self.called.extend(['add_path', path])
+
+class PyInotifyEventsFactoryTestCase(BaseTwistedTestCase):
+    """Test the factory used to receive events."""
+
+    @defer.inlineCallbacks
+    def setUp(self):
+        """Set the diff tests."""
+        yield super(PyInotifyEventsFactoryTestCase, self).setUp()
+        self.processor = FakeProcessor()
+        self.factory = fsevents_daemon.PyInotifyEventsFactory(self.processor)
+
+    def test_path_interesting_not_watched_or_ignored(self):
+        """Test that we do know if the path is not interesting."""
+        path = u'/not/watched/path'
+        self.assertTrue(self.factory.path_is_not_interesting(path))
+
+    def test_path_interesting_watched_not_ignored(self):
+        """Test that we do not know if the path is not interesting."""
+        path = u'/watched/path'
+        self.factory.watched_paths.append(path)
+        self.assertFalse(self.factory.path_is_not_interesting(path))
+
+    def test_path_interesting_watched_but_ignored(self):
+        """Test that we do not know if the path is not interesting."""
+        path = u'/ignored/path'
+        self.factory.watched_paths.append(path)
+        self.factory.ignored_paths.append(path)
+        self.assertTrue(self.factory.path_is_not_interesting(path))
+
+    def test_path_interesting_not_watched_but_ignored(self):
+        """Test that we do not know if the path is not interesting."""
+        path = u'/ignored/path'
+        self.factory.ignored_paths.append(path)
+        self.assertTrue(self.factory.path_is_not_interesting(path))
+
+    def test_is_create_false_rename(self):
+        """Test if we do know when an event is a create."""
+        source_path = u'/other/watched/path'
+        destination_path = u'/watched/path'
+        source_head, _ = os.path.split(source_path)
+        destination_head, _ = os.path.split(destination_path)
+        self.factory.watched_paths.extend([source_head, destination_head])
+        event = FakeDaemonEvent()
+        event.event_paths.extend([source_path, destination_path])
+        self.assertFalse(self.factory.is_create(event))
+
+    def test_is_create_false_delete(self):
+        """Test if we do know when an event is a create."""
+        source_path = u'/watched/path'
+        destination_path = u'/not/watched/path'
+        source_head, _ = os.path.split(source_path)
+        self.factory.watched_paths.append(source_head)
+        event = FakeDaemonEvent()
+        event.event_paths.extend([source_path, destination_path])
+        self.assertFalse(self.factory.is_create(event))
+
+    def test_is_create_true(self):
+        """Test is we do know when an event is a create."""
+        source_path = u'/not/watched/path'
+        destination_path = u'/watched/path'
+        destination_head, _ = os.path.split(destination_path)
+        self.factory.watched_paths.append(destination_head)
+        event = FakeDaemonEvent()
+        event.event_paths.extend([source_path, destination_path])
+        self.assertTrue(self.factory.is_create(event))
+
+    def test_is_delete_false_rename(self):
+        """Test if we do know when an event is a delete."""
+        source_path = u'/other/watched/path'
+        destination_path = u'/watched/path'
+        source_head, _ = os.path.split(source_path)
+        destination_head, _ = os.path.split(destination_path)
+        self.factory.watched_paths.extend([source_head, destination_head])
+        event = FakeDaemonEvent()
+        event.event_paths.extend([source_path, destination_path])
+        self.assertFalse(self.factory.is_delete(event))
+
+    def test_is_delete_false_create(self):
+        """Test if we do know when an event is a delete."""
+        source_path = u'/not/watched/path'
+        destination_path = u'/watched/path'
+        destination_head, _ = os.path.split(destination_path)
+        self.factory.watched_paths.append(destination_head)
+        event = FakeDaemonEvent()
+        event.event_paths.extend([source_path, destination_path])
+        self.assertFalse(self.factory.is_delete(event))
+
+    def test_is_delete_true(self):
+        """Test if we do know when an event is a delete."""
+        source_path = u'/watched/path'
+        destination_path = u'/not/watched/path'
+        source_head, _ = os.path.split(source_path)
+        self.factory.watched_paths.append(source_head)
+        event = FakeDaemonEvent()
+        event.event_paths.extend([source_path, destination_path])
+        self.assertTrue(self.factory.is_delete(event))
+
+    def test_generate_from_event(self):
+        """Test the creation of a fake from event."""
+        cookie = 'cookie'
+        source_path = u'/source/path'
+        destination_path = u'/destination/path'
+        event = FakeDaemonEvent()
+        event.event_paths.extend([source_path, destination_path])
+        pyinotify_event = self.factory.generate_from_event(event, cookie)
+        self.assertEqual(cookie, pyinotify_event.cookie)
+        self.assertEqual(0, pyinotify_event.wd)
+        self.assertEqual(event.is_directory, pyinotify_event.dir)
+        self.assertEqual(IN_MOVED_FROM, pyinotify_event.mask)
+        self.assertEqual(source_path, pyinotify_event.pathname)
+
+    def test_generate_to_event(self):
+        """Test the creation of a fake to event."""
+        cookie = 'cookie'
+        source_path = u'/source/path'
+        destination_path = u'/destination/path'
+        event = FakeDaemonEvent()
+        event.event_paths.extend([source_path, destination_path])
+        pyinotify_event = self.factory.generate_to_event(event, cookie)
+        self.assertEqual(cookie, pyinotify_event.cookie)
+        self.assertEqual(0, pyinotify_event.wd)
+        self.assertEqual(event.is_directory, pyinotify_event.dir)
+        self.assertEqual(IN_MOVED_TO, pyinotify_event.mask)
+        self.assertEqual(destination_path, pyinotify_event.pathname)
+
+    def test_convert_in_pyinotify_event_no_rename(self):
+        """Test the creation of a no rename event."""
+        event_path = u'/path/of/the/event'
+        for action in fsevents_daemon.DARWIN_ACTIONS:
+            event = FakeDaemonEvent()
+            event.event_paths.append(event_path)
+            event.event_type = action
+            converted_events = self.factory.convert_in_pyinotify_event(event)
+            self.assertEqual(1, len(converted_events))
+            pyinotify_event = converted_events[0]
+            self.assertEqual(0, pyinotify_event.wd)
+            self.assertEqual(event.is_directory, pyinotify_event.dir)
+            self.assertEqual(fsevents_daemon.DARWIN_ACTIONS[action],
+                pyinotify_event.mask)
+            self.assertEqual(event_path, pyinotify_event.pathname)
+
+    def test_convert_in_pyinotify_event_rename_create(self):
+        """Test the creation of a rename which is a create."""
+        source_path = u'/not/watched/path'
+        destination_path = u'/watched/path'
+        head, _ = os.path.split(destination_path)
+        self.factory.watched_paths.append(head)
+        event = FakeDaemonEvent()
+        event.event_type = fsevents.FSE_RENAME
+        event.event_paths.extend([source_path, destination_path])
+        converted_events = self.factory.convert_in_pyinotify_event(event)
+        self.assertEqual(1, len(converted_events))
+        pyinotify_event = converted_events[0]
+        self.assertEqual(0, pyinotify_event.wd)
+        self.assertEqual(event.is_directory, pyinotify_event.dir)
+        self.assertEqual(IN_CREATE, pyinotify_event.mask)
+        self.assertEqual(destination_path, pyinotify_event.pathname)
+
+    def test_convert_in_pyinotify_event_rename_delete(self):
+        """Test the creation of a rename which is a delete."""
+        source_path = u'/watched/path'
+        destination_path = u'/not/watched/path'
+        head, _ = os.path.split(source_path)
+        self.factory.watched_paths.append(head)
+        event = FakeDaemonEvent()
+        event.event_type = fsevents.FSE_RENAME
+        event.event_paths.extend([source_path, destination_path])
+        converted_events = self.factory.convert_in_pyinotify_event(event)
+        self.assertEqual(1, len(converted_events))
+        pyinotify_event = converted_events[0]
+        self.assertEqual(0, pyinotify_event.wd)
+        self.assertEqual(event.is_directory, pyinotify_event.dir)
+        self.assertEqual(IN_DELETE, pyinotify_event.mask)
+        self.assertEqual(source_path, pyinotify_event.pathname)
+
+    def test_convert_in_pyinotify_event_rename(self):
+        """Test the creation of a rename event."""
+        source_path = u'/watched/path1'
+        destination_path = u'/watched/path2'
+        head, _ = os.path.split(source_path)
+        self.factory.watched_paths.append(head)
+        event = FakeDaemonEvent()
+        event.event_type = fsevents.FSE_RENAME
+        event.event_paths.extend([source_path, destination_path])
+        converted_events = self.factory.convert_in_pyinotify_event(event)
+        self.assertEqual(2, len(converted_events))
+        from_event = converted_events[0]
+        to_event = converted_events[1]
+        # assert from event
+        self.assertEqual(0, from_event.wd)
+        self.assertEqual(event.is_directory, from_event.dir)
+        self.assertEqual(IN_MOVED_FROM, from_event.mask)
+        self.assertEqual(source_path, from_event.pathname)
+        # assert to event
+        self.assertEqual(0, to_event.wd)
+        self.assertEqual(event.is_directory, to_event.dir)
+        self.assertEqual(IN_MOVED_TO, to_event.mask)
+        self.assertEqual(destination_path, to_event.pathname)
+        # assert cookie
+        self.assertEqual(from_event.cookie, to_event.cookie)
+
+    def test_process_event_ignored_type(self):
+        """Test processing the event of an ignored type."""
+        for action in fsevents_daemon.DARWIN_IGNORED_ACTIONS:
+            event = FakeDaemonEvent()
+            event.event_type = action
+            self.factory.process_event(event)
+        self.assertEqual(0, len(self.processor.processed_events))
+
+    def test_process_event_dropped(self):
+        """Test processing the drop of the events."""
+        func_called = []
+        event = FakeDaemonEvent()
+        event.event_type = fsevents.FSE_EVENTS_DROPPED
+
+        def fake_events_dropped():
+            """A fake events dropped implementation."""
+            func_called.append('fake_events_dropped')
+
+        self.patch(self.factory, 'events_dropper', fake_events_dropped)
+        self.factory.process_event(event)
+        self.assertIn('fake_events_dropped', func_called)
+
+    def test_process_ignored_path(self):
+        """Test processing events from an ignored path."""
+        event_path = u'/path/of/the/event'
+        head, _ = os.path.split(event_path)
+        self.factory.ignored_paths.append(head)
+        event = FakeDaemonEvent()
+        event.event_paths.append(event_path)
+        event.event_type = fsevents.FSE_CREATE_FILE
+        self.factory.process_event(event)
+        self.assertEqual(0, len(self.processor.processed_events))
+
+    def test_process_not_ignored_path(self):
+        """Test processing events that are not ignored."""
+        event_path = u'/path/of/the/event'
+        head, _ = os.path.split(event_path)
+        self.factory.watched_paths.append(head)
+        event = FakeDaemonEvent()
+        event.event_paths.append(event_path)
+        event.event_type = fsevents.FSE_CREATE_FILE
+        self.factory.process_event(event)
+        self.assertEqual(1, len(self.processor.processed_events))
+        self.assertEqual(event_path,
+                self.processor.processed_events[0].pathname)
+
+
+class FilesystemMonitorTestCase(BaseTwistedTestCase):
+    """Test the notify processor."""
+
+    def fake_connect_to_daemon(self):
+        """A fake connection to daemon call."""
+        self.monitor._protocol = self.protocol
+        defer.succeed(self.protocol)
+
+    @defer.inlineCallbacks
+    def setUp(self):
+        """Set the tests."""
+        yield super(FilesystemMonitorTestCase, self).setUp()
+        self.processor = FakeProcessor()
+        self.factory = FakePyInotifyEventsFactory(self.processor)
+        self.protocol = FakeProtocol()
+        self.monitor = fsevents_daemon.FilesystemMonitor(None, None)
+
+        # override default objects
+        self.monitor._processor = self.processor
+        self.monitor._factory = self.factory
+
+        # patch the connect
+        self.patch(fsevents_daemon.FilesystemMonitor, '_connect_to_daemon',
+                self.fake_connect_to_daemon)
+
+    @defer.inlineCallbacks
+    def test_shutdown_protocol(self):
+        """Test shutdown with a protocol."""
+        self.monitor._protocol = self.protocol
+        yield self.monitor.shutdown()
+        self.assertIn('remove_user', self.protocol.called)
+
+    @defer.inlineCallbacks
+    def test_shutdown_no_protocol(self):
+        """Test shutdown without a protocol."""
+        stopped = yield self.monitor.shutdown()
+        self.assertTrue(stopped)
+
+    @defer.inlineCallbacks
+    def test_rm_path_not_root(self):
+        """Test removing a path."""
+        dirpath = '/path/to/remove/'
+        self.factory.watched_paths.append('/path')
+        yield self.monitor.rm_watch(dirpath)
+        self.assertIn(dirpath, self.factory.ignored_paths)
+
+    @defer.inlineCallbacks
+    def test_rm_path_root(self):
+        """Test removing a path that is a root path."""
+        dirpath = '/path/to/remove/'
+        self.factory.watched_paths.append(dirpath)
+        yield self.monitor.rm_watch(dirpath)
+        self.assertIn('remove_path', self.protocol.called)
+        self.assertIn(dirpath, self.protocol.called)
+        self.assertNotIn(dirpath, self.factory.watched_paths)
+
+    @defer.inlineCallbacks
+    def test_add_watch_not_root(self):
+        """Test adding a watch."""
+        dirpath = '/path/to/remove/'
+        self.factory.watched_paths.append('/path')
+        yield self.monitor.add_watch(dirpath)
+        self.assertNotIn('add_path', self.protocol.called)
+
+    @defer.inlineCallbacks
+    def test_add_watch_root(self):
+        """Test adding a watch that is a root."""
+        dirpath = '/path/to/remove/'
+        self.factory.watched_paths.append('/other/path')
+        yield self.monitor.add_watch(dirpath)
+        self.assertIn('add_path', self.protocol.called)
+        self.assertIn(dirpath, self.protocol.called)
+
+    @defer.inlineCallbacks
+    def test_add_watch_ignored(self):
+        """Test adding a watch that was ignored."""
+        dirpath = '/path/to/remove/'
+        self.factory.ignored_paths.append(dirpath)
+        yield self.monitor.add_watch(dirpath)
+        self.assertNotIn('add_path', self.protocol.called)

=== modified file 'ubuntuone/platform/filesystem_notifications/common.py'
--- ubuntuone/platform/filesystem_notifications/common.py	2012-07-13 12:39:33 +0000
+++ ubuntuone/platform/filesystem_notifications/common.py	2012-07-19 14:21:30 +0000
@@ -58,8 +58,10 @@
 )
 
 if sys.platform == 'darwin':
-    from ubuntuone.platform.filesystem_notifications import darwin
-    source = darwin
+    from ubuntuone.platform.filesystem_notifications.darwin import (
+        fsevents_client,
+    )
+    source = fsevents_client
 elif sys.platform == 'win32':
     from ubuntuone.platform.filesystem_notifications import windows
     source = windows
@@ -330,8 +332,8 @@
     @is_valid_os_path(path_indexes=[1])
     def get_wd(self, path):
         """Return the watcher that is used to watch the given path."""
-        if not path.endswith(os.path.sep):
-            path = path + os.path.sep
+        if not path[-1] == os.path.sep:
+            path += os.path.sep
         for current_wd in self._wdm:
             watch_path = self._wdm[current_wd].path
             if ((watch_path == path or watch_path in path)

=== added directory 'ubuntuone/platform/filesystem_notifications/darwin'
=== renamed file 'ubuntuone/platform/filesystem_notifications/darwin.py' => 'ubuntuone/platform/filesystem_notifications/darwin/__init__.py'
--- ubuntuone/platform/filesystem_notifications/darwin.py	2012-07-13 12:39:33 +0000
+++ ubuntuone/platform/filesystem_notifications/darwin/__init__.py	2012-07-19 14:21:30 +0000
@@ -28,132 +28,8 @@
 # files in the program, then also delete it here.
 """Filesystem Notifications module for MAC OS."""
 
-import os
-
-import fsevents
-from twisted.internet import defer, reactor
-
-from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
-    IN_DELETE,
-    IN_CREATE,
-    IN_MODIFY,
-    IN_MOVED_FROM,
-    IN_MOVED_TO,
-)
-
-# a map between the few events that we have on common platforms and those
-# found in pyinotify
-ACTIONS = {
-    fsevents.IN_CREATE: IN_CREATE,
-    fsevents.IN_DELETE: IN_DELETE,
-    fsevents.IN_MODIFY: IN_MODIFY,
-    fsevents.IN_MOVED_FROM: IN_MOVED_FROM,
-    fsevents.IN_MOVED_TO: IN_MOVED_TO,
-}
-
-# a map of the actions to names so that we have better logs.
-ACTIONS_NAMES = {
-    fsevents.IN_CREATE: 'IN_CREATE',
-    fsevents.IN_DELETE: 'IN_DELETE',
-    fsevents.IN_MODIFY: 'IN_MODIFY',
-    fsevents.IN_MOVED_FROM: 'IN_MOVED_FROM',
-    fsevents.IN_MOVED_TO: 'IN_MOVED_TO',
-}
-
-
-def path_is_ignored(path):
-    """Should we ignore this path in the current platform.?"""
-    # don't support links yet
-    if os.path.islink(path):
-        return True
-    return False
-
-
-# The implementation of the code that is provided as the pyinotify substitute
-class Watch(object):
-    """Implement the same functions as pyinotify.Watch."""
-
-    def __init__(self, path, process_events):
-        """Create a new instance for the given path.
-
-        The process_events parameter is a callback to be executed in the main
-        reactor thread to convert events in pyinotify events and add them to
-        the state machine.
-        """
-        self.path = os.path.abspath(path)
-        self.process_events = process_events
-        self.watching = False
-        self.ignore_paths = []
-        # Create stream with folder to watch
-        self.stream = fsevents.Stream(self._process_events,
-            path, file_events=True)
-
-    def _process_events(self, event):
-        """Receive the filesystem event and move it to the main thread."""
-        reactor.callFromThread(self._process_events_in_main_thread, event)
-
-    def _process_events_in_main_thread(self, event):
-        """Process the events from the queue."""
-        action, cookie, file_name = (event.mask, event.cookie, event.name)
-
-        syncdaemon_path = os.path.join(self.path, file_name)
-        self.process_events(action, file_name, cookie,
-                syncdaemon_path)
-
-    def start_watching(self):
-        """Start watching."""
-        self.watching = True
-        return defer.succeed(self.watching)
-
-    def stop_watching(self):
-        """Stop watching."""
-        self.watching = False
-        return defer.succeed(self.watching)
-
-    # For API compatibility
-    @property
-    def started(self):
-        """A deferred that will be called when the watch is running."""
-        return defer.succeed(self.watching)
-
-    @property
-    def stopped(self):
-        """A deferred fired when the watch thread has finished."""
-        return defer.succeed(self.watching)
-
-
-class WatchManager(object):
-    """Implement the same functions as pyinotify.WatchManager.
-
-    All paths passed to methods in this class should be darwin paths.
-
-    """
-
-    def __init__(self, log):
-        """Init the manager to keep track of the different watches."""
-        self.log = log
-        self.observer = fsevents.Observer()
-        self.observer.start()
-
-    def stop_watch(self, watch):
-        """Stop a given watch."""
-        watch.stop_watching()
-        self.observer.unschedule(watch.platform_watch.stream)
-        return defer.succeed(True)
-
-    def stop(self):
-        """Stop the manager."""
-        self.observer.stop()
-        self.observer.join()
-
-    def del_watch(self, watch):
-        """Delete the watch and clean resources."""
-        self.observer.unschedule(watch.platform_watch.stream)
-
-    def add_watch(self, watch):
-        """This method perform actually the action of registering the watch."""
-        self.observer.schedule(watch.platform_watch.stream)
-        return True
-
-    def rm_watch(self, watch):
-        """Remove the the watch with the given wd."""
+from ubuntuone.platform.filesystem_notifications.darwin import fsevents_daemon
+
+
+FilesystemMonitor = fsevents_daemon.FilesystemMonitor
+NotifyProcessor = fsevents_daemon.NotifyProcessor

=== added file 'ubuntuone/platform/filesystem_notifications/darwin/fsevents_client.py'
--- ubuntuone/platform/filesystem_notifications/darwin/fsevents_client.py	1970-01-01 00:00:00 +0000
+++ ubuntuone/platform/filesystem_notifications/darwin/fsevents_client.py	2012-07-19 14:21:30 +0000
@@ -0,0 +1,159 @@
+# -*- coding: utf-8 *-*
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# In addition, as a special exception, the copyright holders give
+# permission to link the code of portions of this program with the
+# OpenSSL library under certain conditions as described in each
+# individual source file, and distribute linked combinations
+# including the two.
+# You must obey the GNU General Public License in all respects
+# for all of the code used other than OpenSSL.  If you modify
+# file(s) with this exception, you may extend this exception to your
+# version of the file(s), but you are not obligated to do so.  If you
+# do not wish to do so, delete this exception statement from your
+# version.  If you delete this exception statement from all source
+# files in the program, then also delete it here.
+"""Filesystem Notifications module for MAC OS."""
+
+import os
+
+import fsevents
+from twisted.internet import defer, reactor
+
+from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
+    IN_DELETE,
+    IN_CREATE,
+    IN_MODIFY,
+    IN_MOVED_FROM,
+    IN_MOVED_TO,
+)
+
+# a map between the few events that we have on common platforms and those
+# found in pyinotify
+ACTIONS = {
+    fsevents.IN_CREATE: IN_CREATE,
+    fsevents.IN_DELETE: IN_DELETE,
+    fsevents.IN_MODIFY: IN_MODIFY,
+    fsevents.IN_MOVED_FROM: IN_MOVED_FROM,
+    fsevents.IN_MOVED_TO: IN_MOVED_TO,
+}
+
+# a map of the actions to names so that we have better logs.
+ACTIONS_NAMES = {
+    fsevents.IN_CREATE: 'IN_CREATE',
+    fsevents.IN_DELETE: 'IN_DELETE',
+    fsevents.IN_MODIFY: 'IN_MODIFY',
+    fsevents.IN_MOVED_FROM: 'IN_MOVED_FROM',
+    fsevents.IN_MOVED_TO: 'IN_MOVED_TO',
+}
+
+
+def path_is_ignored(path):
+    """Should we ignore this path in the current platform.?"""
+    # don't support links yet
+    if os.path.islink(path):
+        return True
+    return False
+
+
+# The implementation of the code that is provided as the pyinotify substitute
+class Watch(object):
+    """Implement the same functions as pyinotify.Watch."""
+
+    def __init__(self, path, process_events):
+        """Create a new instance for the given path.
+
+        The process_events parameter is a callback to be executed in the main
+        reactor thread to convert events in pyinotify events and add them to
+        the state machine.
+        """
+        self.path = os.path.abspath(path)
+        self.process_events = process_events
+        self.watching = False
+        self.ignore_paths = []
+        # Create stream with folder to watch
+        self.stream = fsevents.Stream(self._process_events,
+            path, file_events=True)
+
+    def _process_events(self, event):
+        """Receive the filesystem event and move it to the main thread."""
+        reactor.callFromThread(self._process_events_in_main_thread, event)
+
+    def _process_events_in_main_thread(self, event):
+        """Process the events from the queue."""
+        action, cookie, file_name = (event.mask, event.cookie, event.name)
+
+        syncdaemon_path = os.path.join(self.path, file_name)
+        self.process_events(action, file_name, cookie,
+                syncdaemon_path)
+
+    def start_watching(self):
+        """Start watching."""
+        self.watching = True
+        return defer.succeed(self.watching)
+
+    def stop_watching(self):
+        """Stop watching."""
+        self.watching = False
+        return defer.succeed(self.watching)
+
+    # For API compatibility
+    @property
+    def started(self):
+        """A deferred that will be called when the watch is running."""
+        return defer.succeed(self.watching)
+
+    @property
+    def stopped(self):
+        """A deferred fired when the watch thread has finished."""
+        return defer.succeed(self.watching)
+
+
+class WatchManager(object):
+    """Implement the same functions as pyinotify.WatchManager.
+
+    All paths passed to methods in this class should be darwin paths.
+
+    """
+
+    def __init__(self, log):
+        """Init the manager to keep track of the different watches."""
+        self.log = log
+        self.observer = fsevents.Observer()
+        self.observer.start()
+
+    def stop_watch(self, watch):
+        """Stop a given watch."""
+        watch.stop_watching()
+        self.observer.unschedule(watch.platform_watch.stream)
+        return defer.succeed(True)
+
+    def stop(self):
+        """Stop the manager."""
+        self.observer.stop()
+        self.observer.join()
+
+    def del_watch(self, watch):
+        """Delete the watch and clean resources."""
+        self.observer.unschedule(watch.platform_watch.stream)
+
+    def add_watch(self, watch):
+        """This method perform actually the action of registering the watch."""
+        self.observer.schedule(watch.platform_watch.stream)
+        return True
+
+    def rm_watch(self, watch):
+        """Remove the the watch with the given wd."""

=== added file 'ubuntuone/platform/filesystem_notifications/darwin/fsevents_daemon.py'
--- ubuntuone/platform/filesystem_notifications/darwin/fsevents_daemon.py	1970-01-01 00:00:00 +0000
+++ ubuntuone/platform/filesystem_notifications/darwin/fsevents_daemon.py	2012-07-19 14:21:30 +0000
@@ -0,0 +1,594 @@
+# -*- coding: utf-8 *-*
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# permission to link the code of portions of this program with the
+# OpenSSL library under certain conditions as described in each
+# individual source file, and distribute linked combinations
+# including the two.
+# You must obey the GNU General Public License in all respects
+# for all of the code used other than OpenSSL.  If you modify
+# file(s) with this exception, you may extend this exception to your
+# version of the file(s), but you are not obligated to do so.  If you
+# do not wish to do so, delete this exception statement from your
+# version.  If you delete this exception statement from all source
+# files in the program, then also delete it here.
+"""Filesystem notifications based on the fsevents daemon.."""
+
+import logging
+import os
+import unicodedata
+
+from uuid import uuid4
+
+from twisted.internet import defer, endpoints, reactor
+
+from ubuntuone import logger
+from ubuntuone.darwin import fsevents
+from ubuntuone.platform.filesystem_notifications.pyinotify_agnostic import (
+    Event,
+    ProcessEvent,
+    IN_OPEN,
+    IN_CLOSE_NOWRITE,
+    IN_CLOSE_WRITE,
+    IN_CREATE,
+    IN_IGNORED,
+    IN_ISDIR,
+    IN_DELETE,
+    IN_MOVED_FROM,
+    IN_MOVED_TO,
+    IN_MODIFY)
+
+TRACE = logger.TRACE
+
+# map the fsevents actions to those from pyinotify
+DARWIN_ACTIONS = {
+    fsevents.FSE_CREATE_FILE: IN_CREATE,
+    fsevents.FSE_DELETE: IN_DELETE,
+    fsevents.FSE_STAT_CHANGED: IN_MODIFY,
+    fsevents.FSE_CONTENT_MODIFIED: IN_MODIFY,
+    fsevents.FSE_CREATE_DIR: IN_CREATE,
+}
+
+# list of those events from which we do not care
+DARWIN_IGNORED_ACTIONS = (
+    fsevents.FSE_UNKNOWN,
+    fsevents.FSE_INVALID,
+    fsevents.FSE_EXCHANGE,
+    fsevents.FSE_FINDER_INFO_CHANGED,
+    fsevents.FSE_CHOWN,
+    fsevents.FSE_XATTR_MODIFIED,
+    fsevents.FSE_XATTR_REMOVED,
+)
+
+# translates quickly the event and it's is_dir state to our standard events
+NAME_TRANSLATIONS = {
+    IN_OPEN: 'FS_FILE_OPEN',
+    IN_CLOSE_NOWRITE: 'FS_FILE_CLOSE_NOWRITE',
+    IN_CLOSE_WRITE: 'FS_FILE_CLOSE_WRITE',
+    IN_CREATE: 'FS_FILE_CREATE',
+    IN_CREATE | IN_ISDIR: 'FS_DIR_CREATE',
+    IN_DELETE: 'FS_FILE_DELETE',
+    IN_DELETE | IN_ISDIR: 'FS_DIR_DELETE',
+    IN_MOVED_FROM: 'FS_FILE_DELETE',
+    IN_MOVED_FROM | IN_ISDIR: 'FS_DIR_DELETE',
+    IN_MOVED_TO: 'FS_FILE_CREATE',
+    IN_MOVED_TO | IN_ISDIR: 'FS_DIR_CREATE'}
+
+# TODO: This should be in fsevents to be imported!
+# Path to the socket used by the daemon
+DAEMON_SOCKET = '/var/run/ubuntuone_fsevents_daemon'
+
+def get_syncdaemon_valid_path(path):
+    """Return a valid encoded path."""
+    return unicodedata.normalize('NFC', path).encode('utf-8')
+
+
+class PyInotifyEventsFactory(fsevents.FsEventsFactory):
+    """Factory that process events and converts them in pyinotify ones."""
+
+    def __init__(self, processor, ignored_events=DARWIN_IGNORED_ACTIONS):
+        """Create a new instance."""
+        # old style class
+        fsevents.FsEventsFactory.__init__(self)
+        self._processor = processor
+        self._ignored_events = ignored_events
+        self.watched_paths = []
+        self.ignored_paths = []
+
+    def events_dropper(self):
+        """Deal with the fact that the daemon dropped events."""
+
+    def path_is_not_interesting(self, path):
+        """Return if the factory is interested in the path."""
+        is_watched = any(path.startswith(watched_path)
+                             for watched_path in self.watched_paths)
+        is_ignored = any(path.startswith(ignored_path)
+                             for ignored_path in self.ignored_paths)
+        return not is_watched or (is_watched and is_ignored)
+
+    def is_create(self, event):
+        """Decide if a rename event should be considered a create."""
+        # is a create if the creation path (first path) is either not watched or
+        # in the ignored paths
+        source_path = get_syncdaemon_valid_path(event.event_paths[0])
+        return self.path_is_not_interesting(source_path)
+
+    def is_delete(self, event):
+        """Decide if a rename event should be considered a delete."""
+        # is a delete if the destination path (second path) is either not
+        # watched or in the ignored paths
+        dest_path = get_syncdaemon_valid_path(event.event_paths[1])
+        return self.path_is_not_interesting(dest_path)
+
+    def generate_from_event(self, event, cookie):
+        """Return a fake from event from a rename one."""
+        source_path = get_syncdaemon_valid_path(event.event_paths[0])
+        mask = IN_MOVED_FROM
+        if event.is_directory:
+            mask |= IN_ISDIR
+        head, tail = os.path.split(source_path)
+        event_raw_data = {
+            'wd': 0,  # we only have one factory
+            'dir': event.is_directory,
+            'mask': mask,
+            'name': tail,
+            'cookie': cookie,
+            'path': '.'}
+        move_from_event = Event(event_raw_data)
+        move_from_event.pathname = source_path
+        return move_from_event
+
+    def generate_to_event(self, event, cookie):
+        """Return a fake to event from a rename one."""
+        source_path = get_syncdaemon_valid_path(event.event_paths[0])
+        destination_path = get_syncdaemon_valid_path(event.event_paths[1])
+        mask = IN_MOVED_TO
+        if event.is_directory:
+            mask |= IN_ISDIR
+        source_head, source_tail = os.path.split(source_path)
+        head, tail = os.path.split(destination_path)
+        event_raw_data = {
+            'wd': 0,  # we only have one factory
+            'dir': event.is_directory,
+            'mask': mask,
+            'name': tail,
+            'cookie': cookie,
+            'src_pathname' : source_tail,
+            'path': '.'}
+        move_to_event = Event(event_raw_data)
+        move_to_event.pathname = destination_path
+        return move_to_event
+
+    def convert_in_pyinotify_event(self, event):
+        """Get an event from the daemon and convert it in a pyinotify one."""
+        # the rename is a special type of event because it has to be either
+        # converted is a pair of events or in a single one (CREATE or DELETE)
+        if event.event_type == fsevents.FSE_RENAME:
+            is_create = self.is_create(event)
+            if is_create or self.is_delete(event):
+                mask = IN_CREATE if is_create else IN_DELETE
+                if event.is_directory:
+                    mask |= IN_ISDIR
+                # a create means that we moved from a not watched path to a
+                # watched one and therefore we are interested in the SECOND
+                # path of the event. A delete means that we moved from a
+                # watched path for a not watched one and we care about the
+                # FIRST path of the event
+                path = event.event_paths[1] if is_create\
+                            else event.event_paths[0]
+                path = get_syncdaemon_valid_path(path)
+                head, tail = os.path.split(path)
+                event_raw_data = {
+                    'wd': 0,  # we only have one factory
+                    'dir': event.is_directory,
+                    'mask': mask,
+                    'name': tail,
+                    'path': '.'}
+                pyinotify_event = Event(event_raw_data)
+                pyinotify_event.pathname = path
+                return [pyinotify_event]
+            else:
+                # we do have a rename within watched paths lets generate to
+                # fake events
+                cookie = str(uuid4())
+                return [self.generate_from_event(event, cookie),
+                        self.generate_to_event(event, cookie)]
+        else:
+            mask = DARWIN_ACTIONS[event.event_type]
+            if event.is_directory:
+                mask |= IN_ISDIR
+            # we do know that we are not dealing with a move which are the only
+            # events that have more than one path
+            path = get_syncdaemon_valid_path(event.event_paths[0])
+            head, tail = os.path.split(path)
+            event_raw_data = {
+                'wd': 0,  # we only have one factory
+                'dir': event.is_directory,
+                'mask': mask,
+                'name': tail,
+                'path': '.'}
+            pyinotify_event = Event(event_raw_data)
+            # FIXME: event deduces the pathname wrong and we need to manually
+            # set it
+            pyinotify_event.pathname = path
+            return [pyinotify_event]
+
+    def _is_ignored_path(self, path):
+        """Returns if the path is ignored."""
+        if not path[-1] == os.path.sep:
+            path += os.path.sep
+
+        is_ignored_child = any(ignored in path for ignored in self.ignored_paths)
+        return path in self.ignored_paths or is_ignored_child
+
+    def process_event(self, event):
+        """Process an event from the fsevent daemon."""
+        if event.event_type in self._ignored_events:
+            # Do nothing because sd does not care about such info
+            return
+        if event.event_type == fsevents.FSE_EVENTS_DROPPED:
+            # this should not be very common but we have to deal with it
+            return self.events_dropper()
+        events = self.convert_in_pyinotify_event(event)
+        for pyinotify_event in events:
+            # assert that the path name is valid
+            if not any([pyinotify_event.pathname.startswith(path)
+                        for path in self.ignored_paths]):
+                # by definition we are being callFromThread so we do know that
+                # the  events are executed in the right order \o/
+                if not self._is_ignored_path(pyinotify_event.pathname):
+                    self._processor(pyinotify_event)
+
+
+class NotifyProcessor(ProcessEvent):
+    """Mac OS X implementation of NotifyProcessor"""
+
+    def __init__(self, monitor, ignore_config=None):
+        # XXX: avoid circular imports.
+        from ubuntuone.syncdaemon.filesystem_notifications import (
+            GeneralINotifyProcessor)
+        self.general_processor = GeneralINotifyProcessor(monitor,
+            self.handle_dir_delete, NAME_TRANSLATIONS,
+            self.platform_is_ignored, IN_IGNORED, ignore_config=ignore_config)
+        self.held_event = None
+
+    def rm_from_mute_filter(self, event, paths):
+        """Remove event from the mute filter."""
+        self.general_processor.rm_from_mute_filter(event, paths)
+
+    def add_to_mute_filter(self, event, paths):
+        """Add an event and path(s) to the mute filter."""
+        self.general_processor.add_to_mute_filter(event, paths)
+
+    def platform_is_ignored(self, path):
+        """Should we ignore this path in the current platform.?"""
+        # don't support links yet
+        if os.path.islink(path):
+            return True
+        return False
+
+    def is_ignored(self, path):
+        """Should we ignore this path?"""
+        return self.general_processor.is_ignored(path)
+
+    def release_held_event(self, timed_out=False):
+        """Release the event on hold to fulfill its destiny."""
+        self.general_processor.push_event(self.held_event)
+        self.held_event = None
+
+    def process_IN_MODIFY(self, event):
+        """Capture a modify event and fake an open ^ close write events."""
+        # lets ignore dir changes
+        if event.dir:
+            return
+        # on darwin we just get IN_MODIFY, lets always fake
+        # an OPEN & CLOSE_WRITE couple
+        raw_open = raw_close = {
+           'wd': event.wd,
+           'dir': event.dir,
+           'name': event.name,
+           'path': event.path}
+        # calculate the open mask
+        raw_open['mask'] = IN_OPEN
+        # create the event using the raw data, then fix the pathname param
+        open_event = Event(raw_open)
+        open_event.pathname = event.pathname
+        # push the open
+        self.general_processor.push_event(open_event)
+        raw_close['mask'] = IN_CLOSE_WRITE
+        close_event = Event(raw_close)
+        close_event.pathname = event.pathname
+        # push the close event
+        self.general_processor.push_event(close_event)
+
+    def process_IN_MOVED_FROM(self, event):
+        """Capture the MOVED_FROM to maybe syntethize FILE_MOVED."""
+        if self.held_event is not None:
+            self.general_processor.log.warn('Lost pair event of %s',
+                                            self.held_event)
+        self.held_event = event
+
+    def _fake_create_event(self, event):
+        """Fake the creation of an event."""
+        # this is the case of a MOVE from an ignored path (links for example)
+        # to a valid path
+        if event.dir:
+            evtname = "FS_DIR_"
+        else:
+            evtname = "FS_FILE_"
+        self.general_processor.eq_push(evtname + "CREATE", path=event.pathname)
+        if not event.dir:
+            self.general_processor.eq_push('FS_FILE_CLOSE_WRITE',
+                                            path=event.pathname)
+
+    def _fake_delete_create_event(self, event):
+        """Fake the deletion and the creation."""
+        # this is the case of a MOVE from a watch UDF to a diff UDF which
+        # means that we have to copy the way linux works.
+        if event.dir:
+            evtname = "FS_DIR_"
+        else:
+            evtname = "FS_FILE_"
+        m = "Delete because of different shares: %r"
+        self.log.info(m, self.held_event.pathname)
+        self.general_processor.eq_push(evtname + "DELETE",
+                                       path=self.held_event.pathname)
+        self.general_processor.eq_push(evtname + "CREATE", path=event.pathname)
+        if not event.dir:
+            self.general_processor.eq_push('FS_FILE_CLOSE_WRITE',
+                                            path=event.pathname)
+
+    def process_IN_MOVED_TO(self, event):
+        """Capture the MOVED_TO to maybe syntethize FILE_MOVED."""
+        if self.held_event is not None:
+            if event.cookie == self.held_event.cookie:
+                f_path_dir = os.path.split(self.held_event.pathname)[0]
+                t_path_dir = os.path.split(event.pathname)[0]
+
+                is_from_forreal = not self.is_ignored(self.held_event.pathname)
+                is_to_forreal = not self.is_ignored(event.pathname)
+                if is_from_forreal and is_to_forreal:
+                    f_share_id = self.general_processor.get_path_share_id(
+                        f_path_dir)
+                    t_share_id = self.general_processor.get_path_share_id(
+                        t_path_dir)
+                    if f_share_id != t_share_id:
+                        # if the share_id are != push a delete/create
+                        self._fake_delete_create_event(event)
+                    else:
+                        if event.dir:
+                            evtname = "FS_DIR_"
+                        else:
+                            evtname = "FS_FILE_"
+                        self.general_processor.eq_push(evtname + "MOVE",
+                            path_from=self.held_event.pathname,
+                            path_to=event.pathname)
+                elif is_to_forreal:
+                    # this is the case of a MOVE from something ignored
+                    # to a valid filename
+                    self._fake_create_event(event)
+
+                self.held_event = None
+                return
+            else:
+                # we should never get to this point on darwin, never ever!
+                self.release_held_event()
+                self.general_processor.push_event(event)
+        else:
+            # we should never get to this point on darwin, never ever!
+            self.general_processor.log.warn(
+                            'Cookie does not match the previous held event!')
+            self.general_processor.log.warn('Ignoring %s', event)
+
+    def process_default(self, event):
+        """Push the event into the EventQueue."""
+        if self.held_event is not None:
+            self.release_held_event()
+        self.general_processor.push_event(event)
+
+    def handle_dir_delete(self, fullpath):
+        """Some special work when a directory is deleted."""
+        self.general_processor.rm_watch(fullpath)
+
+        # handle the case of move a dir to a non-watched directory
+        paths = self.general_processor.get_paths_starting_with(fullpath,
+            include_base=False)
+
+        paths.sort(reverse=True)
+        for path, is_dir in paths:
+            m = "Pushing deletion because of parent dir move: (is_dir=%s) %r"
+            self.general_processor.log.info(m, is_dir, path)
+            if is_dir:
+                # same as the above remove
+                self.general_processor.rm_watch(path)
+
+                self.general_processor.eq_push('FS_DIR_DELETE', path=path)
+            else:
+                self.general_processor.eq_push('FS_FILE_DELETE', path=path)
+
+    def freeze_begin(self, path):
+        """Puts in hold all the events for this path."""
+        self.general_processor.freeze_begin(path)
+
+    def freeze_rollback(self):
+        """Unfreezes the frozen path, reseting to idle state."""
+        self.general_processor.freeze_rollback()
+
+    def freeze_commit(self, events):
+        """Unfreezes the frozen path, sending received events if not dirty.
+
+        If events for that path happened:
+            - return True
+        else:
+            - push the here received events, return False
+        """
+        return self.general_processor.freeze_commit(events)
+
+    @property
+    def mute_filter(self):
+        """Return the mute filter used by the processor."""
+        return self.general_processor.filter
+
+    @property
+    def frozen_path(self):
+        """Return the frozen path."""
+        return self.general_processor.frozen_path
+
+    @property
+    def log(self):
+        """Return the logger of the instance."""
+        return self.general_processor.log
+
+
+class FilesystemMonitor(object):
+    """Implementation that allows to receive events from the sustem."""
+
+    def __init__(self, eq, fs, ignore_config=None, timeout=1):
+        self.log = logging.getLogger('ubuntuone.SyncDaemon.FSMonitor')
+        self.log.setLevel(TRACE)
+        self.fs = fs
+        self.eq = eq
+        self._processor = NotifyProcessor(self, ignore_config)
+        self._factory = PyInotifyEventsFactory(self._processor)
+        self._protocol = None
+
+    @defer.inlineCallbacks
+    def _connect_to_daemon(self):
+        """Connect to the daemon so that we can receive events."""
+        description = 'unix:path=%s' % DAEMON_SOCKET
+        client = endpoints.clientFromString(reactor, description)
+        self._protocol = yield client.connect(self._factory)
+        # add the user with no paths
+        yield self._protocol.add_user([])
+
+    def add_to_mute_filter(self, event, **info):
+        """Add info to mute filter in the processor."""
+        self._processor.add_to_mute_filter(event, info)
+
+    def rm_from_mute_filter(self, event, **info):
+        """Remove info to mute filter in the processor."""
+        self._processor.rm_from_mute_filter(event, info)
+
+    def shutdown(self):
+        """Prepares the EQ to be closed."""
+        if self._protocol is not None:
+
+            def on_user_removed(data):
+                """We managed to remove the user."""
+                self._protocol.transport.loseConnection()
+                self._protocol = None
+                return True
+
+            def on_user_not_removed(reason):
+                """We did not manage to remove the user."""
+                return True
+
+            d = self._protocol.remove_user()
+            d.addCallback(on_user_removed)
+            d.addErrback(on_user_not_removed)
+            return d
+        return defer.succeed(True)
+
+    @defer.inlineCallbacks
+    def rm_watch(self, dirpath):
+        """Remove watch from a dir."""
+        # in mac os x we are only watching the parent watches, this is an
+        # important details because we will only send a real remove_path to the
+        # daemon if the path is the parent path else we will filter it in the
+        # factory level
+
+        if not dirpath[-1] == os.path.sep:
+            dirpath += os.path.sep
+
+        if dirpath not in self._factory.watched_paths:
+            # we are watching a parent path but we are not a root one
+            # therefore we are going to add it as an ignored path and
+            # return
+            self._factory.ignored_paths.append(dirpath)
+            defer.returnValue(None)
+
+        # if we got to this point we want to remove a root dir, this is an
+        # important detail to take care of. Connect if needed and tell the
+        # daemon to remove the path
+        if self._protocol is None:
+            # we have not yet connected, lets do it!
+            yield self._connect_to_daemon()
+        was_removed = yield self._protocol.remove_path(dirpath)
+        # only remove it if we really removed it
+        if was_removed:
+            self._factory.watched_paths.remove(dirpath)
+
+    @defer.inlineCallbacks
+    def add_watch(self, dirpath):
+        """Add watch to a dir."""
+        if not dirpath[-1] == os.path.sep:
+            dirpath = dirpath + os.path.sep
+
+        # if we are watching a parent dir we can just ensure that it is not ignored
+        if any(dirpath.startswith(watched_path) for watched_path in
+                self._factory.watched_paths):
+            if dirpath in self._factory.ignored_paths:
+                self._factory.ignored_paths.remove(dirpath)
+            defer.returnValue(True)
+
+        if dirpath in self._factory.ignored_paths:
+            self._factory.ignored_paths.remove(dirpath)
+            defer.returnValue(True)
+
+        if self._protocol is None:
+            # we have not yet connected, lets do it!
+            yield self._connect_to_daemon()
+
+        was_added = yield self._protocol.add_path(dirpath)
+        if was_added:
+            self._factory.watched_paths.append(dirpath)
+            defer.returnValue(True)
+
+    def add_watches_to_udf_ancestors(self, volume):
+        """Add a inotify watch to volume's ancestors if it's an UDF."""
+        # On Mac OS X we do no need to add watches to the ancestors because we
+        # will get the events from them with no problem.
+        return defer.succeed(True)
+
+    def is_frozen(self):
+        """Checks if there's something frozen."""
+        return self._processor.frozen_path is not None
+
+    def freeze_begin(self, path):
+        """Puts in hold all the events for this path."""
+        if self._processor.frozen_path is not None:
+            raise ValueError("There's something already frozen!")
+        self._processor.freeze_begin(path)
+
+    def freeze_rollback(self):
+        """Unfreezes the frozen path, reseting to idle state."""
+        if self._processor.frozen_path is None:
+            raise ValueError("Rolling back with nothing frozen!")
+        self._processor.freeze_rollback()
+
+    def freeze_commit(self, events):
+        """Unfreezes the frozen path, sending received events if not dirty.
+
+        If events for that path happened:
+            - return True
+        else:
+            - push the here received events, return False
+        """
+        if self._processor.frozen_path is None:
+            raise ValueError("Committing with nothing frozen!")
+
+        d = defer.execute(self._processor.freeze_commit, events)
+        return d

=== modified file 'ubuntuone/syncdaemon/filesystem_manager.py'
--- ubuntuone/syncdaemon/filesystem_manager.py	2012-04-09 20:07:05 +0000
+++ ubuntuone/syncdaemon/filesystem_manager.py	2012-07-19 14:21:30 +0000
@@ -744,6 +744,7 @@
                 mdobj = self.fs[mdid]
                 if mdobj["share_id"] == share_id:
                     all_mdobjs.append(_MDObject(**mdobj))
+
         return all_mdobjs
 
     def get_mdobjs_in_dir(self, base_path):