Merge lp:~dobey/ubuntuone-client/update-4-0 into lp:ubuntuone-client/stable-4-0

[dobey]

[Facundo Batista]

    - Decode the gettext string before compare (LP: #1011822).

[Manuel de la Peña]

    - Map the attrib event to an in_modify one (LP: #1043914).
    - Add the pattern to ignore the .DS_Store files coming from darwin (LP: #1034138).
    - Fix logging by escaping % (LP: #1043183).

[Mike McCracken]

    - Fix twisted IPC exceptions for syncdaemon status signals. (LP: #902189)
    - Change socket name for daemon to match reverse DNS naming convention. (LP: #1042785)
    - Add a guard so that async calls are perform after each other and sd is not started more than once (LP: #1043287).

[Rodney Dawes]

    - Skip test_zip_acquire_lock as it fails intermittently with twisted 12.x.
    - Allocate one extra slot for the strv to fit all the GSList entries.

[Roberto Alsina (ralsina)]

+1

review: Approve

[Ubuntu One Auto Pilot (otto-pilot)]

The attempt to merge lp:~dobey/ubuntuone-client/update-4-0 into lp:ubuntuone-client/stable-4-0 failed. Below is the output from the failed tests.

/usr/bin/gnome-autogen.sh
checking for autoconf >= 2.53...
  testing autoconf2.50... not found.
  testing autoconf... found 2.69
checking for automake >= 1.10...
  testing automake-1.12... not found.
  testing automake-1.11... found 1.11.5
checking for libtool >= 1.5...
  testing libtoolize... found 2.4.2
checking for intltool >= 0.30...
  testing intltoolize... found 0.50.2
checking for pkg-config >= 0.14.0...
  testing pkg-config... found 0.26
checking for gtk-doc >= 1.0...
  testing gtkdocize... found 1.18
Checking for required M4 macros...
Checking for forbidden M4 macros...
Processing ./configure.ac
Running libtoolize...
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
Running intltoolize...
Running gtkdocize...
Running aclocal-1.11...
Running autoconf...
Running autoheader...
Running automake-1.11...
Running ./configure --enable-gtk-doc --enable-debug ...
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking how to create a ustar tar archive... gnutar
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for library containing strerror... none required
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the sh...

[lp:~dobey/ubuntuone-client/update-4-0]

1260. By dobey on 2012-09-10

[Facundo Batista]

    - Decode the gettext string before compare (LP: #1011822).

[Manuel de la Peña]

    - Map the attrib event to an in_modify one (LP: #1043914).
    - Add the pattern to ignore the .DS_Store files coming from darwin (LP: #1034138).
    - Fix logging by escaping % (LP: #1043183).

[Mike McCracken]

    - Fix twisted IPC exceptions for syncdaemon status signals. (LP: #902189)
    - Change socket name for daemon to match reverse DNS naming convention. (LP: #1042785)
    - Add a guard so that async calls are perform after each other and sd is not started more than once (LP: #1043287).

[Rodney Dawes]

    - Skip test_zip_acquire_lock as it fails intermittently with twisted 12.x.
    - Allocate one extra slot for the strv to fit all the GSList entries.

=== modified file 'data/syncdaemon.conf'
--- data/syncdaemon.conf	2012-07-17 10:51:43 +0000
+++ data/syncdaemon.conf	2012-09-10 17:35:22 +0000
@@ -86,6 +86,7 @@
                  \A\.~lock\..*#\Z
                  \A\.goutputstream-.*\Z
                  \A.*-Spotlight\Z
+                 \A\.DS_Store\Z
  
 use_trash.default = True
 use_trash.parser = bool

=== modified file 'libsyncdaemon/syncdaemon-shares-interface.c'
--- libsyncdaemon/syncdaemon-shares-interface.c	2012-04-09 20:07:05 +0000
+++ libsyncdaemon/syncdaemon-shares-interface.c	2012-09-10 17:35:22 +0000
@@ -303,7 +303,7 @@
 	} else {
 		GSList *l;
 		gint i;
-		gchar **users_array = g_new0 (gchar *, g_slist_length (usernames));
+		gchar **users_array = g_new0 (gchar *, g_slist_length (usernames) + 1);
 
 		for (l = usernames, i = 0; l != NULL; l = l->next, i++)
 			users_array[i] = g_strdup (l->data);

=== modified file 'tests/platform/credentials/test_linux.py'
--- tests/platform/credentials/test_linux.py	2012-06-14 16:54:45 +0000
+++ tests/platform/credentials/test_linux.py	2012-09-10 17:35:22 +0000
@@ -384,7 +384,10 @@
         yield d
 
         self.assertEqual(self.sso_server._app_name, APP_NAME)
-        params = dict(UI_PARAMS)
+        # convert to unicode for the comparison, as the message is arriving
+        # here as bytes (bad gettext usage!)
+        params = dict((x, y.decode("utf8") if isinstance(y, str) else y)
+                      for x, y in UI_PARAMS.items())
         params.update(self.args)
         self.assertEqual(self.sso_server._args, params)
 
@@ -397,7 +400,10 @@
         yield d
 
         self.assertEqual(self.sso_server._app_name, APP_NAME)
-        params = dict(UI_PARAMS)
+        # convert to unicode for the comparison, as the message is arriving
+        # here as bytes (bad gettext usage!)
+        params = dict((x, y.decode("utf8") if isinstance(y, str) else y)
+                      for x, y in UI_PARAMS.items())
         params.update(self.args)
         self.assertEqual(self.sso_server._args, params)
 

=== modified file 'tests/platform/ipc/test_perspective_broker.py'
--- tests/platform/ipc/test_perspective_broker.py	2012-05-22 15:27:36 +0000
+++ tests/platform/ipc/test_perspective_broker.py	2012-09-10 17:35:22 +0000
@@ -27,6 +27,7 @@
 # version.  If you delete this exception statement from all source
 # files in the program, then also delete it here.
 """IPC tests on perspective broker."""
+import itertools
 import os
 
 from mocker import MockerTestCase, ANY
@@ -62,6 +63,7 @@
     Status,
     SyncDaemon,
 )
+from ubuntuone.platform.ipc import ipc_client
 from ubuntuone.platform.ipc.ipc_client import (
     signal,
     ConfigClient,
@@ -498,6 +500,32 @@
         for signal_name, args in self.signal_mapping:
             self.assert_remote_signal(signal_name, *args)
 
+    def test_remote_signal_calling(self):
+        """Call client functions directly, test for TypeErrors."""
+        if self.client_name is None:
+            return
+
+        def fake_remote_cb(*args, **kwargs):
+            return (args, kwargs)
+
+        client = getattr(self, self.client_name)
+        remote_client = self.client_class(FakeRemoteObject())
+        for signal_name, args in self.signal_mapping:
+
+            remote_client_func_name = client.signal_mapping[signal_name]
+            remote_cb_name = "%s_cb" % remote_client_func_name
+            setattr(remote_client, remote_cb_name, fake_remote_cb)
+            self.addCleanup(delattr, remote_client, remote_cb_name)
+
+            remote_client_func = getattr(remote_client, 
+                                         remote_client_func_name)
+
+            result = remote_client_func(*args)
+
+            # add empty kwargs because signal wrapper will add them:
+            expected = (args, {})
+            self.assertEqual(expected, result)
+
 
 class StatusTestCase(IPCTestCase):
     """Test the status client class."""
@@ -764,3 +792,110 @@
                    lambda _: defer.fail(ipc.AlreadyStartedError()))
         is_running = yield ipc.is_already_running()
         self.assertTrue(is_running, "Should be running by now.")
+
+
+class MultipleConnectionsTestCase(TestCase):
+    """Test the execution of the client with multiple connections."""
+
+    @defer.inlineCallbacks
+    def setUp(self):
+        """Set the different tests."""
+        yield super(MultipleConnectionsTestCase, self).setUp()
+        self.client_connect_d = defer.Deferred()  # called when we connected
+        self.client_root_obj_d = defer.Deferred()  # called when we got root
+        self.remote_obj_d = defer.Deferred()  # called when we got the remotes
+        self.register_to_signals_d = defer.Deferred()  # called with signals
+        self.called = []
+        self.num_clients = 4
+
+        @defer.inlineCallbacks
+        def fake_get_root_object():
+            """Fake getting a root objects."""
+            yield self.client_root_obj_d
+            self.called.append('getRootObject')
+            defer.returnValue(True)
+
+        @defer.inlineCallbacks
+        def fake_ipc_client_connect(factory):
+            """Fake ipc_client_connect."""
+            yield self.client_connect_d
+            self.called.append('ipc_client_connect')
+
+            # lets patch the factory getRootObjects function
+            self.patch(factory, 'getRootObject', fake_get_root_object)
+            defer.returnValue(True)
+
+        self.patch(ipc_client, 'ipc_client_connect', fake_ipc_client_connect)
+
+        @defer.inlineCallbacks
+        def fake_request_remote_objects(my_self, root):
+            """Fake request_remote_objects."""
+            yield self.remote_obj_d
+            self.called.append('_request_remote_objects')
+            defer.returnValue(True)
+
+        self.patch(ipc_client.UbuntuOneClient, '_request_remote_objects',
+                fake_request_remote_objects)
+
+        @defer.inlineCallbacks
+        def fake_register_to_signals(my_self):
+            """Fake registering to signals."""
+            yield self.register_to_signals_d
+            self.called.append('register_to_signals')
+            defer.returnValue(True)
+
+        self.patch(ipc_client.UbuntuOneClient, 'register_to_signals',
+                fake_register_to_signals)
+
+    def grouper(self, n, iterable, fillvalue=None):
+        "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
+        args = [iter(iterable)] * n
+        return itertools.izip_longest(*args, fillvalue=fillvalue)
+
+    @defer.inlineCallbacks
+    def test_multiple_connections(self):
+        """Test that we only perform connect once but all other are correct."""
+        deferreds = [self.client_connect_d, self.client_root_obj_d,
+                     self.remote_obj_d, self.register_to_signals_d]
+
+        # the order in which the calls are expected
+        expected_calls = ('ipc_client_connect','getRootObject',
+                '_request_remote_objects', 'register_to_signals')
+
+        clients = []
+        while len(clients) < self.num_clients:
+            clients.append(ipc_client.UbuntuOneClient())
+
+        connected_d = []
+
+        for num_steps in range(len(deferreds)):
+            # tell the first client to connect
+            connected_d.append(clients[0].connect())
+
+            # perform the number of connection steps so far
+            for index, step_d in enumerate(deferreds):
+                if index > num_steps:
+                    break
+                step_d.callback(True)
+
+            # call connect to all the other clients
+            for client in clients[1:]:
+                connected_d.append(client.connect())
+
+            # perform the rest of steps
+            for step_d in deferreds[num_steps + 1:]:
+                step_d.callback(True)
+
+            yield defer.gatherResults(connected_d)
+            # reset all the deferreds for the next round of testing
+            for index, d_name in enumerate(('client_connect_d',
+                    'client_root_obj_d', 'remote_obj_d',
+                    'register_to_signals_d')):
+                new_d = defer.Deferred()
+                setattr(self, d_name, new_d)
+                deferreds[index] = new_d
+
+            # assert that all connect calls have been done in the correct
+            # order
+            for calls in self.grouper(4, self.called, None):
+                self.assertEqual(expected_calls, calls)

=== modified file 'tests/syncdaemon/test_action_queue.py'
--- tests/syncdaemon/test_action_queue.py	2012-06-14 13:47:02 +0000
+++ tests/syncdaemon/test_action_queue.py	2012-09-10 17:35:22 +0000
@@ -62,6 +62,7 @@
     FakeUpload,
 )
 from ubuntuone.devtools import handlers
+from ubuntuone.devtools.testcases import skipTest
 from ubuntuone import logger, clientdefs
 from ubuntuone.platform import open_file, platform, path_exists
 from ubuntuone.storageprotocol import (
@@ -1022,6 +1023,7 @@
         yield self.zq.zip(upload, 'willbreak')
         self.assertEqual(len(called), 0)
 
+    @skipTest('Intermittently failing on twisted 12: LP: #1031815')
     @defer.inlineCallbacks
     def test_zip_acquire_lock(self):
         """Test that it acquires the lock."""

=== modified file 'ubuntuone/platform/filesystem_notifications/monitor/darwin/fsevents_client.py'
--- ubuntuone/platform/filesystem_notifications/monitor/darwin/fsevents_client.py	2012-08-22 15:03:24 +0000
+++ ubuntuone/platform/filesystem_notifications/monitor/darwin/fsevents_client.py	2012-09-10 17:35:22 +0000
@@ -46,6 +46,7 @@
 ACTIONS = {
     fsevents.IN_CREATE: IN_CREATE,
     fsevents.IN_DELETE: IN_DELETE,
+    fsevents.IN_ATTRIB: IN_MODIFY,
     fsevents.IN_MODIFY: IN_MODIFY,
     fsevents.IN_MOVED_FROM: IN_MOVED_FROM,
     fsevents.IN_MOVED_TO: IN_MOVED_TO,

=== modified file 'ubuntuone/platform/filesystem_notifications/monitor/darwin/fsevents_daemon.py'
--- ubuntuone/platform/filesystem_notifications/monitor/darwin/fsevents_daemon.py	2012-07-23 12:26:42 +0000
+++ ubuntuone/platform/filesystem_notifications/monitor/darwin/fsevents_daemon.py	2012-09-10 17:35:22 +0000
@@ -97,7 +97,7 @@
 
 # TODO: This should be in fseventsd to be imported!
 # Path to the socket used by the daemon
-DAEMON_SOCKET = '/var/run/ubuntuone_fsevents_daemon'
+DAEMON_SOCKET = '/var/run/com.ubuntu.one.fsevents.sock'
 
 
 class DescriptionFactory(object):

=== modified file 'ubuntuone/platform/ipc/ipc_client.py'
--- ubuntuone/platform/ipc/ipc_client.py	2012-08-27 15:22:13 +0000
+++ ubuntuone/platform/ipc/ipc_client.py	2012-09-10 17:35:22 +0000
@@ -234,11 +234,11 @@
         """Emit DownloadStarted."""
 
     @signal
-    def on_download_file_progress(self, download, **info):
+    def on_download_file_progress(self, download, info):
         """Emit DownloadFileProgress."""
 
     @signal
-    def on_download_finished(self, download, **info):
+    def on_download_finished(self, download, info):
         """Emit DownloadFinished."""
 
     @signal
@@ -246,11 +246,11 @@
         """Emit UploadStarted."""
 
     @signal
-    def on_upload_file_progress(self, upload, **info):
+    def on_upload_file_progress(self, upload, info):
         """Emit UploadFileProgress."""
 
     @signal
-    def on_upload_finished(self, upload, *info):
+    def on_upload_finished(self, upload, info):
         """Emit UploadFinished."""
 
     @signal
@@ -261,6 +261,14 @@
     def on_metaqueue_changed(self):
         """Emit MetaQueueChanged."""
 
+    @signal
+    def on_request_queue_added(self, op_name, op_id, data):
+        """Emit RequestQueueAdded."""
+
+    @signal
+    def on_request_queue_removed(self, op_name, op_id, data):
+        """Emit RequestQueueRemoved."""
+
 
 class EventsClient(RemoteClient, Referenceable):
     """Client use to access the status api."""
@@ -719,6 +727,8 @@
 class UbuntuOneClient(object):
     """Root object that provides access to all the remote objects."""
 
+    connection_lock = defer.DeferredLock()
+
     def __init__(self):
         """Create a new instance."""
         self.status = None
@@ -765,17 +775,21 @@
     def connect(self):
         """Connect to the syncdaemon service."""
         # pylint: disable=W0702
+        yield self.connection_lock.acquire()
         try:
-            # connect to the remote objects
-            self.factory = PBClientFactory()
-            self.client = yield ipc_client_connect(self.factory)
-            root = yield self.factory.getRootObject()
-            yield self._request_remote_objects(root)
-            yield self.register_to_signals()
+            if self.client is None:
+                # connect to the remote objects
+                self.factory = PBClientFactory()
+                self.client = yield ipc_client_connect(self.factory)
+                root = yield self.factory.getRootObject()
+                yield self._request_remote_objects(root)
+                yield self.register_to_signals()
             defer.returnValue(self)
         except Exception, e:
             raise SyncDaemonClientConnectionError(
                             'Could not connect to the syncdaemon ipc.', e)
+        finally:
+            self.connection_lock.release()
         # pylint: disable=W0702
 
     @defer.inlineCallbacks

=== modified file 'ubuntuone/platform/ipc/perspective_broker.py'
--- ubuntuone/platform/ipc/perspective_broker.py	2012-08-16 12:00:12 +0000
+++ ubuntuone/platform/ipc/perspective_broker.py	2012-09-10 17:35:22 +0000
@@ -305,7 +305,7 @@
         """Fire a signal to notify about a download progress."""
 
     @signal
-    def DownloadFinished(self, path, *info):
+    def DownloadFinished(self, path, info):
         """Fire a signal to notify that a download has finished."""
 
     @signal
@@ -317,7 +317,7 @@
         """Fire a signal to notify about an upload progress."""
 
     @signal
-    def UploadFinished(self, path, *info):
+    def UploadFinished(self, path, info):
         """Fire a signal to notify that an upload has finished."""
 
     @signal

=== modified file 'ubuntuone/syncdaemon/logger.py'
--- ubuntuone/syncdaemon/logger.py	2012-06-21 18:58:50 +0000
+++ ubuntuone/syncdaemon/logger.py	2012-09-10 17:35:22 +0000
@@ -75,6 +75,7 @@
 
         desc = "%-28s share:%-40r node:%-40r %s(%s) " % (_method, _share,
                                                          _uid, _method, args)
+        desc = desc.replace('%', '%%')
         self.zipped_desc = zlib.compress(desc, 9)
         self.logger = _logger
 

=== modified file 'ubuntuone/syncdaemon/sync.py'
--- ubuntuone/syncdaemon/sync.py	2012-06-13 21:31:24 +0000
+++ ubuntuone/syncdaemon/sync.py	2012-09-10 17:35:22 +0000
@@ -271,7 +271,7 @@
                 path = self.key.safe_get('path')
             extra = dict(message=message,
                          mdid=self.key.safe_get("mdid"),
-                         path=path,
+                         path=path.replace('%', '%%'),   # escape %
                          share_id=self.key.safe_get("share_id") or 'root',
                          node_id=self.key.safe_get("node_id"),
                          hasmd=self.key.has_metadata(),