Merge lp:~mandel/ubuntuone-windows-installer/add_python_u1sync into lp:ubuntuone-windows-installer/beta

[Manuel de la Peña]

Modified u1sync to work on windows and added its packaging with py2exe. This provides a temp solution until we refactor certain parts on the syncdaemon.

[Vincenzo Di Somma (vds)]

Tests Ok, build Ok!

review: Approve

[Rick McBride (rmcbride)]

review: Approve

=== modified file 'README.txt'
--- README.txt	2010-08-19 11:13:02 +0000
+++ README.txt	2010-08-27 14:56:43 +0000
@@ -7,8 +7,21 @@
 	* UbuntuOne Windows Service: Windows service that allows to start, stop, pause and resume the sync daemon.
 	* UbuntuOne Windows Service Brodcaster: Provides a WCF service hosted in a windows service that allows to .Net languages
 	  to communicate with the sync daemon and interact with it.
-	  
-2. Build
+
+2. Enviroment setup
+
+The ubuntuone windows port solution provides a port of the python code that is used on Linux to perform the u1 sync operations. Due to the fact that 
+we did not want user to have to be hunting down the python runtime plus the differen modules that have been used. To simplify the live of the 
+windows users we have opted to use py2exe to create an executable that will carry all the different python dependencies of the code. As you
+may already know py2exe is not perfect and does not support egg files. In order to make sure that you easy_install does extra the eggs files after
+the packages are installed please use the following command:
+
+easy_install -Z %package_name%
+
+In order to be able to build the solution you will need to have python win32, the win32 python extensions and the ubuntu one storage protocol in your system.:
+
+
+3. Build
 
 In order to simplify the buil as much as possible, all the required tools for the compilation of the project are provided in the soruce tree. The 
 compilation uses a nant project that allows to run the following targets:
 
=== modified file 'main.build'
--- main.build	2010-08-24 18:32:48 +0000
+++ main.build	2010-08-27 14:56:43 +0000
@@ -176,10 +176,19 @@
               program="nunit-console.exe"
               commandline="UbuntuOneClient.Tests.dll /xml=../../../../test-results/UbuntuOneClient.Tests-Result.xml" />
 	</target>
-	
+	<target name="package_python"
+            description="Creates the exe binary that embeds the python libs that will be used to perform the sync operation in the windows platform">
+        
+        <exec basedir="${python_path}"
+              managed="true"
+              workingdir="src"
+              program="python.exe"
+              commandline="setup.py py2exe" />
+    </target>
+    
 	<target name="installer"
 	        description="Compiles the solution and create a merge installer that allows to install the solution and other related apps."
-			depends="tests">
+			depends="tests, package_python">
 		
 		<mkdir dir="${build_results}" />
 		
 
=== added file 'src/setup.py'
--- src/setup.py	1970-01-01 00:00:00 +0000
+++ src/setup.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,56 @@
+#!/usr/bin/env python
+# Copyright (C) 2010 Canonical - All Rights Reserved
+
+""" """
+import sys
+
+# ModuleFinder can't handle runtime changes to __path__, but win32com uses them
+try:
+    # py2exe 0.6.4 introduced a replacement modulefinder.
+    # This means we have to add package paths there, not to the built-in
+    # one.  If this new modulefinder gets integrated into Python, then
+    # we might be able to revert this some day.
+    # if this doesn't work, try import modulefinder
+    try:
+        import py2exe.mf as modulefinder
+    except ImportError:
+        import modulefinder
+    import win32com
+    for p in win32com.__path__[1:]:
+        modulefinder.AddPackagePath("win32com", p)
+    for extra in ["win32com.shell"]: #,"win32com.mapi"
+        __import__(extra)
+        m = sys.modules[extra]
+        for p in m.__path__[1:]:
+            modulefinder.AddPackagePath(extra, p)
+except ImportError:
+    # no build path setup, no worries.
+    pass
+
+from distutils.core import setup
+import py2exe
+
+if __name__ == '__main__':
+
+    setup(
+        options = {
+            "py2exe": {
+                "compressed": 1,
+                "optimize": 2}},
+        name='u1sync',
+        version='0.0.1',
+        author = "Canonical Online Services Hackers",
+        description="""u1sync is a utility of Ubuntu One.
+        Ubuntu One is a suite of on-line
+        services. This package contains the a synchronization client for the
+        Ubuntu One file sharing service.""",
+        license='GPLv3',
+        console=['u1sync\\main.py'],
+        requires=[
+            'python (>= 2.5)',
+            'oauth',
+            'twisted.names',
+            'twisted.web',
+            'ubuntuone.storageprotocol (>= 1.3.0)',
+            ],
+        )
 
=== added directory 'src/u1sync'
=== added file 'src/u1sync/__init__.py'
--- src/u1sync/__init__.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/__init__.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,14 @@
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""The guts of the u1sync tool."""
 
=== added file 'src/u1sync/client.py'
--- src/u1sync/client.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/client.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,754 @@
+# ubuntuone.u1sync.client
+#
+# Client/protocol end of u1sync
+#
+# Author: Lucio Torre <[e-mail]>
+# Author: Tim Cole <[e-mail]>
+#
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Pretty API for protocol client."""
+
+from __future__ import with_statement
+
+import os
+import sys
+import shutil
+from Queue import Queue
+from threading import Lock
+import zlib
+import urlparse
+import ConfigParser
+from cStringIO import StringIO
+from twisted.internet import reactor, defer
+from twisted.internet.defer import inlineCallbacks, returnValue
+from ubuntuone.logger import LOGFOLDER
+from ubuntuone.storageprotocol.content_hash import crc32
+from ubuntuone.storageprotocol.context import get_ssl_context
+from u1sync.genericmerge import MergeNode
+from u1sync.utils import should_sync
+
+CONSUMER_KEY = "ubuntuone"
+
+from oauth.oauth import OAuthConsumer
+from ubuntuone.storageprotocol.client import (
+    StorageClientFactory, StorageClient)
+from ubuntuone.storageprotocol import request, volumes
+from ubuntuone.storageprotocol.dircontent_pb2 import \
+    DirectoryContent, DIRECTORY
+import uuid
+import logging
+from logging.handlers import RotatingFileHandler
+import time
+
+def share_str(share_uuid):
+    """Converts a share UUID to a form the protocol likes."""
+    return str(share_uuid) if share_uuid is not None else request.ROOT
+
+LOGFILENAME = os.path.join(LOGFOLDER, 'u1sync.log')
+u1_logger = logging.getLogger("u1sync.timing.log")
+handler = RotatingFileHandler(LOGFILENAME)
+u1_logger.addHandler(handler)
+
+def log_timing(func):
+    def wrapper(*arg, **kwargs):
+        start = time.time()
+        ent = func(*arg, **kwargs)
+        stop = time.time()
+        u1_logger.debug('for %s %0.5f ms elapsed' % (func.func_name, \
+                    (stop-start)*1000.0))
+        return ent
+    return wrapper
+
+
+class ForcedShutdown(Exception):
+    """Client shutdown forced."""
+
+
+class Waiter(object):
+    """Wait object for blocking waits."""
+
+    def __init__(self):
+        """Initializes the wait object."""
+        self.queue = Queue()
+
+    def wake(self, result):
+        """Wakes the waiter with a result."""
+        self.queue.put((result, None))
+
+    def wakeAndRaise(self, exc_info):
+        """Wakes the waiter, raising the given exception in it."""
+        self.queue.put((None, exc_info))
+
+    def wakeWithResult(self, func, *args, **kw):
+        """Wakes the waiter with the result of the given function."""
+        try:
+            result = func(*args, **kw)
+        except Exception:
+            self.wakeAndRaise(sys.exc_info())
+        else:
+            self.wake(result)
+
+    def wait(self):
+        """Waits for wakeup."""
+        (result, exc_info) = self.queue.get()
+        if exc_info:
+            try:
+                raise exc_info[0], exc_info[1], exc_info[2]
+            finally:
+                exc_info = None
+        else:
+            return result
+
+
+class SyncStorageClient(StorageClient):
+    """Simple client that calls a callback on connection."""
+
+    @log_timing
+    def connectionMade(self):
+        """Setup and call callback."""
+        StorageClient.connectionMade(self)
+        if self.factory.current_protocol not in (None, self):
+            self.factory.current_protocol.transport.loseConnection()
+        self.factory.current_protocol = self
+        self.factory.observer.connected()
+
+    @log_timing
+    def connectionLost(self, reason=None):
+        """Callback for established connection lost"""
+        if self.factory.current_protocol is self:
+            self.factory.current_protocol = None
+            self.factory.observer.disconnected(reason)
+
+
+class SyncClientFactory(StorageClientFactory):
+    """A cmd protocol factory."""
+    # no init: pylint: disable-msg=W0232
+
+    protocol = SyncStorageClient
+
+    @log_timing
+    def __init__(self, observer):
+        """Create the factory"""
+        self.observer = observer
+        self.current_protocol = None
+
+    @log_timing
+    def clientConnectionFailed(self, connector, reason):
+        """We failed at connecting."""
+        self.current_protocol = None
+        self.observer.connection_failed(reason)
+
+
+class UnsupportedOperationError(Exception):
+    """The operation is unsupported by the protocol version."""
+
+
+class ConnectionError(Exception):
+    """A connection error."""
+
+
+class AuthenticationError(Exception):
+    """An authentication error."""
+
+
+class NoSuchShareError(Exception):
+    """Error when there is no such share available."""
+
+
+class CapabilitiesError(Exception):
+    """A capabilities set/query related error."""
+
+class Client(object):
+    """U1 storage client facade."""
+    required_caps = frozenset(["no-content", "fix462230"])
+
+    def __init__(self, realm, reactor=reactor):
+        """Create the instance."""
+
+        self.reactor = reactor
+        self.factory = SyncClientFactory(self)
+
+        self._status_lock = Lock()
+        self._status = "disconnected"
+        self._status_reason = None
+        self._status_waiting = []
+        self._active_waiters = set()
+        self.consumer_key = CONSUMER_KEY
+        self.consumer_secret = "hammertime"
+        
+    def force_shutdown(self):
+        """Forces the client to shut itself down."""
+        with self._status_lock:
+            self._status = "forced_shutdown"
+            self._reason = None
+            for waiter in self._active_waiters:
+                waiter.wakeAndRaise((ForcedShutdown("Forced shutdown"),
+                                     None, None))
+            self._active_waiters.clear()
+
+    def _get_waiter_locked(self):
+        """Gets a wait object for blocking waits.  Should be called with the
+        status lock held.
+        """
+        waiter = Waiter()
+        if self._status == "forced_shutdown":
+            raise ForcedShutdown("Forced shutdown")
+        self._active_waiters.add(waiter)
+        return waiter
+
+    def _get_waiter(self):
+        """Get a wait object for blocking waits.  Acquires the status lock."""
+        with self._status_lock:
+            return self._get_waiter_locked()
+
+    def _wait(self, waiter):
+        """Waits for the waiter."""
+        try:
+            return waiter.wait()
+        finally:
+            with self._status_lock:
+                if waiter in self._active_waiters:
+                    self._active_waiters.remove(waiter)
+
+    @log_timing
+    def _change_status(self, status, reason=None):
+        """Changes the client status.  Usually called from the reactor
+        thread.
+
+        """
+        with self._status_lock:
+            if self._status == "forced_shutdown":
+                return
+            self._status = status
+            self._status_reason = reason
+            waiting = self._status_waiting
+            self._status_waiting = []
+        for waiter in waiting:
+            waiter.wake((status, reason))
+
+    @log_timing
+    def _await_status_not(self, *ignore_statuses):
+        """Blocks until the client status changes, returning the new status.
+        Should never be called from the reactor thread.
+
+        """
+        with self._status_lock:
+            status = self._status
+            reason = self._status_reason
+            while status in ignore_statuses:
+                waiter = self._get_waiter_locked()
+                self._status_waiting.append(waiter)
+                self._status_lock.release()
+                try:
+                    status, reason = self._wait(waiter)
+                finally:
+                    self._status_lock.acquire()
+            if status == "forced_shutdown":
+                raise ForcedShutdown("Forced shutdown.")
+            return (status, reason)
+
+    def connection_failed(self, reason):
+        """Notification that connection failed."""
+        self._change_status("disconnected", reason)
+
+    def connected(self):
+        """Notification that connection succeeded."""
+        self._change_status("connected")
+
+    def disconnected(self, reason):
+        """Notification that we were disconnected."""
+        self._change_status("disconnected", reason)
+
+    def defer_from_thread(self, function, *args, **kwargs):
+        """Do twisted defer magic to get results and show exceptions."""
+        waiter = self._get_waiter()
+        @log_timing
+        def runner():
+            """inner."""
+            # we do want to catch all
+            # no init: pylint: disable-msg=W0703
+            try:
+                d = function(*args, **kwargs)
+                if isinstance(d, defer.Deferred):
+                    d.addCallbacks(lambda r: waiter.wake((r, None, None)),
+                                   lambda f: waiter.wake((None, None, f)))
+                else:
+                    waiter.wake((d, None, None))
+            except Exception:
+                waiter.wake((None, sys.exc_info(), None))
+
+        self.reactor.callFromThread(runner)
+        result, exc_info, failure = self._wait(waiter)
+        if exc_info:
+            try:
+                raise exc_info[0], exc_info[1], exc_info[2]
+            finally:
+                exc_info = None
+        elif failure:
+            failure.raiseException()
+        else:
+            return result
+
+    @log_timing
+    def connect(self, host, port):
+        """Connect to host/port."""
+        def _connect():
+            """Deferred part."""
+            self.reactor.connectTCP(host, port, self.factory)
+        self._connect_inner(_connect)
+
+    @log_timing
+    def connect_ssl(self, host, port, no_verify):
+        """Connect to host/port using ssl."""
+        def _connect():
+            """deferred part."""
+            ctx = get_ssl_context(no_verify)
+            self.reactor.connectSSL(host, port, self.factory, ctx)
+        self._connect_inner(_connect)
+
+    @log_timing
+    def _connect_inner(self, _connect):
+        """Helper function for connecting."""
+        self._change_status("connecting")
+        self.reactor.callFromThread(_connect)
+        status, reason = self._await_status_not("connecting")
+        if status != "connected":
+            raise ConnectionError(reason.value)
+
+    @log_timing
+    def disconnect(self):
+        """Disconnect."""
+        if self.factory.current_protocol is not None:
+            self.reactor.callFromThread(
+                self.factory.current_protocol.transport.loseConnection)
+        self._await_status_not("connecting", "connected", "authenticated")
+
+    @log_timing
+    def set_capabilities(self):
+        """Set the capabilities with the server"""
+
+        client = self.factory.current_protocol
+        @log_timing
+        def set_caps_callback(req):
+            "Caps query succeeded"
+            if not req.accepted:
+                de = defer.fail("The server denied setting %s capabilities" % \
+                                req.caps)
+                return de
+
+        @log_timing
+        def query_caps_callback(req):
+            "Caps query succeeded"
+            if req.accepted:
+                set_d = client.set_caps(self.required_caps)
+                set_d.addCallback(set_caps_callback)
+                return set_d
+            else:
+                # the server don't have the requested capabilities.
+                # return a failure for now, in the future we might want
+                # to reconnect to another server
+                de = defer.fail("The server don't have the requested"
+                                " capabilities: %s" % str(req.caps))
+                return de
+
+        @log_timing
+        def _wrapped_set_capabilities():
+            """Wrapped set_capabilities """
+            d = client.query_caps(self.required_caps)
+            d.addCallback(query_caps_callback)
+            return d
+
+        try:
+            self.defer_from_thread(_wrapped_set_capabilities)
+        except request.StorageProtocolError, e:
+            raise CapabilitiesError(e)
+
+    @log_timing
+    def get_root_info(self, volume_uuid):
+        """Returns the UUID of the applicable share root."""
+        if volume_uuid is None:
+            _get_root = self.factory.current_protocol.get_root
+            root = self.defer_from_thread(_get_root)
+            return (uuid.UUID(root), True)
+        else:
+            str_volume_uuid = str(volume_uuid)
+            volume = self._match_volume(lambda v: \
+                                       str(v.volume_id) == str_volume_uuid)
+            if isinstance(volume, volumes.ShareVolume):
+                modify = volume.access_level == "Modify"
+            if isinstance(volume, volumes.UDFVolume):
+                modify = True
+            return (uuid.UUID(str(volume.node_id)), modify)
+
+    @log_timing
+    def resolve_path(self, share_uuid, root_uuid, path):
+        """Resolve path relative to the given root node."""
+
+        @inlineCallbacks
+        def _resolve_worker():
+            """Path resolution worker."""
+            node_uuid = root_uuid
+            local_path = path.strip('/')
+
+            while local_path != '':
+                local_path, name = os.path.split(local_path)
+                hashes = yield self._get_node_hashes(share_uuid, [root_uuid])
+                content_hash = hashes.get(root_uuid, None)
+                if content_hash is None:
+                    raise KeyError, "Content hash not available"
+                entries = yield self._get_raw_dir_entries(share_uuid,
+                                                          root_uuid,
+                                                          content_hash)
+                match_name = name.decode('utf-8')
+                match = None
+                for entry in entries:
+                    if match_name == entry.name:
+                        match = entry
+                        break
+
+                if match is None:
+                    raise KeyError, "Path not found"
+
+                node_uuid = uuid.UUID(match.node)
+
+            returnValue(node_uuid)
+
+        return self.defer_from_thread(_resolve_worker)
+
+    @log_timing
+    def oauth_from_token(self, token):
+        """Perform OAuth authorisation using an existing token."""
+
+        consumer = OAuthConsumer(self.consumer_key, self.consumer_secret)
+
+        def _auth_successful(value):
+            """Callback for successful auth.  Changes status to
+            authenticated."""
+            self._change_status("authenticated")
+            return value
+
+        def _auth_failed(value):
+            """Callback for failed auth.  Disconnects."""
+            self.factory.current_protocol.transport.loseConnection()
+            return value
+
+        def _wrapped_authenticate():
+            """Wrapped authenticate."""
+            d = self.factory.current_protocol.oauth_authenticate(consumer,
+                                                                 token)
+            d.addCallbacks(_auth_successful, _auth_failed)
+            return d
+
+        try:
+            self.defer_from_thread(_wrapped_authenticate)
+        except request.StorageProtocolError, e:
+            raise AuthenticationError(e)
+        status, reason = self._await_status_not("connected")
+        if status != "authenticated":
+            raise AuthenticationError(reason.value)
+            
+    @log_timing
+    def find_volume(self, volume_spec):
+        """Finds a share matching the given UUID.  Looks at both share UUIDs
+        and root node UUIDs."""
+        volume = self._match_volume(lambda s: \
+                                    str(s.volume_id) == volume_spec or \
+                                    str(s.node_id) == volume_spec)
+        return uuid.UUID(str(volume.volume_id))
+
+    @log_timing
+    def _match_volume(self, predicate):
+        """Finds a volume matching the given predicate."""
+        _list_shares = self.factory.current_protocol.list_volumes
+        r = self.defer_from_thread(_list_shares)
+        for volume in r.volumes:
+            if predicate(volume):
+                return volume
+        raise NoSuchShareError()
+
+    @log_timing
+    def build_tree(self, share_uuid, root_uuid):
+        """Builds and returns a tree representing the metadata for the given
+        subtree in the given share.
+
+        @param share_uuid: the share UUID or None for the user's volume
+        @param root_uuid: the root UUID of the subtree (must be a directory)
+        @return: a MergeNode tree
+
+        """
+        root = MergeNode(node_type=DIRECTORY, uuid=root_uuid)
+
+        @log_timing
+        @inlineCallbacks
+        def _get_root_content_hash():
+            """Obtain the content hash for the root node."""
+            result = yield self._get_node_hashes(share_uuid, [root_uuid])
+            returnValue(result.get(root_uuid, None))
+
+        root.content_hash = self.defer_from_thread(_get_root_content_hash)
+        if root.content_hash is None:
+            raise ValueError("No content available for node %s" % root_uuid)
+
+        @log_timing
+        @inlineCallbacks
+        def _get_children(parent_uuid, parent_content_hash):
+            """Obtain a sequence of MergeNodes corresponding to a node's
+            immediate children.
+
+            """
+            entries = yield self._get_raw_dir_entries(share_uuid,
+                                                      parent_uuid,
+                                                      parent_content_hash)
+            children = {}
+            for entry in entries:
+                if should_sync(entry.name):
+                    child = MergeNode(node_type=entry.node_type,
+                                      uuid=uuid.UUID(entry.node))
+                    children[entry.name] = child
+
+            child_uuids = [child.uuid for child in children.itervalues()]
+            content_hashes = yield self._get_node_hashes(share_uuid,
+                                                         child_uuids)
+            for child in children.itervalues():
+                child.content_hash = content_hashes.get(child.uuid, None)
+
+            returnValue(children)
+
+        need_children = [root]
+        while need_children:
+            node = need_children.pop()
+            if node.content_hash is not None:
+                children = self.defer_from_thread(_get_children, node.uuid,
+                                                  node.content_hash)
+                node.children = children
+                for child in children.itervalues():
+                    if child.node_type == DIRECTORY:
+                        need_children.append(child)
+
+        return root
+
+    @log_timing
+    def _get_raw_dir_entries(self, share_uuid, node_uuid, content_hash):
+        """Gets raw dir entries for the given directory."""
+        d = self.factory.current_protocol.get_content(share_str(share_uuid),
+                                                      str(node_uuid),
+                                                      content_hash)
+        d.addCallback(lambda c: zlib.decompress(c.data))
+
+        def _parse_content(raw_content):
+            """Parses directory content into a list of entry objects."""
+            unserialized_content = DirectoryContent()
+            unserialized_content.ParseFromString(raw_content)
+            return list(unserialized_content.entries)
+
+        d.addCallback(_parse_content)
+        return d
+
+    @log_timing
+    def download_string(self, share_uuid, node_uuid, content_hash):
+        """Reads a file from the server into a string."""
+        output = StringIO()
+        self._download_inner(share_uuid=share_uuid, node_uuid=node_uuid,
+                             content_hash=content_hash, output=output)
+        return output.getValue()
+
+    @log_timing
+    def download_file(self, share_uuid, node_uuid, content_hash, filename):
+        """Downloads a file from the server."""
+        # file names have to be quoted to make sure that no issues occur when
+        # spaces exist
+        partial_filename = "%s.u1partial" % filename
+        output = open(partial_filename, "w")
+
+        @log_timing
+        def rename_file():
+            """Renames the temporary file to the final name."""
+            output.close()
+            print "Finished downloading %s" % filename
+            os.rename(partial_filename, filename)
+
+        @log_timing
+        def delete_file():
+            """Deletes the temporary file."""
+            output.close()
+            os.unlink(partial_filename)
+
+        self._download_inner(share_uuid=share_uuid, node_uuid=node_uuid,
+                             content_hash=content_hash, output=output,
+                             on_success=rename_file, on_failure=delete_file)
+
+    @log_timing
+    def _download_inner(self, share_uuid, node_uuid, content_hash, output,
+                        on_success=lambda: None, on_failure=lambda: None):
+        """Helper function for content downloads."""
+        dec = zlib.decompressobj()
+
+        @log_timing
+        def write_data(data):
+            """Helper which writes data to the output file."""
+            uncompressed_data = dec.decompress(data)
+            output.write(uncompressed_data)
+
+        @log_timing
+        def finish_download(value):
+            """Helper which finishes the download."""
+            uncompressed_data = dec.flush()
+            output.write(uncompressed_data)
+            on_success()
+            return value
+
+        @log_timing
+        def abort_download(value):
+            """Helper which aborts the download."""
+            on_failure()
+            return value
+
+        @log_timing
+        def _download():
+            """Async helper."""
+            _get_content = self.factory.current_protocol.get_content
+            d = _get_content(share_str(share_uuid), str(node_uuid),
+                             content_hash, callback=write_data)
+            d.addCallbacks(finish_download, abort_download)
+            return d
+
+        self.defer_from_thread(_download)
+
+    @log_timing
+    def create_directory(self, share_uuid, parent_uuid, name):
+        """Creates a directory on the server."""
+        r = self.defer_from_thread(self.factory.current_protocol.make_dir,
+                                   share_str(share_uuid), str(parent_uuid),
+                                   name)
+        return uuid.UUID(r.new_id)
+
+    @log_timing
+    def create_file(self, share_uuid, parent_uuid, name):
+        """Creates a file on the server."""
+        r = self.defer_from_thread(self.factory.current_protocol.make_file,
+                                   share_str(share_uuid), str(parent_uuid),
+                                   name)
+        return uuid.UUID(r.new_id)
+
+    @log_timing
+    def create_symlink(self, share_uuid, parent_uuid, name, target):
+        """Creates a symlink on the server."""
+        raise UnsupportedOperationError("Protocol does not support symlinks")
+
+    @log_timing
+    def upload_string(self, share_uuid, node_uuid, old_content_hash,
+                      content_hash, content):
+        """Uploads a string to the server as file content."""
+        crc = crc32(content, 0)
+        compressed_content = zlib.compress(content, 9)
+        compressed = StringIO(compressed_content)
+        self.defer_from_thread(self.factory.current_protocol.put_content,
+                               share_str(share_uuid), str(node_uuid),
+                               old_content_hash, content_hash,
+                               crc, len(content), len(compressed_content),
+                               compressed)
+
+    @log_timing
+    def upload_file(self, share_uuid, node_uuid, old_content_hash,
+                    content_hash, filename):
+        """Uploads a file to the server."""
+        parent_dir = os.path.split(filename)[0]
+        unique_filename = os.path.join(parent_dir, "." + str(uuid.uuid4()))
+
+
+        class StagingFile(object):
+            """An object which tracks data being compressed for staging."""
+            def __init__(self, stream):
+                """Initialize a compression object."""
+                self.crc32 = 0
+                self.enc = zlib.compressobj(9)
+                self.size = 0
+                self.compressed_size = 0
+                self.stream = stream
+
+            def write(self, bytes):
+                """Compress bytes, keeping track of length and crc32."""
+                self.size += len(bytes)
+                self.crc32 = crc32(bytes, self.crc32)
+                compressed_bytes = self.enc.compress(bytes)
+                self.compressed_size += len(compressed_bytes)
+                self.stream.write(compressed_bytes)
+
+            def finish(self):
+                """Finish staging compressed data."""
+                compressed_bytes = self.enc.flush()
+                self.compressed_size += len(compressed_bytes)
+                self.stream.write(compressed_bytes)
+
+        with open(unique_filename, "w+") as compressed:
+            os.unlink(unique_filename)
+            with open(filename, "r") as original:
+                staging = StagingFile(compressed)
+                shutil.copyfileobj(original, staging)
+            staging.finish()
+            compressed.seek(0)
+            self.defer_from_thread(self.factory.current_protocol.put_content,
+                                   share_str(share_uuid), str(node_uuid),
+                                   old_content_hash, content_hash,
+                                   staging.crc32,
+                                   staging.size, staging.compressed_size,
+                                   compressed)
+
+    @log_timing
+    def move(self, share_uuid, parent_uuid, name, node_uuid):
+        """Moves a file on the server."""
+        self.defer_from_thread(self.factory.current_protocol.move,
+                               share_str(share_uuid), str(node_uuid),
+                               str(parent_uuid), name)
+
+    @log_timing
+    def unlink(self, share_uuid, node_uuid):
+        """Unlinks a file on the server."""
+        self.defer_from_thread(self.factory.current_protocol.unlink,
+                               share_str(share_uuid), str(node_uuid))
+
+    @log_timing
+    def _get_node_hashes(self, share_uuid, node_uuids):
+        """Fetches hashes for the given nodes."""
+        share = share_str(share_uuid)
+        queries = [(share, str(node_uuid), request.UNKNOWN_HASH) \
+                   for node_uuid in node_uuids]
+        d = self.factory.current_protocol.query(queries)
+
+        @log_timing
+        def _collect_hashes(multi_result):
+            """Accumulate hashes from query replies."""
+            hashes = {}
+            for (success, value) in multi_result:
+                if success:
+                    for node_state in value.response:
+                        node_uuid = uuid.UUID(node_state.node)
+                        hashes[node_uuid] = node_state.hash
+            return hashes
+
+        d.addCallback(_collect_hashes)
+        return d
+
+    @log_timing
+    def get_incoming_shares(self):
+        """Returns a list of incoming shares as (name, uuid, accepted)
+        tuples.
+
+        """
+        _list_shares = self.factory.current_protocol.list_shares
+        r = self.defer_from_thread(_list_shares)
+        return [(s.name, s.id, s.other_visible_name,
+                 s.accepted, s.access_level) \
+                for s in r.shares if s.direction == "to_me"]
 
=== added file 'src/u1sync/constants.py'
--- src/u1sync/constants.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/constants.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,30 @@
+# ubuntuone.u1sync.constants
+#
+# u1sync constants
+#
+# Author: Tim Cole <[e-mail]>
+#
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Assorted constant definitions which don't fit anywhere else."""
+
+import re
+
+# the name of the directory u1sync uses to keep metadata about a mirror
+METADATA_DIR_NAME = u".ubuntuone-sync"
+
+# filenames to ignore
+SPECIAL_FILE_RE = re.compile(".*\\.("
+                             "(u1)?partial|part|"
+                             "(u1)?conflict(\\.[0-9]+)?)$")
 
=== added file 'src/u1sync/genericmerge.py'
--- src/u1sync/genericmerge.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/genericmerge.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,88 @@
+# ubuntuone.u1sync.genericmerge
+#
+# Generic merge function
+#
+# Author: Tim Cole <[e-mail]>
+#
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""A generic abstraction for merge operations on directory trees."""
+
+from itertools import chain
+from ubuntuone.storageprotocol.dircontent_pb2 import DIRECTORY
+
+class MergeNode(object):
+    """A filesystem node.  Should generally be treated as immutable."""
+    def __init__(self, node_type, content_hash=None, uuid=None, children=None,
+                 conflict_info=None):
+        """Initializes a node instance."""
+        self.node_type = node_type
+        self.children = children
+        self.uuid = uuid
+        self.content_hash = content_hash
+        self.conflict_info = conflict_info
+
+    def __eq__(self, other):
+        """Equality test."""
+        if type(other) is not type(self):
+            return False
+        return self.node_type == other.node_type and \
+               self.children == other.children and \
+               self.uuid == other.uuid and \
+               self.content_hash == other.content_hash and \
+               self.conflict_info == other.conflict_info
+
+    def __ne__(self, other):
+        """Non-equality test."""
+        return not self.__eq__(other)
+
+
+def show_tree(tree, indent="", name="/"):
+    """Prints a tree."""
+    # TODO: return string do not print
+    if tree.node_type == DIRECTORY:
+        type_str = "DIR "
+    else:
+        type_str = "FILE"
+    print "%s%-36s %s %s  %s" % (indent, tree.uuid, type_str, name,
+                                 tree.content_hash)
+    if tree.node_type == DIRECTORY and tree.children is not None:
+        for name in sorted(tree.children.keys()):
+            subtree = tree.children[name]
+            show_tree(subtree, indent="  " + indent, name=name)
+
+def generic_merge(trees, pre_merge, post_merge, partial_parent, name):
+    """Generic tree merging function."""
+
+    partial_result = pre_merge(nodes=trees, name=name,
+                               partial_parent=partial_parent)
+
+    def tree_children(tree):
+        """Returns children if tree is not None"""
+        return tree.children if tree is not None else None
+
+    child_dicts = [tree_children(t) or {} for t in trees]
+    child_names = set(chain(*[cs.iterkeys() for cs in child_dicts]))
+    child_results = {}
+    for child_name in child_names:
+        subtrees = [cs.get(child_name, None) for cs in child_dicts]
+        child_result = generic_merge(trees=subtrees,
+                                     pre_merge=pre_merge,
+                                     post_merge=post_merge,
+                                     partial_parent=partial_result,
+                                     name=child_name)
+        child_results[child_name] = child_result
+
+    return post_merge(nodes=trees, partial_result=partial_result,
+                      child_results=child_results)
 
=== added file 'src/u1sync/main.py'
--- src/u1sync/main.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/main.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,360 @@
+# ubuntuone.u1sync.main
+#
+# Prototype directory sync client
+#
+# Author: Lucio Torre <[e-mail]>
+# Author: Tim Cole <[e-mail]>
+#
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""A prototype directory sync client."""
+
+from __future__ import with_statement
+
+import os
+import sys
+import uuid
+import signal
+import logging
+from Queue import Queue
+from errno import EEXIST
+from twisted.internet import reactor
+# import the storeage protocol to allow the communication with the server
+import ubuntuone.storageprotocol.dircontent_pb2 as dircontent_pb2
+from ubuntuone.storageprotocol.dircontent_pb2 import DIRECTORY, SYMLINK
+# import helper modules
+from u1sync.genericmerge import (
+    show_tree, generic_merge)
+from u1sync.client import (
+    ConnectionError, AuthenticationError, NoSuchShareError,
+    ForcedShutdown, Client)
+from u1sync.scan import scan_directory
+from u1sync.merge import (
+    SyncMerge, ClobberServerMerge, ClobberLocalMerge, merge_trees)
+from u1sync.sync import download_tree, upload_tree
+from u1sync.utils import safe_mkdir
+from u1sync import metadata
+from u1sync.constants import METADATA_DIR_NAME
+from u1sync.ubuntuone_optparse import UbuntuOneOptionsParser
+
+# pylint: disable-msg=W0212
+NODE_TYPE_ENUM = dircontent_pb2._NODETYPE
+# pylint: enable-msg=W0212
+def node_type_str(node_type):
+    """Converts a numeric node type to a human-readable string."""
+    return NODE_TYPE_ENUM.values_by_number[node_type].name
+
+
+class ReadOnlyShareError(Exception):
+    """Share is read-only."""
+
+
+class DirectoryAlreadyInitializedError(Exception):
+    """The directory has already been initialized."""
+
+
+class DirectoryNotInitializedError(Exception):
+    """The directory has not been initialized."""
+
+
+class NoParentError(Exception):
+    """A node has no parent."""
+
+
+class TreesDiffer(Exception):
+    """Raised when diff tree differs."""
+    def __init__(self, quiet):
+        self.quiet = quiet
+
+
+MERGE_ACTIONS = {
+    # action: (merge_class, should_upload, should_download)
+    'sync': (SyncMerge, True, True),
+    'clobber-server': (ClobberServerMerge, True, False),
+    'clobber-local': (ClobberLocalMerge, False, True),
+    'upload': (SyncMerge, True, False),
+    'download': (SyncMerge, False, True),
+    'auto': None # special case
+}
+
+DEFAULT_MERGE_ACTION = 'auto'
+
+def do_init(client, share_spec, directory, quiet, subtree_path,
+            metadata=metadata):
+    """Initializes a directory for syncing, and syncs it."""
+    info = metadata.Metadata()
+
+    if share_spec is not None:
+        info.share_uuid = client.find_volume(share_spec)
+    else:
+        info.share_uuid = None
+
+    if subtree_path is not None:
+        info.path = subtree_path
+    else:
+        info.path = "/"
+
+    logging.info("Initializing directory...")
+    safe_mkdir(directory)
+
+    metadata_dir = os.path.join(directory, METADATA_DIR_NAME)
+    try:
+        os.mkdir(metadata_dir)
+    except OSError, e:
+        if e.errno == EEXIST:
+            raise DirectoryAlreadyInitializedError(directory)
+        else:
+            raise
+
+    logging.info("Writing mirror metadata...")
+    metadata.write(metadata_dir, info)
+
+    logging.info("Done.")
+
+def do_sync(client, directory, action, dry_run, quiet):
+    """Synchronizes a directory with the given share."""
+    absolute_path = os.path.abspath(directory)
+    while True:
+        metadata_dir = os.path.join(absolute_path, METADATA_DIR_NAME)
+        if os.path.exists(metadata_dir):
+            break
+        if absolute_path == "/":
+            raise DirectoryNotInitializedError(directory)
+        absolute_path = os.path.split(absolute_path)[0]
+
+    logging.info("Reading mirror metadata...")
+    info = metadata.read(metadata_dir)
+
+    top_uuid, writable = client.get_root_info(info.share_uuid)
+
+    if info.root_uuid is None:
+        info.root_uuid = client.resolve_path(info.share_uuid, top_uuid,
+                                             info.path)
+
+    if action == 'auto':
+        if writable:
+            action = 'sync'
+        else:
+            action = 'download'
+    merge_type, should_upload, should_download = MERGE_ACTIONS[action]
+    if should_upload and not writable:
+        raise ReadOnlyShareError(info.share_uuid)
+        
+    logging.info("Scanning directory...")
+    
+    local_tree = scan_directory(absolute_path, quiet=quiet)
+ 
+    logging.info("Fetching metadata...")
+    
+    remote_tree = client.build_tree(info.share_uuid, info.root_uuid)
+    if not quiet:
+        show_tree(remote_tree)
+
+    logging.info("Merging trees...")
+    merged_tree = merge_trees(old_local_tree=info.local_tree,
+                              local_tree=local_tree,
+                              old_remote_tree=info.remote_tree,
+                              remote_tree=remote_tree,
+                              merge_action=merge_type())
+    if not quiet:
+        show_tree(merged_tree)
+
+    logging.info("Syncing content...")
+    if should_download:
+        info.local_tree = download_tree(merged_tree=merged_tree,
+                                        local_tree=local_tree,
+                                        client=client,
+                                        share_uuid=info.share_uuid,
+                                        path=absolute_path, dry_run=dry_run,
+                                        quiet=quiet)
+    else:
+        info.local_tree = local_tree
+    if should_upload:
+        info.remote_tree = upload_tree(merged_tree=merged_tree,
+                                       remote_tree=remote_tree,
+                                       client=client,
+                                       share_uuid=info.share_uuid,
+                                       path=absolute_path, dry_run=dry_run,
+                                       quiet=quiet)
+    else:
+        info.remote_tree = remote_tree
+
+    if not dry_run:
+        logging.info("Updating mirror metadata...")
+        metadata.write(metadata_dir, info)
+
+    logging.info("Done.")
+
+def do_list_shares(client):
+    """Lists available (incoming) shares."""
+    shares = client.get_incoming_shares()
+    for (name, id, user, accepted, access) in shares:
+        if not accepted:
+            status = " [not accepted]"
+        else:
+            status = ""
+        name = name.encode("utf-8")
+        user = user.encode("utf-8")
+        print "%s  %s (from %s) [%s]%s" % (id, name, user, access, status)
+
+def do_diff(client, share_spec, directory, quiet, subtree_path,
+            ignore_symlinks=True):
+    """Diffs a local directory with the server."""
+    if share_spec is not None:
+        share_uuid = client.find_volume(share_spec)
+    else:
+        share_uuid = None
+    if subtree_path is None:
+        subtree_path = '/'
+    # pylint: disable-msg=W0612
+    root_uuid, writable = client.get_root_info(share_uuid)
+    subtree_uuid = client.resolve_path(share_uuid, root_uuid, subtree_path)
+    local_tree = scan_directory(directory, quiet=True)
+    remote_tree = client.build_tree(share_uuid, subtree_uuid)
+
+    def pre_merge(nodes, name, partial_parent):
+        """Compares nodes and prints differences."""
+        (local_node, remote_node) = nodes
+        # pylint: disable-msg=W0612
+        (parent_display_path, parent_differs) = partial_parent
+        display_path = os.path.join(parent_display_path, name.encode("UTF-8"))
+        differs = True
+        if local_node is None:
+            logging.info("%s missing from client",display_path)
+        elif remote_node is None:
+            if ignore_symlinks and local_node.node_type == SYMLINK:
+                differs = False
+                logging.info("%s missing from server",display_path)
+        elif local_node.node_type != remote_node.node_type:
+            local_type = node_type_str(local_node.node_type)
+            remote_type = node_type_str(remote_node.node_type)
+            logging.info("%s node types differ (client: %s, server: %s)",
+                display_path, local_type, remote_type)
+        elif local_node.node_type != DIRECTORY and \
+             local_node.content_hash != remote_node.content_hash:
+            local_content = local_node.content_hash
+            remote_content = remote_node.content_hash
+            logging.info("%s has different content (client: %s, server: %s)",
+                display_path, local_content, remote_content)
+        else:
+            differs = False
+        return (display_path, differs)
+
+    def post_merge(nodes, partial_result, child_results):
+        """Aggregates 'differs' flags."""
+        # pylint: disable-msg=W0612
+        (display_path, differs) = partial_result
+        return differs or any(child_results.itervalues())
+
+    differs = generic_merge(trees=[local_tree, remote_tree],
+                            pre_merge=pre_merge, post_merge=post_merge,
+                            partial_parent=("", False), name=u"")
+    if differs:
+        raise TreesDiffer(quiet=quiet)
+
+def do_main(argv, options_parser):
+    """The main user-facing portion of the script."""
+    # pass the arguments to ensure that the code works correctly
+    options_parser.get_options(argv)
+    client = Client(realm=options_parser.options.realm, reactor=reactor)
+    # set the logging level to info in the user passed quite to be false
+    if options_parser.options.quiet:
+        logging.basicConfig(level=logging.INFO)
+        
+    signal.signal(signal.SIGINT, lambda s, f: client.force_shutdown())
+    signal.signal(signal.SIGTERM, lambda s, f: client.force_shutdown())
+
+    def run_client():
+        """Run the blocking client."""
+        token = options_parser.options.token
+
+        client.connect_ssl(options_parser.options.host, 
+            int(options_parser.options.port), 
+            options_parser.options.no_ssl_verify)
+
+        try:
+            client.set_capabilities()
+            client.oauth_from_token(options_parser.options.token)
+
+            if options_parser.options.mode == "sync":
+                do_sync(client=client, directory=options_parser.options.directory,
+                        action=options_parser.options.action,
+                        dry_run=options_parser.options.dry_run, 
+                        quiet=options_parser.options.quiet)
+            elif options_parser.options.mode == "init":
+                do_init(client=client, share_spec=options_parser.options.share,
+                        directory=options_parser.options.directory,
+                        quiet=options_parser.options.quiet, subtree_path=options_parser.options.subtree)
+            elif options.mode == "list-shares":
+                do_list_shares(client=client)
+            elif options.mode == "diff":
+                do_diff(client=client, share_spec=options_parser.options.share,
+                        directory=directory,
+                        quiet=options_parser.options.quiet, 
+                        subtree_path=options_parser.options.subtree,
+                        ignore_symlinks=False)
+            elif options_parser.options.mode == "authorize":
+                if not options_parser.options.quiet:
+                    print "Authorized."
+        finally:
+            client.disconnect()
+
+    def capture_exception(queue, func):
+        """Capture the exception from calling func."""
+        try:
+            func()
+        except Exception:
+            queue.put(sys.exc_info())
+        else:
+            queue.put(None)
+        finally:
+            reactor.callWhenRunning(reactor.stop)
+
+    queue = Queue()
+    reactor.callInThread(capture_exception, queue, run_client)
+    reactor.run(installSignalHandlers=False)
+    exc_info = queue.get(True, 0.1)
+    if exc_info:
+        raise exc_info[0], exc_info[1], exc_info[2]
+
+def main(argv):
+    """Top-level main function."""
+    try:
+        do_main(argv, UbuntuOneOptionsParser())
+    except AuthenticationError, e:
+        print "Authentication failed: %s" % e
+    except ConnectionError, e:
+        print "Connection failed: %s" % e
+    except DirectoryNotInitializedError:
+        print "Directory not initialized; " \
+            "use --init [DIRECTORY] to initialize it."
+    except DirectoryAlreadyInitializedError:
+        print "Directory already initialized."
+    except NoSuchShareError:
+        print "No matching share found."
+    except ReadOnlyShareError:
+        print "The selected action isn't possible on a read-only share."
+    except (ForcedShutdown, KeyboardInterrupt):
+        print "Interrupted!"
+    except TreesDiffer, e:
+        if not e.quiet:
+            print "Trees differ."
+    else:
+        return 0
+    return 1
+
+if __name__ == "__main__":
+    # set the name of the process, this just works on windows
+    sys.argv[0] = "Ubuntuone"
+    main(sys.argv[1:])
 \ No newline at end of file
 
=== added file 'src/u1sync/merge.py'
--- src/u1sync/merge.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/merge.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,186 @@
+# ubuntuone.u1sync.merge
+#
+# Tree state merging
+#
+# Author: Tim Cole <[e-mail]>
+#
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Code for merging changes between modified trees."""
+
+from __future__ import with_statement
+
+import os
+
+from ubuntuone.storageprotocol.dircontent_pb2 import DIRECTORY
+from u1sync.genericmerge import (
+    MergeNode, generic_merge)
+import uuid
+
+
+class NodeTypeMismatchError(Exception):
+    """Node types don't match."""
+
+
+def merge_trees(old_local_tree, local_tree, old_remote_tree, remote_tree,
+                merge_action):
+    """Performs a tree merge using the given merge action."""
+
+    def pre_merge(nodes, name, partial_parent):
+        """Accumulates path and determines merged node type."""
+        old_local_node, local_node, old_remote_node, remote_node = nodes
+        # pylint: disable-msg=W0612
+        (parent_path, parent_type) = partial_parent
+        path = os.path.join(parent_path, name.encode("utf-8"))
+        node_type = merge_action.get_node_type(old_local_node=old_local_node,
+                                               local_node=local_node,
+                                               old_remote_node=old_remote_node,
+                                               remote_node=remote_node,
+                                               path=path)
+        return (path, node_type)
+
+    def post_merge(nodes, partial_result, child_results):
+        """Drops deleted children and merges node."""
+        old_local_node, local_node, old_remote_node, remote_node = nodes
+        # pylint: disable-msg=W0612
+        (path, node_type) = partial_result
+        if node_type == DIRECTORY:
+            merged_children = dict([(name, child) for (name, child)
+                                                  in child_results.iteritems()
+                                                  if child is not None])
+        else:
+            merged_children = None
+        return merge_action.merge_node(old_local_node=old_local_node,
+                                       local_node=local_node,
+                                       old_remote_node=old_remote_node,
+                                       remote_node=remote_node,
+                                       node_type=node_type,
+                                       merged_children=merged_children)
+
+    return generic_merge(trees=[old_local_tree, local_tree,
+                                old_remote_tree, remote_tree],
+                         pre_merge=pre_merge, post_merge=post_merge,
+                         name=u"", partial_parent=("", None))
+
+
+class SyncMerge(object):
+    """Performs a bidirectional sync merge."""
+
+    def get_node_type(self, old_local_node, local_node,
+                      old_remote_node, remote_node, path):
+        """Requires that all node types match."""
+        node_type = None
+        for node in (old_local_node, local_node, remote_node):
+            if node is not None:
+                if node_type is not None:
+                    if node.node_type != node_type:
+                        message = "Node types don't match for %s" % path
+                        raise NodeTypeMismatchError(message)
+                else:
+                    node_type = node.node_type
+        return node_type
+
+    def merge_node(self, old_local_node, local_node,
+                   old_remote_node, remote_node, node_type, merged_children):
+        """Performs bidirectional merge of node state."""
+
+        def node_content_hash(node):
+            """Returns node content hash if node is not None"""
+            return node.content_hash if node is not None else None
+
+        old_local_content_hash = node_content_hash(old_local_node)
+        local_content_hash = node_content_hash(local_node)
+        old_remote_content_hash = node_content_hash(old_remote_node)
+        remote_content_hash = node_content_hash(remote_node)
+
+        locally_deleted = old_local_node is not None and local_node is None
+        deleted_on_server = old_remote_node is not None and remote_node is None
+        # updated means modified or created
+        locally_updated = not locally_deleted and \
+                          old_local_content_hash != local_content_hash
+        updated_on_server = not deleted_on_server and \
+                            old_remote_content_hash != remote_content_hash
+
+        has_merged_children = merged_children is not None and \
+                              len(merged_children) > 0
+
+        either_node_exists = local_node is not None or remote_node is not None
+        should_delete = (locally_deleted and not updated_on_server) or \
+                        (deleted_on_server and not locally_updated)
+
+        if (either_node_exists and not should_delete) or has_merged_children:
+            if node_type != DIRECTORY and \
+               locally_updated and updated_on_server and \
+               local_content_hash != remote_content_hash:
+                # local_content_hash will become the merged content_hash;
+                # save remote_content_hash in conflict info
+                conflict_info = (str(uuid.uuid4()), remote_content_hash)
+            else:
+                conflict_info = None
+            node_uuid = remote_node.uuid if remote_node is not None else None
+            if locally_updated:
+                content_hash = local_content_hash or remote_content_hash
+            else:
+                content_hash = remote_content_hash or local_content_hash
+            return MergeNode(node_type=node_type, uuid=node_uuid,
+                            children=merged_children, content_hash=content_hash,
+                            conflict_info=conflict_info)
+        else:
+            return None
+
+
+class ClobberServerMerge(object):
+    """Clobber server to match local state."""
+
+    def get_node_type(self, old_local_node, local_node,
+                      old_remote_node, remote_node, path):
+        """Return local node type."""
+        if local_node is not None:
+            return local_node.node_type
+        else:
+            return None
+
+    def merge_node(self, old_local_node, local_node,
+                   old_remote_node, remote_node, node_type, merged_children):
+        """Copy local node and associate with remote uuid (if applicable)."""
+        if local_node is None:
+            return None
+        if remote_node is not None:
+            node_uuid = remote_node.uuid
+        else:
+            node_uuid = None
+        return MergeNode(node_type=local_node.node_type, uuid=node_uuid,
+                        content_hash=local_node.content_hash,
+                        children=merged_children)
+
+
+class ClobberLocalMerge(object):
+    """Clobber local state to match server."""
+
+    def get_node_type(self, old_local_node, local_node,
+                      old_remote_node, remote_node, path):
+        """Return remote node type."""
+        if remote_node is not None:
+            return remote_node.node_type
+        else:
+            return None
+
+    def merge_node(self, old_local_node, local_node,
+                   old_remote_node, remote_node, node_type, merged_children):
+        """Copy the remote node."""
+        if remote_node is None:
+            return None
+        return MergeNode(node_type=node_type, uuid=remote_node.uuid,
+                        content_hash=remote_node.content_hash,
+                        children=merged_children)
 
=== added file 'src/u1sync/metadata.py'
--- src/u1sync/metadata.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/metadata.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,145 @@
+# ubuntuone.u1sync.metadata
+#
+# u1sync metadata routines
+#
+# Author: Tim Cole <[e-mail]>
+#
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Routines for loading/storing u1sync mirror metadata."""
+
+from __future__ import with_statement
+
+import os
+import cPickle as pickle
+from errno import ENOENT
+from contextlib import contextmanager
+from ubuntuone.storageprotocol.dircontent_pb2 import DIRECTORY
+from u1sync.merge import MergeNode
+from u1sync.utils import safe_unlink
+import uuid
+
+class Metadata(object):
+    """Object representing mirror metadata."""
+    def __init__(self, local_tree=None, remote_tree=None, share_uuid=None,
+                 root_uuid=None, path=None):
+        """Populate fields."""
+        self.local_tree = local_tree
+        self.remote_tree = remote_tree
+        self.share_uuid = share_uuid
+        self.root_uuid = root_uuid
+        self.path = path
+
+def read(metadata_dir):
+    """Read metadata for a mirror rooted at directory."""
+    index_file = os.path.join(metadata_dir, "local-index")
+    share_uuid_file = os.path.join(metadata_dir, "share-uuid")
+    root_uuid_file = os.path.join(metadata_dir, "root-uuid")
+    path_file = os.path.join(metadata_dir, "path")
+
+    index = read_pickle_file(index_file, {})
+    share_uuid = read_uuid_file(share_uuid_file)
+    root_uuid = read_uuid_file(root_uuid_file)
+    path = read_string_file(path_file, '/')
+
+    local_tree = index.get("tree", None)
+    remote_tree = index.get("remote_tree", None)
+
+    if local_tree is None:
+        local_tree = MergeNode(node_type=DIRECTORY, children={})
+    if remote_tree is None:
+        remote_tree = MergeNode(node_type=DIRECTORY, children={})
+
+    return Metadata(local_tree=local_tree, remote_tree=remote_tree,
+                    share_uuid=share_uuid, root_uuid=root_uuid,
+                    path=path)
+
+def write(metadata_dir, info):
+    """Writes all metadata for the mirror rooted at directory."""
+    share_uuid_file = os.path.join(metadata_dir, "share-uuid")
+    root_uuid_file = os.path.join(metadata_dir, "root-uuid")
+    index_file = os.path.join(metadata_dir, "local-index")
+    path_file = os.path.join(metadata_dir, "path")
+    if info.share_uuid is not None:
+        write_uuid_file(share_uuid_file, info.share_uuid)
+    else:
+        safe_unlink(share_uuid_file)
+    if info.root_uuid is not None:
+        write_uuid_file(root_uuid_file, info.root_uuid)
+    else:
+        safe_unlink(root_uuid_file)
+    write_string_file(path_file, info.path)
+    write_pickle_file(index_file, {"tree": info.local_tree,
+                                   "remote_tree": info.remote_tree})
+
+def write_pickle_file(filename, value):
+    """Writes a pickled python object to a file."""
+    with atomic_update_file(filename) as stream:
+        pickle.dump(value, stream, 2)
+
+def write_string_file(filename, value):
+    """Writes a string to a file with an added line feed, or
+    deletes the file if value is None.
+    """
+    if value is not None:
+        with atomic_update_file(filename) as stream:
+            stream.write(value)
+            stream.write('\n')
+    else:
+        safe_unlink(filename)
+
+def write_uuid_file(filename, value):
+    """Writes a UUID to a file."""
+    write_string_file(filename, str(value))
+
+def read_pickle_file(filename, default_value=None):
+    """Reads a pickled python object from a file."""
+    try:
+        with open(filename, "rb") as stream:
+            return pickle.load(stream)
+    except IOError, e:
+        if e.errno != ENOENT:
+            raise
+        return default_value
+
+def read_string_file(filename, default_value=None):
+    """Reads a string from a file, discarding the final character."""
+    try:
+        with open(filename, "r") as stream:
+            return stream.read()[:-1]
+    except IOError, e:
+        if e.errno != ENOENT:
+            raise
+        return default_value
+
+def read_uuid_file(filename, default_value=None):
+    """Reads a UUID from a file."""
+    try:
+        with open(filename, "r") as stream:
+            return uuid.UUID(stream.read()[:-1])
+    except IOError, e:
+        if e.errno != ENOENT:
+            raise
+        return default_value
+
+@contextmanager
+def atomic_update_file(filename):
+    """Returns a context manager for atomically updating a file."""
+    temp_filename = "%s.%s" % (filename, uuid.uuid4())
+    try:
+        with open(temp_filename, "w") as stream:
+            yield stream
+        os.rename(temp_filename, filename)
+    finally:
+        safe_unlink(temp_filename)
 
=== added file 'src/u1sync/scan.py'
--- src/u1sync/scan.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/scan.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,102 @@
+# ubuntuone.u1sync.scan
+#
+# Directory scanning
+#
+# Author: Tim Cole <[e-mail]>
+#
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Code for scanning local directory state."""
+
+from __future__ import with_statement
+
+import os
+import hashlib
+import shutil
+from errno import ENOTDIR, EINVAL
+import sys
+
+EMPTY_HASH = "sha1:%s" % hashlib.sha1().hexdigest()
+
+from ubuntuone.storageprotocol.dircontent_pb2 import \
+    DIRECTORY, FILE, SYMLINK
+from u1sync.genericmerge import MergeNode 
+from u1sync.utils import should_sync
+
+def scan_directory(path, display_path="", quiet=False):
+    """Scans a local directory and builds an in-memory tree from it."""
+    if display_path != "" and not quiet:
+        print display_path
+
+    link_target = None
+    child_names = None
+    try:
+        print "Path is " + str(path)
+        if sys.platform == "win32":
+            if path.endswith(".lnk") or path.endswith(".url"):
+                import win32com.client 
+                import pythoncom
+                pythoncom.CoInitialize()
+                shell = win32com.client.Dispatch("WScript.Shell")
+                shortcut = shell.CreateShortCut(path)
+                print(shortcut.Targetpath)
+                link_target = shortcut.Targetpath
+            else:
+                link_target = None
+                if os.path.isdir(path):
+                    child_names = os.listdir(path)
+        else:
+            link_target = os.readlink(path)
+    except OSError, e:
+        if e.errno != EINVAL:
+            raise
+        try:
+            child_names = os.listdir(path)
+        except OSError, e:
+            if e.errno != ENOTDIR:
+                raise
+
+    if link_target is not None:
+        # symlink
+        sum = hashlib.sha1()
+        sum.update(link_target)
+        content_hash = "sha1:%s" % sum.hexdigest()
+        return MergeNode(node_type=SYMLINK, content_hash=content_hash)
+    elif child_names is not None:
+        # directory
+        child_names = [n for n in child_names if should_sync(n.decode("utf-8"))]
+        child_paths = [(os.path.join(path, child_name),
+                        os.path.join(display_path, child_name)) \
+                       for child_name in child_names]
+        children = [scan_directory(child_path, child_display_path, quiet) \
+                    for (child_path, child_display_path) in child_paths]
+        unicode_child_names = [n.decode("utf-8") for n in child_names]
+        children = dict(zip(unicode_child_names, children))
+        return MergeNode(node_type=DIRECTORY, children=children)
+    else:
+        # regular file
+        sum = hashlib.sha1()
+
+
+        class HashStream(object):
+            """Stream that computes hashes."""
+            def write(self, bytes):
+                """Accumulate bytes."""
+                sum.update(bytes)
+
+
+        with open(path, "r") as stream:
+            shutil.copyfileobj(stream, HashStream())
+        content_hash = "sha1:%s" % sum.hexdigest()
+        return MergeNode(node_type=FILE, content_hash=content_hash)
 
=== added file 'src/u1sync/sync.py'
--- src/u1sync/sync.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/sync.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,384 @@
+# ubuntuone.u1sync.sync
+#
+# State update
+#
+# Author: Tim Cole <[e-mail]>
+#
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""After merging, these routines are used to synchronize state locally and on
+the server to correspond to the merged result."""
+
+from __future__ import with_statement
+
+import os
+
+EMPTY_HASH = ""
+UPLOAD_SYMBOL = u"\u25b2".encode("utf-8")
+DOWNLOAD_SYMBOL = u"\u25bc".encode("utf-8")
+CONFLICT_SYMBOL = "!"
+DELETE_SYMBOL = "X"
+
+from ubuntuone.storageprotocol import request
+from ubuntuone.storageprotocol.dircontent_pb2 import (
+    DIRECTORY, SYMLINK)
+from u1sync.genericmerge import (
+    MergeNode, generic_merge)
+from u1sync.utils import safe_mkdir
+from u1sync.client import UnsupportedOperationError
+
+def get_conflict_path(path, conflict_info):
+    """Returns path for conflict file corresponding to path."""
+    dir, name = os.path.split(path)
+    unique_id = conflict_info[0]
+    return os.path.join(dir, "conflict-%s-%s" % (unique_id, name))
+
+def name_from_path(path):
+    """Returns unicode name from last path component."""
+    return os.path.split(path)[1].decode("UTF-8")
+
+
+class NodeSyncError(Exception):
+    """Error syncing node."""
+
+
+class NodeCreateError(NodeSyncError):
+    """Error creating node."""
+
+
+class NodeUpdateError(NodeSyncError):
+    """Error updating node."""
+
+
+class NodeDeleteError(NodeSyncError):
+    """Error deleting node."""
+
+
+def sync_tree(merged_tree, original_tree, sync_mode, path, quiet):
+    """Performs actual synchronization."""
+
+    def pre_merge(nodes, name, partial_parent):
+        """Create nodes and write content as required."""
+        (merged_node, original_node) = nodes
+        # pylint: disable-msg=W0612
+        (parent_path, parent_display_path, parent_uuid, parent_synced) \
+            = partial_parent
+
+        utf8_name = name.encode("utf-8")
+        path = os.path.join(parent_path, utf8_name)
+        display_path = os.path.join(parent_display_path, utf8_name)
+        node_uuid = None
+
+        synced = False
+        if merged_node is not None:
+            if merged_node.node_type == DIRECTORY:
+                if original_node is not None:
+                    synced = True
+                    node_uuid = original_node.uuid
+                else:
+                    if not quiet:
+                        print "%s   %s" % (sync_mode.symbol, display_path)
+                    try:
+                        create_dir = sync_mode.create_directory
+                        node_uuid = create_dir(parent_uuid=parent_uuid,
+                                               path=path)
+                        synced = True
+                    except NodeCreateError, e:
+                        print e
+            elif merged_node.content_hash is None:
+                if not quiet:
+                    print "?   %s" % display_path
+            elif original_node is None or \
+                 original_node.content_hash != merged_node.content_hash or \
+                 merged_node.conflict_info is not None:
+                conflict_info = merged_node.conflict_info
+                if conflict_info is not None:
+                    conflict_symbol = CONFLICT_SYMBOL
+                else:
+                    conflict_symbol = " "
+                if not quiet:
+                    print "%s %s %s" % (sync_mode.symbol, conflict_symbol,
+                                        display_path)
+                if original_node is not None:
+                    node_uuid = original_node.uuid or merged_node.uuid
+                    original_hash = original_node.content_hash or EMPTY_HASH
+                else:
+                    node_uuid = merged_node.uuid
+                    original_hash = EMPTY_HASH
+                try:
+                    sync_mode.write_file(node_uuid=node_uuid,
+                                         content_hash=
+                                         merged_node.content_hash,
+                                         old_content_hash=original_hash,
+                                         path=path,
+                                         parent_uuid=parent_uuid,
+                                         conflict_info=conflict_info,
+                                         node_type=merged_node.node_type)
+                    synced = True
+                except NodeSyncError, e:
+                    print e
+            else:
+                synced = True
+
+        return (path, display_path, node_uuid, synced)
+
+    def post_merge(nodes, partial_result, child_results):
+        """Delete nodes."""
+        (merged_node, original_node) = nodes
+        # pylint: disable-msg=W0612
+        (path, display_path, node_uuid, synced) = partial_result
+
+        if merged_node is None:
+            assert original_node is not None
+            if not quiet:
+                print "%s %s %s" % (sync_mode.symbol, DELETE_SYMBOL,
+                                    display_path)
+            try:
+                if original_node.node_type == DIRECTORY:
+                    sync_mode.delete_directory(node_uuid=original_node.uuid,
+                                               path=path)
+                else:
+                    # files or symlinks
+                    sync_mode.delete_file(node_uuid=original_node.uuid,
+                                          path=path)
+                synced = True
+            except NodeDeleteError, e:
+                print e
+
+        if synced:
+            model_node = merged_node
+        else:
+            model_node = original_node
+
+        if model_node is not None:
+            if model_node.node_type == DIRECTORY:
+                child_iter = child_results.iteritems()
+                merged_children = dict([(name, child) for (name, child)
+                                                      in child_iter
+                                                      if child is not None])
+            else:
+                # if there are children here it's because they failed to delete
+                merged_children = None
+            return MergeNode(node_type=model_node.node_type,
+                             uuid=model_node.uuid,
+                             children=merged_children,
+                             content_hash=model_node.content_hash)
+        else:
+            return None
+
+    return generic_merge(trees=[merged_tree, original_tree],
+                         pre_merge=pre_merge, post_merge=post_merge,
+                         partial_parent=(path, "", None, True), name=u"")
+
+def download_tree(merged_tree, local_tree, client, share_uuid, path, dry_run,
+                  quiet):
+    """Downloads a directory."""
+    if dry_run:
+        downloader = DryRun(symbol=DOWNLOAD_SYMBOL)
+    else:
+        downloader = Downloader(client=client, share_uuid=share_uuid)
+    return sync_tree(merged_tree=merged_tree, original_tree=local_tree,
+                     sync_mode=downloader, path=path, quiet=quiet)
+
+def upload_tree(merged_tree, remote_tree, client, share_uuid, path, dry_run,
+                quiet):
+    """Uploads a directory."""
+    if dry_run:
+        uploader = DryRun(symbol=UPLOAD_SYMBOL)
+    else:
+        uploader = Uploader(client=client, share_uuid=share_uuid)
+    return sync_tree(merged_tree=merged_tree, original_tree=remote_tree,
+                     sync_mode=uploader, path=path, quiet=quiet)
+
+
+class DryRun(object):
+    """A class which implements the sync interface but does nothing."""
+    def __init__(self, symbol):
+        """Initializes a DryRun instance."""
+        self.symbol = symbol
+
+    def create_directory(self, parent_uuid, path):
+        """Doesn't create a directory."""
+        return None
+
+    def write_file(self, node_uuid, old_content_hash, content_hash,
+                   parent_uuid, path, conflict_info, node_type):
+        """Doesn't write a file."""
+        return None
+
+    def delete_directory(self, node_uuid, path):
+        """Doesn't delete a directory."""
+
+    def delete_file(self, node_uuid, path):
+        """Doesn't delete a file."""
+
+
+class Downloader(object):
+    """A class which implements the download half of syncing."""
+    def __init__(self, client, share_uuid):
+        """Initializes a Downloader instance."""
+        self.client = client
+        self.share_uuid = share_uuid
+        self.symbol = DOWNLOAD_SYMBOL
+
+    def create_directory(self, parent_uuid, path):
+        """Creates a directory."""
+        try:
+            safe_mkdir(path)
+        except OSError, e:
+            raise NodeCreateError("Error creating local directory %s: %s" % \
+                                  (path, e))
+        return None
+
+    def write_file(self, node_uuid, old_content_hash, content_hash,
+                   parent_uuid, path, conflict_info, node_type):
+        """Creates a file and downloads new content for it."""
+        if conflict_info:
+            # download to conflict file rather than overwriting local changes
+            path = get_conflict_path(path, conflict_info)
+            content_hash = conflict_info[1]
+        try:
+            if node_type == SYMLINK:
+                self.client.download_string(share_uuid=
+                                            self.share_uuid,
+                                            node_uuid=node_uuid,
+                                            content_hash=content_hash)
+            else:
+                self.client.download_file(share_uuid=self.share_uuid,
+                                          node_uuid=node_uuid,
+                                          content_hash=content_hash,
+                                          filename=path)
+        except (request.StorageRequestError, UnsupportedOperationError), e:
+            if os.path.exists(path):
+                raise NodeUpdateError("Error downloading content for %s: %s" %\
+                                      (path, e))
+            else:
+                raise NodeCreateError("Error locally creating %s: %s" % \
+                                      (path, e))
+
+    def delete_directory(self, node_uuid, path):
+        """Deletes a directory."""
+        try:
+            os.rmdir(path)
+        except OSError, e:
+            raise NodeDeleteError("Error locally deleting %s: %s" % (path, e))
+
+    def delete_file(self, node_uuid, path):
+        """Deletes a file."""
+        try:
+            os.unlink(path)
+        except OSError, e:
+            raise NodeDeleteError("Error locally deleting %s: %s" % (path, e))
+
+
+class Uploader(object):
+    """A class which implements the upload half of syncing."""
+    def __init__(self, client, share_uuid):
+        """Initializes an uploader instance."""
+        self.client = client
+        self.share_uuid = share_uuid
+        self.symbol = UPLOAD_SYMBOL
+
+    def create_directory(self, parent_uuid, path):
+        """Creates a directory on the server."""
+        name = name_from_path(path)
+        try:
+            return self.client.create_directory(share_uuid=self.share_uuid,
+                                                parent_uuid=parent_uuid,
+                                                name=name)
+        except (request.StorageRequestError, UnsupportedOperationError), e:
+            raise NodeCreateError("Error remotely creating %s: %s" % \
+                                  (path, e))
+
+    def write_file(self, node_uuid, old_content_hash, content_hash,
+                   parent_uuid, path, conflict_info, node_type):
+        """Creates a file on the server and uploads new content for it."""
+
+        if conflict_info:
+            # move conflicting file out of the way on the server
+            conflict_path = get_conflict_path(path, conflict_info)
+            conflict_name = name_from_path(conflict_path)
+            try:
+                self.client.move(share_uuid=self.share_uuid,
+                                 parent_uuid=parent_uuid,
+                                 name=conflict_name,
+                                 node_uuid=node_uuid)
+            except (request.StorageRequestError, UnsupportedOperationError), e:
+                raise NodeUpdateError("Error remotely renaming %s to %s: %s" %\
+                                      (path, conflict_path, e))
+            node_uuid = None
+            old_content_hash = EMPTY_HASH
+
+        if node_type == SYMLINK:
+            try:
+                target = os.readlink(path)
+            except OSError, e:
+                raise NodeCreateError("Error retrieving link target " \
+                                      "for %s: %s" % (path, e))
+        else:
+            target = None
+
+        name = name_from_path(path)
+        if node_uuid is None:
+            try:
+                if node_type == SYMLINK:
+                    node_uuid = self.client.create_symlink(share_uuid=
+                                                           self.share_uuid,
+                                                           parent_uuid=
+                                                           parent_uuid,
+                                                           name=name,
+                                                           target=target)
+                    old_content_hash = content_hash
+                else:
+                    node_uuid = self.client.create_file(share_uuid=
+                                                        self.share_uuid,
+                                                        parent_uuid=
+                                                        parent_uuid,
+                                                        name=name)
+            except (request.StorageRequestError, UnsupportedOperationError), e:
+                raise NodeCreateError("Error remotely creating %s: %s" % \
+                                      (path, e))
+
+        if old_content_hash != content_hash:
+            try:
+                if node_type == SYMLINK:
+                    self.client.upload_string(share_uuid=self.share_uuid,
+                                              node_uuid=node_uuid,
+                                              content_hash=content_hash,
+                                              old_content_hash=
+                                              old_content_hash,
+                                              content=target)
+                else:
+                    self.client.upload_file(share_uuid=self.share_uuid,
+                                            node_uuid=node_uuid,
+                                            content_hash=content_hash,
+                                            old_content_hash=old_content_hash,
+                                            filename=path)
+            except (request.StorageRequestError, UnsupportedOperationError), e:
+                raise NodeUpdateError("Error uploading content for %s: %s" % \
+                                      (path, e))
+
+    def delete_directory(self, node_uuid, path):
+        """Deletes a directory."""
+        try:
+            self.client.unlink(share_uuid=self.share_uuid, node_uuid=node_uuid)
+        except (request.StorageRequestError, UnsupportedOperationError), e:
+            raise NodeDeleteError("Error remotely deleting %s: %s" % (path, e))
+
+    def delete_file(self, node_uuid, path):
+        """Deletes a file."""
+        try:
+            self.client.unlink(share_uuid=self.share_uuid, node_uuid=node_uuid)
+        except (request.StorageRequestError, UnsupportedOperationError), e:
+            raise NodeDeleteError("Error remotely deleting %s: %s" % (path, e))
 
=== added file 'src/u1sync/ubuntuone_optparse.py'
--- src/u1sync/ubuntuone_optparse.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/ubuntuone_optparse.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,202 @@
+# ubuntuone.u1sync.ubuntuone_optparse
+#
+# Prototype directory sync client
+#
+# Author: Manuel de la Pena <[e-mail]>
+#
+# Copyright 2010 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+import uuid
+from oauth.oauth import OAuthToken
+from optparse import OptionParser, SUPPRESS_HELP
+from u1sync.merge import (
+    SyncMerge, ClobberServerMerge, ClobberLocalMerge)
+
+MERGE_ACTIONS = {
+    # action: (merge_class, should_upload, should_download)
+    'sync': (SyncMerge, True, True),
+    'clobber-server': (ClobberServerMerge, True, False),
+    'clobber-local': (ClobberLocalMerge, False, True),
+    'upload': (SyncMerge, True, False),
+    'download': (SyncMerge, False, True),
+    'auto': None # special case
+}
+
+DEFAULT_MERGE_ACTION = 'auto'
+
+class NotParsedOptionsError(Exception):
+    """Exception thrown when there options have not been parsed."""
+    
+class NotValidatedOptionsError(Exception):
+    """Exception thrown when the options have not been validated."""
+    
+class UbuntuOneOptionsParser(OptionParser):
+    """Parse for the options passed in the command line"""
+    
+    def __init__(self):
+        usage = "Usage: %prog [options] [DIRECTORY]\n" \
+                "       %prog --authorize [options]\n" \
+                "       %prog --list-shares [options]\n" \
+                "       %prog --init [--share=SHARE_UUID] [options] DIRECTORY\n" \
+                "       %prog --diff [--share=SHARE_UUID] [options] DIRECTORY"
+        OptionParser.__init__(self, usage=usage)
+        self._was_validated = False
+        self._args = None
+        # add the different options to be used
+        self.add_option("--port", dest="port", metavar="PORT",
+                          default=443,
+                          help="The port on which to connect to the server")
+        self.add_option("--host", dest="host", metavar="HOST",
+                          default='fs-1.one.ubuntu.com',
+                          help="The server address")
+        self.add_option("--realm", dest="realm", metavar="REALM",
+                          default='https://ubuntuone.com',
+                          help="The oauth realm")
+        self.add_option("--oauth", dest="oauth", metavar="KEY:SECRET",
+                          default=None,
+                          help="Explicitly provide OAuth credentials "
+                               "(default is to query keyring)")
+                               
+        action_list = ", ".join(sorted(MERGE_ACTIONS.keys()))
+        self.add_option("--action", dest="action", metavar="ACTION",
+                          default=None,
+                          help="Select a sync action (%s; default is %s)" % \
+                               (action_list, DEFAULT_MERGE_ACTION))
+                               
+        self.add_option("--dry-run", action="store_true", dest="dry_run",
+                          default=False, help="Do a dry run without actually "
+                                              "making changes")
+        self.add_option("--quiet", action="store_true", dest="quiet",
+                          default=False, help="Produces less output")
+        self.add_option("--authorize", action="store_const", dest="mode",
+                          const="authorize",
+                          help="Authorize this machine")
+        self.add_option("--list-shares", action="store_const", dest="mode",
+                          const="list-shares", default="sync",
+                          help="List available shares")
+        self.add_option("--init", action="store_const", dest="mode",
+                          const="init",
+                          help="Initialize a local directory for syncing")
+        self.add_option("--no-ssl-verify", action="store_true",
+                          dest="no_ssl_verify",
+                          default=False, help=SUPPRESS_HELP)
+        self.add_option("--diff", action="store_const", dest="mode",
+                          const="diff",
+                          help="Compare tree on server with local tree " \
+                               "(does not require previous --init)")
+        self.add_option("--share", dest="share", metavar="SHARE_UUID",
+                          default=None,
+                          help="Sync the directory with a share rather than the " \
+                               "user's own volume")
+        self.add_option("--subtree", dest="subtree", metavar="PATH",
+                          default=None,
+                          help="Mirror a subset of the share or volume")
+    
+    def get_options(self, arguments):
+        """Parses the arguments to from the command line."""
+        (self.options, self._args) = \
+            self.parse_args(arguments)
+        self._validate_args()
+    
+    def _validate_args(self):
+        """Validates the args that have been parsed."""
+        self._is_only_share()
+        self._get_directory()
+        self._validate_action_usage()
+        self._validate_authorize_usage()
+        self._validate_subtree_usage()
+        self._validate_action()
+        self._validate_oauth()
+        
+    def _is_only_share(self):
+        """Ensures that the share options is not convined with any other."""
+        if self.options.share is not None and \
+                self.options.mode != "init" and \
+                self.options.mode != "diff":
+            self.error("--share is only valid with --init or --diff")
+    
+    def _get_directory(self):
+        """Gets the directory to be used according to the paramenters."""
+        print self._args
+        if self.options.mode == "sync" or self.options.mode == "init" or \
+           self.options.mode == "diff":
+            if len(self._args) > 2:
+                self.error("Too many arguments")
+            elif len(self._args) < 1:
+                if self.options.mode == "init" or self.options.mode == "diff":
+                    self.error("--%s requires a directory to "
+                                 "be specified" % self.options.mode)
+                else:
+                    self.options.directory = "."
+            else:
+                self.options.directory = self._args[0]
+    
+    def _validate_action_usage(self):
+        """Ensures that the --action option is correctly used"""
+        if self.options.mode == "init" or \
+            self.options.mode == "list-shares" or \
+            self.options.mode == "diff" or \
+            self.options.mode == "authorize":
+            if self.options.action is not None:
+                self.error("--%s does not take the --action parameter" % \
+                             self.options.mode)
+            if self.options.dry_run:
+                self.error("--%s does not take the --dry-run parameter" % \
+                             self.options.mode)
+    
+    def _validate_authorize_usage(self):
+        """Validates the usage of the authorize option.""" 
+        if self.options.mode == "authorize":
+            if self.options.oauth is not None:
+                self.error("--authorize does not take the --oauth parameter")
+        if self.options.mode == "list-shares" or \
+            self.options.mode == "authorize":
+            if len(self._args) != 0:
+                self.error("--list-shares does not take a directory")
+    
+    def _validate_subtree_usage(self):
+        """Validates the usage of the subtree option"""
+        if self.options.mode != "init" and self.options.mode != "diff":
+            if self.options.subtree is not None:
+                self.error("--%s does not take the --subtree parameter" % \
+                    self.options.mode)
+    
+    def _validate_action(self):
+        """Validates the actions passed to the options."""
+        if self.options.action is not None and \
+            self.options.action not in MERGE_ACTIONS:
+            self.error("--action: Unknown action %s" % self.options.action)
+
+        if self.options.action is None:
+            self.options.action = DEFAULT_MERGE_ACTION
+    
+    def _validate_oauth(self):
+        """Validates that the oatuh was passed."""
+        if self.options.oauth is None:
+            self.error("--oauth is currently compulsery.")
+        else:
+            try:
+                (key, secret) = self.options.oauth.split(':', 2)
+            except ValueError:
+                self.error("--oauth requires a key and secret together in the "
+                             " form KEY:SECRET")
+            self.options.token = OAuthToken(key, secret)
+            
+    def _validate_share(self):
+        """Validates the share option"""
+        if self.options.share is not None:
+            try:
+                uuid.UUID(self.options.share)
+            except ValueError, e:
+                self.error("Invalid --share argument: %s" % e)
 \ No newline at end of file
 
=== added file 'src/u1sync/utils.py'
--- src/u1sync/utils.py	1970-01-01 00:00:00 +0000
+++ src/u1sync/utils.py	2010-08-27 14:56:43 +0000
@@ -0,0 +1,50 @@
+# ubuntuone.u1sync.utils
+#
+# Miscellaneous utility functions
+#
+# Author: Tim Cole <[e-mail]>
+#
+# Copyright 2009 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Miscellaneous utility functions."""
+
+import os
+from errno import EEXIST, ENOENT
+from u1sync.constants import (
+    METADATA_DIR_NAME, SPECIAL_FILE_RE)
+
+def should_sync(filename):
+    """Returns True if the filename should be synced.
+
+    @param filename: a unicode filename
+
+    """
+    return filename != METADATA_DIR_NAME and \
+           not SPECIAL_FILE_RE.match(filename)
+
+def safe_mkdir(path):
+    """Creates a directory iff it does not already exist."""
+    try:
+        os.mkdir(path)
+    except OSError, e:
+        if e.errno != EEXIST:
+            raise
+
+def safe_unlink(path):
+    """Unlinks a file iff it exists."""
+    try:
+        os.unlink(path)
+    except OSError, e:
+        if e.errno != ENOENT:
+            raise