Merge lp:~pedronis/u1db/remote-sync-exchange-consolidation into lp:u1db

[Samuele Pedroni]

Support remote sync-exchange:

- refactor how incoming/outgoing documents are processed around sync exchange with some symmetric callback based approach

- let requests and responses support stream of entries (documents) with a callback based approach on the server for both incoming and outgoing streams, and on the client for now for incoming streams

plug the two things together to get remote sync exchange working.

Move away from response objects to an approach where the requests get and deal directly with the responder, calling methods. Yet to support with this: sending error conditions

This was originally two branches/MPs, see the final comments there to see how things were addressed:

* https://code.launchpad.net/~pedronis/u1db/remote-sync-exchange/+merge/80728

* https://code.launchpad.net/~pedronis/u1db/remote-sync-exchange-response-order/+merge/80801

Still to do (for different branches):

- factor out state and sync_exchange decomposition on SyncTarget to a SyncExchange object that can be obtained from local targets

- finish converting docs_info argument to sync_exchange into something callback based as well

- tests for sync_exchange and other RPC calls' bytes-on-the-wire

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/01/2011 09:54 PM, Samuele Pedroni wrote:
> Samuele Pedroni has proposed merging
> lp:~pedronis/u1db/remote-sync-exchange-consolidation into lp:u1db.
>
> Requested reviews: Ubuntu One hackers (ubuntuone-hackers)
>
> For more details, see:
> https://code.launchpad.net/~pedronis/u1db/remote-sync-exchange-consolidation/+merge/80949
>
>
...

> + def _insert_other_doc(self, doc_id, doc_rev, doc): +
> """Try to insert synced over document.

Just mentioning that we should think if this should be named according
to its behavior. Maybe not, but a thought.

> +class EntrySource(object): + + def __init__(self, entries): +
> self.i = 0 + self.entries = entries + + @staticmethod +
> def prepare(self, entry): + return entry + + def
> cb(self): + if self.i >= len(self.entries): +
> return None + entry = self.prepare(self.entries[self.i]) +
> self.i += 1 + return entry +

^- this has a bug, which indicates we are missing some test coverage.
Specifically @staticmethod def prepare(self, entry) would need to be
called with two arguments, what you want is:

@staticmethod
def prepare(entry):
  return entry

I think the idea of EntrySource is nice, the specific implementation
is a bit more of EntryListToSource or something like that. But... not
a big deal.

> +class _DocSource(client.EntrySource): + + @staticmethod +
> def prepare((doc_id, doc_rev, doc)): + return
> dict(doc_id=doc_id, doc_rev=doc_rev, doc=doc)

I'm not a huge fan of unpacking tuples in the args. I'd rather it was
spelled:

def prepare(entry):
  doc_id, doc_rev, doc = entry
  return dict(...)

I'm not fixed on this, it is just a general feeling of mine. (I
checked that you get roughly the same errors of:
ValueError: need more than 1 value to unpack
either way.)

Overall this is really nice.

 merge: approve

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6wZBUACgkQJdeBCYSNAANFvgCgrEbRLDWDR7YrlU6+qNmm/+Rm
tZIAoJPKU1QELk8/ETgXf7MAMpM3OiTf
=1TMx
-----END PGP SIGNATURE-----

review: Approve

[lp:~pedronis/u1db/remote-sync-exchange-consolidation]

107. By Samuele Pedroni on 2011-11-01

fix bare EntrySource.prepare

108. By Samuele Pedroni on 2011-11-01

_insert_other_doc -> insert_doc_from_source|target