Merge lp:~thisfred/u1db/documentation-update into lp:u1db
- documentation-update
- Merge into trunk
Proposed by
Eric Casteleijn
Status: | Merged |
---|---|
Approved by: | Eric Casteleijn |
Approved revision: | 356 |
Merged at revision: | 354 |
Proposed branch: | lp:~thisfred/u1db/documentation-update |
Merge into: | lp:u1db |
Diff against target: |
753 lines (+199/-191) 9 files modified
CMakeLists.txt (+5/-1) doc/sqlite_schema.txt (+5/-5) html-docs/conflicts.rst (+44/-43) html-docs/high-level-api.rst (+73/-70) html-docs/index.rst (+14/-14) html-docs/philosophy.rst (+31/-32) html-docs/quickstart.rst (+14/-14) html-docs/reference-implementation.rst (+4/-4) u1db/__init__.py (+9/-8) |
To merge this branch: | bzr merge lp:~thisfred/u1db/documentation-update |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Lucio Torre (community) | Approve | ||
Review via email: mp+115187@code.launchpad.net |
Commit message
Added make doctest to make check, so the documentation has a higher chance of not lying. Corrected the documentation to pass doctests and tell the truth.
Description of the change
Added make doctest to make check, so the documentation has a higher chance of not lying. Corrected the documentation to pass doctests and tell the truth.
To post a comment you must log in.
Revision history for this message
Lucio Torre (lucio.torre) : | # |
review:
Approve
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === modified file 'CMakeLists.txt' |
2 | --- CMakeLists.txt 2012-07-05 16:00:16 +0000 |
3 | +++ CMakeLists.txt 2012-07-16 17:22:17 +0000 |
4 | @@ -40,9 +40,13 @@ |
5 | ${CMAKE_CURRENT_BINARY_DIR} |
6 | WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} ) |
7 | |
8 | +add_custom_target(doctests |
9 | + COMMAND cd html-docs && make doctest && cd .. |
10 | +) |
11 | + |
12 | add_custom_target(check |
13 | COMMAND python -m testtools.run discover |
14 | - DEPENDS build-inplace |
15 | + DEPENDS build-inplace doctests |
16 | ) |
17 | |
18 | add_custom_target(build-inplace |
19 | |
20 | === modified file 'doc/sqlite_schema.txt' |
21 | --- doc/sqlite_schema.txt 2011-12-02 10:15:55 +0000 |
22 | +++ doc/sqlite_schema.txt 2012-07-16 17:22:17 +0000 |
23 | @@ -25,7 +25,7 @@ |
24 | It is intended to be a list of fields, and possibly mappings on those fields. |
25 | Something like:: |
26 | |
27 | - CREATE_INDEX(mydb, "myindex", ["field", "other.subfield", "number(third)"]) |
28 | + CREATE_INDEX(mydb, "myindex", ["field", "other.subfield", "number(third)"]) |
29 | |
30 | |
31 | Recommended Implementation |
32 | @@ -83,7 +83,7 @@ |
33 | |
34 | {"lastname": "pedroni", "firstname": "john"} |
35 | |
36 | -Which should not match the above query. |
37 | +Which should not match the above query. |
38 | |
39 | We also want an SQL index on this table, something like [#]_:: |
40 | |
41 | @@ -134,7 +134,7 @@ |
42 | 3) It isn't hard to map nested fields into this structure. And you have |
43 | the nice property that you don't have to change the data to add/remove an |
44 | index. |
45 | - |
46 | + |
47 | 4) It isn't 100% clear how we handle mapped fields in this structure. Something |
48 | like ``lower(lastname)``. It is possible that we could only support the set |
49 | of mappings that we can do with SQL on the live data. However, that will |
50 | @@ -147,7 +147,7 @@ |
51 | seem to support turning "SELECT * FROM table WHERE value LIKE 'p%'" into an |
52 | index query. Even though value is in a btree, it doesn't use it. However, |
53 | you could use >= and < to get a range query. Something like:: |
54 | - |
55 | + |
56 | SELECT * FROM table WHERE value >= 'p' AND value < 'q' |
57 | |
58 | Since sqlite supports closed and open ended ranges, we don't have to play |
59 | @@ -168,7 +168,7 @@ |
60 | ----------------- |
61 | |
62 | The same schema as defined above, except you always put every field into the |
63 | -document_fields table. |
64 | +document_fields table. |
65 | |
66 | Discussion |
67 | ~~~~~~~~~~ |
68 | |
69 | === modified file 'html-docs/conflicts.rst' |
70 | --- html-docs/conflicts.rst 2011-12-21 16:09:22 +0000 |
71 | +++ html-docs/conflicts.rst 2012-07-16 17:22:17 +0000 |
72 | @@ -7,26 +7,26 @@ |
73 | Conflicts |
74 | ------------- |
75 | |
76 | -If two u1dbs are synced, and then the same document is changed in different ways |
77 | -in each u1db, and then they are synced again, there will be a *conflict*. This |
78 | -does not block synchronisation: the document is registered as being in conflict, |
79 | -and resolving that is up to the u1db-using application. |
80 | +If two u1dbs are synced, and then the same document is changed in different |
81 | +ways in each u1db, and then they are synced again, there will be a *conflict*. |
82 | +This does not block synchronisation: the document is registered as being in |
83 | +conflict, and resolving that is up to the u1db-using application. |
84 | |
85 | Importantly, **conflicts are not synced**. If *machine A* initiates a sync with |
86 | *machine B*, and this sync results in a conflict, the conflict **only registers |
87 | on machine A**. This policy is sometimes called "other wins": the machine you |
88 | synced *to* wins conflicts, and the document will have machine B's content on |
89 | -both machine A and machine B. However, on machine A the document is marked |
90 | -as having conflicts, and must be resolved there: |
91 | +both machine A and machine B. However, on machine A the document is marked as |
92 | +having conflicts, and must be resolved there: |
93 | |
94 | .. testsetup :: |
95 | |
96 | import u1db, json |
97 | db=u1db.open(':memory:', True) |
98 | docFromA=u1db.Document('test','machineA:1',json.dumps({'camefrom':'machineA'})) |
99 | - db.put_doc_if_newer(docFromA, save_conflict=True) |
100 | + db._put_doc_if_newer(docFromA, save_conflict=True, replica_uid='machineA', replica_gen=1) |
101 | docFromB=u1db.Document('test','machineB:1',json.dumps({'camefrom':'machineB'})) |
102 | - db.put_doc_if_newer(docFromB, save_conflict=True) |
103 | + db._put_doc_if_newer(docFromB, save_conflict=True, replica_uid='machineB', replica_gen=1) |
104 | |
105 | .. doctest :: |
106 | |
107 | @@ -35,66 +35,67 @@ |
108 | >>> docFromB.has_conflicts # the document is in conflict |
109 | True |
110 | >>> conflicts = db.get_doc_conflicts(docFromB.doc_id) |
111 | - >>> print conflicts |
112 | - [(u'machineB:1', u'{"camefrom": "machineB"}'), (u'machineA:1', u'{"camefrom": "machineA"}')] |
113 | - >>> db.resolve_doc(docFromB, [x[0] for x in conflicts]) # resolve in favour of B |
114 | + >>> conflicts |
115 | + [Document(test, machineB:1, conflicted, u'{"camefrom": "machineB"}'), Document(test, machineA:1, u'{"camefrom": "machineA"}')] |
116 | + >>> db.resolve_doc(docFromB, [d.rev for d in conflicts]) # resolve in favour of B |
117 | >>> doc_is_now = db.get_doc("test") |
118 | >>> doc_is_now.content # the content has been updated to doc's content |
119 | - u'{"camefrom": "machineB"}' |
120 | + {u'camefrom': u'machineB'} |
121 | + >>> db.get_doc_conflicts(docFromB.doc_id) |
122 | + [] |
123 | >>> doc_is_now.has_conflicts # and is no longer in conflict |
124 | False |
125 | |
126 | Note that ``put_doc`` will fail because we got conflicts from a sync, but it |
127 | -may also fail for another reason. If you acquire a document before a sync and |
128 | -then sync, and the sync updates that document, then re-putting that document |
129 | -with modified content will also fail, because the revision is not the current |
130 | +may also fail for another reason. If you acquire a document before a sync and |
131 | +then sync, and the sync updates that document, then re-putting that document |
132 | +with modified content will also fail, because the revision is not the current |
133 | one. This will raise a ``RevisionConflict`` error. |
134 | |
135 | Revisions |
136 | ---------- |
137 | |
138 | -As an app developer, you should treat a ``Document``'s ``revision`` as an opaque |
139 | -cookie; do not try and deconstruct it or edit it. It is for your u1db |
140 | +As an app developer, you should treat a ``Document``'s ``revision`` as an |
141 | +opaque cookie; do not try and deconstruct it or edit it. It is for your u1db |
142 | implementation's use. You can therefore ignore the rest of this section. |
143 | |
144 | -If you are writing a new u1db implementation, understanding revisions is |
145 | +If you are writing a new u1db implementation, understanding revisions is |
146 | important, and this is where you find out about them. |
147 | |
148 | To keep track of document revisions u1db uses vector versions. Each |
149 | -synchronized instance of the same database is called a replica and has |
150 | -a unique identifier (``replica uid``) assigned to it (currently the |
151 | -reference implementation by default uses UUID4s for that); a |
152 | -revision is a mapping between ``replica uids`` and ``edit numbers``: ``rev = |
153 | -<replica_uid:edit_num...>``, or using a functional notation |
154 | -``rev(replica_uid) = edit_num``. The current concrete format is a string |
155 | -built out of each ``replica_uid`` concatenated with ``':'`` and with its edit |
156 | -number in decimal, sorted lexicographically by ``replica_uid`` and then |
157 | -all joined with ``'|'``, for example: ``'replicaA:1|replicaB:3'`` . Absent |
158 | -``replica uids`` in a revision mapping are implicitly mapped to edit |
159 | -number 0. |
160 | +synchronized instance of the same database is called a replica and has a unique |
161 | +identifier (``replica uid``) assigned to it (currently the reference |
162 | +implementation by default uses UUID4s for that); a revision is a mapping |
163 | +between ``replica uids`` and ``edit numbers``: ``rev |
164 | += <replica_uid:edit_num...>``, or using a functional notation |
165 | +``rev(replica_uid) = edit_num``. The current concrete format is a string built |
166 | +out of each ``replica_uid`` concatenated with ``':'`` and with its edit number |
167 | +in decimal, sorted lexicographically by ``replica_uid`` and then all joined |
168 | +with ``'|'``, for example: ``'replicaA:1|replicaB:3'`` . Absent ``replica |
169 | +uids`` in a revision mapping are implicitly mapped to edit number 0. |
170 | |
171 | The new revision of a document modified locally in a replica, is the |
172 | -modification of the old revision where the edit number mapped for the |
173 | -editing ``replica uid`` is increased by 1. |
174 | +modification of the old revision where the edit number mapped for the editing |
175 | +``replica uid`` is increased by 1. |
176 | |
177 | -When syncing one needs to establish whether an incoming revision is |
178 | -newer than the current one or in conflict. A revision |
179 | +When syncing one needs to establish whether an incoming revision is newer than |
180 | +the current one or in conflict. A revision |
181 | |
182 | ``rev1 = <replica_1i:edit_num1i|i=1..n>`` |
183 | |
184 | -is newer than a different |
185 | +is newer than a different |
186 | |
187 | ``rev2 = <replica_2j:edit_num2j|j=1..m>`` |
188 | |
189 | -if for all ``i=1..n``, ``rev2(replica_1i) <= edit_num1i`` |
190 | - |
191 | -and for all ``j=1..m``, ``rev1(replica_2j) >= edit_num2j``. |
192 | - |
193 | -Two revisions which are not equal nor one newer than the |
194 | -other are in conflict. |
195 | - |
196 | -When resolving a conflict locally in a replica ``replica_resol``, starting from |
197 | -``rev1...revN`` in conflict, the resulting revision ``rev_resol`` is obtained by: |
198 | +if for all ``i=1..n``, ``rev2(replica_1i) <= edit_num1i`` |
199 | + |
200 | +and for all ``j=1..m``, ``rev1(replica_2j) >= edit_num2j``. |
201 | + |
202 | +Two revisions which are not equal nor one newer than the other are in conflict. |
203 | + |
204 | +When resolving a conflict locally in a replica ``replica_resol``, starting from |
205 | +``rev1...revN`` in conflict, the resulting revision ``rev_resol`` is obtained |
206 | +by: |
207 | |
208 | ``R`` is the set the of all replicas explicitly mentioned in ``rev1..revN`` |
209 | |
210 | |
211 | === modified file 'html-docs/high-level-api.rst' |
212 | --- html-docs/high-level-api.rst 2011-12-21 11:09:04 +0000 |
213 | +++ html-docs/high-level-api.rst 2012-07-16 17:22:17 +0000 |
214 | @@ -4,23 +4,22 @@ |
215 | ################## |
216 | |
217 | The U1DB API has three separate sections: document storage and retrieval, |
218 | -querying, and sync. Here we describe the high-level API. Remember that you |
219 | -will need to choose an implementation, and exactly how this API is defined |
220 | -is implementation-specific, in order that it fits with the language's |
221 | -conventions. |
222 | +querying, and sync. Here we describe the high-level API. Remember that you will |
223 | +need to choose an implementation, and exactly how this API is defined is |
224 | +implementation-specific, in order that it fits with the language's conventions. |
225 | |
226 | Document storage and retrieval |
227 | ############################## |
228 | |
229 | U1DB stores documents. A document is a set of nested key-values; basically, |
230 | -anything you can express with JSON. Implementations are likely to provide a |
231 | -Document object "wrapper" for these documents; exactly how the wrapper works |
232 | +anything you can express with JSON. Implementations are likely to provide |
233 | +a Document object "wrapper" for these documents; exactly how the wrapper works |
234 | is implementation-defined. |
235 | |
236 | Creating and editing documents |
237 | ------------------------------ |
238 | |
239 | -To create a document, use ``create_doc()``. Code examples below are from |
240 | +To create a document, use ``create_doc()``. Code examples below are from |
241 | :ref:`reference-implementation` in Python. |
242 | |
243 | .. testcode :: |
244 | @@ -33,12 +32,12 @@ |
245 | |
246 | .. testoutput :: |
247 | |
248 | - {"key": "value"} |
249 | + {'key': 'value'} |
250 | testdoc |
251 | |
252 | -Editing an *existing* document is done with ``put_doc()``. This is separate from |
253 | -``create_doc()`` so as to avoid accidental overwrites. ``put_doc()`` takes a |
254 | -``Document`` object, because the object encapsulates revision information for |
255 | +Editing an *existing* document is done with ``put_doc()``. This is separate |
256 | +from ``create_doc()`` so as to avoid accidental overwrites. ``put_doc()`` takes |
257 | +a ``Document`` object, because the object encapsulates revision information for |
258 | a particular document. |
259 | |
260 | .. testcode :: |
261 | @@ -52,15 +51,16 @@ |
262 | except u1db.errors.RevisionConflict: |
263 | print "There was a conflict when creating the doc!" |
264 | print "Now editing the doc with the doc object we got back..." |
265 | - data = json.loads(doc1.content) |
266 | - data["key1"] = "edited" |
267 | - doc1.content = json.dumps(data) |
268 | + doc1.content["key1"] = "edited" |
269 | db.put_doc(doc1) |
270 | + doc2 = db.get_doc(doc1.doc_id) |
271 | + print doc2.content |
272 | |
273 | .. testoutput :: |
274 | |
275 | There was a conflict when creating the doc! |
276 | Now editing the doc with the doc object we got back... |
277 | + {u'key1': u'edited'} |
278 | |
279 | Finally, deleting a document is done with ``delete_doc()``. |
280 | |
281 | @@ -70,9 +70,14 @@ |
282 | db = u1db.open(":memory:", create=True) |
283 | doc = db.create_doc(json.dumps({"key": "value"})) |
284 | db.delete_doc(doc) |
285 | + print db.get_doc(doc.doc_id) |
286 | + doc = db.get_doc(doc.doc_id, include_deleted=True) |
287 | + print doc.content |
288 | |
289 | .. testoutput :: |
290 | |
291 | + None |
292 | + None |
293 | |
294 | Retrieving documents |
295 | -------------------- |
296 | @@ -90,7 +95,7 @@ |
297 | |
298 | .. testoutput :: |
299 | |
300 | - {"key": "value"} |
301 | + {u'key': u'value'} |
302 | testdoc |
303 | |
304 | And it's also possible to retrieve many documents by ``doc_id``. |
305 | @@ -140,7 +145,7 @@ |
306 | {"firstname": "Alan", "surname", "Hansen", "position": "defence"} ID ah |
307 | {"firstname": "John", "surname", "Wayne", "position": "filmstar"} ID jw |
308 | |
309 | -an index expression of ``["firstname"]`` will create an index that looks |
310 | +an index expression of ``["firstname"]`` will create an index that looks |
311 | (conceptually) like this |
312 | |
313 | ====================== =========== |
314 | @@ -152,25 +157,25 @@ |
315 | John jw |
316 | ====================== =========== |
317 | |
318 | -and that index is created with ``create_index("by-firstname", ["firstname"])`` - that is, |
319 | -create an index with a name and a list of index expressions. (Exactly how to |
320 | -pass the name and the list of index expressions is something specific to |
321 | -each implementation.) |
322 | +and that index is created with ``create_index("by-firstname", "firstname")`` |
323 | +-- that is, create an index with a name and a list of index expressions. |
324 | +(Exactly how to pass the name and the list of index expressions is something |
325 | +specific to each implementation.) |
326 | |
327 | Index expressions |
328 | ^^^^^^^^^^^^^^^^^ |
329 | |
330 | -An index expression describes how to get data from a document; you can think |
331 | -of it as describing a function which, when given a document, returns a value, |
332 | +An index expression describes how to get data from a document; you can think of |
333 | +it as describing a function which, when given a document, returns a value, |
334 | which is then used as the index key. |
335 | |
336 | **Name a field.** A basic index expression is a dot-delimited list of nesting |
337 | -fieldnames, so the index expression ``field.sub1.sub2`` applied to a document |
338 | +fieldnames, so the index expression ``field.sub1.sub2`` applied to a document |
339 | with ID ``doc1`` and content:: |
340 | |
341 | { |
342 | - "field": { |
343 | - "sub1": { |
344 | + "field": { |
345 | + "sub1": { |
346 | "sub2": "hello" |
347 | "sub3": "not selected" |
348 | } |
349 | @@ -187,11 +192,11 @@ |
350 | |
351 | **Name a list.** If an index expression names a field whose contents is a list |
352 | of strings, the doc will have multiple entries in the index, one per entry in |
353 | -the list. So, the index expression ``field.tags`` applied to a document with |
354 | -ID "doc2" and content:: |
355 | +the list. So, the index expression ``field.tags`` applied to a document with ID |
356 | +"doc2" and content:: |
357 | |
358 | { |
359 | - "field": { |
360 | + "field": { |
361 | "tags": [ "tag1", "tag2", "tag3" ] |
362 | } |
363 | } |
364 | @@ -206,25 +211,30 @@ |
365 | tag3 doc2 |
366 | ========= ====== |
367 | |
368 | -**Transformation functions.** An index expression may be wrapped in any number of |
369 | -transformation functions. A function transforms the result of the contained |
370 | -index expression: for example, if an expression ``name.firstname`` generates |
371 | -"John" when applied to a document, then ``lower(name.firstname)`` generates |
372 | +**Transformation functions.** An index expression may be wrapped in any number |
373 | +of transformation functions. A function transforms the result of the contained |
374 | +index expression: for example, if an expression ``name.firstname`` generates |
375 | +"John" when applied to a document, then ``lower(name.firstname)`` generates |
376 | "john". |
377 | |
378 | Available transformation functions are: |
379 | |
380 | * ``lower(index_expression)`` - lowercase the value |
381 | - * ``splitwords(index_expression)`` - split the value on whitespace; will act like a |
382 | - list and add multiple entries to the index |
383 | - * ``is_null(index_expression)`` - True if value is null or not a string or the field |
384 | - is absent, otherwise false |
385 | + * ``splitwords(index_expression)`` - split the value on whitespace; will act |
386 | + like a list and add multiple entries to the index |
387 | + * ``number(index_expression, width)`` - takes an integer value, and turns it |
388 | + into a string, left padded with zeroes, to make it at least as wide as |
389 | + width. |
390 | + * ``bool(index_expression)`` - takes a boolean value and turns it into '0' if |
391 | + false and '1' if true. |
392 | + * ``is_null(index_expression)`` - True if value is null or not a string or the |
393 | + field is absent, otherwise false |
394 | |
395 | -So, the index expression ``splitwords(lower(field.name))`` applied to a document with |
396 | -ID "doc3" and content:: |
397 | +So, the index expression ``splitwords(lower(field.name))`` applied to |
398 | +a document with ID "doc3" and content:: |
399 | |
400 | { |
401 | - "field": { |
402 | + "field": { |
403 | "name": "Bruce David Grobbelaar" |
404 | } |
405 | } |
406 | @@ -243,25 +253,18 @@ |
407 | Querying an index |
408 | ----------------- |
409 | |
410 | -Pass a list of tuples of index keys to ``get_from_index``; the last index key in |
411 | -each tuple (and *only* the last one) can end with an asterisk, which matches |
412 | -initial substrings. So, querying our ``by-firstname`` index from above:: |
413 | - |
414 | - get_from_index( |
415 | - "by-firstname", # name of index |
416 | - [ # begin the list of index keys |
417 | - ("John", ) # an index key |
418 | - ] # end the list |
419 | - ) |
420 | - |
421 | - |
422 | -will return ``[ 'jw', 'jb' ]`` - that is, a list of document IDs. |
423 | - |
424 | -``get_from_index("by_firstname", [("J*")])`` will match all index keys beginning |
425 | -with "J", and so will return ``[ 'jw', 'jb', 'jm' ]``. |
426 | - |
427 | -``get_from_index("by_firstname", [("Jan"), ("Alan")])`` will match both the |
428 | -queried index keys, and so will return ``[ 'jm', 'ah' ]``. |
429 | +Pass an index key or a tuple of index keys (if the index is on multiple fields) |
430 | +to ``get_from_index``; the last index key in each tuple (and *only* the last |
431 | +one) can end with an asterisk, which matches initial substrings. So, querying |
432 | +our ``by-firstname`` index from above:: |
433 | + |
434 | + get_from_index("by-firstname", "John") |
435 | + |
436 | + |
437 | +will return the documents with ids: 'jw', 'jb'. |
438 | + |
439 | +``get_from_index("by_firstname", "J*")`` will match all index keys beginning |
440 | +with "J", and so will return the documents with ids: 'jw', 'jb', 'jm'. |
441 | |
442 | |
443 | Index functions |
444 | @@ -277,30 +280,30 @@ |
445 | ####### |
446 | |
447 | U1DB is a syncable database. Any U1DB can be synced with any U1DB server; most |
448 | -U1DB implementations are capable of being run as a server. Syncing brings |
449 | -both the server and the client up to date with one another; save data into a |
450 | -local U1DB whether online or offline, and then sync when online. |
451 | +U1DB implementations are capable of being run as a server. Syncing brings both |
452 | +the server and the client up to date with one another; save data into a local |
453 | +U1DB whether online or offline, and then sync when online. |
454 | |
455 | Pass an HTTP URL to sync with that server. |
456 | |
457 | Syncing databases which have been independently changed may produce conflicts. |
458 | Read about the U1DB conflict policy and more about syncing at :ref:`conflicts`. |
459 | |
460 | -Running your own U1DB server is implementation-specific. :ref:`reference-implementation` |
461 | -is able to be run as a server. |
462 | +Running your own U1DB server is implementation-specific. |
463 | +:ref:`reference-implementation` is able to be run as a server. |
464 | |
465 | Dealing with conflicts |
466 | ---------------------- |
467 | |
468 | -Syncing a database can result in conflicts; if your user changes the same |
469 | +Syncing a database can result in conflicts; if your user changes the same |
470 | document in two different places and then syncs again, that document will be |
471 | ''in conflict'', meaning that it has incompatible changes. If this is the case, |
472 | -``doc.has_conflicts`` will be true, and put_doc to a conflicted doc will give a |
473 | -``ConflictedDoc`` error. To get a list of conflicted versions of the |
474 | -document, do ``get_doc_conflicts(doc_id)``. Deciding what the final unconflicted |
475 | -document should look like is obviously specific to the user's application; once |
476 | -decided, call ``resolve_doc(doc, list_of_conflicted_revisions)`` to resolve and |
477 | -set the final resolved content. |
478 | +``doc.has_conflicts`` will be true, and put_doc to a conflicted doc will give |
479 | +a ``ConflictedDoc`` error. To get a list of conflicted versions of the |
480 | +document, do ``get_doc_conflicts(doc_id)``. Deciding what the final |
481 | +unconflicted document should look like is obviously specific to the user's |
482 | +application; once decided, call ``resolve_doc(doc, list_of_conflicted_revisions)`` |
483 | +to resolve and set the final resolved content. |
484 | |
485 | Syncing functions |
486 | ^^^^^^^^^^^^^^^^^ |
487 | |
488 | === modified file 'html-docs/index.rst' |
489 | --- html-docs/index.rst 2011-12-21 13:49:41 +0000 |
490 | +++ html-docs/index.rst 2012-07-16 17:22:17 +0000 |
491 | @@ -1,29 +1,29 @@ |
492 | U1DB |
493 | #### |
494 | |
495 | -U1DB is a database API for synchronised databases of JSON documents. It's |
496 | -simple to use in applications, and allows apps to store documents and |
497 | -synchronise them between machines and devices. U1DB itself is not a database: |
498 | -instead, it's an API which can be backed by any database for storage. This means that you |
499 | -can use u1db on different platforms, from different languages, and backed |
500 | -on to different databases, and sync between all of them. |
501 | +U1DB is a database API for synchronised databases of JSON documents. It's |
502 | +simple to use in applications, and allows apps to store documents and |
503 | +synchronise them between machines and devices. U1DB itself is not a database: |
504 | +instead, it's an API which can be backed by any database for storage. This |
505 | +means that you can use u1db on different platforms, from different languages, |
506 | +and backed on to different databases, and sync between all of them. |
507 | |
508 | The API for U1DB looks similar across all different implementations. This API |
509 | -is described at :ref:`high-level-api`. To actually use U1DB you'll need an |
510 | -implementation; a version of U1DB made available on your choice of platform, |
511 | -in your choice of language, and on your choice of backend database. |
512 | +is described at :ref:`high-level-api`. To actually use U1DB you'll need an |
513 | +implementation; a version of U1DB made available on your choice of platform, in |
514 | +your choice of language, and on your choice of backend database. |
515 | |
516 | -If you're interested in using U1DB in an application, look at |
517 | -:ref:`high-level-api` first, and then choose one of the :ref:`implementations` |
518 | -and read about exactly how the U1DB API is made available in that |
519 | +If you're interested in using U1DB in an application, look at |
520 | +:ref:`high-level-api` first, and then choose one of the :ref:`implementations` |
521 | +and read about exactly how the U1DB API is made available in that |
522 | implementation. Get going quickly with the :ref:`quickstart`. |
523 | |
524 | -If you're interested in hacking on U1DB itself, read about the |
525 | +If you're interested in hacking on U1DB itself, read about the |
526 | :ref:`rules for U1DB <philosophy>` and :ref:`reference-implementation`. |
527 | |
528 | .. toctree:: |
529 | :maxdepth: 1 |
530 | - |
531 | + |
532 | quickstart |
533 | high-level-api |
534 | reference-implementation |
535 | |
536 | === modified file 'html-docs/philosophy.rst' |
537 | --- html-docs/philosophy.rst 2011-12-21 13:11:20 +0000 |
538 | +++ html-docs/philosophy.rst 2012-07-16 17:22:17 +0000 |
539 | @@ -5,52 +5,51 @@ |
540 | |
541 | Some notes on what u1db is for, how it works, and how it should be used. |
542 | |
543 | -U1DB is a cross-platform, cross-device, syncable database API. In order to be this |
544 | -way, there's a philosophy behind it. Key to this philosophy is that u1db can |
545 | -be implemented in many languages and on top of many back ends: this means that |
546 | -the API needs to be, as much as possible, portable between very different |
547 | +U1DB is a cross-platform, cross-device, syncable database API. In order to be |
548 | +this way, there's a philosophy behind it. Key to this philosophy is that u1db |
549 | +can be implemented in many languages and on top of many back ends: this means |
550 | +that the API needs to be, as much as possible, portable between very different |
551 | languages. Each implementation should implement :ref:`high-level-api` in the |
552 | -way appropriate to that language (Python uses tuples all over the place, |
553 | -Vala/C use a Document object for most things, and so on), but it's important |
554 | -that an implementation not diverge from the API. Because u1db is a syncable |
555 | -database, it's quite likely that an app developer using it will be building their |
556 | -app on multiple platforms at once. Knowledge that an app developer has from |
557 | -having built a u1db app on one platform should be transferable to another |
558 | -platform. This means that querying is the same across platforms; storing and |
559 | -retrieving docs is the same across platforms; syncing is the same across |
560 | -platforms. U1DB is also syncable to Ubuntu One, which is a very large |
561 | -server installation; the API needs to be suitable to run at scales from a |
562 | -mobile phone up to a large server installation. |
563 | - |
564 | -For similar reasons, u1db is *schemaless*. Documents stored in u1db do not |
565 | -need to contain any pre-defined list of fields; this way, an application can |
566 | -store whatever it wants, however it wants; development is faster and changing |
567 | -how data is stored is simpler. |
568 | - |
569 | -What this means is that u1db is for user-specific data. A desktop app or a |
570 | -mobile app storing data for a user is the ideal use case. A web app which |
571 | -holds data for many users should be using and syncing a separate u1db for |
572 | -each user. U1DB isn't designed to be the backend database for the next |
573 | -Facebook. |
574 | +way appropriate to that language (Python uses tuples all over the place, Vala/C |
575 | +use a Document object for most things, and so on), but it's important that an |
576 | +implementation not diverge from the API. Because u1db is a syncable database, |
577 | +it's quite likely that an app developer using it will be building their app on |
578 | +multiple platforms at once. Knowledge that an app developer has from having |
579 | +built a u1db app on one platform should be transferable to another platform. |
580 | +This means that querying is the same across platforms; storing and retrieving |
581 | +docs is the same across platforms; syncing is the same across platforms. U1DB |
582 | +is also syncable to Ubuntu One, which is a very large server installation; the |
583 | +API needs to be suitable to run at scales from a mobile phone up to a large |
584 | +server installation. |
585 | + |
586 | +For similar reasons, u1db is *schemaless*. Documents stored in u1db do not need |
587 | +to contain any pre-defined list of fields; this way, an application can store |
588 | +whatever it wants, however it wants; development is faster and changing how |
589 | +data is stored is simpler. |
590 | + |
591 | +What this means is that u1db is for user-specific data. A desktop app or |
592 | +a mobile app storing data for a user is the ideal use case. A web app which |
593 | +holds data for many users should be using and syncing a separate u1db for each |
594 | +user. U1DB isn't designed to be the backend database for the next Facebook. |
595 | |
596 | To this end, there are a few guidelines. Primarily, the guideline the u1db team |
597 | used for the largest u1db is somewhere around 10,000 documents. It's important |
598 | to note that this is not an *enforced* limit; an app dev can store a zillion |
599 | documents in a u1db if they want. However, the implementations are allowed to |
600 | assume that there aren't a zillion documents; in particular, suggestions for |
601 | -API changes which make things more annoying for a 1,000 documents use-case |
602 | -in order to help with a zillion documents are not likely to be adopted. |
603 | +API changes which make things more annoying for a 1,000 documents use-case in |
604 | +order to help with a zillion documents are not likely to be adopted. |
605 | |
606 | Similarly, suggested changes to the high-level API which are very difficult to |
607 | implement in static languages like C are also unlikely to be adopted, in order |
608 | to maintain the goal of knowledge on one platform transferring to another. |
609 | |
610 | -U1DB is designed so that implementations are built by creating small layers on |
611 | +U1DB is designed so that implementations are built by creating small layers on |
612 | top of existing storage solutions. It isn't a database in itself; it's an API |
613 | layer which sits on top of a native database to that platform. This means that |
614 | -the platform provides the actual database functionality and u1db takes advantage |
615 | -of it. SQLite where available, localStorage for JavaScript in the web browser; |
616 | -u1db should work with the platform, not be ported to it. |
617 | +the platform provides the actual database functionality and u1db takes |
618 | +advantage of it. SQLite where available, localStorage for JavaScript in the web |
619 | +browser; u1db should work with the platform, not be ported to it. |
620 | |
621 | It should be easy to sync a u1db from place to place. There is a direct server |
622 | HTTP API, which allows an app to work with a u1db on the server without any |
623 | |
624 | === modified file 'html-docs/quickstart.rst' |
625 | --- html-docs/quickstart.rst 2011-12-21 11:09:04 +0000 |
626 | +++ html-docs/quickstart.rst 2012-07-16 17:22:17 +0000 |
627 | @@ -18,8 +18,8 @@ |
628 | Use from source control |
629 | ^^^^^^^^^^^^^^^^^^^^^^^ |
630 | |
631 | -u1db is `maintained in bazaar in Launchpad <http://launchpad.net/u1db/>`_. To fetch the latest version, |
632 | -`bzr branch lp:u1db`. |
633 | +u1db is `maintained in bazaar in Launchpad <http://launchpad.net/u1db/>`_. To |
634 | +fetch the latest version, `bzr branch lp:u1db`. |
635 | |
636 | Starting u1db |
637 | ------------- |
638 | @@ -28,29 +28,29 @@ |
639 | |
640 | >>> import u1db, json, tempfile |
641 | >>> db = u1db.open(":memory:", create=True) |
642 | - |
643 | + |
644 | >>> content = json.dumps({"name": "Alan Hansen"}) # create a document |
645 | >>> doc = db.create_doc(content) |
646 | - >>> print doc.content |
647 | - {"name": "Alan Hansen"} |
648 | + >>> doc.content |
649 | + {'name': 'Alan Hansen'} |
650 | >>> doc.content = json.dumps({"name": "Alan Hansen", "position": "defence"}) # update the document's content |
651 | >>> rev = db.put_doc(doc) |
652 | - |
653 | + |
654 | >>> content = json.dumps({"name": "John Barnes", "position": "forward"}) # create more documents |
655 | >>> doc2 = db.create_doc(content) |
656 | >>> content = json.dumps({"name": "Ian Rush", "position": "forward"}) |
657 | >>> doc2 = db.create_doc(content) |
658 | - |
659 | - >>> db.create_index("by-position", ("position",)) # create an index by passing an index expression |
660 | - |
661 | - >>> results = db.get_from_index("by-position", [("forward",)]) # query that index by passing a list of tuples of queries |
662 | + |
663 | + >>> db.create_index("by-position", "position") # create an index by passing a field name |
664 | + |
665 | + >>> results = db.get_from_index("by-position", "forward") # query that index by passing a value |
666 | >>> len(results) |
667 | 2 |
668 | - >>> data = [json.loads(result.content) for result in results] |
669 | + >>> data = [result.content for result in results] |
670 | >>> names = [item["name"] for item in data] |
671 | >>> sorted(names) |
672 | [u'Ian Rush', u'John Barnes'] |
673 | - |
674 | + |
675 | Running a server |
676 | ---------------- |
677 | |
678 | @@ -80,7 +80,7 @@ |
679 | >>> import u1db |
680 | >>> db = u1db.open(":memory:", create=True) |
681 | >>> generation = db.sync("http://127.0.0.1:43632/example.u1db") |
682 | - |
683 | + |
684 | or from the command line |
685 | |
686 | .. code-block:: bash |
687 | @@ -88,4 +88,4 @@ |
688 | ~/u1db/trunk$ ./u1db-client init-db someother.u1db |
689 | ~/u1db/trunk$ ./u1db-client sync someother.u1db http://127.0.0.1:43632/example.u1db |
690 | |
691 | - |
692 | + |
693 | |
694 | === modified file 'html-docs/reference-implementation.rst' |
695 | --- html-docs/reference-implementation.rst 2011-12-21 11:09:04 +0000 |
696 | +++ html-docs/reference-implementation.rst 2012-07-16 17:22:17 +0000 |
697 | @@ -4,10 +4,10 @@ |
698 | ############################# |
699 | |
700 | The u1db reference implementation is written in Python, with a SQLite back end. |
701 | -It can be used as a real working implementation by Python code. It is also used |
702 | -to document and test how u1db should work; it has a comprehensive test suite. |
703 | -Implementation authors should port the u1db reference test suite in order to |
704 | -test that their implementation is correct; in particular, sync conformance is |
705 | +It can be used as a real working implementation by Python code. It is also used |
706 | +to document and test how u1db should work; it has a comprehensive test suite. |
707 | +Implementation authors should port the u1db reference test suite in order to |
708 | +test that their implementation is correct; in particular, sync conformance is |
709 | defined as being able to sync with the reference implementation. |
710 | |
711 | Fetch with ``bzr branch lp:u1db`` or from `Launchpad <http://launchpad.net/u1db>`_. |
712 | |
713 | === modified file 'u1db/__init__.py' |
714 | --- u1db/__init__.py 2012-07-12 17:21:15 +0000 |
715 | +++ u1db/__init__.py 2012-07-16 17:22:17 +0000 |
716 | @@ -60,8 +60,9 @@ |
717 | returned as documents by the database. |
718 | |
719 | :param factory: A function that returns an object which at minimum must |
720 | - satisfy the same interface as does the class DocumentBase. Subclassing |
721 | - that class is the easiest way to create such a function. |
722 | + satisfy the same interface as does the class DocumentBase. |
723 | + Subclassing that class is the easiest way to create such |
724 | + a function. |
725 | """ |
726 | raise NotImplementedError(self.set_document_factory) |
727 | |
728 | @@ -169,11 +170,11 @@ |
729 | and the index generated. |
730 | |
731 | :name: A unique name which can be used as a key prefix |
732 | - :index_expressions: index expressions defining the index |
733 | - information. Examples: |
734 | - "fieldname" to index alphabetically sorted on field. |
735 | - "number(fieldname, width)", "lower(fieldname)", |
736 | - "fieldname.subfieldname" |
737 | + :index_expressions: index expressions defining the index information. |
738 | + Examples: |
739 | + "fieldname" to index alphabetically sorted on field. |
740 | + "number(fieldname, width)", "lower(fieldname)", |
741 | + "fieldname.subfieldname" |
742 | """ |
743 | raise NotImplementedError(self.create_index) |
744 | |
745 | @@ -243,7 +244,7 @@ |
746 | raise NotImplementedError(self.get_index_keys) |
747 | |
748 | def get_doc_conflicts(self, doc_id): |
749 | - """Get the list of conflict texts for the given document. |
750 | + """Get the list of conflicts for the given document. |
751 | |
752 | The order of the conflicts is such that the first entry is the value |
753 | that would be returned by "get_doc". |