Merge lp:~thisfred/desktopcouch/fix-fieldmappings into lp:desktopcouch
- fix-fieldmappings
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Eric Casteleijn |
Approved revision: | 206 |
Merged at revision: | 205 |
Proposed branch: | lp:~thisfred/desktopcouch/fix-fieldmappings |
Merge into: | lp:desktopcouch |
Diff against target: |
1532 lines (+273/-188) 13 files modified
.bzrignore (+2/-0) bin/desktopcouch-pair (+41/-41) data/source_desktopcouch.py (+1/-1) desktopcouch/pair/couchdb_pairing/couchdb_io.py (+5/-5) desktopcouch/pair/tests/test_couchdb_io.py (+3/-3) desktopcouch/records/doc/field_registry.txt (+21/-16) desktopcouch/records/field_registry.py (+16/-9) desktopcouch/records/server.py (+16/-14) desktopcouch/records/server_base.py (+53/-32) desktopcouch/records/tests/test_field_registry.py (+50/-15) desktopcouch/records/tests/test_record.py (+20/-20) desktopcouch/records/tests/test_server.py (+44/-31) utilities/lint.sh (+1/-1) |
To merge this branch: | bzr merge lp:~thisfred/desktopcouch/fix-fieldmappings |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Chad Miller (community) | Approve | ||
Vincenzo Di Somma (community) | Approve | ||
dobey (community) | Approve | ||
Review via email: mp+40573@code.launchpad.net |
Commit message
Fixes bug #416963: fieldregistry transformers now return results, and don't need a record or dictionary like object passed in.
Description of the change
Fixes bug #416963: fieldregistry transformers now return results, and don't need a record or dictionary like object passed in. (They will still accept one, to preserve backward compatibility, and to make it possible to initialize some date before the transformation.)
Eric Casteleijn (thisfred) wrote : | # |
Eric Casteleijn (thisfred) wrote : | # |
The meat of the real changes is in field_registry.py and its tests
dobey (dobey) : | # |
Vincenzo Di Somma (vds) : | # |
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
Attempt to merge into lp:desktopcouch failed due to conflicts:
text conflict in desktopcouch/
text conflict in desktopcouch/
text conflict in utilities/lint.sh
Chad Miller (cmiller) wrote : | # |
Approve revno 204.
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.
Preview Diff
1 | === added file '.bzrignore' |
2 | --- .bzrignore 1970-01-01 00:00:00 +0000 |
3 | +++ .bzrignore 2010-11-12 14:40:00 +0000 |
4 | @@ -0,0 +1,2 @@ |
5 | +_trial_temp |
6 | +.coverage |
7 | |
8 | === modified file 'bin/desktopcouch-pair' |
9 | --- bin/desktopcouch-pair 2010-04-05 21:17:42 +0000 |
10 | +++ bin/desktopcouch-pair 2010-11-12 14:40:00 +0000 |
11 | @@ -22,7 +22,7 @@ |
12 | |
13 | A tool to set two local machines to replicate their couchdb instances to each |
14 | other, or to set this machine to replicate to-and-from Ubuntu One (and perhaps |
15 | -other cloud services). |
16 | +other cloud services). |
17 | |
18 | Local-Pairing Authentication |
19 | ---------------------------- |
20 | @@ -37,7 +37,7 @@ |
21 | |
22 | Alice then computes the SHA512 digest of Bob's secret and compares it to be |
23 | sure that the other machine is indeed the user's. Alice then concatenates |
24 | -Bob's secret and the public seed, and sends the resulting hex digest back to |
25 | +Bob's secret and the public seed, and sends the resulting hex digest back to |
26 | Bob to prove that she received the secret from the user. Alice sets herself to |
27 | replicate to Bob. |
28 | |
29 | @@ -50,7 +50,7 @@ |
30 | import logging |
31 | import getpass |
32 | import gettext |
33 | -# gettext implements "_" function. pylint: disable-msg=E0602 |
34 | +# gettext implements "_" function. pylint: disable=E0602 |
35 | import random |
36 | import cgi |
37 | |
38 | @@ -122,13 +122,13 @@ |
39 | |
40 | We see listeners on the network and send invitations to pair with us. |
41 | We generate a secret message and a public seed. We get the SHA512 hex |
42 | - digest of the secret message, append the public seed and send it to |
43 | - Alice. We also display the cleartext of the secret message to the |
44 | + digest of the secret message, append the public seed and send it to |
45 | + Alice. We also display the cleartext of the secret message to the |
46 | screen, so that the user can take it to the machine he thinks is Alice. |
47 | |
48 | - Eventually, we receive a message back from Alice. We compute the |
49 | - secret message we started with concatenated with the public seed, and |
50 | - if that matches Alice's message, then Alice must know the secret we |
51 | + Eventually, we receive a message back from Alice. We compute the |
52 | + secret message we started with concatenated with the public seed, and |
53 | + if that matches Alice's message, then Alice must know the secret we |
54 | displayed to the user. We then set outselves to replicate to Alice.""" |
55 | |
56 | def delete_event(self, widget, event, data=None): |
57 | @@ -167,7 +167,7 @@ |
58 | self.secret_message = secret_message |
59 | self.public_seed = generate_secret() |
60 | |
61 | - self.inviter = network_io.start_send_invitation(hostname, port, |
62 | + self.inviter = network_io.start_send_invitation(hostname, port, |
63 | self.auth_completed, self.secret_message, self.public_seed, |
64 | self.on_close, couchdb_io.get_my_host_unique_id(create=True)[0], |
65 | local_files.get_oauth_tokens()) |
66 | @@ -180,7 +180,7 @@ |
67 | """us, and to prove veracity of the invitation we\n""" + |
68 | """sent, it is waiting for you to tell it this secret:\n""" + |
69 | """<span font-size="xx-large" color="blue" weight="bold">""" + |
70 | - """<tt>%s</tt></span> .""") % |
71 | + """<tt>%s</tt></span> .""") % |
72 | (cgi.escape(service), cgi.escape(self.secret_message))) |
73 | text.set_justify(gtk.JUSTIFY_CENTER) |
74 | top_vbox.pack_start(text, False, False, 10) |
75 | @@ -197,7 +197,7 @@ |
76 | |
77 | class AcceptInvitation: |
78 | """We're part of 'Alice' in this module's story. |
79 | - |
80 | + |
81 | We've received an invitation. We now send the other end a secret key. The |
82 | secret should make its way back to us via meatspace. We open a dialog |
83 | asking for that secret here, which we validate. |
84 | @@ -258,7 +258,7 @@ |
85 | |
86 | self.result = gtk.Label("") |
87 | self.result.show() |
88 | - self.entry_box.connect("changed", |
89 | + self.entry_box.connect("changed", |
90 | lambda widget: self.result.set_text("")) |
91 | top_vbox.pack_start(self.result, False, False, 0) |
92 | |
93 | @@ -300,7 +300,7 @@ |
94 | |
95 | class Listening: |
96 | """We're part of 'Alice' in this module's story. |
97 | - |
98 | + |
99 | Window that starts listening for other machines to pick *us* to pair |
100 | with. There must be at least one of these on the network for pairing to |
101 | happen. We listen for a finite amount of time, and then stop.""" |
102 | @@ -337,7 +337,7 @@ |
103 | we're listening.""" |
104 | |
105 | self.listener = network_io.ListenForInvitations( |
106 | - self.receive_invitation_challenge, |
107 | + self.receive_invitation_challenge, |
108 | lambda: self.window.destroy(), |
109 | couchdb_io.get_my_host_unique_id(create=True)[0], |
110 | local_files.get_oauth_tokens()) |
111 | @@ -407,16 +407,16 @@ |
112 | text = gtk.Label() |
113 | text.set_markup( |
114 | _("""We're listening for invitations! From another\n""" + |
115 | - """machine on this local network, run this\n""" + |
116 | - """same tool and find the machine called\n""" + |
117 | - """<span font-size="xx-large" weight="bold"><tt>""" + |
118 | - """%s-%s-%d</tt></span> .""") % |
119 | - (cgi.escape(hostid), cgi.escape(userid), |
120 | + """machine on this local network, run this\n""" + |
121 | + """same tool and find the machine called\n""" + |
122 | + """<span font-size="xx-large" weight="bold"><tt>""" + |
123 | + """%s-%s-%d</tt></span> .""") % |
124 | + (cgi.escape(hostid), cgi.escape(userid), |
125 | listen_port)) |
126 | text.set_justify(gtk.JUSTIFY_CENTER) |
127 | top_vbox.pack_start(text, False, False, 10) |
128 | text.show() |
129 | - |
130 | + |
131 | self.update_counter_view() |
132 | self.window.show_all() |
133 | |
134 | @@ -444,7 +444,7 @@ |
135 | |
136 | self.update_counter_view() |
137 | return True |
138 | - |
139 | + |
140 | |
141 | class PickOrListen: |
142 | """Main top-level window that represents the life of the application.""" |
143 | @@ -459,7 +459,7 @@ |
144 | |
145 | def create_pick_pane(self, container): |
146 | """Set up the pane that contains what's necessary to choose an |
147 | - already-listening tool instance. This sets up a "Bob" in the |
148 | + already-listening tool instance. This sets up a "Bob" in the |
149 | module's story.""" |
150 | |
151 | # positions: host id, descr, host, port, cloud_name |
152 | @@ -473,15 +473,15 @@ |
153 | srv = getattr(services, srv_name) |
154 | try: |
155 | if srv.is_active(): |
156 | - all_paired_cloud_servers = [x.key for x in |
157 | + all_paired_cloud_servers = [x.key for x in |
158 | couchdb_io.get_pairings()] |
159 | if not srv_name in all_paired_cloud_servers: |
160 | - self.listening_hosts.append(None, |
161 | + self.listening_hosts.append(None, |
162 | [srv.name, srv.description, "", 0, srv_name]) |
163 | except Exception, e: |
164 | self.logging.exception("service %r has errors", srv_name) |
165 | |
166 | - self.inviting = None # pylint: disable-msg=W0201 |
167 | + self.inviting = None # pylint: disable=W0201 |
168 | |
169 | hostname_col = gtk.TreeViewColumn(_("hostname")) |
170 | hostid_col = gtk.TreeViewColumn(_("service name")) |
171 | @@ -500,7 +500,7 @@ |
172 | tv.show() |
173 | |
174 | def clicked(selection): |
175 | - """An item in the list of services was clicked, so now we go |
176 | + """An item in the list of services was clicked, so now we go |
177 | about inviting it to pair with us.""" |
178 | |
179 | model, iter = selection.get_selected() |
180 | @@ -511,7 +511,7 @@ |
181 | hostname = model.get_value(iter, 2) |
182 | port = model.get_value(iter, 3) |
183 | service_name = model.get_value(iter, 4) |
184 | - |
185 | + |
186 | if service_name: |
187 | # Pairing with a cloud service, which doesn't do key exchange |
188 | pair_with_cloud_service(service_name, self.window) |
189 | @@ -519,7 +519,7 @@ |
190 | self.listening_hosts.remove(iter) |
191 | # add to already-paired list |
192 | srv = getattr(services, service_name) |
193 | - self.already_paired_hosts.append(None, |
194 | + self.already_paired_hosts.append(None, |
195 | [service, _("paired just now"), hostname, port, service_name, None]) |
196 | return |
197 | |
198 | @@ -548,7 +548,7 @@ |
199 | self.listening_hosts.append(None, [name, description, host, port, None]) |
200 | |
201 | def remove_service_from_list(name): |
202 | - """When a zeroconf service disappears, this finds it in the |
203 | + """When a zeroconf service disappears, this finds it in the |
204 | listing and removes it as an option for picking.""" |
205 | |
206 | it = self.listening_hosts.get_iter_first() |
207 | @@ -604,7 +604,7 @@ |
208 | already_paired_record.value.get("ctime", |
209 | _("unknown date")) |
210 | try: |
211 | - self.already_paired_hosts.append(None, |
212 | + self.already_paired_hosts.append(None, |
213 | [srv.name, nice_description, "", 0, srv_name, pid]) |
214 | except Exception, e: |
215 | logging.error("Service %s had an error", srv_name, e) |
216 | @@ -613,7 +613,7 @@ |
217 | nice_description = _("paired ") + \ |
218 | already_paired_record.value.get("ctime", |
219 | _("unknown date")) |
220 | - self.already_paired_hosts.append(None, |
221 | + self.already_paired_hosts.append(None, |
222 | [hostname, nice_description, None, 0, None, pid]) |
223 | else: |
224 | logging.error("unknown pairing record %s", |
225 | @@ -635,7 +635,7 @@ |
226 | tv.show() |
227 | |
228 | def clicked(selection): |
229 | - """An item in the list of services was clicked, so now we go |
230 | + """An item in the list of services was clicked, so now we go |
231 | about inviting it to pair with us.""" |
232 | |
233 | model, iter = selection.get_selected() |
234 | @@ -646,17 +646,17 @@ |
235 | port = model.get_value(iter, 3) |
236 | service_name = model.get_value(iter, 4) |
237 | pid = model.get_value(iter, 5) |
238 | - |
239 | + |
240 | if service_name: |
241 | # delete record |
242 | for record in couchdb_io.get_pairings(): |
243 | couchdb_io.remove_pairing(record.id, True) |
244 | - |
245 | + |
246 | # remove from already-paired list |
247 | self.already_paired_hosts.remove(iter) |
248 | # add to listening list |
249 | srv = getattr(services, service_name) |
250 | - self.listening_hosts.append(None, [service, srv.description, |
251 | + self.listening_hosts.append(None, [service, srv.description, |
252 | hostname, port, service_name]) |
253 | return |
254 | |
255 | @@ -665,7 +665,7 @@ |
256 | if record.value["pairing_identifier"] == pid: |
257 | couchdb_io.remove_pairing(record.id, False) |
258 | break |
259 | - |
260 | + |
261 | # remove from already-paired list |
262 | self.already_paired_hosts.remove(iter) |
263 | # do not add to listening list -- if it's listening then zeroconf |
264 | @@ -699,7 +699,7 @@ |
265 | |
266 | def create_single_listen_pane(self, container): |
267 | """This sets up an "Alice" from the module's story. |
268 | - |
269 | + |
270 | This assumes we're pairing a single, known local CouchDB instance, |
271 | instead of generic instances that we'd need more information to talk |
272 | about. Instead of using this function, one might use another that |
273 | @@ -749,7 +749,7 @@ |
274 | #some_row_in_list.connect("clicked", self.listen, target_db_info) |
275 | |
276 | def __init__(self): |
277 | - |
278 | + |
279 | |
280 | self.logging = logging.getLogger(self.__class__.__name__) |
281 | |
282 | @@ -802,8 +802,8 @@ |
283 | fail_note.run() |
284 | fail_note.destroy() |
285 | return |
286 | - |
287 | - success_note = gtk.Dialog(title=_("Paired with %(hostname)s") % locals(), |
288 | + |
289 | + success_note = gtk.Dialog(title=_("Paired with %(hostname)s") % locals(), |
290 | parent=parent, |
291 | flags=gtk.DIALOG_DESTROY_WITH_PARENT, |
292 | buttons=(gtk.STOCK_OK, gtk.RESPONSE_OK,)) |
293 | @@ -838,7 +838,7 @@ |
294 | fail_note.run() |
295 | fail_note.destroy() |
296 | return |
297 | - |
298 | + |
299 | success_note = gtk.MessageDialog( |
300 | parent=parent, |
301 | flags=gtk.DIALOG_DESTROY_WITH_PARENT, |
302 | |
303 | === modified file 'data/source_desktopcouch.py' |
304 | --- data/source_desktopcouch.py 2010-02-03 19:39:17 +0000 |
305 | +++ data/source_desktopcouch.py 2010-11-12 14:40:00 +0000 |
306 | @@ -14,7 +14,7 @@ |
307 | # You should have received a copy of the GNU General Public License along |
308 | # with this program. If not, see <http://www.gnu.org/licenses/>. |
309 | """Stub for Apport""" |
310 | -# pylint: disable-msg=F0401,C0103 |
311 | +# pylint: disable=F0401,C0103 |
312 | # shut up about apport. We know. We aren't going to backport it for pqm |
313 | import apport |
314 | from apport.hookutils import attach_file_if_exists, packaging |
315 | |
316 | === modified file 'desktopcouch/pair/couchdb_pairing/couchdb_io.py' |
317 | --- desktopcouch/pair/couchdb_pairing/couchdb_io.py 2010-11-09 17:37:32 +0000 |
318 | +++ desktopcouch/pair/couchdb_pairing/couchdb_io.py 2010-11-12 14:40:00 +0000 |
319 | @@ -55,7 +55,7 @@ |
320 | protocol = "https" if has_ssl else "http" |
321 | if auth_pair: |
322 | auth = (":".join( |
323 | - map(urllib.quote, auth_pair)) + "@") # pylint: disable-msg=W0141 |
324 | + map(urllib.quote, auth_pair)) + "@") # pylint: disable=W0141 |
325 | else: |
326 | auth = "" |
327 | if (protocol, port) in (("http", 80), ("https", 443)): |
328 | @@ -108,7 +108,7 @@ |
329 | push_to_server=True, server=hostname, ctx=ctx) |
330 | |
331 | |
332 | -def get_static_paired_hosts(uri=None, # pylint: disable-msg=R0914 |
333 | +def get_static_paired_hosts(uri=None, # pylint: disable=R0914 |
334 | ctx=local_files.DEFAULT_CONTEXT, port=None): |
335 | """Retreive a list of static hosts' information in the form of |
336 | (ID, service name, to_push, to_pull) .""" |
337 | @@ -251,7 +251,7 @@ |
338 | oauth_tokens=oauth_tokens) |
339 | |
340 | |
341 | -def replicate(source_database, target_database, # pylint: disable-msg=R0914 |
342 | +def replicate(source_database, target_database, # pylint: disable=R0914 |
343 | target_host=None, target_port=None, source_host=None, |
344 | source_port=None, source_ssl=False, target_ssl=False, |
345 | source_oauth=None, target_oauth=None, local_uri=None): |
346 | @@ -268,7 +268,7 @@ |
347 | else: |
348 | server.CouchDatabase(target_database, create=True, uri=local_uri) |
349 | logging.debug("db exists, and we're ready to replicate") |
350 | - except: # pylint: disable-msg=W0702 |
351 | + except: # pylint: disable=W0702 |
352 | logging.exception( |
353 | "can't create/verify %r %s:%d oauth=%s", target_database, |
354 | target_host, target_port, obsfuscate(target_oauth)) |
355 | @@ -309,7 +309,7 @@ |
356 | content=record) |
357 | logging.debug( |
358 | "replicate result: %r %r", obsfuscate(resp), obsfuscate(data)) |
359 | - except: # pylint: disable-msg=W0702 |
360 | + except: # pylint: disable=W0702 |
361 | logging.exception("can't replicate %r %r <== %r", source_database, |
362 | local_uri, obsfuscate(record)) |
363 | |
364 | |
365 | === modified file 'desktopcouch/pair/tests/test_couchdb_io.py' |
366 | --- desktopcouch/pair/tests/test_couchdb_io.py 2010-11-09 17:37:32 +0000 |
367 | +++ desktopcouch/pair/tests/test_couchdb_io.py 2010-11-12 14:40:00 +0000 |
368 | @@ -67,8 +67,8 @@ |
369 | |
370 | def tearDown(self): |
371 | """tear down each test""" |
372 | - del self.mgt_database._server['management'] # pylint: disable-msg=W0212 |
373 | - del self.mgt_database._server['foo'] # pylint: disable-msg=W0212 |
374 | + del self.mgt_database._server['management'] # pylint: disable=W0212 |
375 | + del self.mgt_database._server['foo'] # pylint: disable=W0212 |
376 | |
377 | def test_obsfuscation(self): |
378 | """Test the obfuscation of sensitive data.""" |
379 | @@ -98,7 +98,7 @@ |
380 | self.assertEqual(couchdb_io.obsfuscate({1: {}}), {1: {}}) |
381 | self.assertEqual(couchdb_io.obsfuscate({1: 1}), {1: 1}) |
382 | |
383 | - def test_put_static_paired_service(self): # pylint: disable-msg=R0201 |
384 | + def test_put_static_paired_service(self): # pylint: disable=R0201 |
385 | """Test putting a static paired service.""" |
386 | service_name = "dummyfortest" |
387 | oauth_data = { |
388 | |
389 | === modified file 'desktopcouch/records/doc/field_registry.txt' |
390 | --- desktopcouch/records/doc/field_registry.txt 2009-10-05 13:36:43 +0000 |
391 | +++ desktopcouch/records/doc/field_registry.txt 2010-11-12 14:40:00 +0000 |
392 | @@ -9,7 +9,16 @@ |
393 | >>> from desktopcouch.records.record import Record |
394 | |
395 | Say we have a very simple audiofile record type that defines 'artist' |
396 | -and 'title' string fields. Now also say we have an application that |
397 | +and 'title' string fields. |
398 | + |
399 | +>>> class AudioFile(Record): |
400 | +... """An audio file desktopcouch record.""" |
401 | +... |
402 | +... def __init__(self, data=None): |
403 | +... super(AudioFile, self).__init__( |
404 | +... record_type='http://example.org/record_types/audio_file', data=data) |
405 | + |
406 | +Now also say we have an application that |
407 | wants to interact with records of this type called 'My Awesome Music |
408 | Player' or MAMP. The developers of MAMP use a data structure that has |
409 | the same fields, but uses slightly different names for them: |
410 | @@ -23,7 +32,8 @@ |
411 | |
412 | and instantiate a Transformer object: |
413 | |
414 | ->>> my_transformer = Transformer('My Awesome Music Player', my_registry) |
415 | +>>> my_transformer = Transformer( |
416 | +... 'My Awesome Music Player', my_registry, record_class=AudioFile) |
417 | |
418 | If MAMP has the following song object (a plain dictionary): |
419 | |
420 | @@ -36,8 +46,7 @@ |
421 | object: |
422 | |
423 | >>> AUDIO_FILE_RECORD_TYPE = 'http://example.org/record_types/audio_file' |
424 | ->>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE) |
425 | ->>> my_transformer.from_app(my_song, new_record) |
426 | +>>> new_record = my_transformer.from_app(my_song) |
427 | |
428 | Now we can look at the underlying data: |
429 | |
430 | @@ -63,8 +72,7 @@ |
431 | this data. Let's see what happens if we run the transformation with |
432 | this field present, but undefined in the field registry: |
433 | |
434 | ->>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE) |
435 | ->>> my_transformer.from_app(my_song, new_record) |
436 | +>>> new_record = my_transformer.from_app(my_song) |
437 | |
438 | >>> new_record._data #doctest: +NORMALIZE_WHITESPACE |
439 | {'record_type': 'http://example.org/record_types/audio_file', |
440 | @@ -111,9 +119,9 @@ |
441 | ... default_values={'description': 'subject'}), |
442 | ... } |
443 | |
444 | ->>> my_transformer = Transformer('My Awesome Music Player', my_registry) |
445 | ->>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE) |
446 | ->>> my_transformer.from_app(my_song, new_record) |
447 | +>>> my_transformer = Transformer( |
448 | +... 'My Awesome Music Player', my_registry, record_class=AudioFile) |
449 | +>>> new_record = my_transformer.from_app(my_song) |
450 | |
451 | Since _data will now contain lots of uuids to keep references intact, |
452 | it's less readable, and a less clear example, so I'll show you what |
453 | @@ -141,8 +149,7 @@ |
454 | |
455 | and now look at transforming in the other direction: |
456 | |
457 | ->>> new_song = {} |
458 | ->>> my_transformer.to_app(new_record, new_song) |
459 | +>>> new_song = my_transformer.to_app(new_record) |
460 | >>> new_song #doctest: +NORMALIZE_WHITESPACE |
461 | {'tag_title': 'shaking it', |
462 | 'tag_subject': 'talking', |
463 | @@ -193,7 +200,8 @@ |
464 | ... 'songtitle': SimpleFieldMapping('title'), |
465 | ... 'stars': StarIntMapping('score'), |
466 | ... } |
467 | ->>> my_transformer = Transformer('My Awesome Music Player', my_registry) |
468 | +>>> my_transformer = Transformer( |
469 | +... 'My Awesome Music Player', my_registry, record_class=AudioFile) |
470 | |
471 | Create a song with a rating: |
472 | |
473 | @@ -204,10 +212,7 @@ |
474 | ... 'number_of_times_played_in_mamp': 23 |
475 | ... } |
476 | |
477 | ->>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE) |
478 | ->>> my_transformer.from_app(my_song, new_record) |
479 | +>>> new_record = my_transformer.from_app(my_song) |
480 | >>> new_record['score'] |
481 | 100 |
482 | |
483 | -And, I don't know if you've ever heard the song in question, but that |
484 | -is in fact correct! ;) |
485 | |
486 | === modified file 'desktopcouch/records/field_registry.py' |
487 | --- desktopcouch/records/field_registry.py 2010-11-02 23:10:39 +0000 |
488 | +++ desktopcouch/records/field_registry.py 2010-11-12 14:40:00 +0000 |
489 | @@ -21,7 +21,7 @@ |
490 | |
491 | import copy |
492 | |
493 | -from desktopcouch.records.record import MergeableList |
494 | +from desktopcouch.records.record import MergeableList |
495 | ANNOTATION_NAMESPACE = 'application_annotations' |
496 | |
497 | |
498 | @@ -30,12 +30,15 @@ |
499 | data. |
500 | """ |
501 | |
502 | - def __init__(self, app_name, field_registry): |
503 | + def __init__(self, app_name, field_registry, record_class=None): |
504 | self.app_name = app_name |
505 | self.field_registry = field_registry |
506 | + self.record_class = record_class |
507 | |
508 | - def from_app(self, data, record): |
509 | + def from_app(self, data, record=None): |
510 | """Transform from application data to record data.""" |
511 | + if record is None: |
512 | + record = self.record_class() |
513 | for key, val in data.items(): |
514 | if key in self.field_registry: |
515 | self.field_registry[key].setValue(record, val) |
516 | @@ -43,15 +46,19 @@ |
517 | record.application_annotations.setdefault( |
518 | self.app_name, {}).setdefault( |
519 | 'application_fields', {})[key] = val |
520 | + return record |
521 | |
522 | - def to_app(self, record, data): |
523 | + def to_app(self, record, data=None): |
524 | """Transform from record data to application data.""" |
525 | + if data is None: |
526 | + data = {} |
527 | annotations = record.application_annotations.get(self.app_name, {}).get( |
528 | 'application_fields', {}) |
529 | for key, value in annotations.items(): |
530 | data[key] = value |
531 | for key in self.field_registry: |
532 | data[key] = self.field_registry[key].getValue(record) |
533 | + return data |
534 | |
535 | |
536 | class SimpleFieldMapping(object): |
537 | @@ -81,8 +88,8 @@ |
538 | class MergeableListFieldMapping(object): |
539 | """Mapping between MergeableLists and application fields.""" |
540 | |
541 | - def __init__( |
542 | - self, app_name, uuid_field, root_list, field_name, default_values=None): |
543 | + def __init__(self, app_name, uuid_field, root_list, field_name, |
544 | + default_values=None): |
545 | """initialize the default values""" |
546 | self._app_name = app_name |
547 | self._uuid_field = uuid_field |
548 | @@ -119,8 +126,10 @@ |
549 | uuid_key = self._uuidLookup(record) |
550 | if not uuid_key: |
551 | return |
552 | + # pylint: disable=W0212 |
553 | if self._field_name in root_list._data.get(uuid_key, []): |
554 | - del root_list._data[uuid_key][self._field_name] |
555 | + del root_list._data[uuid_key][self._field_name] |
556 | + # pylint: enable=W0212 |
557 | |
558 | def setValue(self, record, value): |
559 | """set the value for the registered field""" |
560 | @@ -155,5 +164,3 @@ |
561 | application_annotations[self._uuid_field] = uuid_key |
562 | return |
563 | record_dict[self._field_name] = value |
564 | - |
565 | - |
566 | |
567 | === modified file 'desktopcouch/records/server.py' |
568 | --- desktopcouch/records/server.py 2010-11-04 11:01:31 +0000 |
569 | +++ desktopcouch/records/server.py 2010-11-12 14:40:00 +0000 |
570 | @@ -18,10 +18,11 @@ |
571 | # Mark G. Saye <mark.saye@canonical.com> |
572 | # Stuart Langridge <stuart.langridge@canonical.com> |
573 | # Chad Miller <chad.miller@canonical.com> |
574 | - |
575 | + |
576 | """The Desktop Couch Records API.""" |
577 | |
578 | -import copy, uuid |
579 | +import copy |
580 | +import uuid |
581 | |
582 | from couchdb import Server |
583 | from couchdb.client import Resource |
584 | @@ -32,9 +33,10 @@ |
585 | |
586 | DCTRASH = 'dctrash' |
587 | |
588 | + |
589 | class OAuthCapableServer(Server): |
590 | """Subclass Server to provide oauth magic""" |
591 | - # pylint: disable-msg=W0231 |
592 | + # pylint: disable=W0231 |
593 | # __init__ method from base class is not called |
594 | def __init__(self, uri, oauth_tokens=None, ctx=None): |
595 | """Subclass of couchdb.client.Server which creates a custom |
596 | @@ -47,17 +49,17 @@ |
597 | if oauth_tokens is None: |
598 | oauth_tokens = desktopcouch.local_files.get_oauth_tokens(ctx) |
599 | (consumer_key, consumer_secret, token, token_secret) = ( |
600 | - oauth_tokens["consumer_key"], oauth_tokens["consumer_secret"], |
601 | + oauth_tokens["consumer_key"], oauth_tokens["consumer_secret"], |
602 | oauth_tokens["token"], oauth_tokens["token_secret"]) |
603 | http.add_oauth_tokens( |
604 | consumer_key, consumer_secret, token, token_secret) |
605 | self.resource = Resource(http, uri) |
606 | - # pylint: enable-msg=W0231 |
607 | - |
608 | - |
609 | + # pylint: enable=W0231 |
610 | + |
611 | + |
612 | class CouchDatabase(server_base.CouchDatabaseBase): |
613 | """An small records specific abstraction over a couch db database.""" |
614 | - |
615 | + |
616 | def __init__(self, database, uri=None, record_factory=None, create=False, |
617 | server_class=OAuthCapableServer, oauth_tokens=None, |
618 | ctx=desktopcouch.local_files.DEFAULT_CONTEXT): |
619 | @@ -67,7 +69,7 @@ |
620 | database, uri, record_factory=record_factory, create=create, |
621 | server_class=server_class, oauth_tokens=oauth_tokens, ctx=ctx) |
622 | |
623 | - # pylint: disable-msg=W0212 |
624 | + # pylint: disable=W0212 |
625 | #Access to a protected member |
626 | def delete_record(self, record_id): |
627 | """Delete record with given id""" |
628 | @@ -79,9 +81,9 @@ |
629 | create=True, |
630 | **self._server_class_extras) |
631 | new_record.record_id = str(uuid.uuid4()) |
632 | - del new_record._data['_rev'] |
633 | + del new_record._data['_rev'] |
634 | try: |
635 | - del new_record._data['_attachments'] |
636 | + del new_record._data['_attachments'] |
637 | except KeyError: |
638 | pass |
639 | new_record.application_annotations['desktopcouch'] = { |
640 | @@ -90,9 +92,9 @@ |
641 | 'original_id': record_id}} |
642 | del self.db[record_id] |
643 | return dctrash.put_record(new_record) |
644 | - # pylint: enable-msg=W0212 |
645 | + # pylint: enable=W0212 |
646 | |
647 | - # pylint: disable-msg=W0221 |
648 | + # pylint: disable=W0221 |
649 | # Arguments number differs from overridden method |
650 | def _reconnect(self): |
651 | if not self.server_uri: |
652 | @@ -101,4 +103,4 @@ |
653 | else: |
654 | uri = self.server_uri |
655 | super(CouchDatabase, self)._reconnect(uri=uri) |
656 | - # pylint: enable-msg=W0221 |
657 | + # pylint: enable=W0221 |
658 | |
659 | === modified file 'desktopcouch/records/server_base.py' |
660 | --- desktopcouch/records/server_base.py 2010-11-03 22:57:06 +0000 |
661 | +++ desktopcouch/records/server_base.py 2010-11-12 14:40:00 +0000 |
662 | @@ -22,31 +22,37 @@ |
663 | |
664 | """The Desktop Couch Records API.""" |
665 | |
666 | -import httplib2, urlparse, cgi, copy, warnings |
667 | +import cgi |
668 | +import copy |
669 | +import httplib2 |
670 | +import urlparse |
671 | +import warnings |
672 | + |
673 | from time import time |
674 | |
675 | # please keep desktopcouch python 2.5 compatible for now |
676 | |
677 | # pylint can't deal with failing imports even when they're handled |
678 | -# pylint: disable-msg=F0401 |
679 | +# pylint: disable=F0401 |
680 | try: |
681 | # Python 2.5 |
682 | import simplejson as json |
683 | except ImportError: |
684 | # Python 2.6+ |
685 | import json |
686 | -# pylint: enable-msg=F0401 |
687 | +# pylint: enable=F0401 |
688 | |
689 | from oauth import oauth |
690 | |
691 | from couchdb import Server |
692 | from couchdb.client import ResourceNotFound, ResourceConflict, uri as couchdburi |
693 | from couchdb.design import ViewDefinition |
694 | -from record import Record |
695 | +from desktopcouch.records.record import Record |
696 | import logging |
697 | |
698 | DEFAULT_DESIGN_DOCUMENT = None # each view in its own eponymous design doc. |
699 | |
700 | + |
701 | def get_changes(self, changes_since): |
702 | """This method is used to monkey patch the database to provide a |
703 | get_changes method""" |
704 | @@ -58,6 +64,7 @@ |
705 | resp, data = self.resource.http.request(uri, "GET", "", {}) |
706 | return resp, data |
707 | |
708 | + |
709 | def transform_to_records(view_results): |
710 | """Transform view resulst into Record objects.""" |
711 | for result in view_results: |
712 | @@ -67,10 +74,10 @@ |
713 | class FieldsConflict(Exception): |
714 | """Raised in case of an unrecoverable couchdb conflict.""" |
715 | |
716 | - #pylint: disable-msg=W0231 |
717 | + #pylint: disable=W0231 |
718 | def __init__(self, conflicts): |
719 | self.conflicts = conflicts |
720 | - #pylint: enable-msg=W0231 |
721 | + #pylint: enable=W0231 |
722 | |
723 | def __str__(self): |
724 | return "<CouchDB Conflict Error: %s>" % self.conflicts |
725 | @@ -97,6 +104,7 @@ |
726 | httplib2.Authentication.__init__(self, None, host, request_uri, |
727 | headers, response, content, http) |
728 | |
729 | + # pylint: disable=R0914 |
730 | def request(self, method, request_uri, headers, content): |
731 | """Modify the request headers to add the appropriate |
732 | Authorization header.""" |
733 | @@ -106,35 +114,43 @@ |
734 | self.oauth_data['token_secret']) |
735 | sig_method = oauth.OAuthSignatureMethod_HMAC_SHA1 |
736 | full_http_url = "%s://%s%s" % (self.scheme, self.host, request_uri) |
737 | - schema, netloc, path, params, query, fragment = \ |
738 | - urlparse.urlparse(full_http_url) |
739 | + # pylint: disable=W0612 |
740 | + # better than using dummy variables, in case we want to use |
741 | + # any of them later on |
742 | + schema, netloc, path, params, query, fragment = urlparse.urlparse( |
743 | + full_http_url) |
744 | + # pylint: enable=W0612 |
745 | querystr_as_dict = dict(cgi.parse_qsl(query)) |
746 | req = oauth.OAuthRequest.from_consumer_and_token( |
747 | consumer, |
748 | access_token, |
749 | - http_method = method, |
750 | - http_url = full_http_url, |
751 | - parameters = querystr_as_dict |
752 | - ) |
753 | + http_method=method, |
754 | + http_url=full_http_url, |
755 | + parameters=querystr_as_dict) |
756 | req.sign_request(sig_method(), consumer, access_token) |
757 | + # pylint: disable=W0212 |
758 | headers.update(httplib2._normalize_headers(req.to_header())) |
759 | + # pylint: enable=W0212 |
760 | + # pylint: enable=R0914 |
761 | |
762 | |
763 | class OAuthCapableHttp(httplib2.Http): |
764 | """Subclass of httplib2.Http which specifically uses our OAuth |
765 | Authentication subclass (because httplib2 doesn't know about it)""" |
766 | - def __init__(self, scheme="http", cache=None, timeout=None, proxy_info=None): |
767 | + def __init__(self, scheme="http", cache=None, timeout=None, |
768 | + proxy_info=None): |
769 | self.__scheme = scheme |
770 | + self.oauth_data = None |
771 | super(OAuthCapableHttp, self).__init__(cache, timeout, proxy_info) |
772 | |
773 | def add_oauth_tokens(self, consumer_key, consumer_secret, |
774 | token, token_secret): |
775 | + """Add the OAuth tokens to the Http object.""" |
776 | self.oauth_data = { |
777 | "consumer_key": consumer_key, |
778 | "consumer_secret": consumer_secret, |
779 | "token": token, |
780 | - "token_secret": token_secret |
781 | - } |
782 | + "token_secret": token_secret} |
783 | |
784 | def _auth_from_challenge(self, host, request_uri, headers, response, |
785 | content): |
786 | @@ -144,6 +160,7 @@ |
787 | yield OAuthAuthentication(self.oauth_data, host, request_uri, headers, |
788 | response, content, self, self.__scheme) |
789 | |
790 | + |
791 | def row_is_deleted(row): |
792 | """Test if a row is marked as deleted. Smart views 'maps' should not |
793 | return rows that are marked as deleted, so this function is not often |
794 | @@ -174,10 +191,12 @@ |
795 | |
796 | @staticmethod |
797 | def _is_reconnection_fail(ex): |
798 | + """Check whether this is the bug in httplib.""" |
799 | return isinstance(ex, AttributeError) and \ |
800 | ex.args == ("'NoneType' object has no attribute 'makefile'",) |
801 | |
802 | def _reconnect(self, uri=None): |
803 | + """Reconnect after losing connection.""" |
804 | logging.info("Connecting to %s.", |
805 | self.server_uri or "discovered local port") |
806 | self._server = self._server_class(uri or self.server_uri, |
807 | @@ -211,8 +230,9 @@ |
808 | """Closure storing the database for lower levels to use when needed. |
809 | """ |
810 | def getter(): |
811 | - return source_db.get_attachment(document_id, attachment_name), \ |
812 | - content_type |
813 | + """Get the attachment and content type.""" |
814 | + return source_db.get_attachment( |
815 | + document_id, attachment_name), content_type |
816 | return getter |
817 | |
818 | try: |
819 | @@ -240,7 +260,7 @@ |
820 | from uuid import uuid4 |
821 | record.record_id = uuid4().hex |
822 | try: |
823 | - self.db[record.record_id] = record._data |
824 | + self.db[record.record_id] = record._data # pylint: disable=W0212 |
825 | except ResourceConflict: |
826 | if record.record_revision is None and not row_is_deleted(record): |
827 | old_record = self.db[record.record_id] |
828 | @@ -249,14 +269,16 @@ |
829 | # but we have marked deleted internally. Instead of |
830 | # complaining, pull up the previous revision ID and |
831 | # add that to the user's record, and re-send it. |
832 | + # pylint: disable=W0212 |
833 | record._set_record_revision(old_record.rev) |
834 | - |
835 | self.db[record.record_id] = record._data |
836 | + # pylint: enable=W0212 |
837 | else: |
838 | raise |
839 | else: |
840 | raise |
841 | |
842 | + # pylint: disable=W0212 |
843 | for attachment_name in getattr(record, "_detached", []): |
844 | self.db.delete_attachment(record._data, attachment_name) |
845 | |
846 | @@ -266,6 +288,7 @@ |
847 | data, |
848 | attachment_name, |
849 | content_type) |
850 | + # pylint: enable=W0212 |
851 | |
852 | return record.record_id |
853 | |
854 | @@ -282,6 +305,7 @@ |
855 | # although with a single record we need to test for the |
856 | # revisison, with a batch we do not, but we have to make sure |
857 | # that we did not get an error |
858 | + # pylint: disable=W0212 |
859 | batch_put_result = self.db.update([record._data for record in batch]) |
860 | for current_tuple in batch_put_result: |
861 | success, docid, rev_or_exc = current_tuple |
862 | @@ -289,15 +313,14 @@ |
863 | record = records_hash[docid] |
864 | # set the new rev |
865 | record._data["_rev"] = rev_or_exc |
866 | - |
867 | for attachment_name in getattr(record, "_detached", []): |
868 | self.db.delete_attachment(record._data, attachment_name) |
869 | - |
870 | for attachment_name in record.list_attachments(): |
871 | data, content_type = record.attachment_data(attachment_name) |
872 | self.db.put_attachment( |
873 | - {"_id":record.record_id, "_rev":record["_rev"]}, |
874 | + {"_id": record.record_id, "_rev": record["_rev"]}, |
875 | data, attachment_name, content_type) |
876 | + # pylint: enable=W0212 |
877 | # all success record have the blobs added we return result of |
878 | # update |
879 | return batch_put_result |
880 | @@ -322,7 +345,7 @@ |
881 | if cached_record is None: |
882 | cached_record = self.db[record_id] |
883 | if isinstance(cached_record, Record): |
884 | - cached_record = cached_record._data |
885 | + cached_record = cached_record._data # pylint: disable=W0212 |
886 | record = copy.deepcopy(cached_record) |
887 | # Loop until either failure or success has been determined |
888 | while True: |
889 | @@ -404,7 +427,7 @@ |
890 | # No atomic updates. Only read & mutate & write. Le sigh. |
891 | # First, get current contents. |
892 | try: |
893 | - view_container = self.db[doc_id]["views"] |
894 | + view_container = self.db[doc_id]["views"] |
895 | except (KeyError, ResourceNotFound): |
896 | raise KeyError |
897 | |
898 | @@ -414,9 +437,7 @@ |
899 | # Construct a new list of objects representing all views to have. |
900 | views = [ |
901 | ViewDefinition(design_doc, k, v.get("map"), v.get("reduce")) |
902 | - for k, v |
903 | - in view_container.iteritems() |
904 | - ] |
905 | + for k, v in view_container.iteritems()] |
906 | # Push back a new batch of view. Pray to Eris that this doesn't |
907 | # clobber anything we want. |
908 | |
909 | @@ -446,9 +467,8 @@ |
910 | view_id_fmt = "_design/%(design_doc)s/_view/%(view_name)s" |
911 | return self.db.view(view_id_fmt % locals(), **params) |
912 | |
913 | - |
914 | def add_view(self, view_name, map_js, reduce_js, |
915 | - design_doc=DEFAULT_DESIGN_DOCUMENT): |
916 | + design_doc=DEFAULT_DESIGN_DOCUMENT): |
917 | """Create a view, given a name and the two parts (map and reduce). |
918 | Return the document id.""" |
919 | if design_doc is None: |
920 | @@ -476,7 +496,7 @@ |
921 | """Return a list of view names for a given design document. There is |
922 | no error if the design document does not exist or if there are no views |
923 | in it.""" |
924 | - doc_id = "_design/%(design_doc)s" % locals() |
925 | + doc_id = "_design/%s" % design_doc |
926 | try: |
927 | return list(self.db[doc_id]["views"]) |
928 | except (KeyError, ResourceNotFound): |
929 | @@ -542,12 +562,14 @@ |
930 | return viewdata[record_type] |
931 | |
932 | def get_view_results_as_records(self, view_name, record_type=None): |
933 | + """Get the results of a view as a list of Record objects.""" |
934 | view_results = self.execute_view(view_name, view_name) |
935 | if record_type: |
936 | view_results = view_results[record_type] |
937 | return [record for record in transform_to_records(view_results)] |
938 | |
939 | def get_all_records(self, record_type=None): |
940 | + """Get all records from the database, optionally by record type.""" |
941 | view_name = "get_records_and_type" |
942 | try: |
943 | return self.get_view_results_as_records( |
944 | @@ -568,7 +590,6 @@ |
945 | return self.get_view_results_as_records( |
946 | view_name, record_type=record_type) |
947 | |
948 | - |
949 | def get_changes(self, niceness=10): |
950 | """Get a list of database changes. This is the sister function of |
951 | report_changes that returns a list instead of calling a function for |
952 | @@ -610,7 +631,7 @@ |
953 | raise IOError( |
954 | "HTTP response code %s.\n%s" % (resp["status"], data)) |
955 | structure = json.loads(data) |
956 | - for change in structure.get("results"): |
957 | + for change in structure.get("results"): # pylint: disable=E1103 |
958 | # kw-args can't have unicode keys |
959 | change_encoded_keys = dict( |
960 | (k.encode("utf8"), v) for k, v in change.iteritems()) |
961 | |
962 | === modified file 'desktopcouch/records/tests/test_field_registry.py' |
963 | --- desktopcouch/records/tests/test_field_registry.py 2010-11-11 19:01:41 +0000 |
964 | +++ desktopcouch/records/tests/test_field_registry.py 2010-11-12 14:40:00 +0000 |
965 | @@ -17,7 +17,10 @@ |
966 | |
967 | """Test cases for field mapping""" |
968 | |
969 | -import copy, doctest, os |
970 | +import copy |
971 | +import doctest |
972 | +import os |
973 | + |
974 | from testtools import TestCase |
975 | |
976 | import desktopcouch |
977 | @@ -34,25 +37,35 @@ |
978 | 'not_the_field': 'different value'}, |
979 | 'd6d2c23b-279c-45c8-afb2-ec84ee7c81c3': { |
980 | 'the field': 'another value'}}, |
981 | - 'application_annotations': {'Test App': {'private_application_annotations':{ |
982 | - 'test_field': 'e47455fb-da05-481e-a2c7-88f14d5cc163'}}}} |
983 | + 'application_annotations': {'Test App': { |
984 | + 'private_application_annotations': { |
985 | + 'test_field': 'e47455fb-da05-481e-a2c7-88f14d5cc163'}}}} |
986 | |
987 | APP_RECORD = { |
988 | 'record_type': 'http://example.com/test', |
989 | 'simpleField': 23, |
990 | - 'strawberryField': 'the value',} |
991 | + 'strawberryField': 'the value'} |
992 | |
993 | FIELD_REGISTRY = { |
994 | 'simpleField': SimpleFieldMapping('simple_field'), |
995 | 'strawberryField': MergeableListFieldMapping( |
996 | - 'Test App', 'test_field', 'test_fields', 'the_field'),} |
997 | + 'Test App', 'test_field', 'test_fields', 'the_field')} |
998 | + |
999 | + |
1000 | +class TestRecord(Record): |
1001 | + """A test record type.""" |
1002 | + |
1003 | + def __init__(self, data=None): |
1004 | + super(TestRecord, self).__init__( |
1005 | + data=data, record_type='http://example.com/test') |
1006 | |
1007 | |
1008 | class AppTransformer(Transformer): |
1009 | """A test transformer class.""" |
1010 | |
1011 | def __init__(self): |
1012 | - super(AppTransformer, self).__init__('Test App', FIELD_REGISTRY) |
1013 | + super(AppTransformer, self).__init__( |
1014 | + 'Test App', FIELD_REGISTRY, record_class=TestRecord) |
1015 | |
1016 | |
1017 | class TestFieldMapping(TestCase): |
1018 | @@ -83,16 +96,16 @@ |
1019 | |
1020 | def test_mergeable_list_field_mapping1(self): |
1021 | """Test the MergeableListFieldMapping object.""" |
1022 | - # pylint: disable-msg=W0212 |
1023 | + # pylint: disable=W0212 |
1024 | record = Record(self.test_record) |
1025 | mapping = MergeableListFieldMapping( |
1026 | 'Test App', 'test_field', 'test_fields', 'the_field') |
1027 | self.assertEqual('the value', mapping.getValue(record)) |
1028 | - del record._data['test_fields'][ |
1029 | + del record._data['test_fields'][ # pylint: disable=W0212 |
1030 | 'e47455fb-da05-481e-a2c7-88f14d5cc163'] |
1031 | mapping.deleteValue(record) |
1032 | self.assertEqual(None, mapping.getValue(record)) |
1033 | - # pylint: enable-msg=W0212 |
1034 | + # pylint: enable=W0212 |
1035 | |
1036 | def test_mergeable_list_field_mapping_empty_field(self): |
1037 | """Test setting empty values in the MergeableListFieldMapping object.""" |
1038 | @@ -112,21 +125,44 @@ |
1039 | self.transformer = AppTransformer() |
1040 | |
1041 | def test_from_app(self): |
1042 | - """Test transformation from app to Ubuntu One.""" |
1043 | - # pylint: disable-msg=W0212 |
1044 | + """Test transformation from app to desktopcouch.""" |
1045 | + record = self.transformer.from_app(APP_RECORD) |
1046 | + underlying = record._data # pylint: disable=W0212 |
1047 | + self.assertEqual(23, record['simple_field']) |
1048 | + the_uuid = record.application_annotations['Test App']\ |
1049 | + ['private_application_annotations']['test_field'] |
1050 | + self.assertEqual( |
1051 | + {'the_field': 'the value'}, |
1052 | + underlying['test_fields'][the_uuid]) |
1053 | + |
1054 | + def test_from_app_with_record(self): |
1055 | + """Test transformation from app to desktopcouch when passing in |
1056 | + an existing record. |
1057 | + |
1058 | + """ |
1059 | record = Record(record_type="http://example.com/test") |
1060 | self.transformer.from_app(APP_RECORD, record) |
1061 | - underlying = record._data |
1062 | + underlying = record._data # pylint: disable=W0212 |
1063 | self.assertEqual(23, record['simple_field']) |
1064 | the_uuid = record.application_annotations['Test App']\ |
1065 | ['private_application_annotations']['test_field'] |
1066 | self.assertEqual( |
1067 | {'the_field': 'the value'}, |
1068 | underlying['test_fields'][the_uuid]) |
1069 | - # pylint: enable-msg=W0212 |
1070 | + # pylint: enable=W0212 |
1071 | |
1072 | def test_to_app(self): |
1073 | - """Test transformation to app from Ubuntu One.""" |
1074 | + """Test transformation to app from desktopcouch.""" |
1075 | + record = Record(TEST_RECORD) |
1076 | + data = self.transformer.to_app(record) |
1077 | + self.assertEqual( |
1078 | + {'simpleField': 23, 'strawberryField': 'the value'}, data) |
1079 | + |
1080 | + def test_to_app_with_dictionary(self): |
1081 | + """Test transformation to app from desktopcouch when passing |
1082 | + in an existing dictionary. |
1083 | + |
1084 | + """ |
1085 | record = Record(TEST_RECORD) |
1086 | data = {} |
1087 | self.transformer.to_app(record, data) |
1088 | @@ -145,5 +181,4 @@ |
1089 | '../desktopcouch/records/doc/field_registry.txt' |
1090 | results = doctest.testfile(field_registry_tests_path, |
1091 | module_relative=False) |
1092 | - |
1093 | self.assertEqual(0, results.failed) |
1094 | |
1095 | === modified file 'desktopcouch/records/tests/test_record.py' |
1096 | --- desktopcouch/records/tests/test_record.py 2010-11-11 19:01:41 +0000 |
1097 | +++ desktopcouch/records/tests/test_record.py 2010-11-12 14:40:00 +0000 |
1098 | @@ -19,11 +19,12 @@ |
1099 | |
1100 | """Tests for the RecordDict object on which the Contacts API is built.""" |
1101 | |
1102 | -import doctest, os |
1103 | +import doctest |
1104 | +import os |
1105 | from testtools import TestCase |
1106 | |
1107 | # pylint does not like relative imports from containing packages |
1108 | -# pylint: disable-msg=F0401 |
1109 | +# pylint: disable=F0401 |
1110 | import desktopcouch |
1111 | from desktopcouch.records.server import CouchDatabase |
1112 | from desktopcouch.records.record import (Record, RecordDict, MergeableList, |
1113 | @@ -34,7 +35,7 @@ |
1114 | class TestRecords(TestCase): |
1115 | """Test the record functionality""" |
1116 | |
1117 | - # pylint: disable-msg=C0103 |
1118 | + # pylint: disable=C0103 |
1119 | def setUp(self): |
1120 | """Test setup.""" |
1121 | super(TestRecords, self).setUp() |
1122 | @@ -47,7 +48,7 @@ |
1123 | "subfield_uuid": { |
1124 | "e47455fb-da05-481e-a2c7-88f14d5cc163": { |
1125 | "field11": "value11", |
1126 | - "field12": "value12",}, |
1127 | + "field12": "value12"}, |
1128 | "d6d2c23b-279c-45c8-afb2-ec84ee7c81c3": { |
1129 | "field21": "value21", |
1130 | "field22": "value22"}}, |
1131 | @@ -56,12 +57,13 @@ |
1132 | "record_type": "http://fnord.org/smorgasbord", |
1133 | } |
1134 | self.record = Record(self.dict) |
1135 | - # pylint: enable-msg=C0103 |
1136 | + # pylint: enable=C0103 |
1137 | |
1138 | def test_revision(self): |
1139 | """Test document always has a revision field and that the revision |
1140 | changes when the document is updated""" |
1141 | self.assertEquals(self.record.record_revision, None) |
1142 | + |
1143 | def set_rev(rec): |
1144 | """Set revision.""" |
1145 | rec.record_revision = "1" |
1146 | @@ -126,8 +128,7 @@ |
1147 | ['a', 'b', 'record_type', 'subfield', 'subfield_uuid'], |
1148 | sorted(self.record.keys())) |
1149 | self.assertIn("a", self.record) |
1150 | - self.assertTrue(self.record.has_key("a")) |
1151 | - self.assertFalse(self.record.has_key("f")) |
1152 | + self.assertNotIn('f', self.record) |
1153 | self.assertNotIn("_id", self.record) # is internal. play dumb. |
1154 | |
1155 | def test_application_annotations(self): |
1156 | @@ -249,7 +250,7 @@ |
1157 | value = [1, 2, 3, 4, 5] |
1158 | self.record["subfield_uuid"] = value |
1159 | self.assertRaises(IndexError, self.record["subfield_uuid"].pop, |
1160 | - len(value)*2) |
1161 | + len(value) * 2) |
1162 | |
1163 | def test_mergeable_list_pop_last(self): |
1164 | """Test that exception is raised when poping last item""" |
1165 | @@ -272,9 +273,9 @@ |
1166 | |
1167 | def test_dictionary_access_to_mergeable_list(self): |
1168 | """Test that appropriate errors are raised.""" |
1169 | - # pylint: disable-msg=W0212 |
1170 | + # pylint: disable=W0212 |
1171 | keys = self.record["subfield_uuid"]._data.keys() |
1172 | - # pylint: enable-msg=W0212 |
1173 | + # pylint: enable=W0212 |
1174 | self.assertRaises( |
1175 | TypeError, |
1176 | self.record["subfield_uuid"].__getitem__, keys[0]) |
1177 | @@ -298,9 +299,9 @@ |
1178 | |
1179 | def test_uuid_like_keys(self): |
1180 | """Test that appropriate errors are raised.""" |
1181 | - # pylint: disable-msg=W0212 |
1182 | + # pylint: disable=W0212 |
1183 | keys = self.record["subfield_uuid"]._data.keys() |
1184 | - # pylint: enable-msg=W0212 |
1185 | + # pylint: enable=W0212 |
1186 | self.assertRaises( |
1187 | IllegalKeyException, |
1188 | self.record["subfield"].__setitem__, keys[0], 'stuff') |
1189 | @@ -313,7 +314,7 @@ |
1190 | def test_run_doctests(self): |
1191 | """Run all doc tests from here to set the proper context (ctx)""" |
1192 | ctx = test_environment.test_context |
1193 | - globs = { "db": CouchDatabase('testing', create=True, ctx=ctx) } |
1194 | + globs = {"db": CouchDatabase('testing', create=True, ctx=ctx)} |
1195 | |
1196 | records_tests_path = os.path.dirname( |
1197 | desktopcouch.__file__) + '/records/doc/records.txt' |
1198 | @@ -333,25 +334,24 @@ |
1199 | results = doctest.testfile( |
1200 | '../desktopcouch/records/doc/an_example_application.txt', |
1201 | module_relative=False) |
1202 | - |
1203 | self.assertEqual(0, results.failed) |
1204 | |
1205 | def test_record_id(self): |
1206 | """Test all passible way to assign a record id""" |
1207 | - data = {"_id":"recordid"} |
1208 | + data = {"_id": "recordid"} |
1209 | record = Record(data, record_type="url") |
1210 | self.assertEqual(data["_id"], record.record_id) |
1211 | data = {} |
1212 | record_id = "recordid" |
1213 | record = Record(data, record_type="url", record_id=record_id) |
1214 | self.assertEqual(record_id, record.record_id) |
1215 | - data = {"_id":"differentid"} |
1216 | + data = {"_id": "differentid"} |
1217 | self.assertRaises(ValueError, |
1218 | Record, data, record_id=record_id, record_type="url") |
1219 | |
1220 | def test_record_type_version(self): |
1221 | """Test record type version support""" |
1222 | - data = {"_id":"recordid"} |
1223 | + data = {"_id": "recordid"} |
1224 | record1 = Record(data, record_type="url") |
1225 | self.assertIs(None, record1.record_type_version) |
1226 | record2 = Record(data, record_type="url", record_type_version=1) |
1227 | @@ -361,7 +361,7 @@ |
1228 | class TestRecordFactory(TestCase): |
1229 | """Test Record/Mergeable List factories.""" |
1230 | |
1231 | - # pylint: disable-msg=C0103 |
1232 | + # pylint: disable=C0103 |
1233 | def setUp(self): |
1234 | """Test setup.""" |
1235 | super(TestRecordFactory, self).setUp() |
1236 | @@ -370,7 +370,7 @@ |
1237 | "b": "B", |
1238 | "subfield": { |
1239 | "field11s": "value11s", |
1240 | - "field12s": "value12s",}, |
1241 | + "field12s": "value12s"}, |
1242 | "subfield_uuid": |
1243 | [ |
1244 | {"field11": "value11", |
1245 | @@ -379,7 +379,7 @@ |
1246 | "field22": "value22"}], |
1247 | "record_type": "http://fnord.org/smorgasbord", |
1248 | } |
1249 | - # pylint: enable-msg=C0103 |
1250 | + # pylint: enable=C0103 |
1251 | |
1252 | def test_build(self): |
1253 | """Test RecordDict/MergeableList factory method.""" |
1254 | |
1255 | === modified file 'desktopcouch/records/tests/test_server.py' |
1256 | --- desktopcouch/records/tests/test_server.py 2010-11-03 22:57:06 +0000 |
1257 | +++ desktopcouch/records/tests/test_server.py 2010-11-12 14:40:00 +0000 |
1258 | @@ -32,27 +32,23 @@ |
1259 | from desktopcouch.platform import find_pid |
1260 | |
1261 | # pylint can't deal with failing imports even when they're handled |
1262 | -# pylint: disable-msg=F0401 |
1263 | +# pylint: disable=F0401 |
1264 | try: |
1265 | from io import StringIO |
1266 | except ImportError: |
1267 | from cStringIO import StringIO as StringIO |
1268 | -# pylint: enable-msg=F0401 |
1269 | +# pylint: enable=F0401 |
1270 | |
1271 | DCTRASH = 'dctrash' |
1272 | |
1273 | -FAKE_RECORD_TYPE = "http://example.org/test" |
1274 | - |
1275 | -js = """ |
1276 | -function(doc) { |
1277 | - if (doc.record_type == '%s') { |
1278 | - emit(doc._id, null); |
1279 | - } |
1280 | -}""" % FAKE_RECORD_TYPE |
1281 | - |
1282 | |
1283 | def get_test_context(): |
1284 | - return test_environment.test_context |
1285 | + """Return test context.""" |
1286 | + return test_environment.test_context |
1287 | + |
1288 | +# pylint: disable=W0212 |
1289 | +# I don't care about private members in tests. |
1290 | + |
1291 | |
1292 | class TestCouchDatabaseDeprecated(testtools.TestCase): |
1293 | """Test specific for CouchDatabase""" |
1294 | @@ -84,9 +80,11 @@ |
1295 | super(TestCouchDatabaseDeprecated, self).tearDown() |
1296 | |
1297 | def maybe_die(self): |
1298 | + """Method that could kill couchdb. Or could it? Or COULD it?""" |
1299 | pass |
1300 | |
1301 | - def wait_until_server_dead(self, pid=None): |
1302 | + def wait_until_server_dead(self, pid=None): # pylint: disable=R0201 |
1303 | + """Wait until the server is no longer breathing.""" |
1304 | if pid is not None: |
1305 | pid = find_pid( |
1306 | start_if_not_running=False, ctx=get_test_context()) |
1307 | @@ -102,11 +100,12 @@ |
1308 | def test_get_records_by_record_type_save_view(self): |
1309 | """Test getting mutliple records by type""" |
1310 | records = self.database.get_records( |
1311 | - record_type="test.com",create_view=True) |
1312 | + record_type="test.com", create_view=True) |
1313 | self.maybe_die() # should be able to survive couchdb death |
1314 | self.assertEqual(3, len(records)) |
1315 | |
1316 | def test_func_get_records(self): |
1317 | + """Functional test of get_records.""" |
1318 | record_ids_we_care_about = set() |
1319 | good_record_type = "http://example.com/unittest/good" |
1320 | other_record_type = "http://example.com/unittest/bad" |
1321 | @@ -145,18 +144,19 @@ |
1322 | design_doc="mustNotExist", create_view=False) |
1323 | |
1324 | def test_get_view_by_type_new_but_already(self): |
1325 | + """Test calling the view.""" |
1326 | self.database.get_records(create_view=True) |
1327 | self.maybe_die() # should be able to survive couchdb death |
1328 | self.database.get_records(create_view=True) |
1329 | # No exceptions on second run? Yay. |
1330 | |
1331 | def test_get_view_by_type_createxcl_fail(self): |
1332 | + """Test that calling the view without creating it fails.""" |
1333 | self.database.get_records(create_view=True) |
1334 | self.maybe_die() # should be able to survive couchdb death |
1335 | self.assertRaises(KeyError, self.database.get_records, create_view=None) |
1336 | |
1337 | |
1338 | - |
1339 | class TestCouchDatabase(testtools.TestCase): |
1340 | """tests specific for CouchDatabase""" |
1341 | |
1342 | @@ -187,9 +187,11 @@ |
1343 | super(TestCouchDatabase, self).tearDown() |
1344 | |
1345 | def maybe_die(self): |
1346 | + """Method that could kill couchdb. Or could it? Or COULD it?""" |
1347 | pass |
1348 | |
1349 | - def wait_until_server_dead(self, pid=None): |
1350 | + def wait_until_server_dead(self, pid=None): # pylint: disable=R0201 |
1351 | + """Wait until the server is no longer breathing.""" |
1352 | if pid is not None: |
1353 | pid = find_pid( |
1354 | start_if_not_running=False, ctx=get_test_context()) |
1355 | @@ -203,6 +205,7 @@ |
1356 | break |
1357 | |
1358 | def test_database_not_exists(self): |
1359 | + """Test that the database does not exist.""" |
1360 | self.assertRaises( |
1361 | NoSuchDatabase, CouchDatabase, "this-must-not-exist", create=False) |
1362 | |
1363 | @@ -251,7 +254,7 @@ |
1364 | # put the batch and ensure that the records have been added |
1365 | batch_result = self.database.put_records_batch(batch) |
1366 | for current_tuple in batch_result: |
1367 | - success, docid, rev_or_exc = current_tuple |
1368 | + success, docid, rev_or_exc = current_tuple # pylint: disable=W0612 |
1369 | if success: |
1370 | self.assertTrue(self.database._server[self.dbname][docid]) |
1371 | else: |
1372 | @@ -299,15 +302,6 @@ |
1373 | |
1374 | self.database.delete_record(new_record_id) |
1375 | |
1376 | - def test_get_deleted_record(self): |
1377 | - """Test (not) getting a deleted record.""" |
1378 | - record = Record({'record_number': 0}, record_type="http://example.com/") |
1379 | - record_id = self.database.put_record(record) |
1380 | - self.database.delete_record(record_id) |
1381 | - self.maybe_die() # should be able to survive couchdb death |
1382 | - retrieved_record = self.database.get_record(record_id) |
1383 | - self.assertEqual(None, retrieved_record) |
1384 | - |
1385 | def test_record_exists(self): |
1386 | """Test checking whether a record exists.""" |
1387 | record = Record({'record_number': 0}, record_type="http://example.com/") |
1388 | @@ -338,6 +332,7 @@ |
1389 | self.assertEqual(3, working_copy['field3']) |
1390 | |
1391 | def test_view_add_and_delete(self): |
1392 | + """Test adding and deleting a view.""" |
1393 | design_doc = "design" |
1394 | view1_name = "unit_tests_are_wonderful" |
1395 | view2_name = "unit_tests_are_marvelous" |
1396 | @@ -365,6 +360,7 @@ |
1397 | KeyError, self.database.delete_view, view2_name, design_doc) |
1398 | |
1399 | def test_func_get_all_records(self): |
1400 | + """Functional test of get_all_records.""" |
1401 | record_ids_we_care_about = set() |
1402 | good_record_type = "http://example.com/unittest/good" |
1403 | other_record_type = "http://example.com/unittest/bad" |
1404 | @@ -401,6 +397,7 @@ |
1405 | self.assertTrue(len(record_ids_we_care_about) == 0, "expected zero") |
1406 | |
1407 | def test_list_views(self): |
1408 | + """Test the list views.""" |
1409 | design_doc = "d" |
1410 | self.assertEqual(self.database.list_views(design_doc), []) |
1411 | |
1412 | @@ -415,12 +412,14 @@ |
1413 | self.assertEqual(self.database.list_views(design_doc), []) |
1414 | |
1415 | def test_get_view_by_type_new_but_already(self): |
1416 | + """Test calling the view.""" |
1417 | self.database.get_all_records() |
1418 | self.maybe_die() # should be able to survive couchdb death |
1419 | self.database.get_all_records() |
1420 | # No exceptions on second run? Yay. |
1421 | |
1422 | def test_get_changes(self): |
1423 | + """Test get_changes.""" |
1424 | self.test_put_record() |
1425 | self.test_update_fields() |
1426 | self.test_delete_record() |
1427 | @@ -435,7 +434,10 @@ |
1428 | self.failUnless("id" in change) |
1429 | |
1430 | def test_report_changes_polite(self): |
1431 | + """Test reporting changes.""" |
1432 | + |
1433 | def rep(**kwargs): |
1434 | + """Check the keyword arguments for changes and id.""" |
1435 | self.failUnless("changes" in kwargs) |
1436 | self.failUnless("id" in kwargs) |
1437 | |
1438 | @@ -453,7 +455,10 @@ |
1439 | self.assertEqual(0, count) |
1440 | |
1441 | def test_report_changes_exceptions(self): |
1442 | + """Test reporting changes in the face of exceptions.""" |
1443 | + |
1444 | def rep(**kwargs): |
1445 | + """Check the keyword arguments for changes and id.""" |
1446 | self.failUnless("changes" in kwargs) |
1447 | self.failUnless("id" in kwargs) |
1448 | |
1449 | @@ -470,7 +475,7 @@ |
1450 | |
1451 | # Exceptions in our callbacks do not consume an event. |
1452 | self.assertRaises( |
1453 | - ZeroDivisionError, self.database.report_changes, lambda **kw: 1/0) |
1454 | + ZeroDivisionError, self.database.report_changes, lambda **kw: 1 / 0) |
1455 | |
1456 | # Ensure pos'n is same. |
1457 | self.assertEqual(saved_position, self.database._changes_since) |
1458 | @@ -488,7 +493,10 @@ |
1459 | self.assertEqual(saved_position + 1, self.database._changes_since) |
1460 | |
1461 | def test_report_changes_all_ops_give_known_keys(self): |
1462 | + """Test reporting changes results have the right keys.""" |
1463 | + |
1464 | def rep(**kwargs): |
1465 | + """Check the keyword arguments for changes and id.""" |
1466 | self.failUnless("changes" in kwargs) |
1467 | self.failUnless("id" in kwargs) |
1468 | |
1469 | @@ -498,7 +506,10 @@ |
1470 | self.database.report_changes(rep) |
1471 | |
1472 | def test_report_changes_nochanges(self): |
1473 | + """Test report changes when there are none.""" |
1474 | + |
1475 | def rep(**kwargs): |
1476 | + """Check the keyword arguments for changes and id.""" |
1477 | self.failUnless("changes" in kwargs) |
1478 | self.failUnless("id" in kwargs) |
1479 | |
1480 | @@ -515,6 +526,7 @@ |
1481 | self.assertEqual(saved_position, self.database._changes_since) |
1482 | |
1483 | def test_attachments(self): |
1484 | + """Test attachments.""" |
1485 | content = StringIO("0123456789\n==========\n\n" * 5) |
1486 | |
1487 | constructed_record = Record( |
1488 | @@ -529,12 +541,12 @@ |
1489 | constructed_record.attach("string", "another document", "text/plain") |
1490 | |
1491 | self.maybe_die() # should be able to survive couchdb death |
1492 | - constructed_record.attach("XXXXXXXXX", "never used", "text/plain") |
1493 | + constructed_record.attach( |
1494 | + "SOMETHINGSOMETHING", "never used", "text/plain") |
1495 | constructed_record.detach("never used") # detach works before commit. |
1496 | |
1497 | # We can read from a document that we constructed. |
1498 | - out_file, out_content_type = \ |
1499 | - constructed_record.attachment_data("nu/mbe/rs") |
1500 | + _, out_content_type = constructed_record.attachment_data("nu/mbe/rs") |
1501 | self.assertEqual(out_content_type, "text/plain") |
1502 | |
1503 | # One can not put another document of the same name. |
1504 | @@ -604,6 +616,7 @@ |
1505 | self.database.delete_record(record_id) |
1506 | |
1507 | def test_view_fetch(self): |
1508 | + """Test view fetch.""" |
1509 | design_doc = "test_view_fetch" |
1510 | view1_name = "unit_tests_are_great_yeah" |
1511 | |
1512 | @@ -641,7 +654,7 @@ |
1513 | non_working_copy['field3'] = 3 |
1514 | self.database.put_record(non_working_copy) |
1515 | self.database.update_fields( |
1516 | - record_id, {'field1': 11,('nested', 'sub2'): 's2-changed'}, |
1517 | + record_id, {'field1': 11, ('nested', 'sub2'): 's2-changed'}, |
1518 | cached_record=record) |
1519 | working_copy = self.database.get_record(record_id) |
1520 | self.assertEqual(0, working_copy['record_number']) |
1521 | |
1522 | === modified file 'utilities/lint.sh' |
1523 | --- utilities/lint.sh 2010-11-11 19:01:41 +0000 |
1524 | +++ utilities/lint.sh 2010-11-12 14:40:00 +0000 |
1525 | @@ -37,7 +37,7 @@ |
1526 | fi |
1527 | |
1528 | export PYTHONPATH="/usr/share/pycentral/pylint/site-packages:desktopcouch:$PYTHONPATH" |
1529 | -pylint="`which pylint` -r n -i y --variable-rgx=[a-z_][a-z0-9_]{0,30} --method-rgx=[a-z_][a-z0-9_]{2,} -d I0011,R0904,R0913" |
1530 | +pylint="`which pylint` -r n -i y --variable-rgx=[a-z_][a-z0-9_]{0,30} --attr-rgx=[a-z_][a-z0-9_]{1,30} --method-rgx=[a-z_][a-z0-9_]{2,} -d I0011,R0902,R0904,R0913,W0142" |
1531 | |
1532 | pylint_notices=`$pylint $pyfiles` |
1533 |
90% of the diff comes from delinting, you'll be happy to know.