Merge lp:~jderose/dmedia/core-api into lp:dmedia
- core-api
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 183 |
Proposed branch: | lp:~jderose/dmedia/core-api |
Merge into: | lp:dmedia |
Diff against target: |
3658 lines (+1603/-1016) 32 files modified
MANIFEST.in (+4/-3) dmedia-cli (+69/-167) dmedia-gtk (+46/-37) dmedia-importer-service (+47/-46) dmedia-service (+72/-0) dmedia/__init__.py (+1/-0) dmedia/abstractcouch.py (+22/-34) dmedia/api.py (+87/-0) dmedia/constants.py (+3/-1) dmedia/core.py (+195/-0) dmedia/filestore.py (+31/-36) dmedia/gtkui/client.py (+19/-19) dmedia/gtkui/service.py (+23/-28) dmedia/gtkui/tests/test_client.py (+10/-9) dmedia/gtkui/tests/test_service.py (+0/-53) dmedia/importer.py (+5/-14) dmedia/schema.py (+19/-3) dmedia/service/__init__.py (+94/-0) dmedia/tests/couch.py (+5/-3) dmedia/tests/test_abstractcouch.py (+72/-143) dmedia/tests/test_api.py (+125/-0) dmedia/tests/test_core.py (+336/-0) dmedia/tests/test_downloader.py (+8/-8) dmedia/tests/test_filestore.py (+106/-77) dmedia/tests/test_importer.py (+4/-4) dmedia/tests/test_transcoder.py (+2/-2) dmedia/tests/test_views.py (+65/-136) dmedia/views.py (+76/-162) dmedia/workers.py (+6/-6) setup.py (+46/-23) share/org.freedesktop.DMedia.service (+3/-0) share/org.freedesktop.DMediaImporter.service (+2/-2) |
To merge this branch: | bzr merge lp:~jderose/dmedia/core-api |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
James Raymond | Approve | ||
Review via email: mp+57148@code.launchpad.net |
Commit message
Description of the change
This is just for fun so everyone (including myself) can gasp at how many lines this diff is probably going to be.
But *all* the unit tests pass, including that one pesky client => dbus test that has been failing for the last 2 months. I've beat up on this a lot, has been solid. Now it's time for the daily build users to abuse it as best they can. Huge change, sorry. Now I'll get back to saner merges, promise.
Lots of awesome changes, too tired to explain them all right now. Oh, but one cool one... you can now run dmedia against system-wide CouchDB if you want. Just launch `dmedia-service` with the --no-dc option. This wont work with DBus itself launching it, though, must be manual. Will need to add a config file or somesuch.
Jason Gerard DeRose (jderose) wrote : | # |
Thanks James, you deserve to be Knighted for reading through all that!
Preview Diff
1 | === modified file 'MANIFEST.in' |
2 | --- MANIFEST.in 2011-03-27 14:12:36 +0000 |
3 | +++ MANIFEST.in 2011-04-11 12:02:24 +0000 |
4 | @@ -1,4 +1,5 @@ |
5 | -include AUTHORS COPYING HACKING.txt README.txt |
6 | -include dmedia-service |
7 | -include dmedia/webui/data/* |
8 | +include AUTHORS COPYING HACKING.txt |
9 | +include dmedia/tests/*.py |
10 | include dmedia/tests/data/* |
11 | +include dmedia/webui/tests/*.py |
12 | +include dmedia/gtkui/tests/*.py |
13 | |
14 | === modified file 'dmedia-cli' |
15 | --- dmedia-cli 2010-12-29 13:58:00 +0000 |
16 | +++ dmedia-cli 2011-04-11 12:02:24 +0000 |
17 | @@ -5,7 +5,7 @@ |
18 | # David Green <david4dev@gmail.com> |
19 | # |
20 | # dmedia: distributed media library |
21 | -# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com> |
22 | +# Copyright (C) 2010, 2011 Jason Gerard DeRose <jderose@novacut.com> |
23 | # |
24 | # This file is part of `dmedia`. |
25 | # |
26 | @@ -24,174 +24,76 @@ |
27 | |
28 | |
29 | """ |
30 | -WARNING: the dmedia content-hash and schema are *not* yet stable, may change |
31 | -wildly and without warning! |
32 | - |
33 | -This will eventually be turned into a VCS-style script with several commands. |
34 | -For now it has a single function, to recursively import media files from a |
35 | -directory. At this point, is is a quick-and-dirty demo of how media files might |
36 | -be stored and how their meta-data might be stored. |
37 | - |
38 | -For example, say we scan all the JPEG images in the '/usr/share/backgrounds' |
39 | -directory: |
40 | - |
41 | - dmedia /usr/share/backgrounds jpg |
42 | - |
43 | -Media files are identified by their unique Id which is generated by their |
44 | -base32-encoded sha1 content-hash. In the above example, the |
45 | -'Life_by_Paco_Espinoza.jpg' file happens to have a sha1 content-hash of |
46 | -'6BRRXCGRM2GKVPTREJPGRNGUR2GF2L4K'. As such, this file will be stored at: |
47 | - |
48 | - ~/.dmedia/6B/RRXCGRM2GKVPTREJPGRNGUR2GF2L4K.jpg |
49 | - |
50 | -Meta-data for the media files is stored in CouchDB using desktopcouch. Each |
51 | -media file has its own document. The sha1 content-hash is used as the document |
52 | -'_id'. For example, the 'Life_by_Paco_Espinoza.jpg' file has a document that |
53 | -looks like this: |
54 | - |
55 | - { |
56 | - "_id": "6BRRXCGRM2GKVPTREJPGRNGUR2GF2L4K", |
57 | - "_rev": "1-c19ea015eb53ede147d63d55f3967d13", |
58 | - "name": "Life_by_Paco_Espinoza.jpg", |
59 | - "record_type": "http://example.com/dmedia", |
60 | - "bytes": 360889, |
61 | - "height": 1500, |
62 | - "shutter": "1/400", |
63 | - "width": 2000, |
64 | - "ext": "jpg", |
65 | - "camera": "DSC-H5", |
66 | - "iso": 125, |
67 | - "focal_length": "6.0 mm", |
68 | - "mtime": 1284394022, |
69 | - "aperture": 4 |
70 | - } |
71 | - |
72 | -All media files will have the following fields: |
73 | - |
74 | - bytes - File size in bytes |
75 | - mtime - Value of path.getmtime() at time of import |
76 | - name - The path.basename() part of the original source file |
77 | - ext - The extension of the original source file, normalized to lower-case |
78 | - |
79 | -Additional fields depend upon the type of media file. For example, image and |
80 | -video files will always have 'width' and 'height', whereas video and audio files |
81 | -will always have a 'duration'. |
82 | - |
83 | -You can browse through the dmedia database using a standard web-browser, like |
84 | -this: |
85 | - |
86 | - xdg-open ~/.local/share/desktop-couch/couchdb.html |
87 | - |
88 | -Note that the sha1 hash is only being used as a stop-gap. dmedia will use the |
89 | -Skein hash after its final constant tweaks are made. See: |
90 | - |
91 | - http://blog.novacut.com/2010/09/how-about-that-skein-hash.html |
92 | +Command line tool for talking to dmedia DBus services. |
93 | """ |
94 | |
95 | from __future__ import print_function |
96 | |
97 | -import sys |
98 | -import os |
99 | -from os import path |
100 | -import optparse |
101 | -import xdg.BaseDirectory |
102 | -import gobject |
103 | +import argparse |
104 | +import json |
105 | + |
106 | import dmedia |
107 | -from dmedia.client import Client |
108 | - |
109 | -script = path.basename(sys.argv[0]) |
110 | - |
111 | -parser = optparse.OptionParser( |
112 | - version=dmedia.__version__, |
113 | - usage='Usage: %s DIRECTORY [EXTENSIONS...]' % script, |
114 | - epilog='Example: %s /media/EOS_DIGITAL jpg cr2 mov' % script, |
115 | -) |
116 | -parser.add_option('--kill', |
117 | - action='store_true', |
118 | - default=False, |
119 | - help='shutdown dmedia-service daemon and exit', |
120 | -) |
121 | -parser.add_option('--quick', |
122 | - dest='extract', |
123 | - action='store_false', |
124 | - default=True, |
125 | - help='import without metadata extraction or thumbnail generation', |
126 | -) |
127 | -parser.add_option('--type', |
128 | - action='append', |
129 | - default=[], |
130 | - help='import all files of TYPE ("video", "audio", or "image")' |
131 | -) |
132 | -(options, args) = parser.parse_args() |
133 | - |
134 | - |
135 | -# Check if they just want us to kill the daemon |
136 | -if options.kill: |
137 | - print('Telling dmedia-service daemon to shutdown...') |
138 | - Client().kill() |
139 | - print('Shutdown.') |
140 | - sys.exit() |
141 | - |
142 | - |
143 | -if len(args) < 1: |
144 | - parser.print_usage() |
145 | - sys.exit('ERROR: must provide DIRECTORY') |
146 | -base = path.abspath(args[0]) |
147 | -if not path.isdir(base): |
148 | - parser.print_usage() |
149 | - sys.exit('ERROR: not a directory: %r' % base) |
150 | - |
151 | - |
152 | -extensions = list(a.lower().strip('.') for a in args[1:]) |
153 | - |
154 | - |
155 | -class CLI(object): |
156 | - def __init__(self, base): |
157 | - self.base = base |
158 | - self.mainloop = gobject.MainLoop() |
159 | - self.client = Client() |
160 | - self.client.connect('import_started', self.on_import_started) |
161 | - self.client.connect('import_count', self.on_import_count) |
162 | - self.client.connect('import_progress', self.on_import_progress) |
163 | - self.client.connect('import_finished', self.on_import_finished) |
164 | - |
165 | - def on_import_started(self, client, base, import_id): |
166 | - if base != self.base: |
167 | - return |
168 | - print('Started import of %s' % base) |
169 | - print (' import_id = %s' % import_id) |
170 | - |
171 | - def on_import_count(self, client, base, import_id, total): |
172 | - if base != self.base: |
173 | - return |
174 | - print('Found %d files' % total) |
175 | - |
176 | - def on_import_progress(self, c, base, import_id, completed, total, info): |
177 | - if base != self.base: |
178 | - return |
179 | - print(' %(action)s %(src)s' % info) |
180 | - |
181 | - def on_import_finished(self, c, base, import_id, stats): |
182 | - if base != self.base: |
183 | - return |
184 | - print('-' * 80) |
185 | - for key in sorted(stats): |
186 | - print('%s=%d' % (key, stats[key])) |
187 | - self.mainloop.quit() |
188 | - |
189 | - def run(self, options): |
190 | - try: |
191 | - self.client.start_import(self.base, options.extract) |
192 | - self.mainloop.run() |
193 | - except KeyboardInterrupt: |
194 | - self.client.stop_import(self.base) |
195 | - print('\nImport stopped') |
196 | - |
197 | - couchdb_data_path = xdg.BaseDirectory.save_data_path('desktop-couch') |
198 | - dc = path.join(couchdb_data_path, 'couchdb.html') |
199 | - print('\nBrowse the `dmedia` database in CouchDB:') |
200 | - print(' file://%s\n' % dc) |
201 | - |
202 | - |
203 | -cli = CLI(base) |
204 | -cli.run(options) |
205 | +from dmedia.constants import BUS |
206 | + |
207 | + |
208 | +parser = argparse.ArgumentParser( |
209 | + description='Execute methods on dmedia DBus services', |
210 | +) |
211 | +parser.add_argument('--version', action='version', version=dmedia.__version__) |
212 | +parser.add_argument('--bus', |
213 | + help='DBus bus name; default is %(default)r', |
214 | + default=BUS, |
215 | +) |
216 | + |
217 | +subparsers = parser.add_subparsers( |
218 | + title='Commands from {!r}'.format(BUS) |
219 | +) |
220 | + |
221 | + |
222 | +p_version = subparsers.add_parser('version', |
223 | + help='get version of running dmedia service', |
224 | +) |
225 | +def do_version(dm, args): |
226 | + print( |
227 | + '{} {}'.format(args.bus, dm.version()) |
228 | + ) |
229 | +p_version.set_defaults(func=do_version) |
230 | + |
231 | + |
232 | +p_kill = subparsers.add_parser('kill', |
233 | + help='kill `dmedia-service`', |
234 | +) |
235 | +def do_kill(dm, args): |
236 | + print('Killing {}...'.format(args.bus)) |
237 | + dm.kill() |
238 | +p_kill.set_defaults(func=do_kill) |
239 | + |
240 | + |
241 | +p_get_env = subparsers.add_parser('get-env', |
242 | + help='echo out JSON encoded env dict', |
243 | +) |
244 | +def do_get_env(dm, args): |
245 | + print(json.dumps(dm.get_env(), sort_keys=True, indent=2)) |
246 | +p_get_env.set_defaults(func=do_get_env) |
247 | + |
248 | + |
249 | +p_get_auth_url = subparsers.add_parser('get-auth-url', |
250 | + help='echo desktopcouch basic auth URL', |
251 | +) |
252 | +def do_get_auth_url(dm, args): |
253 | + print(dm.get_auth_url()) |
254 | +p_get_auth_url.set_defaults(func=do_get_auth_url) |
255 | + |
256 | + |
257 | +p_has_app = subparsers.add_parser('has-app', |
258 | + help='show whether the WebUI app is available', |
259 | +) |
260 | +def do_has_app(dm, args): |
261 | + print(bool(dm.has_app())) |
262 | +p_has_app.set_defaults(func=do_has_app) |
263 | + |
264 | + |
265 | +args = parser.parse_args() |
266 | +from dmedia.api import DMedia |
267 | +dm = DMedia(args.bus) |
268 | +args.func(dm, args) |
269 | |
270 | === modified file 'dmedia-gtk' |
271 | --- dmedia-gtk 2011-03-28 21:55:42 +0000 |
272 | +++ dmedia-gtk 2011-04-11 12:02:24 +0000 |
273 | @@ -4,7 +4,7 @@ |
274 | # Jason Gerard DeRose <jderose@novacut.com> |
275 | # |
276 | # dmedia: distributed media library |
277 | -# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com> |
278 | +# Copyright (C) 2010, 2011 Jason Gerard DeRose <jderose@novacut.com> |
279 | # |
280 | # This file is part of `dmedia`. |
281 | # |
282 | @@ -21,48 +21,58 @@ |
283 | # You should have received a copy of the GNU Affero General Public License along |
284 | # with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
285 | |
286 | -import sys |
287 | -from subprocess import check_call |
288 | -import optparse |
289 | +from __future__ import print_function |
290 | + |
291 | +import argparse |
292 | +from gettext import gettext as _ |
293 | + |
294 | import dmedia |
295 | - |
296 | -parser = optparse.OptionParser( |
297 | - version=dmedia.__version__, |
298 | +from dmedia.constants import BUS |
299 | +from dmedia.api import DMedia |
300 | + |
301 | + |
302 | +BROWSER = 'dmedia/app/browser' |
303 | + |
304 | +parser = argparse.ArgumentParser( |
305 | + description='Manage media files in your dmedia library', |
306 | ) |
307 | -parser.add_option('--browser', |
308 | +parser.add_argument('--version', action='version', version=dmedia.__version__) |
309 | +parser.add_argument('--browser', |
310 | + help='open dmedia HTML5 UI in default browser', |
311 | action='store_true', |
312 | default=False, |
313 | - help='open dmedia HTML5 UI in default browser', |
314 | -) |
315 | -(options, args) = parser.parse_args() |
316 | - |
317 | -from dmedia.webui.app import App |
318 | -from dmedia.abstractcouch import get_env |
319 | -from dmedia.metastore import MetaStore |
320 | - |
321 | -app = App() |
322 | -env = get_env() |
323 | -store = MetaStore(env) |
324 | -store.update(app.get_doc()) |
325 | - |
326 | - |
327 | -if options.browser: |
328 | - uri = store.get_auth_uri() + '/dmedia/app/browser' |
329 | - check_call(['xdg-open', uri]) |
330 | - sys.exit() |
331 | - |
332 | -uri = store.get_uri() + '/dmedia/app/browser' |
333 | - |
334 | +) |
335 | +parser.add_argument('--bus', |
336 | + help='DBus bus name; default is %(default)r', |
337 | + default=BUS, |
338 | +) |
339 | +parser.add_argument('--att', |
340 | + help='CouchDB attachment path; default is %(default)r', |
341 | + default=BROWSER, |
342 | +) |
343 | + |
344 | +args = parser.parse_args() |
345 | +dm = DMedia(args.bus) |
346 | + |
347 | +if not dm.has_app(): |
348 | + print('Oops, cannot import dmedia.webui.app!') |
349 | + raise SystemExit(2) |
350 | + |
351 | + |
352 | +if args.browser: |
353 | + url = dm.get_auth_url() + args.att |
354 | + import subprocess |
355 | + subprocess.check_call(['/usr/bin/xdg-open', url]) |
356 | + raise SystemExit() |
357 | + |
358 | + |
359 | +env = dm.get_env() |
360 | |
361 | from dmedia.gtkui.widgets import CouchView |
362 | from gi.repository import Gtk, GObject |
363 | |
364 | - |
365 | -GObject.threads_init() |
366 | - |
367 | - |
368 | window = Gtk.Window() |
369 | -window.set_title('test') |
370 | +window.set_title(_('Media Browser')) |
371 | window.set_default_size(960, 540) |
372 | window.connect('destroy', Gtk.main_quit) |
373 | |
374 | @@ -70,11 +80,10 @@ |
375 | scroll.set_policy(Gtk.PolicyType.AUTOMATIC, Gtk.PolicyType.AUTOMATIC) |
376 | window.add(scroll) |
377 | |
378 | -view = CouchView(env['url'], env['oauth']) |
379 | +view = CouchView(env['url'], env.get('oauth')) |
380 | scroll.add(view) |
381 | |
382 | window.show_all() |
383 | -view.load_uri(uri) |
384 | - |
385 | +view.load_uri(env['url'] + args.att) |
386 | |
387 | Gtk.main() |
388 | |
389 | === renamed file 'dmedia-service' => 'dmedia-importer-service' |
390 | --- dmedia-service 2011-03-27 10:45:13 +0000 |
391 | +++ dmedia-importer-service 2011-04-11 12:02:24 +0000 |
392 | @@ -21,53 +21,54 @@ |
393 | # You should have received a copy of the GNU Affero General Public License along |
394 | # with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
395 | |
396 | -import sys |
397 | -import optparse |
398 | - |
399 | -import dbus |
400 | -from dbus.mainloop.glib import DBusGMainLoop |
401 | - |
402 | -DBusGMainLoop(set_as_default=True) |
403 | +""" |
404 | +dmedia importer DBus service at "org.freedesktop.DMedia.Importer". |
405 | +""" |
406 | + |
407 | +import argparse |
408 | +import json |
409 | +import time |
410 | |
411 | import dmedia |
412 | -from dmedia.constants import BUS, DBNAME |
413 | -from dmedia.abstractcouch import get_env |
414 | - |
415 | - |
416 | -parser = optparse.OptionParser(version=dmedia.__version__) |
417 | -parser.add_option('--bus', |
418 | - default=BUS, |
419 | - help='D-Bus bus name; default is %r' % BUS, |
420 | -) |
421 | -parser.add_option('--dbname', |
422 | - metavar='DIR', |
423 | - default=DBNAME, |
424 | - help='CouchDB database; default is %r' % DBNAME, |
425 | -) |
426 | -parser.add_option('--no-gui', |
427 | +from dmedia.constants import BUS, IMPORT_BUS |
428 | + |
429 | + |
430 | +parser = argparse.ArgumentParser( |
431 | + description='DBus service @{}'.format(IMPORT_BUS), |
432 | +) |
433 | +parser.add_argument('--version', action='version', version=dmedia.__version__) |
434 | +parser.add_argument('--no-gui', |
435 | + help='run without NotifyOSD and Application Indicators', |
436 | action='store_true', |
437 | default=False, |
438 | - help='run without NotifyOSD and Application Indicators', |
439 | -) |
440 | - |
441 | - |
442 | -def exit(msg, code=1): |
443 | - parser.print_help() |
444 | - print('ERROR: ' + msg) |
445 | - sys.exit(code) |
446 | - |
447 | - |
448 | -if __name__ == '__main__': |
449 | - (options, args) = parser.parse_args() |
450 | - |
451 | - dmedia.configure_logging('service') |
452 | - |
453 | - env = get_env(options.dbname) |
454 | - env['bus'] = options.bus |
455 | - env['no_gui'] = options.no_gui |
456 | - |
457 | - from dmedia.gtkui import service |
458 | - from gi.repository import GObject |
459 | - mainloop = GObject.MainLoop() |
460 | - obj = service.DMedia(env, mainloop.quit) |
461 | - mainloop.run() |
462 | +) |
463 | +parser.add_argument('--bus', |
464 | + help='DBus bus name; default is %(default)r', |
465 | + default=IMPORT_BUS, |
466 | +) |
467 | +parser.add_argument('--env', |
468 | + help='dmedia environment as JSON serialized string', |
469 | + metavar='JSON', |
470 | +) |
471 | + |
472 | +args = parser.parse_args() |
473 | +dmedia.configure_logging('importer') |
474 | + |
475 | +from dbus.mainloop.glib import DBusGMainLoop |
476 | +DBusGMainLoop(set_as_default=True) |
477 | + |
478 | +from gi.repository import GObject |
479 | +GObject.threads_init() |
480 | + |
481 | +from dmedia import api |
482 | +dm = api.DMedia() |
483 | + |
484 | +env = dm.get_env(args.env) |
485 | +if args.no_gui: |
486 | + env['no_gui'] = True |
487 | + |
488 | +from dmedia.gtkui.service import DMedia |
489 | + |
490 | +mainloop = GObject.MainLoop() |
491 | +obj = DMedia(env, args.bus, mainloop.quit) |
492 | +mainloop.run() |
493 | |
494 | === added file 'dmedia-service' |
495 | --- dmedia-service 1970-01-01 00:00:00 +0000 |
496 | +++ dmedia-service 2011-04-11 12:02:24 +0000 |
497 | @@ -0,0 +1,72 @@ |
498 | +#!/usr/bin/env python |
499 | + |
500 | +# Authors: |
501 | +# Jason Gerard DeRose <jderose@novacut.com> |
502 | +# |
503 | +# dmedia: distributed media library |
504 | +# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com> |
505 | +# |
506 | +# This file is part of `dmedia`. |
507 | +# |
508 | +# `dmedia` is free software: you can redistribute it and/or modify it under the |
509 | +# terms of the GNU Affero General Public License as published by the Free |
510 | +# Software Foundation, either version 3 of the License, or (at your option) any |
511 | +# later version. |
512 | +# |
513 | +# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY |
514 | +# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR |
515 | +# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more |
516 | +# details. |
517 | +# |
518 | +# You should have received a copy of the GNU Affero General Public License along |
519 | +# with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
520 | + |
521 | +""" |
522 | +Core dmedia DBus service at "org.freedesktop.DMedia". |
523 | +""" |
524 | + |
525 | +import argparse |
526 | +import time |
527 | + |
528 | +import dmedia |
529 | +from dmedia.constants import BUS, DBNAME |
530 | + |
531 | + |
532 | +parser = argparse.ArgumentParser( |
533 | + description='DBus service @{}'.format(BUS), |
534 | +) |
535 | +parser.add_argument('--version', action='version', version=dmedia.__version__) |
536 | +parser.add_argument('--no-dc', |
537 | + action='store_true', |
538 | + default=False, |
539 | + help='Use system-wide CouchDB instead of desktopcouch', |
540 | +) |
541 | +parser.add_argument('--bus', |
542 | + default=BUS, |
543 | + help='DBus bus name; default is %(default)r', |
544 | +) |
545 | +parser.add_argument('--dbname', |
546 | + metavar='DB', |
547 | + default=DBNAME, |
548 | + help='CouchDB database; default is %(default)r', |
549 | +) |
550 | +parser.add_argument('--env', |
551 | + help='dmedia environment as JSON serialized string', |
552 | + metavar='JSON', |
553 | +) |
554 | + |
555 | +args = parser.parse_args() |
556 | +dmedia.configure_logging('service') |
557 | + |
558 | +from dbus.mainloop.glib import DBusGMainLoop |
559 | +DBusGMainLoop(set_as_default=True) |
560 | + |
561 | +from gi.repository import GObject |
562 | +GObject.threads_init() |
563 | + |
564 | +from dmedia.service import DMedia |
565 | + |
566 | +mainloop = GObject.MainLoop() |
567 | +couchargs = (args.dbname, args.no_dc, args.env) |
568 | +obj = DMedia(couchargs, args.bus, mainloop.quit, start=time.time()) |
569 | +mainloop.run() |
570 | |
571 | === modified file 'dmedia/__init__.py' |
572 | --- dmedia/__init__.py 2011-04-01 01:28:38 +0000 |
573 | +++ dmedia/__init__.py 2011-04-11 12:02:24 +0000 |
574 | @@ -58,3 +58,4 @@ |
575 | level=logging.DEBUG, |
576 | format='\t'.join(format), |
577 | ) |
578 | + logging.info('dmedia %s, namespace=%r', __version__, namespace) |
579 | |
580 | === modified file 'dmedia/abstractcouch.py' |
581 | --- dmedia/abstractcouch.py 2011-02-21 11:49:14 +0000 |
582 | +++ dmedia/abstractcouch.py 2011-04-11 12:02:24 +0000 |
583 | @@ -37,6 +37,7 @@ |
584 | https://bugs.launchpad.net/dmedia/+bug/722035 |
585 | """ |
586 | |
587 | +import json |
588 | import logging |
589 | |
590 | from couchdb import Server, ResourceNotFound |
591 | @@ -44,15 +45,12 @@ |
592 | from desktopcouch.records.http import OAuthSession |
593 | except ImportError: |
594 | OAuthSession = None |
595 | -if OAuthSession is not None: |
596 | - from desktopcouch.application.platform import find_port |
597 | - from desktopcouch.application.local_files import get_oauth_tokens |
598 | |
599 | |
600 | log = logging.getLogger() |
601 | |
602 | |
603 | -def get_couchdb_server(env): |
604 | +def get_server(env): |
605 | """ |
606 | Return `couchdb.Server` for desktopcouch or system-wide CouchDB. |
607 | |
608 | @@ -98,11 +96,9 @@ |
609 | The goal is to have all the needed information is one easily serialized |
610 | piece of data (important for testing across multiple processes). |
611 | """ |
612 | - url = env.get('url', 'http://localhost:5984/') |
613 | + url = env['url'] |
614 | log.info('CouchDB server is %r', url) |
615 | - if env.get('oauth') is None: |
616 | - session = None |
617 | - else: |
618 | + if 'oauth' in env: |
619 | if OAuthSession is None: |
620 | raise ValueError( |
621 | "provided env['oauth'] but OAuthSession not available: %r" % |
622 | @@ -110,29 +106,27 @@ |
623 | ) |
624 | log.info('Using desktopcouch `OAuthSession`') |
625 | session = OAuthSession(credentials=env['oauth']) |
626 | + else: |
627 | + session = None |
628 | return Server(url, session=session) |
629 | |
630 | |
631 | -def get_dmedia_db(env, server=None): |
632 | +def get_db(env, server=None): |
633 | """ |
634 | - Return the dmedia database specified by *env*. |
635 | + Return the CouchDB database specified by *env*. |
636 | |
637 | - The database name is determined by ``env['dbname']``. If the ``"dbname"`` |
638 | - key is missing or is ``None``, the default database name ``"dmedia"`` is |
639 | - used. |
640 | + The database name is determined by ``env['dbname']``. |
641 | |
642 | If the database does not exist, it will be created. |
643 | |
644 | If *server* is ``None``, one is created based on *env* by calling |
645 | - `get_couchdb_server()`. |
646 | + `get_server()`. |
647 | |
648 | Returns a ``couchdb.Database`` instance. |
649 | """ |
650 | if server is None: |
651 | - server = get_couchdb_server(env) |
652 | - dbname = env.get('dbname') |
653 | - if dbname is None: |
654 | - dbname = 'dmedia' |
655 | + server = get_server(env) |
656 | + dbname = env['dbname'] |
657 | log.info('CouchDB database is %r', dbname) |
658 | try: |
659 | return server[dbname] |
660 | @@ -140,19 +134,13 @@ |
661 | return server.create(dbname) |
662 | |
663 | |
664 | -def get_env(dbname=None): |
665 | - """ |
666 | - Return default *env*. |
667 | - |
668 | - This will return an appropriate *env* based on whether desktopcouch is |
669 | - available. Not a perfect solution, but works for now. |
670 | - """ |
671 | - if OAuthSession is None: |
672 | - return {'dbname': dbname} |
673 | - port = find_port() |
674 | - return { |
675 | - 'port': port, |
676 | - 'url': 'http://localhost:%d/' % port, |
677 | - 'oauth': get_oauth_tokens(), |
678 | - 'dbname': dbname, |
679 | - } |
680 | +def load_env(env_s): |
681 | + env = json.loads(env_s) |
682 | + # FIXME: hack to work-around for Python oauth not working with unicode, |
683 | + # which is what we get when the env is retrieved over D-Bus as JSON |
684 | + if 'oauth' in env: |
685 | + env['oauth'] = dict( |
686 | + (k.encode('ascii'), v.encode('ascii')) |
687 | + for (k, v) in env['oauth'].iteritems() |
688 | + ) |
689 | + return env |
690 | |
691 | === added file 'dmedia/api.py' |
692 | --- dmedia/api.py 1970-01-01 00:00:00 +0000 |
693 | +++ dmedia/api.py 2011-04-11 12:02:24 +0000 |
694 | @@ -0,0 +1,87 @@ |
695 | +# Authors: |
696 | +# Jason Gerard DeRose <jderose@novacut.com> |
697 | +# |
698 | +# dmedia: distributed media library |
699 | +# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com> |
700 | +# |
701 | +# This file is part of `dmedia`. |
702 | +# |
703 | +# `dmedia` is free software: you can redistribute it and/or modify it under the |
704 | +# terms of the GNU Affero General Public License as published by the Free |
705 | +# Software Foundation, either version 3 of the License, or (at your option) any |
706 | +# later version. |
707 | +# |
708 | +# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY |
709 | +# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR |
710 | +# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more |
711 | +# details. |
712 | +# |
713 | +# You should have received a copy of the GNU Affero General Public License along |
714 | +# with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
715 | + |
716 | +""" |
717 | +Python convenience API for talking to dmedia components over DBus. |
718 | +""" |
719 | + |
720 | +import json |
721 | + |
722 | +import dbus |
723 | + |
724 | +from dmedia.constants import BUS |
725 | + |
726 | +# On my system, this takes around 0.15 seconds, almost all because of |
727 | +# >>> from desktopcouch.records.http import OAuthSession |
728 | +# |
729 | +#from dmedia.abstractcouch import load_env |
730 | + |
731 | + |
732 | +def load_env(env_s): |
733 | + env = json.loads(env_s) |
734 | + # FIXME: hack to work-around for Python oauth not working with unicode, |
735 | + # which is what we get when the env is retrieved over D-Bus as JSON |
736 | + if 'oauth' in env: |
737 | + env['oauth'] = dict( |
738 | + (k.encode('ascii'), v.encode('ascii')) |
739 | + for (k, v) in env['oauth'].iteritems() |
740 | + ) |
741 | + return env |
742 | + |
743 | + |
744 | +class DMedia(object): |
745 | + """ |
746 | + Talk to "org.freedesktop.DMedia". |
747 | + """ |
748 | + def __init__(self, bus=BUS): |
749 | + self.bus = bus |
750 | + self.conn = dbus.SessionBus() |
751 | + self._proxy = None |
752 | + |
753 | + @property |
754 | + def proxy(self): |
755 | + if self._proxy is None: |
756 | + self._proxy = self.conn.get_object(self.bus, '/') |
757 | + return self._proxy |
758 | + |
759 | + def version(self): |
760 | + return self.proxy.Version() |
761 | + |
762 | + def kill(self): |
763 | + self.proxy.Kill() |
764 | + self._proxy = None |
765 | + |
766 | + def get_env(self, env_s=None): |
767 | + if not env_s: |
768 | + env_s = self.proxy.GetEnv() |
769 | + return load_env(env_s) |
770 | + |
771 | + def get_auth_url(self): |
772 | + return self.proxy.GetAuthURL() |
773 | + |
774 | + def has_app(self): |
775 | + return self.proxy.HasApp() |
776 | + |
777 | + |
778 | +class DMediaImporter(object): |
779 | + """ |
780 | + Talk to "org.freedesktop.DMediaImporter". |
781 | + """ |
782 | |
783 | === modified file 'dmedia/constants.py' |
784 | --- dmedia/constants.py 2011-02-25 16:38:29 +0000 |
785 | +++ dmedia/constants.py 2011-04-11 12:02:24 +0000 |
786 | @@ -48,7 +48,9 @@ |
787 | |
788 | # D-Bus releated: |
789 | BUS = 'org.freedesktop.DMedia' |
790 | -INTERFACE = BUS |
791 | +IFACE = BUS |
792 | +IMPORT_BUS = 'org.freedesktop.DMediaImporter' |
793 | +IMPORT_IFACE = IMPORT_BUS |
794 | DC_BUS = 'org.desktopcouch.CouchDB' |
795 | DC_INTERFACE = DC_BUS |
796 | |
797 | |
798 | === added file 'dmedia/core.py' |
799 | --- dmedia/core.py 1970-01-01 00:00:00 +0000 |
800 | +++ dmedia/core.py 2011-04-11 12:02:24 +0000 |
801 | @@ -0,0 +1,195 @@ |
802 | +# Authors: |
803 | +# Jason Gerard DeRose <jderose@novacut.com> |
804 | +# |
805 | +# dmedia: distributed media library |
806 | +# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com> |
807 | +# |
808 | +# This file is part of `dmedia`. |
809 | +# |
810 | +# `dmedia` is free software: you can redistribute it and/or modify it under the |
811 | +# terms of the GNU Affero General Public License as published by the Free |
812 | +# Software Foundation, either version 3 of the License, or (at your option) any |
813 | +# later version. |
814 | +# |
815 | +# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY |
816 | +# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR |
817 | +# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more |
818 | +# details. |
819 | +# |
820 | +# You should have received a copy of the GNU Affero General Public License along |
821 | +# with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
822 | + |
823 | +""" |
824 | +Core dmedia entry-point/API - start here! |
825 | + |
826 | +For background, please see: |
827 | + |
828 | + https://bugs.launchpad.net/dmedia/+bug/753260 |
829 | + |
830 | + |
831 | +Security note on /dmedia/_local/dmedia |
832 | +====================================== |
833 | + |
834 | +When `DMedia.init_filestores()` is called, it creates `FileStore` instances |
835 | +based solely on information in the non-replicated /dmedia/_local/dmedia |
836 | +document... despite the fact that the exact same information is also available |
837 | +in the corresponding 'dmedia/store' documents. |
838 | + |
839 | +When it comes to deciding what files dmedia will read and write, it's prudent to |
840 | +assume that replicated documents are untrustworthy. |
841 | + |
842 | +The dangerous approach would be to use a view to get all the 'dmedia/store' |
843 | +documents with a machine_id that matches this machine_id, and initialize those `FileStore`. But the problem is that if an attacker gained control of just one |
844 | +of your replicating peers or services, they could insert arbitrary |
845 | +'dmedia/store' documents, and have dmedia happily initialize `FileStore` at |
846 | +those mount points. And that would be "a bad thing". |
847 | + |
848 | +So although the corresponding 'dmedia/store' records are created (if they don't |
849 | +already exists), they are completely ignored when it comes to deciding what |
850 | +filestores and mount points are configured. |
851 | +""" |
852 | + |
853 | +import logging |
854 | +from copy import deepcopy |
855 | +import os |
856 | +from os import path |
857 | + |
858 | +from couchdb import ResourceNotFound |
859 | +try: |
860 | + import desktopcouch |
861 | + from desktopcouch.application.platform import find_port |
862 | + from desktopcouch.application.local_files import get_oauth_tokens |
863 | +except ImportError: |
864 | + desktopcouch = None |
865 | + |
866 | +try: |
867 | + from dmedia.webui.app import App |
868 | +except ImportError: |
869 | + App = None |
870 | + |
871 | +from .constants import DBNAME |
872 | +from .abstractcouch import get_server, get_db, load_env |
873 | +from .schema import random_id, create_machine, create_store |
874 | +from .views import init_views |
875 | + |
876 | + |
877 | +log = logging.getLogger() |
878 | + |
879 | + |
880 | +def get_env(dbname=DBNAME, no_dc=False): |
881 | + """ |
882 | + Return default CouchDB environment. |
883 | + |
884 | + This will return an appropriate environment based on whether desktopcouch is |
885 | + available. If you supply ``no_dc=True``, the environment for the default |
886 | + system wide CouchDB will be returned, even if desktopcouch is available. |
887 | + |
888 | + For example: |
889 | + |
890 | + >>> get_env(no_dc=True) |
891 | + {'url': 'http://localhost:5984/', 'dbname': 'dmedia', 'port': 5984} |
892 | + >>> get_env(dbname='foo', no_dc=True) |
893 | + {'url': 'http://localhost:5984/', 'dbname': 'foo', 'port': 5984} |
894 | + |
895 | + Not a perfect solution, but works for now. |
896 | + """ |
897 | + if desktopcouch is None or no_dc: |
898 | + return { |
899 | + 'dbname': dbname, |
900 | + 'port': 5984, |
901 | + 'url': 'http://localhost:5984/', |
902 | + } |
903 | + port = find_port() |
904 | + return { |
905 | + 'dbname': dbname, |
906 | + 'port': port, |
907 | + 'url': 'http://localhost:%d/' % port, |
908 | + 'oauth': get_oauth_tokens(), |
909 | + } |
910 | + |
911 | + |
912 | +class LocalStores(object): |
913 | + def by_id(self, _id): |
914 | + pass |
915 | + |
916 | + def by_path(self, parentdir): |
917 | + pass |
918 | + |
919 | + def by_device(self, device): |
920 | + pass |
921 | + |
922 | + |
923 | +class Core(object): |
924 | + def __init__(self, dbname=DBNAME, no_dc=False, env_s=None): |
925 | + if env_s: |
926 | + self.env = load_env(env_s) |
927 | + else: |
928 | + self.env = get_env(dbname, no_dc) |
929 | + self.home = path.abspath(os.environ['HOME']) |
930 | + if not path.isdir(self.home): |
931 | + raise ValueError('HOME is not a dir: {!}'.format(self.home)) |
932 | + self.server = get_server(self.env) |
933 | + self.db = get_db(self.env, self.server) |
934 | + self._has_app = None |
935 | + |
936 | + def bootstrap(self): |
937 | + (self.local, self.machine) = self.init_local() |
938 | + self.machine_id = self.machine['_id'] |
939 | + self.env['machine_id'] = self.machine_id |
940 | + store = self.init_filestores() |
941 | + self.env['filestore'] = {'_id': store['_id'], 'path': store['path']} |
942 | + init_views(self.db) |
943 | + |
944 | + def init_local(self): |
945 | + """ |
946 | + Get the /dmedia/_local/dmedia document, creating it if needed. |
947 | + """ |
948 | + local_id = '_local/dmedia' |
949 | + try: |
950 | + local = self.db[local_id] |
951 | + except ResourceNotFound: |
952 | + machine = create_machine() |
953 | + local = { |
954 | + '_id': local_id, |
955 | + 'machine': deepcopy(machine), |
956 | + 'filestores': {}, |
957 | + } |
958 | + self.db.save(local) |
959 | + self.db.save(machine) |
960 | + else: |
961 | + try: |
962 | + machine = self.db[local['machine']['_id']] |
963 | + except ResourceNotFound: |
964 | + machine = deepcopy(local['machine']) |
965 | + self.db.save(machine) |
966 | + return (local, machine) |
967 | + |
968 | + def init_filestores(self): |
969 | + if not self.local['filestores']: |
970 | + store = create_store(self.home, self.machine_id) |
971 | + self.local['filestores'][store['_id']] = deepcopy(store) |
972 | + self.local['default_filestore'] = store['_id'] |
973 | + self.db.save(self.local) |
974 | + self.db.save(store) |
975 | + return self.local['filestores'][self.local['default_filestore']] |
976 | + |
977 | + def init_app(self): |
978 | + if App is None: |
979 | + log.info('init_app(): `dmedia.webui.app` not available') |
980 | + return False |
981 | + log.info('init_app(): creating /dmedia/app document') |
982 | + doc = App().get_doc() |
983 | + _id = doc['_id'] |
984 | + assert '_rev' not in doc |
985 | + try: |
986 | + old = self.db[_id] |
987 | + doc['_rev'] = old['_rev'] |
988 | + self.db.save(doc) |
989 | + except ResourceNotFound: |
990 | + self.db.save(doc) |
991 | + return True |
992 | + |
993 | + def has_app(self): |
994 | + if self._has_app is None: |
995 | + self._has_app = self.init_app() |
996 | + return self._has_app |
997 | |
998 | === modified file 'dmedia/filestore.py' |
999 | --- dmedia/filestore.py 2011-04-06 20:56:54 +0000 |
1000 | +++ dmedia/filestore.py 2011-04-11 12:02:24 +0000 |
1001 | @@ -164,7 +164,6 @@ |
1002 | from threading import Thread |
1003 | from Queue import Queue |
1004 | |
1005 | -from .schema import create_store |
1006 | from .errors import AmbiguousPath, FileStoreTraversal |
1007 | from .errors import DuplicateFile, IntegrityError |
1008 | from .constants import LEAF_SIZE, TYPE_ERROR, EXT_PAT |
1009 | @@ -559,7 +558,7 @@ |
1010 | To create a `FileStore`, you give it the directory that will be its base on |
1011 | the filesystem: |
1012 | |
1013 | - >>> fs = FileStore('/home/jderose/.dmedia') #doctest: +SKIP |
1014 | + >>> fs = FileStore('/home/jderose') #doctest: +SKIP |
1015 | >>> fs.base #doctest: +SKIP |
1016 | '/home/jderose/.dmedia' |
1017 | |
1018 | @@ -567,7 +566,7 @@ |
1019 | |
1020 | >>> fs = FileStore() |
1021 | >>> fs.base #doctest: +ELLIPSIS |
1022 | - '/tmp/store...' |
1023 | + '/tmp/.../.dmedia' |
1024 | |
1025 | You can add files to the store using `FileStore.import_file()`: |
1026 | |
1027 | @@ -580,7 +579,7 @@ |
1028 | path of the file using `FileStore.path()`: |
1029 | |
1030 | >>> fs.path('HIGJPQWY4PI7G7IFOB2G4TKY6PMTJSI7', 'mov') #doctest: +ELLIPSIS |
1031 | - '/tmp/store.../HI/GJPQWY4PI7G7IFOB2G4TKY6PMTJSI7.mov' |
1032 | + '/tmp/.../.dmedia/HI/GJPQWY4PI7G7IFOB2G4TKY6PMTJSI7.mov' |
1033 | |
1034 | As the files are assumed to be read-only and unchanging, moving a file into |
1035 | its canonical location must be atomic. There are 2 scenarios that must be |
1036 | @@ -601,35 +600,31 @@ |
1037 | `fallocate()` function, which calls the Linux ``fallocate`` command. |
1038 | """ |
1039 | |
1040 | - def __init__(self, base=None, machine_id=None): |
1041 | - if base is None: |
1042 | - base = tempfile.mkdtemp(prefix='store.') |
1043 | - self.base = safe_path(base) |
1044 | + def __init__(self, parent=None, dotdir='.dmedia'): |
1045 | + if parent is None: |
1046 | + parent = tempfile.mkdtemp(prefix='store.') |
1047 | + self.parent = safe_path(parent) |
1048 | + if not path.isdir(self.parent): |
1049 | + raise ValueError('%s.parent not a directory: %r' % |
1050 | + (self.__class__.__name__, parent) |
1051 | + ) |
1052 | + self.base = path.join(self.parent, dotdir) |
1053 | try: |
1054 | - os.makedirs(self.base) |
1055 | + os.mkdir(self.base) |
1056 | except OSError: |
1057 | pass |
1058 | if not path.isdir(self.base): |
1059 | raise ValueError('%s.base not a directory: %r' % |
1060 | (self.__class__.__name__, self.base) |
1061 | ) |
1062 | - |
1063 | - # FIXME: This is too high-level for FileStore, should instead be deault |
1064 | - # with by the core API entry point as FileStore are first initialized |
1065 | - self.record = path.join(self.base, 'store.json') |
1066 | - try: |
1067 | - fp = open(self.record, 'rb') |
1068 | - doc = json.load(fp) |
1069 | - except IOError: |
1070 | - fp = open(self.record, 'wb') |
1071 | - doc = create_store(self.base, machine_id) |
1072 | - json.dump(doc, fp, sort_keys=True, indent=4) |
1073 | - fp.close() |
1074 | - self._doc = doc |
1075 | - self._id = doc['_id'] |
1076 | + if path.islink(self.base): |
1077 | + raise ValueError('{!r} is symlink to {!r}'.format( |
1078 | + self.base, os.readlink(self.base) |
1079 | + ) |
1080 | + ) |
1081 | |
1082 | def __repr__(self): |
1083 | - return '%s(%r)' % (self.__class__.__name__, self.base) |
1084 | + return '%s(%r)' % (self.__class__.__name__, self.parent) |
1085 | |
1086 | ############################################ |
1087 | # Methods to prevent path traversals attacks |
1088 | @@ -656,7 +651,7 @@ |
1089 | |
1090 | >>> fs = FileStore() |
1091 | >>> fs.join('NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') #doctest: +ELLIPSIS |
1092 | - '/tmp/store.../NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW' |
1093 | + '/tmp/.../.dmedia/NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW' |
1094 | |
1095 | However, a `FileStoreTraversal` is raised if *parts* cause a path |
1096 | traversal outside of the `FileStore` base directory: |
1097 | @@ -664,14 +659,14 @@ |
1098 | >>> fs.join('../ssh') #doctest: +ELLIPSIS |
1099 | Traceback (most recent call last): |
1100 | ... |
1101 | - FileStoreTraversal: '/tmp/ssh' outside base '/tmp/store...' |
1102 | + FileStoreTraversal: '/tmp/.../ssh' outside base '/tmp/.../.dmedia' |
1103 | |
1104 | Or Likewise if an absolute path is included in *parts*: |
1105 | |
1106 | >>> fs.join('NW', '/etc', 'ssh') #doctest: +ELLIPSIS |
1107 | Traceback (most recent call last): |
1108 | ... |
1109 | - FileStoreTraversal: '/etc/ssh' outside base '/tmp/store...' |
1110 | + FileStoreTraversal: '/etc/ssh' outside base '/tmp/.../.dmedia' |
1111 | |
1112 | Also see `FileStore.create_parent()`. |
1113 | """ |
1114 | @@ -689,14 +684,14 @@ |
1115 | >>> fs.create_parent('/foo/my.ogv') #doctest: +ELLIPSIS |
1116 | Traceback (most recent call last): |
1117 | ... |
1118 | - FileStoreTraversal: '/foo/my.ogv' outside base '/tmp/store...' |
1119 | + FileStoreTraversal: '/foo/my.ogv' outside base '/tmp/.../.dmedia' |
1120 | |
1121 | It also protects against malicious filenames like this: |
1122 | |
1123 | >>> fs.create_parent('/foo/../bar/my.ogv') #doctest: +ELLIPSIS |
1124 | Traceback (most recent call last): |
1125 | ... |
1126 | - FileStoreTraversal: '/bar/my.ogv' outside base '/tmp/store...' |
1127 | + FileStoreTraversal: '/bar/my.ogv' outside base '/tmp/.../.dmedia' |
1128 | |
1129 | If doesn't already exists, the directory containing *filename* is |
1130 | created. Returns the directory containing *filename*. |
1131 | @@ -747,12 +742,12 @@ |
1132 | |
1133 | >>> fs = FileStore() |
1134 | >>> fs.path('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW') #doctest: +ELLIPSIS |
1135 | - '/tmp/store.../NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW' |
1136 | + '/tmp/.../.dmedia/NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW' |
1137 | |
1138 | Or with a file extension: |
1139 | |
1140 | >>> fs.path('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW', 'txt') #doctest: +ELLIPSIS |
1141 | - '/tmp/store.../NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW.txt' |
1142 | + '/tmp/.../.dmedia/NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW.txt' |
1143 | |
1144 | If called with ``create=True``, the parent directory is created with |
1145 | `FileStore.create_parent()`. |
1146 | @@ -854,12 +849,12 @@ |
1147 | |
1148 | >>> fs = FileStore() |
1149 | >>> fs.tmp('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW') #doctest: +ELLIPSIS |
1150 | - '/tmp/store.../transfers/NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW' |
1151 | + '/tmp/.../.dmedia/transfers/NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW' |
1152 | |
1153 | Or with a file extension: |
1154 | |
1155 | >>> fs.tmp('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW', 'txt') #doctest: +ELLIPSIS |
1156 | - '/tmp/store.../transfers/NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW.txt' |
1157 | + '/tmp/.../.dmedia/transfers/NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW.txt' |
1158 | |
1159 | If called with ``create=True``, the parent directory is created with |
1160 | `FileStore.create_parent()`. |
1161 | @@ -979,7 +974,7 @@ |
1162 | >>> tmp_fp = open(fs.join('foo.mov'), 'wb') |
1163 | >>> chash = 'ZR765XWSF6S7JQHLUI4GCG5BHGPE252O' |
1164 | >>> fs.tmp_move(tmp_fp, chash, 'mov') #doctest: +ELLIPSIS |
1165 | - '/tmp/store.../ZR/765XWSF6S7JQHLUI4GCG5BHGPE252O.mov' |
1166 | + '/tmp/.../.dmedia/ZR/765XWSF6S7JQHLUI4GCG5BHGPE252O.mov' |
1167 | |
1168 | Note, however, that this method does *not* verify the content hash of |
1169 | the temporary file! This is by design as many operations will compute |
1170 | @@ -1081,7 +1076,7 @@ |
1171 | >>> fs = FileStore() |
1172 | >>> tmp = fs.tmp('TGX33XXWU3EVHEEY5J7NBOJGKBFXLEBK', 'mov', create=True) |
1173 | >>> tmp #doctest: +ELLIPSIS |
1174 | - '/tmp/store.../transfers/TGX33XXWU3EVHEEY5J7NBOJGKBFXLEBK.mov' |
1175 | + '/tmp/.../.dmedia/transfers/TGX33XXWU3EVHEEY5J7NBOJGKBFXLEBK.mov' |
1176 | |
1177 | Then the downloader will write to the temporary file as it's being |
1178 | downloaded: |
1179 | @@ -1102,7 +1097,7 @@ |
1180 | |
1181 | >>> dst = fs.tmp_verify_move('TGX33XXWU3EVHEEY5J7NBOJGKBFXLEBK', 'mov') |
1182 | >>> dst #doctest: +ELLIPSIS |
1183 | - '/tmp/store.../TG/X33XXWU3EVHEEY5J7NBOJGKBFXLEBK.mov' |
1184 | + '/tmp/.../.dmedia/TG/X33XXWU3EVHEEY5J7NBOJGKBFXLEBK.mov' |
1185 | |
1186 | The return value is the absolute path of the canonical file. |
1187 | |
1188 | |
1189 | === modified file 'dmedia/gtkui/client.py' |
1190 | --- dmedia/gtkui/client.py 2011-03-27 10:45:13 +0000 |
1191 | +++ dmedia/gtkui/client.py 2011-04-11 12:02:24 +0000 |
1192 | @@ -27,7 +27,7 @@ |
1193 | import dbus.mainloop.glib |
1194 | from gi.repository import GObject |
1195 | from gi.repository.GObject import TYPE_PYOBJECT |
1196 | -from dmedia.constants import BUS, INTERFACE, EXTENSIONS |
1197 | +from dmedia.constants import IMPORT_BUS, IMPORT_IFACE, EXTENSIONS |
1198 | |
1199 | |
1200 | # We need mainloop integration to test signals: |
1201 | @@ -122,7 +122,7 @@ |
1202 | |
1203 | def __init__(self, bus=None): |
1204 | super(Client, self).__init__() |
1205 | - self._bus = (BUS if bus is None else bus) |
1206 | + self._bus = (IMPORT_BUS if bus is None else bus) |
1207 | self._conn = dbus.SessionBus() |
1208 | self._proxy = None |
1209 | |
1210 | @@ -137,27 +137,27 @@ |
1211 | return self._proxy |
1212 | |
1213 | def _call(self, name, *args): |
1214 | - method = self.proxy.get_dbus_method(name, dbus_interface=INTERFACE) |
1215 | + method = self.proxy.get_dbus_method(name, dbus_interface=IMPORT_IFACE) |
1216 | return method(*args) |
1217 | |
1218 | def _connect_signals(self): |
1219 | self.proxy.connect_to_signal( |
1220 | - 'BatchStarted', self._on_BatchStarted, INTERFACE |
1221 | - ) |
1222 | - self.proxy.connect_to_signal( |
1223 | - 'BatchFinished', self._on_BatchFinished, INTERFACE |
1224 | - ) |
1225 | - self.proxy.connect_to_signal( |
1226 | - 'ImportStarted', self._on_ImportStarted, INTERFACE |
1227 | - ) |
1228 | - self.proxy.connect_to_signal( |
1229 | - 'ImportCount', self._on_ImportCount, INTERFACE |
1230 | - ) |
1231 | - self.proxy.connect_to_signal( |
1232 | - 'ImportProgress', self._on_ImportProgress, INTERFACE |
1233 | - ) |
1234 | - self.proxy.connect_to_signal( |
1235 | - 'ImportFinished', self._on_ImportFinished, INTERFACE |
1236 | + 'BatchStarted', self._on_BatchStarted, IMPORT_IFACE |
1237 | + ) |
1238 | + self.proxy.connect_to_signal( |
1239 | + 'BatchFinished', self._on_BatchFinished, IMPORT_IFACE |
1240 | + ) |
1241 | + self.proxy.connect_to_signal( |
1242 | + 'ImportStarted', self._on_ImportStarted, IMPORT_IFACE |
1243 | + ) |
1244 | + self.proxy.connect_to_signal( |
1245 | + 'ImportCount', self._on_ImportCount, IMPORT_IFACE |
1246 | + ) |
1247 | + self.proxy.connect_to_signal( |
1248 | + 'ImportProgress', self._on_ImportProgress, IMPORT_IFACE |
1249 | + ) |
1250 | + self.proxy.connect_to_signal( |
1251 | + 'ImportFinished', self._on_ImportFinished, IMPORT_IFACE |
1252 | ) |
1253 | |
1254 | def _on_BatchStarted(self, batch_id): |
1255 | |
1256 | === modified file 'dmedia/gtkui/service.py' |
1257 | --- dmedia/gtkui/service.py 2011-03-28 12:38:29 +0000 |
1258 | +++ dmedia/gtkui/service.py 2011-04-11 12:02:24 +0000 |
1259 | @@ -22,7 +22,11 @@ |
1260 | # with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
1261 | |
1262 | """ |
1263 | -Makes dmedia functionality avaible over D-Bus. |
1264 | +D-Bus service implementing Pro File Import UX. |
1265 | + |
1266 | +For details, please see: |
1267 | + |
1268 | + https://wiki.ubuntu.com/AyatanaDmediaLovefest |
1269 | """ |
1270 | |
1271 | from os import path |
1272 | @@ -35,9 +39,8 @@ |
1273 | import dbus.mainloop.glib |
1274 | |
1275 | from dmedia import __version__ |
1276 | -from dmedia.constants import BUS, INTERFACE, DBNAME, EXT_MAP |
1277 | +from dmedia.constants import IMPORT_BUS, IMPORT_IFACE, EXT_MAP |
1278 | from dmedia.importer import ImportManager |
1279 | -from dmedia.metastore import MetaStore |
1280 | |
1281 | from .util import NotifyManager, Timer, import_started, batch_finished |
1282 | |
1283 | @@ -71,11 +74,12 @@ |
1284 | 'ImportFinished', |
1285 | ]) |
1286 | |
1287 | - def __init__(self, env, killfunc=None): |
1288 | + def __init__(self, env, bus, killfunc): |
1289 | + assert callable(killfunc) |
1290 | self._env = env |
1291 | + self._bus = bus |
1292 | self._killfunc = killfunc |
1293 | - self._bus = env.get('bus', BUS) |
1294 | - self._dbname = env.get('dbname', DBNAME) |
1295 | + self._bus = bus |
1296 | self._no_gui = env.get('no_gui', False) |
1297 | log.info('Starting service on %r', self._bus) |
1298 | self._conn = dbus.SessionBus() |
1299 | @@ -120,21 +124,12 @@ |
1300 | self._indicator.set_menu(self._menu) |
1301 | self._indicator.set_status(AppIndicator.IndicatorStatus.ACTIVE) |
1302 | |
1303 | - self._metastore = None |
1304 | self._manager = None |
1305 | |
1306 | @property |
1307 | - def metastore(self): |
1308 | - if self._metastore is None: |
1309 | - self._metastore = MetaStore(self._env) |
1310 | - return self._metastore |
1311 | - |
1312 | - @property |
1313 | def manager(self): |
1314 | if self._manager is None: |
1315 | - self._manager = ImportManager( |
1316 | - self.metastore.get_env(), self._on_signal |
1317 | - ) |
1318 | + self._manager = ImportManager(self._env, self._on_signal) |
1319 | self._manager.start() |
1320 | return self._manager |
1321 | |
1322 | @@ -164,7 +159,7 @@ |
1323 | except Exception: |
1324 | log.exception('Could not open dmedia database in Futon') |
1325 | |
1326 | - @dbus.service.signal(INTERFACE, signature='s') |
1327 | + @dbus.service.signal(IMPORT_IFACE, signature='s') |
1328 | def BatchStarted(self, batch_id): |
1329 | """ |
1330 | Fired at transition from idle to at least one active import. |
1331 | @@ -181,7 +176,7 @@ |
1332 | self._indicator.set_menu(self._menu) |
1333 | self._timer.start() |
1334 | |
1335 | - @dbus.service.signal(INTERFACE, signature='sa{sx}') |
1336 | + @dbus.service.signal(IMPORT_IFACE, signature='sa{sx}') |
1337 | def BatchFinished(self, batch_id, stats): |
1338 | """ |
1339 | Fired at transition from at least one active import to idle. |
1340 | @@ -201,7 +196,7 @@ |
1341 | self._timer.stop() |
1342 | self._current.hide() |
1343 | |
1344 | - @dbus.service.signal(INTERFACE, signature='ss') |
1345 | + @dbus.service.signal(IMPORT_IFACE, signature='ss') |
1346 | def ImportStarted(self, base, import_id): |
1347 | """ |
1348 | Fired when card is inserted. |
1349 | @@ -219,19 +214,19 @@ |
1350 | # via FireWire or USB |
1351 | self._notify.replace(summary, body, 'notification-device-usb') |
1352 | |
1353 | - @dbus.service.signal(INTERFACE, signature='ssx') |
1354 | + @dbus.service.signal(IMPORT_IFACE, signature='ssx') |
1355 | def ImportCount(self, base, import_id, total): |
1356 | pass |
1357 | |
1358 | - @dbus.service.signal(INTERFACE, signature='ssiia{ss}') |
1359 | + @dbus.service.signal(IMPORT_IFACE, signature='ssiia{ss}') |
1360 | def ImportProgress(self, base, import_id, completed, total, info): |
1361 | pass |
1362 | |
1363 | - @dbus.service.signal(INTERFACE, signature='ssa{sx}') |
1364 | + @dbus.service.signal(IMPORT_IFACE, signature='ssa{sx}') |
1365 | def ImportFinished(self, base, import_id, stats): |
1366 | pass |
1367 | |
1368 | - @dbus.service.method(INTERFACE, in_signature='', out_signature='') |
1369 | + @dbus.service.method(IMPORT_IFACE, in_signature='', out_signature='') |
1370 | def Kill(self): |
1371 | """ |
1372 | Kill the dmedia service process. |
1373 | @@ -243,14 +238,14 @@ |
1374 | log.info('Calling killfunc()') |
1375 | self._killfunc() |
1376 | |
1377 | - @dbus.service.method(INTERFACE, in_signature='', out_signature='s') |
1378 | + @dbus.service.method(IMPORT_IFACE, in_signature='', out_signature='s') |
1379 | def Version(self): |
1380 | """ |
1381 | Return dmedia version. |
1382 | """ |
1383 | return __version__ |
1384 | |
1385 | - @dbus.service.method(INTERFACE, in_signature='as', out_signature='as') |
1386 | + @dbus.service.method(IMPORT_IFACE, in_signature='as', out_signature='as') |
1387 | def GetExtensions(self, types): |
1388 | """ |
1389 | Get a list of extensions based on broad categories in *types*. |
1390 | @@ -267,7 +262,7 @@ |
1391 | extensions.update(EXT_MAP[key]) |
1392 | return sorted(extensions) |
1393 | |
1394 | - @dbus.service.method(INTERFACE, in_signature='sb', out_signature='s') |
1395 | + @dbus.service.method(IMPORT_IFACE, in_signature='sb', out_signature='s') |
1396 | def StartImport(self, base, extract): |
1397 | """ |
1398 | Start import of card mounted at *base*. |
1399 | @@ -289,7 +284,7 @@ |
1400 | return 'started' |
1401 | return 'already_running' |
1402 | |
1403 | - @dbus.service.method(INTERFACE, in_signature='s', out_signature='s') |
1404 | + @dbus.service.method(IMPORT_IFACE, in_signature='s', out_signature='s') |
1405 | def StopImport(self, base): |
1406 | """ |
1407 | In running, stop the import of directory or file at *base*. |
1408 | @@ -298,7 +293,7 @@ |
1409 | return 'stopped' |
1410 | return 'not_running' |
1411 | |
1412 | - @dbus.service.method(INTERFACE, in_signature='', out_signature='as') |
1413 | + @dbus.service.method(IMPORT_IFACE, in_signature='', out_signature='as') |
1414 | def ListImports(self): |
1415 | """ |
1416 | Return list of currently running imports. |
1417 | |
1418 | === modified file 'dmedia/gtkui/tests/test_client.py' |
1419 | --- dmedia/gtkui/tests/test_client.py 2011-03-27 10:45:13 +0000 |
1420 | +++ dmedia/gtkui/tests/test_client.py 2011-04-11 12:02:24 +0000 |
1421 | @@ -27,6 +27,7 @@ |
1422 | from os import path |
1423 | from subprocess import Popen |
1424 | import time |
1425 | +import json |
1426 | |
1427 | import dbus |
1428 | from dbus.proxies import ProxyObject |
1429 | @@ -44,7 +45,7 @@ |
1430 | |
1431 | tree = path.dirname(path.dirname(path.abspath(dmedia.__file__))) |
1432 | assert path.isfile(path.join(tree, 'setup.py')) |
1433 | -script = path.join(tree, 'dmedia-service') |
1434 | +script = path.join(tree, 'dmedia-importer-service') |
1435 | assert path.isfile(script) |
1436 | |
1437 | |
1438 | @@ -73,7 +74,7 @@ |
1439 | self.handlers[name] = callback |
1440 | |
1441 | |
1442 | -class test_Client(CouchCase): |
1443 | +class TestClient(CouchCase): |
1444 | klass = client.Client |
1445 | |
1446 | def setUp(self): |
1447 | @@ -87,17 +88,17 @@ |
1448 | How do people usually unit test dbus services? This works, but not sure |
1449 | if there is a better idiom in common use. --jderose |
1450 | """ |
1451 | - super(test_Client, self).setUp() |
1452 | + super(TestClient, self).setUp() |
1453 | self.bus = random_bus() |
1454 | cmd = [script, '--no-gui', |
1455 | - '--dbname', self.dbname, |
1456 | '--bus', self.bus, |
1457 | + '--env', json.dumps(self.env), |
1458 | ] |
1459 | self.service = Popen(cmd) |
1460 | time.sleep(1) # Give dmedia-service time to start |
1461 | |
1462 | def tearDown(self): |
1463 | - super(test_Client, self).tearDown() |
1464 | + super(TestClient, self).tearDown() |
1465 | try: |
1466 | self.service.terminate() |
1467 | self.service.wait() |
1468 | @@ -112,7 +113,7 @@ |
1469 | def test_init(self): |
1470 | # Test with no bus |
1471 | inst = self.klass() |
1472 | - self.assertEqual(inst._bus, 'org.freedesktop.DMedia') |
1473 | + self.assertEqual(inst._bus, 'org.freedesktop.DMediaImporter') |
1474 | self.assertTrue(isinstance(inst._conn, dbus.SessionBus)) |
1475 | self.assertTrue(inst._proxy is None) |
1476 | |
1477 | @@ -243,19 +244,19 @@ |
1478 | self.assertEqual( |
1479 | signals.messages[3], |
1480 | ('import_progress', inst, base, import_id, 1, 3, |
1481 | - dict(action='imported', src=src1, _id=mov_hash) |
1482 | + dict(action='imported', src=src1) |
1483 | ) |
1484 | ) |
1485 | self.assertEqual( |
1486 | signals.messages[4], |
1487 | ('import_progress', inst, base, import_id, 2, 3, |
1488 | - dict(action='imported', src=src2, _id=thm_hash) |
1489 | + dict(action='imported', src=src2) |
1490 | ) |
1491 | ) |
1492 | self.assertEqual( |
1493 | signals.messages[5], |
1494 | ('import_progress', inst, base, import_id, 3, 3, |
1495 | - dict(action='skipped', src=dup1, _id=mov_hash) |
1496 | + dict(action='skipped', src=dup1) |
1497 | ) |
1498 | ) |
1499 | self.assertEqual( |
1500 | |
1501 | === removed file 'dmedia/gtkui/tests/test_service.py' |
1502 | --- dmedia/gtkui/tests/test_service.py 2011-03-27 09:56:34 +0000 |
1503 | +++ dmedia/gtkui/tests/test_service.py 1970-01-01 00:00:00 +0000 |
1504 | @@ -1,53 +0,0 @@ |
1505 | -# Authors: |
1506 | -# Jason Gerard DeRose <jderose@novacut.com> |
1507 | -# |
1508 | -# dmedia: distributed media library |
1509 | -# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com> |
1510 | -# |
1511 | -# This file is part of `dmedia`. |
1512 | -# |
1513 | -# `dmedia` is free software: you can redistribute it and/or modify it under the |
1514 | -# terms of the GNU Affero General Public License as published by the Free |
1515 | -# Software Foundation, either version 3 of the License, or (at your option) any |
1516 | -# later version. |
1517 | -# |
1518 | -# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY |
1519 | -# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR |
1520 | -# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more |
1521 | -# details. |
1522 | -# |
1523 | -# You should have received a copy of the GNU Affero General Public License along |
1524 | -# with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
1525 | - |
1526 | -""" |
1527 | -Unit tests for `dmedia.metastore` module. |
1528 | -""" |
1529 | - |
1530 | -from unittest import TestCase |
1531 | - |
1532 | -from dmedia.tests.helpers import TempDir, random_bus |
1533 | -from dmedia.tests.couch import CouchCase |
1534 | -from dmedia import importer |
1535 | -from dmedia.gtkui import service |
1536 | - |
1537 | - |
1538 | -class test_DMedia(CouchCase): |
1539 | - klass = service.DMedia |
1540 | - |
1541 | - def test_init(self): |
1542 | - bus = random_bus() |
1543 | - self.env['bus'] = bus |
1544 | - self.env['no_gui'] = True |
1545 | - def kill(): |
1546 | - pass |
1547 | - inst = self.klass(self.env, killfunc=kill) |
1548 | - self.assertTrue(inst._killfunc is kill) |
1549 | - self.assertTrue(inst._bus is bus) |
1550 | - self.assertTrue(inst._dbname is self.dbname) |
1551 | - self.assertTrue(inst._no_gui) |
1552 | - self.assertEqual(inst._manager, None) |
1553 | - |
1554 | - m = inst.manager |
1555 | - self.assertTrue(inst._manager is m) |
1556 | - self.assertTrue(isinstance(m, importer.ImportManager)) |
1557 | - self.assertEqual(m._callback, inst._on_signal) |
1558 | |
1559 | === modified file 'dmedia/importer.py' |
1560 | --- dmedia/importer.py 2011-04-08 20:59:58 +0000 |
1561 | +++ dmedia/importer.py 2011-04-11 12:02:24 +0000 |
1562 | @@ -40,8 +40,6 @@ |
1563 | ) |
1564 | from .filestore import FileStore, quick_id, safe_open, safe_ext, pack_leaves |
1565 | from .extractor import merge_metadata |
1566 | -from .abstractcouch import get_env, get_couchdb_server, get_dmedia_db |
1567 | - |
1568 | |
1569 | mimetypes.init() |
1570 | DOTDIR = '.dmedia' |
1571 | @@ -165,15 +163,8 @@ |
1572 | def __init__(self, env, q, key, args): |
1573 | super(ImportWorker, self).__init__(env, q, key, args) |
1574 | (self.base, self.extract) = args |
1575 | - self.home = path.abspath(os.environ['HOME']) |
1576 | - self.filestore = FileStore( |
1577 | - path.join(self.home, DOTDIR), |
1578 | - self.env.get('machine_id') |
1579 | - ) |
1580 | - try: |
1581 | - self.db.save(self.filestore._doc) |
1582 | - except couchdb.ResourceConflict: |
1583 | - pass |
1584 | + self.filestore = FileStore(self.env['filestore']['path']) |
1585 | + self.filestore_id = self.env['filestore']['_id'] |
1586 | |
1587 | self.filetuples = None |
1588 | self._processed = [] |
1589 | @@ -269,8 +260,8 @@ |
1590 | |
1591 | try: |
1592 | doc = self.db[chash] |
1593 | - if self.filestore._id not in doc['stored']: |
1594 | - doc['stored'][self.filestore._id] = { |
1595 | + if self.filestore_id not in doc['stored']: |
1596 | + doc['stored'][self.filestore_id] = { |
1597 | 'copies': 1, |
1598 | 'time': time.time(), |
1599 | } |
1600 | @@ -279,7 +270,7 @@ |
1601 | except couchdb.ResourceNotFound as e: |
1602 | pass |
1603 | |
1604 | - doc = create_file(stat.st_size, leaves, self.filestore._id, |
1605 | + doc = create_file(stat.st_size, leaves, self.filestore_id, |
1606 | copies=1, ext=ext |
1607 | ) |
1608 | assert doc['_id'] == chash |
1609 | |
1610 | === modified file 'dmedia/schema.py' |
1611 | --- dmedia/schema.py 2011-04-07 03:03:07 +0000 |
1612 | +++ dmedia/schema.py 2011-04-11 12:02:24 +0000 |
1613 | @@ -315,6 +315,8 @@ |
1614 | from base64 import b32encode, b32decode, b64encode |
1615 | import re |
1616 | import time |
1617 | +import socket |
1618 | +import platform |
1619 | |
1620 | from .constants import TYPE_ERROR, EXT_PAT |
1621 | |
1622 | @@ -1284,7 +1286,7 @@ |
1623 | 'bytes': file_size, |
1624 | 'ext': ext, |
1625 | 'origin': origin, |
1626 | - 'stored': { |
1627 | + 'stored': { |
1628 | store: { |
1629 | 'copies': copies, |
1630 | 'time': ts, |
1631 | @@ -1293,7 +1295,21 @@ |
1632 | } |
1633 | |
1634 | |
1635 | -def create_store(base, machine_id, copies=1): |
1636 | +def create_machine(): |
1637 | + """ |
1638 | + Create a 'dmedia/machine' document. |
1639 | + """ |
1640 | + return { |
1641 | + '_id': random_id(), |
1642 | + 'ver': 0, |
1643 | + 'type': 'dmedia/machine', |
1644 | + 'time': time.time(), |
1645 | + 'hostname': socket.gethostname(), |
1646 | + 'distribution': list(platform.linux_distribution()), |
1647 | + } |
1648 | + |
1649 | + |
1650 | +def create_store(parentdir, machine_id, copies=1): |
1651 | """ |
1652 | Create a 'dmedia/store' document. |
1653 | """ |
1654 | @@ -1304,7 +1320,7 @@ |
1655 | 'time': time.time(), |
1656 | 'plugin': 'filestore', |
1657 | 'copies': copies, |
1658 | - 'path': base, |
1659 | + 'path': parentdir, |
1660 | 'machine_id': machine_id, |
1661 | } |
1662 | |
1663 | |
1664 | === added directory 'dmedia/service' |
1665 | === added file 'dmedia/service/__init__.py' |
1666 | --- dmedia/service/__init__.py 1970-01-01 00:00:00 +0000 |
1667 | +++ dmedia/service/__init__.py 2011-04-11 12:02:24 +0000 |
1668 | @@ -0,0 +1,94 @@ |
1669 | +# Authors: |
1670 | +# Jason Gerard DeRose <jderose@novacut.com> |
1671 | +# |
1672 | +# dmedia: distributed media library |
1673 | +# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com> |
1674 | +# |
1675 | +# This file is part of `dmedia`. |
1676 | +# |
1677 | +# `dmedia` is free software: you can redistribute it and/or modify it under the |
1678 | +# terms of the GNU Affero General Public License as published by the Free |
1679 | +# Software Foundation, either version 3 of the License, or (at your option) any |
1680 | +# later version. |
1681 | +# |
1682 | +# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY |
1683 | +# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR |
1684 | +# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more |
1685 | +# details. |
1686 | +# |
1687 | +# You should have received a copy of the GNU Affero General Public License along |
1688 | +# with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
1689 | + |
1690 | +""" |
1691 | +Core dmedia DBus service at org.freedesktop.DMedia. |
1692 | +""" |
1693 | + |
1694 | +import logging |
1695 | +import json |
1696 | +import time |
1697 | + |
1698 | +import gnomekeyring |
1699 | +import dbus |
1700 | +import dbus.service |
1701 | + |
1702 | +from dmedia import __version__ |
1703 | +from dmedia.constants import IFACE |
1704 | +from dmedia.core import Core |
1705 | + |
1706 | + |
1707 | +log = logging.getLogger() |
1708 | + |
1709 | + |
1710 | +class DMedia(dbus.service.Object): |
1711 | + def __init__(self, couchargs, bus, killfunc, start=None): |
1712 | + self._bus = bus |
1713 | + self._killfunc = killfunc |
1714 | + log.info('Starting dmedia core service on %r', bus) |
1715 | + self._conn = dbus.SessionBus() |
1716 | + super(DMedia, self).__init__(self._conn, object_path='/') |
1717 | + self._busname = dbus.service.BusName(bus, self._conn) |
1718 | + self._core = Core(*couchargs) |
1719 | + self._core.bootstrap() |
1720 | + self._env_s = json.dumps(self._core.env) |
1721 | + if start is not None: |
1722 | + log.info('Started service in %.3f seconds', time.time() - start) |
1723 | + |
1724 | + @dbus.service.method(IFACE, in_signature='', out_signature='s') |
1725 | + def Version(self): |
1726 | + """ |
1727 | + Return dmedia version. |
1728 | + """ |
1729 | + return __version__ |
1730 | + |
1731 | + @dbus.service.method(IFACE, in_signature='', out_signature='') |
1732 | + def Kill(self): |
1733 | + """ |
1734 | + Kill the `dmedia-service` process. |
1735 | + """ |
1736 | + log.info('Killing dmedia core service on %r', self._bus) |
1737 | + self._killfunc() |
1738 | + |
1739 | + @dbus.service.method(IFACE, in_signature='', out_signature='s') |
1740 | + def GetEnv(self): |
1741 | + """ |
1742 | + Return dmedia env as JSON string. |
1743 | + """ |
1744 | + return self._env_s |
1745 | + |
1746 | + @dbus.service.method(IFACE, in_signature='', out_signature='s') |
1747 | + def GetAuthURL(self): |
1748 | + """ |
1749 | + Get URL with basic auth user and password. |
1750 | + """ |
1751 | + data = gnomekeyring.find_items_sync( |
1752 | + gnomekeyring.ITEM_GENERIC_SECRET, |
1753 | + {'desktopcouch': 'basic'} |
1754 | + ) |
1755 | + (user, password) = data[0].secret.split(':') |
1756 | + return 'http://{user}:{password}@localhost:{port}/'.format( |
1757 | + user=user, password=password, port=self._core.env['port'] |
1758 | + ) |
1759 | + |
1760 | + @dbus.service.method(IFACE, in_signature='', out_signature='b') |
1761 | + def HasApp(self): |
1762 | + return self._core.has_app() |
1763 | |
1764 | === modified file 'dmedia/tests/couch.py' |
1765 | --- dmedia/tests/couch.py 2011-03-27 09:56:34 +0000 |
1766 | +++ dmedia/tests/couch.py 2011-04-11 12:02:24 +0000 |
1767 | @@ -27,7 +27,8 @@ |
1768 | |
1769 | import couchdb |
1770 | |
1771 | -from dmedia.abstractcouch import get_env, get_couchdb_server |
1772 | +from dmedia.core import get_env |
1773 | +from dmedia.abstractcouch import get_server |
1774 | from dmedia.schema import random_id |
1775 | |
1776 | from .helpers import TempHome |
1777 | @@ -48,15 +49,16 @@ |
1778 | |
1779 | def setUp(self): |
1780 | self.home = TempHome() |
1781 | - self.dbname = 'dmedia_test' |
1782 | + self.dbname = 'test_dmedia' |
1783 | self.env = get_env(self.dbname) |
1784 | - server = get_couchdb_server(self.env) |
1785 | + server = get_server(self.env) |
1786 | try: |
1787 | del server[self.dbname] |
1788 | except couchdb.ResourceNotFound: |
1789 | pass |
1790 | self.machine_id = random_id() |
1791 | self.env['machine_id'] = self.machine_id |
1792 | + self.env['filestore'] = {'_id': random_id(), 'path': self.home.path} |
1793 | |
1794 | def tearDown(self): |
1795 | self.home = None |
1796 | |
1797 | === modified file 'dmedia/tests/test_abstractcouch.py' |
1798 | --- dmedia/tests/test_abstractcouch.py 2011-02-21 10:55:29 +0000 |
1799 | +++ dmedia/tests/test_abstractcouch.py 2011-04-11 12:02:24 +0000 |
1800 | @@ -24,8 +24,10 @@ |
1801 | """ |
1802 | |
1803 | from unittest import TestCase |
1804 | +import json |
1805 | |
1806 | import couchdb |
1807 | +from couchdb import ResourceNotFound |
1808 | from desktopcouch.application.platform import find_port |
1809 | from desktopcouch.application.local_files import get_oauth_tokens |
1810 | from desktopcouch.records.http import OAuthSession |
1811 | @@ -34,29 +36,34 @@ |
1812 | |
1813 | from .helpers import raises |
1814 | |
1815 | + |
1816 | +def dc_env(dbname='test_dmedia'): |
1817 | + """ |
1818 | + Create desktopcouch environment. |
1819 | + """ |
1820 | + port = find_port() |
1821 | + return { |
1822 | + 'dbname': dbname, |
1823 | + 'port': port, |
1824 | + 'url': 'http://localhost:%d/' % port, |
1825 | + 'oauth': get_oauth_tokens(), |
1826 | + } |
1827 | + |
1828 | + |
1829 | +def json_env(dbname='test_dmedia'): |
1830 | + """ |
1831 | + For testing with dc env that has gone though json.dumps() + json.loads(). |
1832 | + """ |
1833 | + return json.loads(json.dumps(dc_env(dbname))) |
1834 | + |
1835 | + |
1836 | class test_functions(TestCase): |
1837 | def tearDown(self): |
1838 | if abstractcouch.OAuthSession is None: |
1839 | abstractcouch.OAuthSession = OAuthSession |
1840 | |
1841 | - def dc_env(self): |
1842 | - """ |
1843 | - Create an *env* for desktopcouch. |
1844 | - """ |
1845 | - port = find_port() |
1846 | - return { |
1847 | - 'port': port, |
1848 | - 'url': 'http://localhost:%d/' % port, |
1849 | - 'oauth': get_oauth_tokens(), |
1850 | - } |
1851 | - |
1852 | - def test_get_couchdb_server(self): |
1853 | - f = abstractcouch.get_couchdb_server |
1854 | - |
1855 | - # Test with empty env |
1856 | - s = f({}) |
1857 | - self.assertTrue(isinstance(s, couchdb.Server)) |
1858 | - self.assertEqual(repr(s), "<Server 'http://localhost:5984/'>") |
1859 | + def test_get_server(self): |
1860 | + f = abstractcouch.get_server |
1861 | |
1862 | # Test with only url |
1863 | s = f({'url': 'http://localhost:5984/'}) |
1864 | @@ -64,7 +71,7 @@ |
1865 | self.assertEqual(repr(s), "<Server 'http://localhost:5984/'>") |
1866 | |
1867 | # Test with desktopcouch |
1868 | - env = self.dc_env() |
1869 | + env = dc_env() |
1870 | s = f(env) |
1871 | self.assertTrue(isinstance(s, couchdb.Server)) |
1872 | self.assertEqual( |
1873 | @@ -81,127 +88,49 @@ |
1874 | ) |
1875 | |
1876 | # Test when OAuthSession is not imported, oauth not provided |
1877 | - s = f({}) |
1878 | - self.assertTrue(isinstance(s, couchdb.Server)) |
1879 | - self.assertEqual(repr(s), "<Server 'http://localhost:5984/'>") |
1880 | - s = f({'url': 'http://localhost:5984/'}) |
1881 | - self.assertTrue(isinstance(s, couchdb.Server)) |
1882 | - self.assertEqual(repr(s), "<Server 'http://localhost:5984/'>") |
1883 | - |
1884 | - def test_get_dmedia_db(self): |
1885 | - f = abstractcouch.get_dmedia_db |
1886 | - |
1887 | - # Test when server is not provided |
1888 | - env = self.dc_env() |
1889 | - |
1890 | - assert 'dmedia' not in env |
1891 | - d = f(env) |
1892 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1893 | - self.assertEqual(repr(d), "<Database 'dmedia'>") |
1894 | - self.assertEqual(d.info()['db_name'], 'dmedia') |
1895 | - |
1896 | - env['dbname'] = None |
1897 | - d = f(env) |
1898 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1899 | - self.assertEqual(repr(d), "<Database 'dmedia'>") |
1900 | - self.assertEqual(d.info()['db_name'], 'dmedia') |
1901 | - |
1902 | - env['dbname'] = 'dmedia' |
1903 | - d = f(env) |
1904 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1905 | - self.assertEqual(repr(d), "<Database 'dmedia'>") |
1906 | - self.assertEqual(d.info()['db_name'], 'dmedia') |
1907 | - |
1908 | - env['dbname'] = 'dmedia_test' |
1909 | - d = f(env) |
1910 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1911 | - self.assertEqual(repr(d), "<Database 'dmedia_test'>") |
1912 | - self.assertEqual(d.info()['db_name'], 'dmedia_test') |
1913 | - |
1914 | - |
1915 | - # Test when server *is* provided |
1916 | - env = self.dc_env() |
1917 | - server = abstractcouch.get_couchdb_server(env) |
1918 | - |
1919 | - assert 'dmedia' not in env |
1920 | - d = f(env, server) |
1921 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1922 | - self.assertEqual(repr(d), "<Database 'dmedia'>") |
1923 | - self.assertEqual(d.info()['db_name'], 'dmedia') |
1924 | - |
1925 | - env['dbname'] = None |
1926 | - d = f(env, server) |
1927 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1928 | - self.assertEqual(repr(d), "<Database 'dmedia'>") |
1929 | - self.assertEqual(d.info()['db_name'], 'dmedia') |
1930 | - |
1931 | - env['dbname', server] = 'dmedia' |
1932 | - d = f(env) |
1933 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1934 | - self.assertEqual(repr(d), "<Database 'dmedia'>") |
1935 | - self.assertEqual(d.info()['db_name'], 'dmedia') |
1936 | - |
1937 | - env['dbname'] = 'dmedia_test' |
1938 | - d = f(env, server) |
1939 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1940 | - self.assertEqual(repr(d), "<Database 'dmedia_test'>") |
1941 | - self.assertEqual(d.info()['db_name'], 'dmedia_test') |
1942 | - |
1943 | - |
1944 | - # Test when server=None is explicitly provided |
1945 | - env = self.dc_env() |
1946 | - |
1947 | - assert 'dmedia' not in env |
1948 | - d = f(env, server=None) |
1949 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1950 | - self.assertEqual(repr(d), "<Database 'dmedia'>") |
1951 | - self.assertEqual(d.info()['db_name'], 'dmedia') |
1952 | - |
1953 | - env['dbname'] = None |
1954 | - d = f(env, server=None) |
1955 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1956 | - self.assertEqual(repr(d), "<Database 'dmedia'>") |
1957 | - self.assertEqual(d.info()['db_name'], 'dmedia') |
1958 | - |
1959 | - env['dbname'] = 'dmedia' |
1960 | - d = f(env, server=None) |
1961 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1962 | - self.assertEqual(repr(d), "<Database 'dmedia'>") |
1963 | - self.assertEqual(d.info()['db_name'], 'dmedia') |
1964 | - |
1965 | - env['dbname'] = 'dmedia_test' |
1966 | - d = f(env, server=None) |
1967 | - self.assertTrue(isinstance(d, couchdb.Database)) |
1968 | - self.assertEqual(repr(d), "<Database 'dmedia_test'>") |
1969 | - self.assertEqual(d.info()['db_name'], 'dmedia_test') |
1970 | - |
1971 | - def test_get_env(self): |
1972 | - f = abstractcouch.get_env |
1973 | - port = find_port() |
1974 | - url = 'http://localhost:%d/' % port |
1975 | - oauth = get_oauth_tokens() |
1976 | - |
1977 | - # Test when OAuthSession is available |
1978 | - self.assertEqual( |
1979 | - f(), |
1980 | - {'port': port, 'url': url, 'oauth': oauth, 'dbname': None} |
1981 | - ) |
1982 | - self.assertEqual( |
1983 | - f(dbname=None), |
1984 | - {'port': port, 'url': url, 'oauth': oauth, 'dbname': None} |
1985 | - ) |
1986 | - self.assertEqual( |
1987 | - f(dbname='dmedia'), |
1988 | - {'port': port, 'url': url, 'oauth': oauth, 'dbname': 'dmedia'} |
1989 | - ) |
1990 | - self.assertEqual( |
1991 | - f(dbname='dmedia_test'), |
1992 | - {'port': port, 'url': url, 'oauth': oauth, 'dbname': 'dmedia_test'} |
1993 | - ) |
1994 | - |
1995 | - # Test when OAuthSession is *not* available |
1996 | - abstractcouch.OAuthSession = None |
1997 | - self.assertEqual(f(), {'dbname': None}) |
1998 | - self.assertEqual(f(dbname=None), {'dbname': None}) |
1999 | - self.assertEqual(f(dbname='dmedia'), {'dbname': 'dmedia'}) |
2000 | - self.assertEqual(f(dbname='dmedia_test'), {'dbname': 'dmedia_test'}) |
2001 | + s = f({'url': 'http://localhost:666/'}) |
2002 | + self.assertTrue(isinstance(s, couchdb.Server)) |
2003 | + self.assertEqual(repr(s), "<Server 'http://localhost:666/'>") |
2004 | + |
2005 | + |
2006 | + def test_get_db(self): |
2007 | + f = abstractcouch.get_db |
2008 | + env = dc_env('test_dmedia') |
2009 | + server = abstractcouch.get_server(env) |
2010 | + |
2011 | + # Make sure database doesn't exist: |
2012 | + try: |
2013 | + server.delete('test_dmedia') |
2014 | + except ResourceNotFound: |
2015 | + pass |
2016 | + |
2017 | + # Test when db does not exist, server not provided |
2018 | + self.assertNotIn('test_dmedia', server) |
2019 | + db = f(env) |
2020 | + self.assertTrue(isinstance(db, couchdb.Database)) |
2021 | + self.assertEqual(repr(db), "<Database 'test_dmedia'>") |
2022 | + self.assertEqual(db.info()['db_name'], 'test_dmedia') |
2023 | + self.assertIn('test_dmedia', server) |
2024 | + |
2025 | + # Test when db exists, server not provided |
2026 | + db = f(env) |
2027 | + self.assertTrue(isinstance(db, couchdb.Database)) |
2028 | + self.assertEqual(repr(db), "<Database 'test_dmedia'>") |
2029 | + self.assertEqual(db.info()['db_name'], 'test_dmedia') |
2030 | + self.assertIn('test_dmedia', server) |
2031 | + |
2032 | + # Test when db does not exist, server *is* provided |
2033 | + server.delete('test_dmedia') |
2034 | + self.assertNotIn('test_dmedia', server) |
2035 | + db = f(env, server=server) |
2036 | + self.assertTrue(isinstance(db, couchdb.Database)) |
2037 | + self.assertEqual(repr(db), "<Database 'test_dmedia'>") |
2038 | + self.assertEqual(db.info()['db_name'], 'test_dmedia') |
2039 | + self.assertIn('test_dmedia', server) |
2040 | + |
2041 | + # Test when db exists, server *is* provided |
2042 | + db = f(env, server=server) |
2043 | + self.assertTrue(isinstance(db, couchdb.Database)) |
2044 | + self.assertEqual(repr(db), "<Database 'test_dmedia'>") |
2045 | + self.assertEqual(db.info()['db_name'], 'test_dmedia') |
2046 | + self.assertIn('test_dmedia', server) |
2047 | |
2048 | === added file 'dmedia/tests/test_api.py' |
2049 | --- dmedia/tests/test_api.py 1970-01-01 00:00:00 +0000 |
2050 | +++ dmedia/tests/test_api.py 2011-04-11 12:02:24 +0000 |
2051 | @@ -0,0 +1,125 @@ |
2052 | +# Authors: |
2053 | +# Jason Gerard DeRose <jderose@novacut.com> |
2054 | +# |
2055 | +# dmedia: distributed media library |
2056 | +# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com> |
2057 | +# |
2058 | +# This file is part of `dmedia`. |
2059 | +# |
2060 | +# `dmedia` is free software: you can redistribute it and/or modify it under the |
2061 | +# terms of the GNU Affero General Public License as published by the Free |
2062 | +# Software Foundation, either version 3 of the License, or (at your option) any |
2063 | +# later version. |
2064 | +# |
2065 | +# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY |
2066 | +# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR |
2067 | +# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more |
2068 | +# details. |
2069 | +# |
2070 | +# You should have received a copy of the GNU Affero General Public License along |
2071 | +# with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
2072 | + |
2073 | +""" |
2074 | +Unit tests for `dmedia.api` module. |
2075 | +""" |
2076 | + |
2077 | +import os |
2078 | +from os import path |
2079 | +from subprocess import Popen |
2080 | +import json |
2081 | +import time |
2082 | + |
2083 | +import gnomekeyring |
2084 | + |
2085 | +import dmedia |
2086 | +from dmedia.abstractcouch import get_db |
2087 | +from dmedia import api |
2088 | + |
2089 | +from .couch import CouchCase |
2090 | +from .helpers import random_bus |
2091 | + |
2092 | + |
2093 | +tree = path.dirname(path.dirname(path.abspath(dmedia.__file__))) |
2094 | +assert path.isfile(path.join(tree, 'setup.py')) |
2095 | +script = path.join(tree, 'dmedia-service') |
2096 | +assert path.isfile(script) |
2097 | + |
2098 | + |
2099 | +def get_auth(): |
2100 | + data = gnomekeyring.find_items_sync( |
2101 | + gnomekeyring.ITEM_GENERIC_SECRET, |
2102 | + {'desktopcouch': 'basic'} |
2103 | + ) |
2104 | + (user, password) = data[0].secret.split(':') |
2105 | + return (user, password) |
2106 | + |
2107 | + |
2108 | +class TestDMedia(CouchCase): |
2109 | + klass = api.DMedia |
2110 | + |
2111 | + def setUp(self): |
2112 | + """ |
2113 | + Launch dmedia dbus service using a random bus name. |
2114 | + |
2115 | + This will launch dmedia-service with a random bus name like this: |
2116 | + |
2117 | + dmedia-service --bus org.test3ISHAWZVSWVN5I5S.DMedia |
2118 | + |
2119 | + How do people usually unit test dbus services? This works, but not sure |
2120 | + if there is a better idiom in common use. --jderose |
2121 | + """ |
2122 | + super(TestDMedia, self).setUp() |
2123 | + self.bus = random_bus() |
2124 | + cmd = [script, |
2125 | + '--bus', self.bus, |
2126 | + '--env', json.dumps(self.env), |
2127 | + ] |
2128 | + self.service = Popen(cmd) |
2129 | + time.sleep(1) # Give dmedia-service time to start |
2130 | + |
2131 | + def tearDown(self): |
2132 | + super(TestDMedia, self).tearDown() |
2133 | + try: |
2134 | + self.service.terminate() |
2135 | + self.service.wait() |
2136 | + except OSError: |
2137 | + pass |
2138 | + finally: |
2139 | + self.service = None |
2140 | + |
2141 | + def test_all(self): |
2142 | + inst = self.klass(self.bus) |
2143 | + |
2144 | + # DMedia.Version() |
2145 | + self.assertEqual(inst.version(), dmedia.__version__) |
2146 | + |
2147 | + # DMedia.GetEnv() |
2148 | + env = inst.get_env() |
2149 | + self.assertEqual(env['oauth'], self.env['oauth']) |
2150 | + self.assertEqual(env['port'], self.env['port']) |
2151 | + self.assertEqual(env['url'], self.env['url']) |
2152 | + self.assertEqual(env['dbname'], self.env['dbname']) |
2153 | + |
2154 | + # DMedia.GetAuthURL() |
2155 | + (user, password) = get_auth() |
2156 | + self.assertEqual( |
2157 | + inst.get_auth_url(), |
2158 | + 'http://{user}:{password}@localhost:{port}/'.format( |
2159 | + user=user, password=password, port=self.env['port'] |
2160 | + ) |
2161 | + ) |
2162 | + |
2163 | + # DMedia.HasApp() |
2164 | + db = get_db(self.env) |
2165 | + self.assertNotIn('app', db) |
2166 | + self.assertTrue(inst.has_app()) |
2167 | + self.assertTrue(db['app']['_rev'].startswith('1-')) |
2168 | + self.assertTrue(inst.has_app()) |
2169 | + self.assertTrue(db['app']['_rev'].startswith('1-')) |
2170 | + |
2171 | + # DMedia.Kill() |
2172 | + self.assertIsNone(self.service.poll(), None) |
2173 | + inst.kill() |
2174 | + self.assertTrue(inst._proxy is None) |
2175 | + time.sleep(1) # Give dmedia-service time to shutdown |
2176 | + self.assertEqual(self.service.poll(), 0) |
2177 | |
2178 | === added file 'dmedia/tests/test_core.py' |
2179 | --- dmedia/tests/test_core.py 1970-01-01 00:00:00 +0000 |
2180 | +++ dmedia/tests/test_core.py 2011-04-11 12:02:24 +0000 |
2181 | @@ -0,0 +1,336 @@ |
2182 | +# Authors: |
2183 | +# Jason Gerard DeRose <jderose@novacut.com> |
2184 | +# |
2185 | +# dmedia: distributed media library |
2186 | +# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com> |
2187 | +# |
2188 | +# This file is part of `dmedia`. |
2189 | +# |
2190 | +# `dmedia` is free software: you can redistribute it and/or modify it under the |
2191 | +# terms of the GNU Affero General Public License as published by the Free |
2192 | +# Software Foundation, either version 3 of the License, or (at your option) any |
2193 | +# later version. |
2194 | +# |
2195 | +# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY |
2196 | +# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR |
2197 | +# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more |
2198 | +# details. |
2199 | +# |
2200 | +# You should have received a copy of the GNU Affero General Public License along |
2201 | +# with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
2202 | + |
2203 | +""" |
2204 | +Unit tests for `dmedia.core` module. |
2205 | +""" |
2206 | + |
2207 | +from unittest import TestCase |
2208 | +import json |
2209 | + |
2210 | +import couchdb |
2211 | +import desktopcouch |
2212 | +from desktopcouch.application.platform import find_port |
2213 | +from desktopcouch.application.local_files import get_oauth_tokens |
2214 | + |
2215 | +from dmedia.webui.app import App |
2216 | +from dmedia.schema import random_id |
2217 | +from dmedia import core |
2218 | + |
2219 | +from .couch import CouchCase |
2220 | + |
2221 | + |
2222 | +def dc_env(dbname): |
2223 | + """ |
2224 | + Create desktopcouch environment. |
2225 | + """ |
2226 | + port = find_port() |
2227 | + return { |
2228 | + 'dbname': dbname, |
2229 | + 'port': port, |
2230 | + 'url': 'http://localhost:%d/' % port, |
2231 | + 'oauth': get_oauth_tokens(), |
2232 | + } |
2233 | + |
2234 | + |
2235 | +class TestFunctions(TestCase): |
2236 | + def tearDown(self): |
2237 | + if core.desktopcouch is None: |
2238 | + core.desktopcouch = desktopcouch |
2239 | + |
2240 | + def test_get_env(self): |
2241 | + f = core.get_env |
2242 | + |
2243 | + # Test when desktopcouch is available |
2244 | + self.assertEqual(f(), dc_env('dmedia')) |
2245 | + self.assertEqual(f('foo'), dc_env('foo')) |
2246 | + self.assertEqual(f(dbname='bar'), dc_env('bar')) |
2247 | + |
2248 | + # Test when desktopcouch is available but no_dc=True |
2249 | + self.assertEqual( |
2250 | + f(no_dc=True), |
2251 | + { |
2252 | + 'dbname': 'dmedia', |
2253 | + 'port': 5984, |
2254 | + 'url': 'http://localhost:5984/', |
2255 | + } |
2256 | + ) |
2257 | + self.assertEqual( |
2258 | + f(dbname='foo', no_dc=True), |
2259 | + { |
2260 | + 'dbname': 'foo', |
2261 | + 'port': 5984, |
2262 | + 'url': 'http://localhost:5984/', |
2263 | + } |
2264 | + ) |
2265 | + self.assertEqual( |
2266 | + f('bar', True), |
2267 | + { |
2268 | + 'dbname': 'bar', |
2269 | + 'port': 5984, |
2270 | + 'url': 'http://localhost:5984/', |
2271 | + } |
2272 | + ) |
2273 | + |
2274 | + # Test when desktopcouch is *not* available |
2275 | + core.desktopcouch = None |
2276 | + self.assertEqual( |
2277 | + f(), |
2278 | + { |
2279 | + 'dbname': 'dmedia', |
2280 | + 'port': 5984, |
2281 | + 'url': 'http://localhost:5984/', |
2282 | + } |
2283 | + ) |
2284 | + self.assertEqual( |
2285 | + f('foo'), |
2286 | + { |
2287 | + 'dbname': 'foo', |
2288 | + 'port': 5984, |
2289 | + 'url': 'http://localhost:5984/', |
2290 | + } |
2291 | + ) |
2292 | + self.assertEqual( |
2293 | + f(dbname='bar'), |
2294 | + { |
2295 | + 'dbname': 'bar', |
2296 | + 'port': 5984, |
2297 | + 'url': 'http://localhost:5984/', |
2298 | + } |
2299 | + ) |
2300 | + |
2301 | + |
2302 | +class TestCore(CouchCase): |
2303 | + klass = core.Core |
2304 | + |
2305 | + def tearDown(self): |
2306 | + super(TestCore, self).tearDown() |
2307 | + if core.App is None: |
2308 | + core.App = App |
2309 | + |
2310 | + def test_init(self): |
2311 | + inst = self.klass(self.dbname) |
2312 | + self.assertNotEqual(inst.env, self.env) |
2313 | + self.assertEqual( |
2314 | + set(inst.env), |
2315 | + set(['port', 'url', 'dbname', 'oauth']) |
2316 | + ) |
2317 | + self.assertEqual(inst.env['dbname'], self.dbname) |
2318 | + self.assertEqual(inst.env['port'], self.env['port']) |
2319 | + self.assertEqual(inst.env['url'], self.env['url']) |
2320 | + self.assertEqual(inst.env['oauth'], self.env['oauth']) |
2321 | + self.assertEqual(inst.home, self.home.path) |
2322 | + self.assertIsInstance(inst.db, couchdb.Database) |
2323 | + |
2324 | + inst = self.klass(env_s=json.dumps(self.env)) |
2325 | + self.assertEqual(inst.env, self.env) |
2326 | + |
2327 | + def test_bootstrap(self): |
2328 | + inst = self.klass(self.dbname) |
2329 | + self.assertNotIn('machine_id', inst.env) |
2330 | + self.assertIsNone(inst.bootstrap()) |
2331 | + self.assertEqual(inst.env['machine_id'], inst.machine_id) |
2332 | + |
2333 | + def test_init_local(self): |
2334 | + inst = self.klass(self.dbname) |
2335 | + |
2336 | + # Test when _local/dmedia doesn't exist: |
2337 | + (local, machine) = inst.init_local() |
2338 | + |
2339 | + self.assertIsInstance(local, dict) |
2340 | + self.assertEqual( |
2341 | + set(local), |
2342 | + set([ |
2343 | + '_id', |
2344 | + '_rev', |
2345 | + 'machine', |
2346 | + 'filestores', |
2347 | + ]) |
2348 | + ) |
2349 | + self.assertEqual(local['filestores'], {}) |
2350 | + self.assertEqual(local, inst.db['_local/dmedia']) |
2351 | + |
2352 | + self.assertIsInstance(machine, dict) |
2353 | + self.assertEqual( |
2354 | + set(machine), |
2355 | + set([ |
2356 | + '_id', |
2357 | + '_rev', |
2358 | + 'ver', |
2359 | + 'type', |
2360 | + 'time', |
2361 | + 'hostname', |
2362 | + 'distribution', |
2363 | + ]) |
2364 | + ) |
2365 | + self.assertEqual(machine, inst.db[local['machine']['_id']]) |
2366 | + |
2367 | + # Test when _local/machine exists but 'dmedia/machine' doc doesn't: |
2368 | + inst.db.delete(machine) |
2369 | + (local2, machine2) = inst.init_local() |
2370 | + self.assertEqual(local2, local) |
2371 | + self.assertTrue(machine2['_rev'].startswith('3-')) |
2372 | + self.assertNotEqual(machine2['_rev'], machine['_rev']) |
2373 | + d = dict(machine2) |
2374 | + d.pop('_rev') |
2375 | + self.assertEqual(d, local['machine']) |
2376 | + |
2377 | + # Test when both _local/dmedia and 'dmedia/machine' doc exist: |
2378 | + local3 = { |
2379 | + '_id': '_local/dmedia', |
2380 | + '_rev': local2['_rev'], |
2381 | + 'machine': { |
2382 | + '_id': 'foobar', |
2383 | + 'hello': 'world', |
2384 | + } |
2385 | + } |
2386 | + inst.db.save(local3) |
2387 | + machine3 = { |
2388 | + '_id': 'foobar', |
2389 | + '_rev': machine2['_rev'], |
2390 | + 'hello': 'naughty nurse', |
2391 | + } |
2392 | + inst.db.save(machine3) |
2393 | + (local4, machine4) = inst.init_local() |
2394 | + self.assertEqual(local4, local3) |
2395 | + self.assertEqual(machine4, machine3) |
2396 | + |
2397 | + def test_init_filestores(self): |
2398 | + inst = self.klass(self.dbname) |
2399 | + (inst.local, inst.machine) = inst.init_local() |
2400 | + inst.machine_id = inst.machine['_id'] |
2401 | + inst.env['machine_id'] = inst.machine_id |
2402 | + |
2403 | + self.assertEqual(inst.local['filestores'], {}) |
2404 | + self.assertNotIn('default_filestore', inst.local) |
2405 | + lstore = inst.init_filestores() |
2406 | + self.assertEqual(inst.local, inst.db['_local/dmedia']) |
2407 | + self.assertEqual(len(inst.local['filestores']), 1) |
2408 | + _id = inst.local['default_filestore'] |
2409 | + self.assertEqual(inst.local['filestores'][_id], lstore) |
2410 | + self.assertEqual( |
2411 | + set(lstore), |
2412 | + set([ |
2413 | + '_id', |
2414 | + 'ver', |
2415 | + 'type', |
2416 | + 'time', |
2417 | + 'plugin', |
2418 | + 'copies', |
2419 | + 'path', |
2420 | + 'machine_id', |
2421 | + ]) |
2422 | + ) |
2423 | + self.assertEqual(lstore['ver'], 0) |
2424 | + self.assertEqual(lstore['type'], 'dmedia/store') |
2425 | + self.assertEqual(lstore['plugin'], 'filestore') |
2426 | + self.assertEqual(lstore['copies'], 1) |
2427 | + self.assertEqual(lstore['path'], self.home.path) |
2428 | + self.assertEqual(lstore['machine_id'], inst.machine_id) |
2429 | + |
2430 | + store = inst.db[_id] |
2431 | + self.assertTrue(store['_rev'].startswith('1-')) |
2432 | + store.pop('_rev') |
2433 | + self.assertEqual(store, lstore) |
2434 | + |
2435 | + # Try again when docs already exist: |
2436 | + self.assertEqual(inst.init_filestores(), lstore) |
2437 | + |
2438 | + def test_init_app(self): |
2439 | + inst = self.klass(self.dbname) |
2440 | + |
2441 | + # App is available |
2442 | + self.assertNotIn('app', inst.db) |
2443 | + self.assertIs(inst.init_app(), True) |
2444 | + self.assertIn('app', inst.db) |
2445 | + self.assertIs(inst.init_app(), True) |
2446 | + |
2447 | + # App is not available |
2448 | + core.App = None |
2449 | + self.assertIs(inst.init_app(), False) |
2450 | + |
2451 | + # App is available again (make sure there is no state) |
2452 | + core.App = App |
2453 | + self.assertIs(inst.init_app(), True) |
2454 | + self.assertIs(inst.init_app(), True) |
2455 | + |
2456 | + def test_has_app(self): |
2457 | + class Sub(self.klass): |
2458 | + _calls = 0 |
2459 | + |
2460 | + def init_app(self): |
2461 | + self._calls += 1 |
2462 | + return 'A' * self._calls |
2463 | + |
2464 | + inst = Sub(self.dbname) |
2465 | + self.assertIsNone(inst._has_app) |
2466 | + |
2467 | + inst._has_app = 'foo' |
2468 | + self.assertEqual(inst.has_app(), 'foo') |
2469 | + self.assertEqual(inst._calls, 0) |
2470 | + |
2471 | + inst._has_app = None |
2472 | + self.assertEqual(inst.has_app(), 'A') |
2473 | + self.assertEqual(inst._calls, 1) |
2474 | + self.assertEqual(inst._has_app, 'A') |
2475 | + |
2476 | + self.assertEqual(inst.has_app(), 'A') |
2477 | + self.assertEqual(inst._calls, 1) |
2478 | + self.assertEqual(inst._has_app, 'A') |
2479 | + |
2480 | + inst._has_app = None |
2481 | + self.assertEqual(inst.has_app(), 'AA') |
2482 | + self.assertEqual(inst._calls, 2) |
2483 | + self.assertEqual(inst._has_app, 'AA') |
2484 | + |
2485 | + self.assertEqual(inst.has_app(), 'AA') |
2486 | + self.assertEqual(inst._calls, 2) |
2487 | + self.assertEqual(inst._has_app, 'AA') |
2488 | + |
2489 | + |
2490 | + # Test the real thing, no App |
2491 | + core.App = None |
2492 | + inst = self.klass(self.dbname) |
2493 | + self.assertIs(inst.has_app(), False) |
2494 | + self.assertIs(inst._has_app, False) |
2495 | + self.assertNotIn('app', inst.db) |
2496 | + |
2497 | + |
2498 | + # Test the real thing, App available |
2499 | + core.App = App |
2500 | + inst = self.klass(self.dbname) |
2501 | + |
2502 | + self.assertNotIn('app', inst.db) |
2503 | + self.assertIs(inst.has_app(), True) |
2504 | + self.assertIs(inst._has_app, True) |
2505 | + rev = inst.db['app']['_rev'] |
2506 | + self.assertTrue(rev.startswith('1-')) |
2507 | + |
2508 | + self.assertIs(inst.has_app(), True) |
2509 | + self.assertIs(inst._has_app, True) |
2510 | + self.assertEqual(inst.db['app']['_rev'], rev) |
2511 | + |
2512 | + inst._has_app = None |
2513 | + self.assertIs(inst.has_app(), True) |
2514 | + self.assertIs(inst._has_app, True) |
2515 | + rev2 = inst.db['app']['_rev'] |
2516 | + self.assertNotEqual(rev2, rev) |
2517 | + self.assertTrue(rev2.startswith('2-')) |
2518 | |
2519 | === modified file 'dmedia/tests/test_downloader.py' |
2520 | --- dmedia/tests/test_downloader.py 2011-02-22 14:07:47 +0000 |
2521 | +++ dmedia/tests/test_downloader.py 2011-04-11 12:02:24 +0000 |
2522 | @@ -175,8 +175,8 @@ |
2523 | tmp = TempDir() |
2524 | fs = FileStore(tmp.path) |
2525 | inst = self.klass('', fs, mov_hash, 'mov') |
2526 | - d = tmp.join('transfers') |
2527 | - f = tmp.join('transfers', mov_hash + '.mov') |
2528 | + d = tmp.join('.dmedia', 'transfers') |
2529 | + f = tmp.join('.dmedia', 'transfers', mov_hash + '.mov') |
2530 | self.assertFalse(path.exists(d)) |
2531 | self.assertFalse(path.exists(f)) |
2532 | self.assertEqual(inst.get_tmp(), f) |
2533 | @@ -187,8 +187,8 @@ |
2534 | tmp = TempDir() |
2535 | fs = FileStore(tmp.path) |
2536 | inst = self.klass('', fs, mov_hash) |
2537 | - d = tmp.join('transfers') |
2538 | - f = tmp.join('transfers', mov_hash) |
2539 | + d = tmp.join('.dmedia', 'transfers') |
2540 | + f = tmp.join('.dmedia', 'transfers', mov_hash) |
2541 | self.assertFalse(path.exists(d)) |
2542 | self.assertFalse(path.exists(f)) |
2543 | self.assertEqual(inst.get_tmp(), f) |
2544 | @@ -200,10 +200,10 @@ |
2545 | fs = FileStore(tmp.path) |
2546 | inst = self.klass('', fs, mov_hash, 'mov') |
2547 | |
2548 | - src_d = tmp.join('transfers') |
2549 | - src = tmp.join('transfers', mov_hash + '.mov') |
2550 | - dst_d = tmp.join(mov_hash[:2]) |
2551 | - dst = tmp.join(mov_hash[:2], mov_hash[2:] + '.mov') |
2552 | + src_d = tmp.join('.dmedia', 'transfers') |
2553 | + src = tmp.join('.dmedia', 'transfers', mov_hash + '.mov') |
2554 | + dst_d = tmp.join('.dmedia', mov_hash[:2]) |
2555 | + dst = tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov') |
2556 | |
2557 | # Test when transfers/ dir doesn't exist: |
2558 | e = raises(IOError, inst.finalize) |
2559 | |
2560 | === modified file 'dmedia/tests/test_filestore.py' |
2561 | --- dmedia/tests/test_filestore.py 2011-02-27 03:47:55 +0000 |
2562 | +++ dmedia/tests/test_filestore.py 2011-04-11 12:02:24 +0000 |
2563 | @@ -39,7 +39,7 @@ |
2564 | from dmedia.errors import AmbiguousPath, FileStoreTraversal |
2565 | from dmedia.errors import DuplicateFile, IntegrityError |
2566 | from dmedia.filestore import HashList |
2567 | -from dmedia import filestore, constants, schema |
2568 | +from dmedia import filestore, constants |
2569 | from dmedia.constants import TYPE_ERROR, EXT_PAT, LEAF_SIZE |
2570 | |
2571 | |
2572 | @@ -60,7 +60,6 @@ |
2573 | # Test with normalized absolute path: |
2574 | self.assertEqual(f('/home/jderose/.dmedia'), '/home/jderose/.dmedia') |
2575 | |
2576 | - |
2577 | def test_safe_open(self): |
2578 | f = filestore.safe_open |
2579 | tmp = TempDir() |
2580 | @@ -501,43 +500,66 @@ |
2581 | self.assertEqual(e.pathname, '/foo/bar/../../root') |
2582 | self.assertEqual(e.abspath, '/root') |
2583 | |
2584 | - # Test when base is a file |
2585 | + # Test when parent does not exist |
2586 | + tmp = TempDir() |
2587 | + parent = tmp.join('foo') |
2588 | + e = raises(ValueError, self.klass, parent) |
2589 | + self.assertEqual( |
2590 | + str(e), |
2591 | + 'FileStore.parent not a directory: %r' % parent |
2592 | + ) |
2593 | + |
2594 | + # Test when parent is a file |
2595 | + tmp = TempDir() |
2596 | + parent = tmp.touch('foo') |
2597 | + e = raises(ValueError, self.klass, parent) |
2598 | + self.assertEqual( |
2599 | + str(e), |
2600 | + 'FileStore.parent not a directory: %r' % parent |
2601 | + ) |
2602 | + |
2603 | + # Test when parent=None |
2604 | + inst = self.klass() |
2605 | + self.assertTrue(path.isdir(inst.parent)) |
2606 | + self.assertTrue(inst.parent.startswith('/tmp/store.')) |
2607 | + base = path.join(inst.parent, '.dmedia') |
2608 | + self.assertTrue(path.isdir(base)) |
2609 | + |
2610 | + # Test when base does not exist |
2611 | + tmp = TempDir() |
2612 | + base = tmp.join('.dmedia') |
2613 | + inst = self.klass(tmp.path) |
2614 | + self.assertEqual(inst.parent, tmp.path) |
2615 | + self.assertEqual(inst.base, base) |
2616 | + self.assertTrue(path.isdir(inst.base)) |
2617 | + |
2618 | + # Test when base exists and is a directory |
2619 | + tmp = TempDir() |
2620 | + base = tmp.makedirs('.dmedia') |
2621 | + inst = self.klass(tmp.path) |
2622 | + self.assertEqual(inst.parent, tmp.path) |
2623 | + self.assertEqual(inst.base, base) |
2624 | + self.assertTrue(path.isdir(inst.base)) |
2625 | + |
2626 | + # Test when base exists and is a file |
2627 | tmp = TempDir() |
2628 | base = tmp.touch('.dmedia') |
2629 | - e = raises(ValueError, self.klass, base) |
2630 | + e = raises(ValueError, self.klass, tmp.path) |
2631 | self.assertEqual( |
2632 | str(e), |
2633 | 'FileStore.base not a directory: %r' % base |
2634 | ) |
2635 | |
2636 | - # Test when base does not exist |
2637 | + # Test when base exists and is a symlink to a dir |
2638 | tmp = TempDir() |
2639 | + d = tmp.makedirs('foo') |
2640 | base = tmp.join('.dmedia') |
2641 | - record = tmp.join('.dmedia', 'store.json') |
2642 | - inst = self.klass(base) |
2643 | - self.assertEqual(inst.base, base) |
2644 | - self.assertTrue(path.isdir(inst.base)) |
2645 | - self.assertEqual(inst.record, record) |
2646 | - self.assertTrue(path.isfile(record)) |
2647 | - store_s = open(record, 'rb').read() |
2648 | - doc = json.loads(store_s) |
2649 | - self.assertEqual(schema.check_dmedia_store(doc), None) |
2650 | - self.assertEqual(inst._doc, doc) |
2651 | - self.assertEqual(inst._id, doc['_id']) |
2652 | - |
2653 | - # Test when base exists and is a directory |
2654 | - inst = self.klass(base) |
2655 | - self.assertEqual(inst.base, base) |
2656 | - self.assertTrue(path.isdir(inst.base)) |
2657 | - self.assertEqual(inst.record, record) |
2658 | - self.assertTrue(path.isfile(record)) |
2659 | - self.assertEqual(open(record, 'rb').read(), store_s) |
2660 | - |
2661 | - # Test when base=None |
2662 | - inst = self.klass() |
2663 | - self.assertTrue(path.isdir(inst.base)) |
2664 | - self.assertTrue(inst.base.startswith('/tmp/store.')) |
2665 | - self.assertEqual(inst.record, path.join(inst.base, 'store.json')) |
2666 | + os.symlink(d, base) |
2667 | + e = raises(ValueError, self.klass, tmp.path) |
2668 | + self.assertEqual( |
2669 | + str(e), |
2670 | + '{!r} is symlink to {!r}'.format(base, d) |
2671 | + ) |
2672 | |
2673 | def test_repr(self): |
2674 | tmp = TempDir() |
2675 | @@ -551,29 +573,31 @@ |
2676 | # Methods to prevent path traversals attacks |
2677 | def test_check_path(self): |
2678 | tmp = TempDir() |
2679 | - base = tmp.join('foo', 'bar') |
2680 | - inst = self.klass(base) |
2681 | + parent = tmp.makedirs('foo') |
2682 | + base = tmp.join('foo', '.dmedia') |
2683 | + inst = self.klass(parent) |
2684 | |
2685 | - bad = tmp.join('foo', 'barNone', 'stuff') |
2686 | + bad = tmp.join('foo', '.dmedia2', 'stuff') |
2687 | e = raises(FileStoreTraversal, inst.check_path, bad) |
2688 | self.assertEqual(e.pathname, bad) |
2689 | self.assertEqual(e.abspath, bad) |
2690 | self.assertEqual(e.base, base) |
2691 | |
2692 | - bad = tmp.join('foo', 'bar', '..', 'barNone') |
2693 | + bad = tmp.join('foo', '.dmedia', '..', '.dmedia2') |
2694 | assert '..' in bad |
2695 | e = raises(FileStoreTraversal, inst.check_path, bad) |
2696 | self.assertEqual(e.pathname, bad) |
2697 | - self.assertEqual(e.abspath, tmp.join('foo', 'barNone')) |
2698 | + self.assertEqual(e.abspath, tmp.join('foo', '.dmedia2')) |
2699 | self.assertEqual(e.base, base) |
2700 | |
2701 | - good = tmp.join('foo', 'bar', 'stuff') |
2702 | + good = tmp.join('foo', '.dmedia', 'stuff') |
2703 | self.assertEqual(inst.check_path(good), good) |
2704 | |
2705 | def test_join(self): |
2706 | tmp = TempDir() |
2707 | - base = tmp.join('foo', 'bar') |
2708 | - inst = self.klass(base) |
2709 | + parent = tmp.makedirs('foo') |
2710 | + base = tmp.join('foo', '.dmedia') |
2711 | + inst = self.klass(parent) |
2712 | |
2713 | # Test with an absolute path in parts: |
2714 | e = raises(FileStoreTraversal, inst.join, 'dmedia', '/root') |
2715 | @@ -585,31 +609,32 @@ |
2716 | e = raises(FileStoreTraversal, inst.join, 'NW', '..', '..', '.ssh') |
2717 | self.assertEqual( |
2718 | e.pathname, |
2719 | - tmp.join('foo', 'bar', 'NW', '..', '..', '.ssh') |
2720 | + tmp.join('foo', '.dmedia', 'NW', '..', '..', '.ssh') |
2721 | ) |
2722 | self.assertEqual(e.abspath, tmp.join('foo', '.ssh')) |
2723 | self.assertEqual(e.base, base) |
2724 | |
2725 | # Test for former security issue! See: |
2726 | # https://bugs.launchpad.net/dmedia/+bug/708663 |
2727 | - e = raises(FileStoreTraversal, inst.join, '..', 'barNone', 'stuff') |
2728 | + e = raises(FileStoreTraversal, inst.join, '..', '.dmedia2', 'stuff') |
2729 | self.assertEqual( |
2730 | e.pathname, |
2731 | - tmp.join('foo', 'bar', '..', 'barNone', 'stuff') |
2732 | + tmp.join('foo', '.dmedia', '..', '.dmedia2', 'stuff') |
2733 | ) |
2734 | - self.assertEqual(e.abspath, tmp.join('foo', 'barNone', 'stuff')) |
2735 | + self.assertEqual(e.abspath, tmp.join('foo', '.dmedia2', 'stuff')) |
2736 | self.assertEqual(e.base, base) |
2737 | |
2738 | # Test with some correct parts: |
2739 | self.assertEqual( |
2740 | inst.join('NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW'), |
2741 | - tmp.join('foo', 'bar', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2742 | + tmp.join('foo', '.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2743 | ) |
2744 | |
2745 | def test_create_parent(self): |
2746 | tmp = TempDir() |
2747 | tmp2 = TempDir() |
2748 | inst = self.klass(tmp.path) |
2749 | + base = tmp.join('.dmedia') |
2750 | |
2751 | # Test with a normpath but outside of base: |
2752 | f = tmp2.join('foo', 'bar') |
2753 | @@ -619,7 +644,7 @@ |
2754 | e = raises(FileStoreTraversal, inst.create_parent, f) |
2755 | self.assertEqual(e.pathname, f) |
2756 | self.assertEqual(e.abspath, f) |
2757 | - self.assertEqual(e.base, tmp.path) |
2758 | + self.assertEqual(e.base, base) |
2759 | self.assertFalse(path.exists(f)) |
2760 | self.assertFalse(path.exists(d)) |
2761 | |
2762 | @@ -632,13 +657,13 @@ |
2763 | e = raises(FileStoreTraversal, inst.create_parent, f) |
2764 | self.assertEqual(e.pathname, f) |
2765 | self.assertEqual(e.abspath, tmp2.join('baz', 'f')) |
2766 | - self.assertEqual(e.base, tmp.path) |
2767 | + self.assertEqual(e.base, base) |
2768 | self.assertFalse(path.exists(f)) |
2769 | self.assertFalse(path.exists(d)) |
2770 | |
2771 | # Test with some correct parts: |
2772 | - f = tmp.join('NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2773 | - d = tmp.join('NW') |
2774 | + f = tmp.join('.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2775 | + d = tmp.join('.dmedia', 'NW') |
2776 | self.assertFalse(path.exists(f)) |
2777 | self.assertFalse(path.exists(d)) |
2778 | self.assertEqual(inst.create_parent(f), d) |
2779 | @@ -649,8 +674,8 @@ |
2780 | self.assertTrue(path.isdir(d)) |
2781 | |
2782 | # Confirm that it's using os.makedirs(), not os.mkdir() |
2783 | - f = tmp.join('OM', 'LU', 'WE', 'IP') |
2784 | - d = tmp.join('OM', 'LU', 'WE') |
2785 | + f = tmp.join('.dmedia', 'OM', 'LU', 'WE', 'IP') |
2786 | + d = tmp.join('.dmedia', 'OM', 'LU', 'WE') |
2787 | self.assertFalse(path.exists(f)) |
2788 | self.assertFalse(path.exists(d)) |
2789 | self.assertEqual(inst.create_parent(f), d) |
2790 | @@ -661,18 +686,19 @@ |
2791 | self.assertTrue(path.isdir(d)) |
2792 | |
2793 | # Test with 1-deep: |
2794 | - f = tmp.join('woot') |
2795 | + f = tmp.join('.dmedia', 'woot') |
2796 | self.assertFalse(path.exists(f)) |
2797 | - self.assertEqual(inst.create_parent(f), tmp.path) |
2798 | + self.assertEqual(inst.create_parent(f), base) |
2799 | self.assertFalse(path.exists(f)) |
2800 | |
2801 | # Test for former security issue! See: |
2802 | # https://bugs.launchpad.net/dmedia/+bug/708663 |
2803 | tmp = TempDir() |
2804 | - base = tmp.join('foo', 'bar') |
2805 | - bad = tmp.join('foo', 'barNone', 'stuff') |
2806 | - baddir = tmp.join('foo', 'barNone') |
2807 | - inst = self.klass(base) |
2808 | + parent = tmp.makedirs('foo') |
2809 | + base = tmp.join('foo', '.dmedia') |
2810 | + bad = tmp.join('foo', '.dmedia2', 'stuff') |
2811 | + baddir = tmp.join('foo', '.dmedia2') |
2812 | + inst = self.klass(parent) |
2813 | e = raises(FileStoreTraversal, inst.create_parent, bad) |
2814 | self.assertEqual(e.pathname, bad) |
2815 | self.assertEqual(e.abspath, bad) |
2816 | @@ -717,16 +743,17 @@ |
2817 | |
2818 | def test_path(self): |
2819 | tmp = TempDir() |
2820 | - base = tmp.join('foo', 'bar') |
2821 | - inst = self.klass(base) |
2822 | + parent = tmp.makedirs('foo') |
2823 | + base = tmp.join('foo', '.dmedia') |
2824 | + inst = self.klass(parent) |
2825 | |
2826 | self.assertEqual( |
2827 | inst.path('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW'), |
2828 | - tmp.join('foo', 'bar', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2829 | + tmp.join('foo', '.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2830 | ) |
2831 | self.assertEqual( |
2832 | inst.path('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW', ext='ogv'), |
2833 | - tmp.join('foo', 'bar', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW.ogv') |
2834 | + tmp.join('foo', '.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW.ogv') |
2835 | ) |
2836 | |
2837 | # Test to make sure hashes are getting checked with safe_b32(): |
2838 | @@ -755,8 +782,8 @@ |
2839 | tmp = TempDir() |
2840 | inst = self.klass(tmp.path) |
2841 | |
2842 | - f = tmp.join('NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2843 | - d = tmp.join('NW') |
2844 | + f = tmp.join('.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2845 | + d = tmp.join('.dmedia', 'NW') |
2846 | self.assertFalse(path.exists(f)) |
2847 | self.assertFalse(path.exists(d)) |
2848 | self.assertEqual( |
2849 | @@ -780,7 +807,7 @@ |
2850 | self.assertFalse(inst.exists(mov_hash, 'mov')) |
2851 | |
2852 | # File exists |
2853 | - tmp.touch(mov_hash[:2], mov_hash[2:] + '.mov') |
2854 | + tmp.touch('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov') |
2855 | self.assertTrue(inst.exists(mov_hash, 'mov')) |
2856 | |
2857 | def test_open(self): |
2858 | @@ -834,7 +861,7 @@ |
2859 | self.assertEqual(e.errno, 2) |
2860 | |
2861 | # File exists |
2862 | - f = tmp.touch(mov_hash[:2], mov_hash[2:] + '.mov') |
2863 | + f = tmp.touch('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov') |
2864 | self.assertTrue(path.isfile(f)) |
2865 | inst.remove(mov_hash, 'mov') |
2866 | self.assertFalse(path.exists(f)) |
2867 | @@ -880,11 +907,13 @@ |
2868 | |
2869 | self.assertEqual( |
2870 | inst.tmp('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW'), |
2871 | - tmp.join('transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2872 | + tmp.join('.dmedia', 'transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2873 | ) |
2874 | self.assertEqual( |
2875 | inst.tmp('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW', ext='ogv'), |
2876 | - tmp.join('transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW.ogv') |
2877 | + tmp.join( |
2878 | + '.dmedia', 'transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW.ogv' |
2879 | + ) |
2880 | ) |
2881 | |
2882 | # Test to make sure hashes are getting checked with safe_b32(): |
2883 | @@ -913,8 +942,8 @@ |
2884 | tmp = TempDir() |
2885 | inst = self.klass(tmp.path) |
2886 | |
2887 | - f = tmp.join('transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2888 | - d = tmp.join('transfers') |
2889 | + f = tmp.join('.dmedia', 'transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW') |
2890 | + d = tmp.join('.dmedia', 'transfers') |
2891 | self.assertFalse(path.exists(f)) |
2892 | self.assertFalse(path.exists(d)) |
2893 | self.assertEqual( |
2894 | @@ -935,7 +964,7 @@ |
2895 | |
2896 | tmp = TempDir() |
2897 | inst = self.klass(tmp.path) |
2898 | - filename = tmp.join('transfers', chash) |
2899 | + filename = tmp.join('.dmedia', 'transfers', chash) |
2900 | |
2901 | # Test when file dosen't yet exist |
2902 | fp = inst.allocate_for_transfer(2311, chash) |
2903 | @@ -965,7 +994,7 @@ |
2904 | |
2905 | tmp = TempDir() |
2906 | inst = self.klass(tmp.path) |
2907 | - filename = tmp.join('transfers', chash + '.mov') |
2908 | + filename = tmp.join('.dmedia', 'transfers', chash + '.mov') |
2909 | |
2910 | # Test when file dosen't yet exist |
2911 | fp = inst.allocate_for_transfer(2311, chash, ext='mov') |
2912 | @@ -992,7 +1021,7 @@ |
2913 | |
2914 | def test_allocate_for_import(self): |
2915 | tmp = TempDir() |
2916 | - imports = tmp.join('imports') |
2917 | + imports = tmp.join('.dmedia', 'imports') |
2918 | |
2919 | inst = self.klass(tmp.path) |
2920 | self.assertFalse(path.isdir(imports)) |
2921 | @@ -1020,7 +1049,7 @@ |
2922 | |
2923 | def test_allocate_for_write(self): |
2924 | tmp = TempDir() |
2925 | - writes = tmp.join('writes') |
2926 | + writes = tmp.join('.dmedia', 'writes') |
2927 | |
2928 | inst = self.klass(tmp.path) |
2929 | self.assertFalse(path.isdir(writes)) |
2930 | @@ -1052,7 +1081,7 @@ |
2931 | base = tmp.join('.dmedia') |
2932 | dst_d = tmp.join('.dmedia', mov_hash[:2]) |
2933 | dst = tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov') |
2934 | - inst = self.klass(base) |
2935 | + inst = self.klass(tmp.path) |
2936 | |
2937 | # Test with wrong tmp_fp type |
2938 | e = raises(TypeError, inst.tmp_move, 17, mov_hash, 'mov') |
2939 | @@ -1157,10 +1186,10 @@ |
2940 | tmp = TempDir() |
2941 | inst = self.klass(tmp.path) |
2942 | |
2943 | - src_d = tmp.join('transfers') |
2944 | - src = tmp.join('transfers', mov_hash + '.mov') |
2945 | - dst_d = tmp.join(mov_hash[:2]) |
2946 | - dst = tmp.join(mov_hash[:2], mov_hash[2:] + '.mov') |
2947 | + src_d = tmp.join('.dmedia', 'transfers') |
2948 | + src = tmp.join('.dmedia', 'transfers', mov_hash + '.mov') |
2949 | + dst_d = tmp.join('.dmedia', mov_hash[:2]) |
2950 | + dst = tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov') |
2951 | |
2952 | # Test when transfers/ dir doesn't exist: |
2953 | e = raises(IOError, inst.tmp_verify_move, mov_hash, 'mov') |
2954 | @@ -1235,7 +1264,7 @@ |
2955 | dst = tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov') |
2956 | shutil.copy(sample_mov, src) |
2957 | |
2958 | - inst = self.klass(base) |
2959 | + inst = self.klass(tmp.path) |
2960 | self.assertTrue(path.isfile(src)) |
2961 | self.assertTrue(path.isdir(base)) |
2962 | self.assertFalse(path.exists(dst)) |
2963 | |
2964 | === modified file 'dmedia/tests/test_importer.py' |
2965 | --- dmedia/tests/test_importer.py 2011-04-06 12:35:49 +0000 |
2966 | +++ dmedia/tests/test_importer.py 2011-04-11 12:02:24 +0000 |
2967 | @@ -40,7 +40,7 @@ |
2968 | from dmedia.filestore import FileStore |
2969 | from dmedia.schema import random_id |
2970 | from dmedia import importer, schema |
2971 | -from dmedia.abstractcouch import get_env, get_dmedia_db |
2972 | +from dmedia.abstractcouch import get_db |
2973 | from .helpers import TempDir, TempHome, raises |
2974 | from .helpers import DummyQueue, DummyCallback, prep_import_source |
2975 | from .helpers import sample_mov, sample_thm |
2976 | @@ -203,8 +203,8 @@ |
2977 | self.assertTrue(isinstance(inst.server, couchdb.Server)) |
2978 | self.assertTrue(isinstance(inst.db, couchdb.Database)) |
2979 | |
2980 | - self.assertEqual(inst.home, self.home.path) |
2981 | self.assertTrue(isinstance(inst.filestore, FileStore)) |
2982 | + self.assertEqual(inst.filestore.parent, self.home.path) |
2983 | self.assertEqual(inst.filestore.base, self.home.join('.dmedia')) |
2984 | |
2985 | # Test with extract = False |
2986 | @@ -302,7 +302,7 @@ |
2987 | self.assertTrue(inst.doc is None) |
2988 | _id = inst.start() |
2989 | self.assertEqual(len(_id), 24) |
2990 | - db = get_dmedia_db(self.env) |
2991 | + db = get_db(self.env) |
2992 | self.assertEqual(inst.doc, db[_id]) |
2993 | self.assertEqual( |
2994 | set(inst.doc), |
2995 | @@ -459,7 +459,7 @@ |
2996 | old['stored'] = {rid: {'copies': 2, 'time': 1234567890}} |
2997 | inst.db.save(old) |
2998 | (action, doc) = inst._import_file(src2) |
2999 | - fid = inst.filestore._id |
3000 | + fid = inst.filestore_id |
3001 | self.assertEqual(action, 'skipped') |
3002 | self.assertEqual(set(doc['stored']), set([rid, fid])) |
3003 | t = doc['stored'][fid]['time'] |
3004 | |
3005 | === modified file 'dmedia/tests/test_transcoder.py' |
3006 | --- dmedia/tests/test_transcoder.py 2011-02-27 13:29:05 +0000 |
3007 | +++ dmedia/tests/test_transcoder.py 2011-04-11 12:02:24 +0000 |
3008 | @@ -224,13 +224,13 @@ |
3009 | self.assertEqual(inst.src_fp.mode, 'rb') |
3010 | self.assertEqual( |
3011 | inst.src_fp.name, |
3012 | - self.tmp.join(mov_hash[:2], mov_hash[2:] + '.mov') |
3013 | + self.tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov') |
3014 | ) |
3015 | |
3016 | self.assertTrue(isinstance(inst.dst_fp, file)) |
3017 | self.assertEqual(inst.dst_fp.mode, 'r+b') |
3018 | self.assertTrue( |
3019 | - inst.dst_fp.name.startswith(self.tmp.join('writes')) |
3020 | + inst.dst_fp.name.startswith(self.tmp.join('.dmedia', 'writes')) |
3021 | ) |
3022 | self.assertTrue(inst.dst_fp.name.endswith('.ogv')) |
3023 | |
3024 | |
3025 | === renamed file 'dmedia/tests/test_metastore.py' => 'dmedia/tests/test_views.py' |
3026 | --- dmedia/tests/test_metastore.py 2011-02-20 19:18:43 +0000 |
3027 | +++ dmedia/tests/test_views.py 2011-04-11 12:02:24 +0000 |
3028 | @@ -2,7 +2,7 @@ |
3029 | # Jason Gerard DeRose <jderose@novacut.com> |
3030 | # |
3031 | # dmedia: distributed media library |
3032 | -# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com> |
3033 | +# Copyright (C) 2010, 2011 Jason Gerard DeRose <jderose@novacut.com> |
3034 | # |
3035 | # This file is part of `dmedia`. |
3036 | # |
3037 | @@ -20,150 +20,79 @@ |
3038 | # with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
3039 | |
3040 | """ |
3041 | -Unit tests for `dmedia.metastore` module. |
3042 | +Unit tests for `dmedia.views` module. |
3043 | """ |
3044 | |
3045 | from unittest import TestCase |
3046 | -import os |
3047 | -import shutil |
3048 | -import socket |
3049 | -import platform |
3050 | - |
3051 | -import couchdb |
3052 | - |
3053 | -from dmedia import metastore |
3054 | -from .helpers import TempDir, TempHome |
3055 | + |
3056 | +from dmedia.abstractcouch import get_db |
3057 | +from dmedia import views |
3058 | + |
3059 | from .couch import CouchCase |
3060 | |
3061 | |
3062 | class test_functions(TestCase): |
3063 | def test_build_design_doc(self): |
3064 | - f = metastore.build_design_doc |
3065 | - views = ( |
3066 | + f = views.build_design_doc |
3067 | + views_ = ( |
3068 | ('bytes', 'foo', '_sum'), |
3069 | ('mtime', 'bar', None), |
3070 | ) |
3071 | - self.assertEqual(f('file', views), |
3072 | - ( |
3073 | - '_design/file', |
3074 | - { |
3075 | - '_id': '_design/file', |
3076 | - 'language': 'javascript', |
3077 | - 'views': { |
3078 | - 'bytes': { |
3079 | - 'map': 'foo', |
3080 | - 'reduce': '_sum', |
3081 | - }, |
3082 | - 'mtime': { |
3083 | - 'map': 'bar', |
3084 | - }, |
3085 | - } |
3086 | + self.assertEqual(f('file', views_), |
3087 | + { |
3088 | + '_id': '_design/file', |
3089 | + 'language': 'javascript', |
3090 | + 'views': { |
3091 | + 'bytes': { |
3092 | + 'map': 'foo', |
3093 | + 'reduce': '_sum', |
3094 | + }, |
3095 | + 'mtime': { |
3096 | + 'map': 'bar', |
3097 | + }, |
3098 | } |
3099 | - ) |
3100 | - ) |
3101 | - |
3102 | - def test_create_machine(self): |
3103 | - f = metastore.create_machine |
3104 | - doc = f() |
3105 | - self.assertTrue(isinstance(doc, dict)) |
3106 | - self.assertEqual( |
3107 | - set(doc), |
3108 | - set([ |
3109 | - '_id', |
3110 | - 'machine_id', |
3111 | - 'type', |
3112 | - 'time', |
3113 | - 'hostname', |
3114 | - 'distribution', |
3115 | - ]) |
3116 | - ) |
3117 | - self.assertEqual(doc['type'], 'dmedia/machine') |
3118 | - self.assertEqual(doc['_id'], '_local/machine') |
3119 | - self.assertEqual(doc['hostname'], socket.gethostname()) |
3120 | - self.assertEqual(doc['distribution'], platform.linux_distribution()) |
3121 | - |
3122 | - |
3123 | -class test_MetaStore(CouchCase): |
3124 | - klass = metastore.MetaStore |
3125 | - |
3126 | - def new(self): |
3127 | - return self.klass(self.env) |
3128 | - |
3129 | - def test_init(self): |
3130 | - inst = self.new() |
3131 | - self.assertEqual(inst.env, self.env) |
3132 | - self.assertTrue(isinstance(inst.server, couchdb.Server)) |
3133 | - self.assertTrue(isinstance(inst.db, couchdb.Database)) |
3134 | - |
3135 | - def update(self): |
3136 | - inst = self.new() |
3137 | - '_local/app' |
3138 | - inst.update(dict(_id=_id, foo='bar')) |
3139 | - old = inst.db[_id] |
3140 | - inst.update(dict(_id=_id, foo='bar')) |
3141 | - self.assertEqual(inst.db[_id]['_rev'], old['_rev']) |
3142 | - inst.update(dict(_id=_id, foo='baz')) |
3143 | - self.assertNotEqual(inst.db[_id]['_rev'], old['_rev']) |
3144 | - |
3145 | - def test_create_machine(self): |
3146 | - inst = self.new() |
3147 | - self.assertFalse('_local/machine' in inst.db) |
3148 | - _id = inst.create_machine() |
3149 | - self.assertTrue('_local/machine' in inst.db) |
3150 | - self.assertTrue(_id in inst.db) |
3151 | - loc = inst.db['_local/machine'] |
3152 | - doc = inst.db[_id] |
3153 | - self.assertEqual(set(loc), set(doc)) |
3154 | - self.assertEqual(loc['machine_id'], doc['machine_id']) |
3155 | - self.assertEqual(loc['time'], doc['time']) |
3156 | - |
3157 | - self.assertEqual(inst._machine_id, None) |
3158 | - self.assertEqual(inst.machine_id, _id) |
3159 | - self.assertEqual(inst._machine_id, _id) |
3160 | - |
3161 | - def test_total_bytes(self): |
3162 | - inst = self.new() |
3163 | - self.assertEqual(inst.total_bytes(), 0) |
3164 | - total = 0 |
3165 | - for exp in xrange(20, 31): |
3166 | - size = 2 ** exp + 1 |
3167 | - total += size |
3168 | - inst.db.create({'bytes': size, 'type': 'dmedia/file'}) |
3169 | - self.assertEqual(inst.total_bytes(), total) |
3170 | - |
3171 | - def test_extensions(self): |
3172 | - inst = self.new() |
3173 | - self.assertEqual(list(inst.extensions()), []) |
3174 | - for i in xrange(17): |
3175 | - inst.db.create({'ext': 'mov', 'type': 'dmedia/file'}) |
3176 | - inst.db.create({'ext': 'jpg', 'type': 'dmedia/file'}) |
3177 | - inst.db.create({'ext': 'cr2', 'type': 'dmedia/file'}) |
3178 | - self.assertEqual( |
3179 | - list(inst.extensions()), |
3180 | - [ |
3181 | - ('cr2', 17), |
3182 | - ('jpg', 17), |
3183 | - ('mov', 17), |
3184 | - ] |
3185 | - ) |
3186 | - for i in xrange(27): |
3187 | - inst.db.create({'ext': 'mov', 'type': 'dmedia/file'}) |
3188 | - inst.db.create({'ext': 'jpg', 'type': 'dmedia/file'}) |
3189 | - self.assertEqual( |
3190 | - list(inst.extensions()), |
3191 | - [ |
3192 | - ('cr2', 17), |
3193 | - ('jpg', 44), |
3194 | - ('mov', 44), |
3195 | - ] |
3196 | - ) |
3197 | - for i in xrange(25): |
3198 | - inst.db.create({'ext': 'mov', 'type': 'dmedia/file'}) |
3199 | - self.assertEqual( |
3200 | - list(inst.extensions()), |
3201 | - [ |
3202 | - ('cr2', 17), |
3203 | - ('jpg', 44), |
3204 | - ('mov', 69), |
3205 | - ] |
3206 | - ) |
3207 | + } |
3208 | + ) |
3209 | + |
3210 | + |
3211 | +class TestCouchFunctions(CouchCase): |
3212 | + def test_update_design_doc(self): |
3213 | + f = views.update_design_doc |
3214 | + db = get_db(self.env) |
3215 | + |
3216 | + # Test when design doesn't exist: |
3217 | + doc = views.build_design_doc('user', |
3218 | + [('video', views.user_video, None)] |
3219 | + ) |
3220 | + self.assertEqual(f(db, doc), 'new') |
3221 | + self.assertTrue(db['_design/user']['_rev'].startswith('1-')) |
3222 | + |
3223 | + # Test when design is same: |
3224 | + doc = views.build_design_doc('user', |
3225 | + [('video', views.user_video, None)] |
3226 | + ) |
3227 | + self.assertEqual(f(db, doc), 'same') |
3228 | + self.assertTrue(db['_design/user']['_rev'].startswith('1-')) |
3229 | + |
3230 | + # Test when design is changed: |
3231 | + doc = views.build_design_doc('user', |
3232 | + [('video', views.user_audio, None)] |
3233 | + ) |
3234 | + self.assertEqual(f(db, doc), 'changed') |
3235 | + self.assertTrue(db['_design/user']['_rev'].startswith('2-')) |
3236 | + |
3237 | + # Again test when design is same: |
3238 | + doc = views.build_design_doc('user', |
3239 | + [('video', views.user_audio, None)] |
3240 | + ) |
3241 | + self.assertEqual(f(db, doc), 'same') |
3242 | + self.assertTrue(db['_design/user']['_rev'].startswith('2-')) |
3243 | + |
3244 | + def test_init_views(self): |
3245 | + db = get_db(self.env) |
3246 | + views.init_views(db) |
3247 | + for (name, views_) in views.designs: |
3248 | + doc = views.build_design_doc(name, views_) |
3249 | + saved = db[doc['_id']] |
3250 | + doc['_rev'] = saved['_rev'] |
3251 | + self.assertEqual(saved, doc) |
3252 | |
3253 | === renamed file 'dmedia/metastore.py' => 'dmedia/views.py' |
3254 | --- dmedia/metastore.py 2011-04-06 18:27:00 +0000 |
3255 | +++ dmedia/views.py 2011-04-11 12:02:24 +0000 |
3256 | @@ -2,7 +2,7 @@ |
3257 | # Jason Gerard DeRose <jderose@novacut.com> |
3258 | # |
3259 | # dmedia: distributed media library |
3260 | -# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com> |
3261 | +# Copyright (C) 2010, 2011 Jason Gerard DeRose <jderose@novacut.com> |
3262 | # |
3263 | # This file is part of `dmedia`. |
3264 | # |
3265 | @@ -20,19 +20,15 @@ |
3266 | # with `dmedia`. If not, see <http://www.gnu.org/licenses/>. |
3267 | |
3268 | """ |
3269 | -Store meta-data in desktop-couch. |
3270 | +Defines the dmedia CouchDB views. |
3271 | """ |
3272 | |
3273 | -from os import path |
3274 | -import time |
3275 | -import socket |
3276 | -import platform |
3277 | - |
3278 | -import gnomekeyring |
3279 | -from couchdb import ResourceNotFound, ResourceConflict |
3280 | - |
3281 | -from .abstractcouch import get_couchdb_server, get_dmedia_db |
3282 | -from .schema import random_id |
3283 | +import logging |
3284 | + |
3285 | +from couchdb import ResourceNotFound |
3286 | + |
3287 | + |
3288 | +log = logging.getLogger() |
3289 | |
3290 | |
3291 | _sum = '_sum' |
3292 | @@ -189,156 +185,74 @@ |
3293 | """ |
3294 | |
3295 | |
3296 | +designs = ( |
3297 | + ('type', ( |
3298 | + ('type', type_type, _count), |
3299 | + )), |
3300 | + |
3301 | + ('batch', ( |
3302 | + ('time', batch_time, None), |
3303 | + )), |
3304 | + |
3305 | + ('import', ( |
3306 | + ('time', import_time, None), |
3307 | + )), |
3308 | + |
3309 | + ('file', ( |
3310 | + ('stored', file_stored, _sum), |
3311 | + ('import_id', file_import_id, None), |
3312 | + ('bytes', file_bytes, _sum), |
3313 | + ('ext', file_ext, _count), |
3314 | + ('mime', file_mime, _count), |
3315 | + ('mtime', file_mtime, None), |
3316 | + )), |
3317 | + |
3318 | + ('user', ( |
3319 | + ('copies', user_copies, None), |
3320 | + ('media', user_media, _count), |
3321 | + ('tags', user_tags, _count), |
3322 | + ('all', user_all, None), |
3323 | + ('video', user_video, None), |
3324 | + ('image', user_image, None), |
3325 | + ('audio', user_audio, None), |
3326 | + )), |
3327 | +) |
3328 | + |
3329 | + |
3330 | +def iter_views(views): |
3331 | + for (name, map_, reduce_) in views: |
3332 | + if reduce_ is None: |
3333 | + yield (name, {'map': map_.strip()}) |
3334 | + else: |
3335 | + yield (name, {'map': map_.strip(), 'reduce': reduce_.strip()}) |
3336 | + |
3337 | + |
3338 | def build_design_doc(design, views): |
3339 | - _id = '_design/' + design |
3340 | - d = {} |
3341 | - for (view, map_, reduce_) in views: |
3342 | - d[view] = {'map': map_.strip()} |
3343 | - if reduce_ is not None: |
3344 | - d[view]['reduce'] = reduce_.strip() |
3345 | doc = { |
3346 | - '_id': _id, |
3347 | + '_id': '_design/' + design, |
3348 | 'language': 'javascript', |
3349 | - 'views': d, |
3350 | - } |
3351 | - return (_id, doc) |
3352 | - |
3353 | - |
3354 | -def create_machine(): |
3355 | - # FIXME: this '_local/machine' business probably isn't a very good approach. |
3356 | - # Plus this doesn't directly conform with schema.check_dmedia() |
3357 | - return { |
3358 | - '_id': '_local/machine', |
3359 | - 'machine_id': random_id(), |
3360 | - 'type': 'dmedia/machine', |
3361 | - 'time': time.time(), |
3362 | - 'hostname': socket.gethostname(), |
3363 | - 'distribution': platform.linux_distribution(), |
3364 | - } |
3365 | - |
3366 | - |
3367 | -def docs_equal(doc, old): |
3368 | - if old is None: |
3369 | - return False |
3370 | - doc['_rev'] = old['_rev'] |
3371 | - doc_att = doc.get('_attachments', {}) |
3372 | - old_att = old.get('_attachments', {}) |
3373 | - for key in old_att: |
3374 | - if key in doc_att: |
3375 | - doc_att[key]['revpos'] = old_att[key]['revpos'] |
3376 | - return doc == old |
3377 | - |
3378 | - |
3379 | - |
3380 | -class MetaStore(object): |
3381 | - designs = ( |
3382 | - ('type', ( |
3383 | - ('type', type_type, _count), |
3384 | - )), |
3385 | - |
3386 | - ('batch', ( |
3387 | - ('time', batch_time, None), |
3388 | - )), |
3389 | - |
3390 | - ('import', ( |
3391 | - ('time', import_time, None), |
3392 | - )), |
3393 | - |
3394 | - ('file', ( |
3395 | - ('stored', file_stored, _sum), |
3396 | - ('import_id', file_import_id, None), |
3397 | - ('bytes', file_bytes, _sum), |
3398 | - ('ext', file_ext, _count), |
3399 | - ('mime', file_mime, _count), |
3400 | - ('mtime', file_mtime, None), |
3401 | - )), |
3402 | - |
3403 | - ('user', ( |
3404 | - ('copies', user_copies, None), |
3405 | - ('media', user_media, _count), |
3406 | - ('tags', user_tags, _count), |
3407 | - ('all', user_all, None), |
3408 | - ('video', user_video, None), |
3409 | - ('image', user_image, None), |
3410 | - ('audio', user_audio, None), |
3411 | - )), |
3412 | - ) |
3413 | - |
3414 | - def __init__(self, env): |
3415 | - self.env = env |
3416 | - self.server = get_couchdb_server(env) |
3417 | - self.db = get_dmedia_db(env, self.server) |
3418 | - self.create_views() |
3419 | - self._machine_id = None |
3420 | - |
3421 | - def get_env(self): |
3422 | - env = dict(self.env) |
3423 | - env['machine_id'] = self.machine_id |
3424 | - return env |
3425 | - |
3426 | - def get_basic_auth(self): |
3427 | - data = gnomekeyring.find_items_sync( |
3428 | - gnomekeyring.ITEM_GENERIC_SECRET, |
3429 | - {'desktopcouch': 'basic'} |
3430 | - ) |
3431 | - (user, password) = data[0].secret.split(':') |
3432 | - return (user, password) |
3433 | - |
3434 | - def get_port(self): |
3435 | - return self.env.get('port') |
3436 | - |
3437 | - def get_uri(self): |
3438 | - return 'http://localhost:%s' % self.get_port() |
3439 | - |
3440 | - def get_auth_uri(self): |
3441 | - (user, password) = self.get_basic_auth() |
3442 | - return 'http://%s:%s@localhost:%s' % ( |
3443 | - user, password, self.get_port() |
3444 | - ) |
3445 | - |
3446 | - def create_machine(self): |
3447 | - try: |
3448 | - loc = self.db['_local/machine'] |
3449 | - except ResourceNotFound: |
3450 | - loc = self.sync(create_machine()) |
3451 | - doc = dict(loc) |
3452 | - doc['_id'] = doc['machine_id'] |
3453 | - try: |
3454 | - self.db[doc['_id']] = doc |
3455 | - except ResourceConflict: |
3456 | - pass |
3457 | - return loc['machine_id'] |
3458 | - |
3459 | - @property |
3460 | - def machine_id(self): |
3461 | - if self._machine_id is None: |
3462 | - self._machine_id = self.create_machine() |
3463 | - return self._machine_id |
3464 | - |
3465 | - def update(self, doc): |
3466 | - """ |
3467 | - Create *doc* if it doesn't exists, update doc only if different. |
3468 | - """ |
3469 | - _id = doc['_id'] |
3470 | - old = self.db.get(_id, attachments=True) |
3471 | - if not docs_equal(doc, old): |
3472 | - self.db[_id] = doc |
3473 | - |
3474 | - def sync(self, doc): |
3475 | - _id = doc['_id'] |
3476 | - self.db[_id] = doc |
3477 | - return self.db[_id] |
3478 | - |
3479 | - def create_views(self): |
3480 | - for (name, views) in self.designs: |
3481 | - (_id, doc) = build_design_doc(name, views) |
3482 | - self.update(doc) |
3483 | - |
3484 | - def total_bytes(self): |
3485 | - for row in self.db.view('_design/file/_view/bytes'): |
3486 | - return row.value |
3487 | - return 0 |
3488 | - |
3489 | - def extensions(self): |
3490 | - for row in self.db.view('_design/file/_view/ext', group=True): |
3491 | - yield (row.key, row.value) |
3492 | + 'views': dict(iter_views(views)), |
3493 | + } |
3494 | + return doc |
3495 | + |
3496 | + |
3497 | +def update_design_doc(db, doc): |
3498 | + assert '_rev' not in doc |
3499 | + try: |
3500 | + old = db[doc['_id']] |
3501 | + doc['_rev'] = old['_rev'] |
3502 | + if doc != old: |
3503 | + db.save(doc) |
3504 | + return 'changed' |
3505 | + else: |
3506 | + return 'same' |
3507 | + except ResourceNotFound: |
3508 | + db.save(doc) |
3509 | + return 'new' |
3510 | + |
3511 | + |
3512 | +def init_views(db): |
3513 | + log.info('Initializing views in %r', db) |
3514 | + for (name, views) in designs: |
3515 | + doc = build_design_doc(name, views) |
3516 | + update_design_doc(db, doc) |
3517 | |
3518 | === modified file 'dmedia/workers.py' |
3519 | --- dmedia/workers.py 2011-02-20 18:54:27 +0000 |
3520 | +++ dmedia/workers.py 2011-04-11 12:02:24 +0000 |
3521 | @@ -30,7 +30,7 @@ |
3522 | import logging |
3523 | |
3524 | from .constants import TYPE_ERROR |
3525 | -from .abstractcouch import get_couchdb_server, get_dmedia_db |
3526 | +from .abstractcouch import get_server, get_db |
3527 | |
3528 | |
3529 | log = logging.getLogger() |
3530 | @@ -101,7 +101,7 @@ |
3531 | inst = klass(env, q, key, args) |
3532 | inst.run() |
3533 | except Exception as e: |
3534 | - log.exception('exception in procces %d, worker=%r', pid) |
3535 | + log.exception('exception in procces %d, worker=%r', pid, worker) |
3536 | q.put(dict( |
3537 | signal='error', |
3538 | args=(key, exception_name(e), str(e)), |
3539 | @@ -159,8 +159,8 @@ |
3540 | class CouchWorker(Worker): |
3541 | def __init__(self, env, q, key, args): |
3542 | super(CouchWorker, self).__init__(env, q, key, args) |
3543 | - self.server = get_couchdb_server(env) |
3544 | - self.db = get_dmedia_db(env, self.server) |
3545 | + self.server = get_server(env) |
3546 | + self.db = get_db(env, self.server) |
3547 | |
3548 | |
3549 | class Manager(object): |
3550 | @@ -272,5 +272,5 @@ |
3551 | class CouchManager(Manager): |
3552 | def __init__(self, env, callback=None): |
3553 | super(CouchManager, self).__init__(env, callback) |
3554 | - self.server = get_couchdb_server(env) |
3555 | - self.db = get_dmedia_db(env, self.server) |
3556 | + self.server = get_server(env) |
3557 | + self.db = get_db(env, self.server) |
3558 | |
3559 | === modified file 'setup.py' |
3560 | --- setup.py 2011-03-27 14:12:36 +0000 |
3561 | +++ setup.py 2011-04-11 12:02:24 +0000 |
3562 | @@ -133,31 +133,54 @@ |
3563 | license='AGPLv3+', |
3564 | cmdclass={'test': Test}, |
3565 | |
3566 | - scripts=['dmedia-cli', 'dmedia-import', 'dmedia-gtk'], |
3567 | - packages=['dmedia', 'dmedia.webui', 'dmedia.gtkui'], |
3568 | - package_data={'dmedia.webui': ['data/*']}, |
3569 | - |
3570 | + scripts=[ |
3571 | + 'dmedia-cli', |
3572 | + 'dmedia-import', |
3573 | + 'dmedia-gtk', |
3574 | + ], |
3575 | + packages=[ |
3576 | + 'dmedia', |
3577 | + 'dmedia.service', |
3578 | + 'dmedia.webui', |
3579 | + 'dmedia.gtkui', |
3580 | + ], |
3581 | + package_data={ |
3582 | + 'dmedia.webui': ['data/*'] |
3583 | + }, |
3584 | data_files=[ |
3585 | - ('share/man/man1', ['data/dmedia-cli.1']), |
3586 | - ('share/applications', ['data/dmedia-import.desktop']), |
3587 | - #^ this enables Nautilus to use dmedia-import as a handler for |
3588 | - #media devices such as cameras. `sudo update-desktop-database` |
3589 | - #may need to run for this to show up in the Nautilus |
3590 | - #media handling preferences. |
3591 | - ('share/pixmaps', ['data/dmedia.svg']), |
3592 | + ('share/man/man1', |
3593 | + ['share/dmedia-cli.1'] |
3594 | + ), |
3595 | + ('share/applications', |
3596 | + ['share/dmedia-import.desktop'] |
3597 | + ), |
3598 | + ('share/pixmaps', |
3599 | + ['share/dmedia.svg'] |
3600 | + ), |
3601 | ('share/pixmaps/dmedia', |
3602 | [ |
3603 | - 'data/indicator-rendermenu.svg', |
3604 | - 'data/indicator-rendermenu-att.svg', |
3605 | - ] |
3606 | - ), |
3607 | - ('share/icons/hicolor/scalable/status/', |
3608 | - [ |
3609 | - 'data/indicator-rendermenu.svg', |
3610 | - 'data/indicator-rendermenu-att.svg', |
3611 | - ] |
3612 | - ), #enables status icons to be referenced by icon name |
3613 | - ('share/dbus-1/services', ['data/org.freedesktop.DMedia.service']), |
3614 | - ('lib/dmedia', ['dmedia-service', 'dummy-client']), |
3615 | + 'share/indicator-rendermenu.svg', |
3616 | + 'share/indicator-rendermenu-att.svg', |
3617 | + ] |
3618 | + ), |
3619 | + ('share/icons/hicolor/scalable/status', |
3620 | + [ |
3621 | + 'share/indicator-rendermenu.svg', |
3622 | + 'share/indicator-rendermenu-att.svg', |
3623 | + ] |
3624 | + ), |
3625 | + ('share/dbus-1/services', |
3626 | + [ |
3627 | + 'share/org.freedesktop.DMedia.service', |
3628 | + 'share/org.freedesktop.DMediaImporter.service', |
3629 | + ] |
3630 | + ), |
3631 | + ('lib/dmedia', |
3632 | + [ |
3633 | + 'dmedia-service', |
3634 | + 'dmedia-importer-service', |
3635 | + 'dummy-client', |
3636 | + ] |
3637 | + ), |
3638 | ], |
3639 | ) |
3640 | |
3641 | === renamed directory 'data' => 'share' |
3642 | === added file 'share/org.freedesktop.DMedia.service' |
3643 | --- share/org.freedesktop.DMedia.service 1970-01-01 00:00:00 +0000 |
3644 | +++ share/org.freedesktop.DMedia.service 2011-04-11 12:02:24 +0000 |
3645 | @@ -0,0 +1,3 @@ |
3646 | +[D-BUS Service] |
3647 | +Name=org.freedesktop.DMedia |
3648 | +Exec=/usr/lib/dmedia/dmedia-service |
3649 | |
3650 | === renamed file 'data/org.freedesktop.DMedia.service' => 'share/org.freedesktop.DMediaImporter.service' |
3651 | --- data/org.freedesktop.DMedia.service 2011-02-01 06:00:48 +0000 |
3652 | +++ share/org.freedesktop.DMediaImporter.service 2011-04-11 12:02:24 +0000 |
3653 | @@ -1,3 +1,3 @@ |
3654 | [D-BUS Service] |
3655 | -Name=org.freedesktop.DMedia |
3656 | -Exec=/usr/lib/dmedia/dmedia-service |
3657 | +Name=org.freedesktop.DMediaImporter |
3658 | +Exec=/usr/lib/dmedia/dmedia-importer-service |
Retrospective review says "Looks dandy!"