Merge lp:~jderose/dmedia/core-api into lp:dmedia

Proposed by Jason Gerard DeRose
Status: Merged
Merged at revision: 183
Proposed branch: lp:~jderose/dmedia/core-api
Merge into: lp:dmedia
Diff against target: 3658 lines (+1603/-1016)
32 files modified
MANIFEST.in (+4/-3)
dmedia-cli (+69/-167)
dmedia-gtk (+46/-37)
dmedia-importer-service (+47/-46)
dmedia-service (+72/-0)
dmedia/__init__.py (+1/-0)
dmedia/abstractcouch.py (+22/-34)
dmedia/api.py (+87/-0)
dmedia/constants.py (+3/-1)
dmedia/core.py (+195/-0)
dmedia/filestore.py (+31/-36)
dmedia/gtkui/client.py (+19/-19)
dmedia/gtkui/service.py (+23/-28)
dmedia/gtkui/tests/test_client.py (+10/-9)
dmedia/gtkui/tests/test_service.py (+0/-53)
dmedia/importer.py (+5/-14)
dmedia/schema.py (+19/-3)
dmedia/service/__init__.py (+94/-0)
dmedia/tests/couch.py (+5/-3)
dmedia/tests/test_abstractcouch.py (+72/-143)
dmedia/tests/test_api.py (+125/-0)
dmedia/tests/test_core.py (+336/-0)
dmedia/tests/test_downloader.py (+8/-8)
dmedia/tests/test_filestore.py (+106/-77)
dmedia/tests/test_importer.py (+4/-4)
dmedia/tests/test_transcoder.py (+2/-2)
dmedia/tests/test_views.py (+65/-136)
dmedia/views.py (+76/-162)
dmedia/workers.py (+6/-6)
setup.py (+46/-23)
share/org.freedesktop.DMedia.service (+3/-0)
share/org.freedesktop.DMediaImporter.service (+2/-2)
To merge this branch: bzr merge lp:~jderose/dmedia/core-api
Reviewer Review Type Date Requested Status
James Raymond Approve
Review via email: mp+57148@code.launchpad.net

Description of the change

This is just for fun so everyone (including myself) can gasp at how many lines this diff is probably going to be.

But *all* the unit tests pass, including that one pesky client => dbus test that has been failing for the last 2 months. I've beat up on this a lot, has been solid. Now it's time for the daily build users to abuse it as best they can. Huge change, sorry. Now I'll get back to saner merges, promise.

Lots of awesome changes, too tired to explain them all right now. Oh, but one cool one... you can now run dmedia against system-wide CouchDB if you want. Just launch `dmedia-service` with the --no-dc option. This wont work with DBus itself launching it, though, must be manual. Will need to add a config file or somesuch.

To post a comment you must log in.
Revision history for this message
James Raymond (jamesmr) wrote :

Retrospective review says "Looks dandy!"

review: Approve
Revision history for this message
Jason Gerard DeRose (jderose) wrote :

Thanks James, you deserve to be Knighted for reading through all that!

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'MANIFEST.in'
2--- MANIFEST.in 2011-03-27 14:12:36 +0000
3+++ MANIFEST.in 2011-04-11 12:02:24 +0000
4@@ -1,4 +1,5 @@
5-include AUTHORS COPYING HACKING.txt README.txt
6-include dmedia-service
7-include dmedia/webui/data/*
8+include AUTHORS COPYING HACKING.txt
9+include dmedia/tests/*.py
10 include dmedia/tests/data/*
11+include dmedia/webui/tests/*.py
12+include dmedia/gtkui/tests/*.py
13
14=== modified file 'dmedia-cli'
15--- dmedia-cli 2010-12-29 13:58:00 +0000
16+++ dmedia-cli 2011-04-11 12:02:24 +0000
17@@ -5,7 +5,7 @@
18 # David Green <david4dev@gmail.com>
19 #
20 # dmedia: distributed media library
21-# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com>
22+# Copyright (C) 2010, 2011 Jason Gerard DeRose <jderose@novacut.com>
23 #
24 # This file is part of `dmedia`.
25 #
26@@ -24,174 +24,76 @@
27
28
29 """
30-WARNING: the dmedia content-hash and schema are *not* yet stable, may change
31-wildly and without warning!
32-
33-This will eventually be turned into a VCS-style script with several commands.
34-For now it has a single function, to recursively import media files from a
35-directory. At this point, is is a quick-and-dirty demo of how media files might
36-be stored and how their meta-data might be stored.
37-
38-For example, say we scan all the JPEG images in the '/usr/share/backgrounds'
39-directory:
40-
41- dmedia /usr/share/backgrounds jpg
42-
43-Media files are identified by their unique Id which is generated by their
44-base32-encoded sha1 content-hash. In the above example, the
45-'Life_by_Paco_Espinoza.jpg' file happens to have a sha1 content-hash of
46-'6BRRXCGRM2GKVPTREJPGRNGUR2GF2L4K'. As such, this file will be stored at:
47-
48- ~/.dmedia/6B/RRXCGRM2GKVPTREJPGRNGUR2GF2L4K.jpg
49-
50-Meta-data for the media files is stored in CouchDB using desktopcouch. Each
51-media file has its own document. The sha1 content-hash is used as the document
52-'_id'. For example, the 'Life_by_Paco_Espinoza.jpg' file has a document that
53-looks like this:
54-
55- {
56- "_id": "6BRRXCGRM2GKVPTREJPGRNGUR2GF2L4K",
57- "_rev": "1-c19ea015eb53ede147d63d55f3967d13",
58- "name": "Life_by_Paco_Espinoza.jpg",
59- "record_type": "http://example.com/dmedia",
60- "bytes": 360889,
61- "height": 1500,
62- "shutter": "1/400",
63- "width": 2000,
64- "ext": "jpg",
65- "camera": "DSC-H5",
66- "iso": 125,
67- "focal_length": "6.0 mm",
68- "mtime": 1284394022,
69- "aperture": 4
70- }
71-
72-All media files will have the following fields:
73-
74- bytes - File size in bytes
75- mtime - Value of path.getmtime() at time of import
76- name - The path.basename() part of the original source file
77- ext - The extension of the original source file, normalized to lower-case
78-
79-Additional fields depend upon the type of media file. For example, image and
80-video files will always have 'width' and 'height', whereas video and audio files
81-will always have a 'duration'.
82-
83-You can browse through the dmedia database using a standard web-browser, like
84-this:
85-
86- xdg-open ~/.local/share/desktop-couch/couchdb.html
87-
88-Note that the sha1 hash is only being used as a stop-gap. dmedia will use the
89-Skein hash after its final constant tweaks are made. See:
90-
91- http://blog.novacut.com/2010/09/how-about-that-skein-hash.html
92+Command line tool for talking to dmedia DBus services.
93 """
94
95 from __future__ import print_function
96
97-import sys
98-import os
99-from os import path
100-import optparse
101-import xdg.BaseDirectory
102-import gobject
103+import argparse
104+import json
105+
106 import dmedia
107-from dmedia.client import Client
108-
109-script = path.basename(sys.argv[0])
110-
111-parser = optparse.OptionParser(
112- version=dmedia.__version__,
113- usage='Usage: %s DIRECTORY [EXTENSIONS...]' % script,
114- epilog='Example: %s /media/EOS_DIGITAL jpg cr2 mov' % script,
115-)
116-parser.add_option('--kill',
117- action='store_true',
118- default=False,
119- help='shutdown dmedia-service daemon and exit',
120-)
121-parser.add_option('--quick',
122- dest='extract',
123- action='store_false',
124- default=True,
125- help='import without metadata extraction or thumbnail generation',
126-)
127-parser.add_option('--type',
128- action='append',
129- default=[],
130- help='import all files of TYPE ("video", "audio", or "image")'
131-)
132-(options, args) = parser.parse_args()
133-
134-
135-# Check if they just want us to kill the daemon
136-if options.kill:
137- print('Telling dmedia-service daemon to shutdown...')
138- Client().kill()
139- print('Shutdown.')
140- sys.exit()
141-
142-
143-if len(args) < 1:
144- parser.print_usage()
145- sys.exit('ERROR: must provide DIRECTORY')
146-base = path.abspath(args[0])
147-if not path.isdir(base):
148- parser.print_usage()
149- sys.exit('ERROR: not a directory: %r' % base)
150-
151-
152-extensions = list(a.lower().strip('.') for a in args[1:])
153-
154-
155-class CLI(object):
156- def __init__(self, base):
157- self.base = base
158- self.mainloop = gobject.MainLoop()
159- self.client = Client()
160- self.client.connect('import_started', self.on_import_started)
161- self.client.connect('import_count', self.on_import_count)
162- self.client.connect('import_progress', self.on_import_progress)
163- self.client.connect('import_finished', self.on_import_finished)
164-
165- def on_import_started(self, client, base, import_id):
166- if base != self.base:
167- return
168- print('Started import of %s' % base)
169- print (' import_id = %s' % import_id)
170-
171- def on_import_count(self, client, base, import_id, total):
172- if base != self.base:
173- return
174- print('Found %d files' % total)
175-
176- def on_import_progress(self, c, base, import_id, completed, total, info):
177- if base != self.base:
178- return
179- print(' %(action)s %(src)s' % info)
180-
181- def on_import_finished(self, c, base, import_id, stats):
182- if base != self.base:
183- return
184- print('-' * 80)
185- for key in sorted(stats):
186- print('%s=%d' % (key, stats[key]))
187- self.mainloop.quit()
188-
189- def run(self, options):
190- try:
191- self.client.start_import(self.base, options.extract)
192- self.mainloop.run()
193- except KeyboardInterrupt:
194- self.client.stop_import(self.base)
195- print('\nImport stopped')
196-
197- couchdb_data_path = xdg.BaseDirectory.save_data_path('desktop-couch')
198- dc = path.join(couchdb_data_path, 'couchdb.html')
199- print('\nBrowse the `dmedia` database in CouchDB:')
200- print(' file://%s\n' % dc)
201-
202-
203-cli = CLI(base)
204-cli.run(options)
205+from dmedia.constants import BUS
206+
207+
208+parser = argparse.ArgumentParser(
209+ description='Execute methods on dmedia DBus services',
210+)
211+parser.add_argument('--version', action='version', version=dmedia.__version__)
212+parser.add_argument('--bus',
213+ help='DBus bus name; default is %(default)r',
214+ default=BUS,
215+)
216+
217+subparsers = parser.add_subparsers(
218+ title='Commands from {!r}'.format(BUS)
219+)
220+
221+
222+p_version = subparsers.add_parser('version',
223+ help='get version of running dmedia service',
224+)
225+def do_version(dm, args):
226+ print(
227+ '{} {}'.format(args.bus, dm.version())
228+ )
229+p_version.set_defaults(func=do_version)
230+
231+
232+p_kill = subparsers.add_parser('kill',
233+ help='kill `dmedia-service`',
234+)
235+def do_kill(dm, args):
236+ print('Killing {}...'.format(args.bus))
237+ dm.kill()
238+p_kill.set_defaults(func=do_kill)
239+
240+
241+p_get_env = subparsers.add_parser('get-env',
242+ help='echo out JSON encoded env dict',
243+)
244+def do_get_env(dm, args):
245+ print(json.dumps(dm.get_env(), sort_keys=True, indent=2))
246+p_get_env.set_defaults(func=do_get_env)
247+
248+
249+p_get_auth_url = subparsers.add_parser('get-auth-url',
250+ help='echo desktopcouch basic auth URL',
251+)
252+def do_get_auth_url(dm, args):
253+ print(dm.get_auth_url())
254+p_get_auth_url.set_defaults(func=do_get_auth_url)
255+
256+
257+p_has_app = subparsers.add_parser('has-app',
258+ help='show whether the WebUI app is available',
259+)
260+def do_has_app(dm, args):
261+ print(bool(dm.has_app()))
262+p_has_app.set_defaults(func=do_has_app)
263+
264+
265+args = parser.parse_args()
266+from dmedia.api import DMedia
267+dm = DMedia(args.bus)
268+args.func(dm, args)
269
270=== modified file 'dmedia-gtk'
271--- dmedia-gtk 2011-03-28 21:55:42 +0000
272+++ dmedia-gtk 2011-04-11 12:02:24 +0000
273@@ -4,7 +4,7 @@
274 # Jason Gerard DeRose <jderose@novacut.com>
275 #
276 # dmedia: distributed media library
277-# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com>
278+# Copyright (C) 2010, 2011 Jason Gerard DeRose <jderose@novacut.com>
279 #
280 # This file is part of `dmedia`.
281 #
282@@ -21,48 +21,58 @@
283 # You should have received a copy of the GNU Affero General Public License along
284 # with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
285
286-import sys
287-from subprocess import check_call
288-import optparse
289+from __future__ import print_function
290+
291+import argparse
292+from gettext import gettext as _
293+
294 import dmedia
295-
296-parser = optparse.OptionParser(
297- version=dmedia.__version__,
298+from dmedia.constants import BUS
299+from dmedia.api import DMedia
300+
301+
302+BROWSER = 'dmedia/app/browser'
303+
304+parser = argparse.ArgumentParser(
305+ description='Manage media files in your dmedia library',
306 )
307-parser.add_option('--browser',
308+parser.add_argument('--version', action='version', version=dmedia.__version__)
309+parser.add_argument('--browser',
310+ help='open dmedia HTML5 UI in default browser',
311 action='store_true',
312 default=False,
313- help='open dmedia HTML5 UI in default browser',
314-)
315-(options, args) = parser.parse_args()
316-
317-from dmedia.webui.app import App
318-from dmedia.abstractcouch import get_env
319-from dmedia.metastore import MetaStore
320-
321-app = App()
322-env = get_env()
323-store = MetaStore(env)
324-store.update(app.get_doc())
325-
326-
327-if options.browser:
328- uri = store.get_auth_uri() + '/dmedia/app/browser'
329- check_call(['xdg-open', uri])
330- sys.exit()
331-
332-uri = store.get_uri() + '/dmedia/app/browser'
333-
334+)
335+parser.add_argument('--bus',
336+ help='DBus bus name; default is %(default)r',
337+ default=BUS,
338+)
339+parser.add_argument('--att',
340+ help='CouchDB attachment path; default is %(default)r',
341+ default=BROWSER,
342+)
343+
344+args = parser.parse_args()
345+dm = DMedia(args.bus)
346+
347+if not dm.has_app():
348+ print('Oops, cannot import dmedia.webui.app!')
349+ raise SystemExit(2)
350+
351+
352+if args.browser:
353+ url = dm.get_auth_url() + args.att
354+ import subprocess
355+ subprocess.check_call(['/usr/bin/xdg-open', url])
356+ raise SystemExit()
357+
358+
359+env = dm.get_env()
360
361 from dmedia.gtkui.widgets import CouchView
362 from gi.repository import Gtk, GObject
363
364-
365-GObject.threads_init()
366-
367-
368 window = Gtk.Window()
369-window.set_title('test')
370+window.set_title(_('Media Browser'))
371 window.set_default_size(960, 540)
372 window.connect('destroy', Gtk.main_quit)
373
374@@ -70,11 +80,10 @@
375 scroll.set_policy(Gtk.PolicyType.AUTOMATIC, Gtk.PolicyType.AUTOMATIC)
376 window.add(scroll)
377
378-view = CouchView(env['url'], env['oauth'])
379+view = CouchView(env['url'], env.get('oauth'))
380 scroll.add(view)
381
382 window.show_all()
383-view.load_uri(uri)
384-
385+view.load_uri(env['url'] + args.att)
386
387 Gtk.main()
388
389=== renamed file 'dmedia-service' => 'dmedia-importer-service'
390--- dmedia-service 2011-03-27 10:45:13 +0000
391+++ dmedia-importer-service 2011-04-11 12:02:24 +0000
392@@ -21,53 +21,54 @@
393 # You should have received a copy of the GNU Affero General Public License along
394 # with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
395
396-import sys
397-import optparse
398-
399-import dbus
400-from dbus.mainloop.glib import DBusGMainLoop
401-
402-DBusGMainLoop(set_as_default=True)
403+"""
404+dmedia importer DBus service at "org.freedesktop.DMedia.Importer".
405+"""
406+
407+import argparse
408+import json
409+import time
410
411 import dmedia
412-from dmedia.constants import BUS, DBNAME
413-from dmedia.abstractcouch import get_env
414-
415-
416-parser = optparse.OptionParser(version=dmedia.__version__)
417-parser.add_option('--bus',
418- default=BUS,
419- help='D-Bus bus name; default is %r' % BUS,
420-)
421-parser.add_option('--dbname',
422- metavar='DIR',
423- default=DBNAME,
424- help='CouchDB database; default is %r' % DBNAME,
425-)
426-parser.add_option('--no-gui',
427+from dmedia.constants import BUS, IMPORT_BUS
428+
429+
430+parser = argparse.ArgumentParser(
431+ description='DBus service @{}'.format(IMPORT_BUS),
432+)
433+parser.add_argument('--version', action='version', version=dmedia.__version__)
434+parser.add_argument('--no-gui',
435+ help='run without NotifyOSD and Application Indicators',
436 action='store_true',
437 default=False,
438- help='run without NotifyOSD and Application Indicators',
439-)
440-
441-
442-def exit(msg, code=1):
443- parser.print_help()
444- print('ERROR: ' + msg)
445- sys.exit(code)
446-
447-
448-if __name__ == '__main__':
449- (options, args) = parser.parse_args()
450-
451- dmedia.configure_logging('service')
452-
453- env = get_env(options.dbname)
454- env['bus'] = options.bus
455- env['no_gui'] = options.no_gui
456-
457- from dmedia.gtkui import service
458- from gi.repository import GObject
459- mainloop = GObject.MainLoop()
460- obj = service.DMedia(env, mainloop.quit)
461- mainloop.run()
462+)
463+parser.add_argument('--bus',
464+ help='DBus bus name; default is %(default)r',
465+ default=IMPORT_BUS,
466+)
467+parser.add_argument('--env',
468+ help='dmedia environment as JSON serialized string',
469+ metavar='JSON',
470+)
471+
472+args = parser.parse_args()
473+dmedia.configure_logging('importer')
474+
475+from dbus.mainloop.glib import DBusGMainLoop
476+DBusGMainLoop(set_as_default=True)
477+
478+from gi.repository import GObject
479+GObject.threads_init()
480+
481+from dmedia import api
482+dm = api.DMedia()
483+
484+env = dm.get_env(args.env)
485+if args.no_gui:
486+ env['no_gui'] = True
487+
488+from dmedia.gtkui.service import DMedia
489+
490+mainloop = GObject.MainLoop()
491+obj = DMedia(env, args.bus, mainloop.quit)
492+mainloop.run()
493
494=== added file 'dmedia-service'
495--- dmedia-service 1970-01-01 00:00:00 +0000
496+++ dmedia-service 2011-04-11 12:02:24 +0000
497@@ -0,0 +1,72 @@
498+#!/usr/bin/env python
499+
500+# Authors:
501+# Jason Gerard DeRose <jderose@novacut.com>
502+#
503+# dmedia: distributed media library
504+# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com>
505+#
506+# This file is part of `dmedia`.
507+#
508+# `dmedia` is free software: you can redistribute it and/or modify it under the
509+# terms of the GNU Affero General Public License as published by the Free
510+# Software Foundation, either version 3 of the License, or (at your option) any
511+# later version.
512+#
513+# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY
514+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
515+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
516+# details.
517+#
518+# You should have received a copy of the GNU Affero General Public License along
519+# with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
520+
521+"""
522+Core dmedia DBus service at "org.freedesktop.DMedia".
523+"""
524+
525+import argparse
526+import time
527+
528+import dmedia
529+from dmedia.constants import BUS, DBNAME
530+
531+
532+parser = argparse.ArgumentParser(
533+ description='DBus service @{}'.format(BUS),
534+)
535+parser.add_argument('--version', action='version', version=dmedia.__version__)
536+parser.add_argument('--no-dc',
537+ action='store_true',
538+ default=False,
539+ help='Use system-wide CouchDB instead of desktopcouch',
540+)
541+parser.add_argument('--bus',
542+ default=BUS,
543+ help='DBus bus name; default is %(default)r',
544+)
545+parser.add_argument('--dbname',
546+ metavar='DB',
547+ default=DBNAME,
548+ help='CouchDB database; default is %(default)r',
549+)
550+parser.add_argument('--env',
551+ help='dmedia environment as JSON serialized string',
552+ metavar='JSON',
553+)
554+
555+args = parser.parse_args()
556+dmedia.configure_logging('service')
557+
558+from dbus.mainloop.glib import DBusGMainLoop
559+DBusGMainLoop(set_as_default=True)
560+
561+from gi.repository import GObject
562+GObject.threads_init()
563+
564+from dmedia.service import DMedia
565+
566+mainloop = GObject.MainLoop()
567+couchargs = (args.dbname, args.no_dc, args.env)
568+obj = DMedia(couchargs, args.bus, mainloop.quit, start=time.time())
569+mainloop.run()
570
571=== modified file 'dmedia/__init__.py'
572--- dmedia/__init__.py 2011-04-01 01:28:38 +0000
573+++ dmedia/__init__.py 2011-04-11 12:02:24 +0000
574@@ -58,3 +58,4 @@
575 level=logging.DEBUG,
576 format='\t'.join(format),
577 )
578+ logging.info('dmedia %s, namespace=%r', __version__, namespace)
579
580=== modified file 'dmedia/abstractcouch.py'
581--- dmedia/abstractcouch.py 2011-02-21 11:49:14 +0000
582+++ dmedia/abstractcouch.py 2011-04-11 12:02:24 +0000
583@@ -37,6 +37,7 @@
584 https://bugs.launchpad.net/dmedia/+bug/722035
585 """
586
587+import json
588 import logging
589
590 from couchdb import Server, ResourceNotFound
591@@ -44,15 +45,12 @@
592 from desktopcouch.records.http import OAuthSession
593 except ImportError:
594 OAuthSession = None
595-if OAuthSession is not None:
596- from desktopcouch.application.platform import find_port
597- from desktopcouch.application.local_files import get_oauth_tokens
598
599
600 log = logging.getLogger()
601
602
603-def get_couchdb_server(env):
604+def get_server(env):
605 """
606 Return `couchdb.Server` for desktopcouch or system-wide CouchDB.
607
608@@ -98,11 +96,9 @@
609 The goal is to have all the needed information is one easily serialized
610 piece of data (important for testing across multiple processes).
611 """
612- url = env.get('url', 'http://localhost:5984/')
613+ url = env['url']
614 log.info('CouchDB server is %r', url)
615- if env.get('oauth') is None:
616- session = None
617- else:
618+ if 'oauth' in env:
619 if OAuthSession is None:
620 raise ValueError(
621 "provided env['oauth'] but OAuthSession not available: %r" %
622@@ -110,29 +106,27 @@
623 )
624 log.info('Using desktopcouch `OAuthSession`')
625 session = OAuthSession(credentials=env['oauth'])
626+ else:
627+ session = None
628 return Server(url, session=session)
629
630
631-def get_dmedia_db(env, server=None):
632+def get_db(env, server=None):
633 """
634- Return the dmedia database specified by *env*.
635+ Return the CouchDB database specified by *env*.
636
637- The database name is determined by ``env['dbname']``. If the ``"dbname"``
638- key is missing or is ``None``, the default database name ``"dmedia"`` is
639- used.
640+ The database name is determined by ``env['dbname']``.
641
642 If the database does not exist, it will be created.
643
644 If *server* is ``None``, one is created based on *env* by calling
645- `get_couchdb_server()`.
646+ `get_server()`.
647
648 Returns a ``couchdb.Database`` instance.
649 """
650 if server is None:
651- server = get_couchdb_server(env)
652- dbname = env.get('dbname')
653- if dbname is None:
654- dbname = 'dmedia'
655+ server = get_server(env)
656+ dbname = env['dbname']
657 log.info('CouchDB database is %r', dbname)
658 try:
659 return server[dbname]
660@@ -140,19 +134,13 @@
661 return server.create(dbname)
662
663
664-def get_env(dbname=None):
665- """
666- Return default *env*.
667-
668- This will return an appropriate *env* based on whether desktopcouch is
669- available. Not a perfect solution, but works for now.
670- """
671- if OAuthSession is None:
672- return {'dbname': dbname}
673- port = find_port()
674- return {
675- 'port': port,
676- 'url': 'http://localhost:%d/' % port,
677- 'oauth': get_oauth_tokens(),
678- 'dbname': dbname,
679- }
680+def load_env(env_s):
681+ env = json.loads(env_s)
682+ # FIXME: hack to work-around for Python oauth not working with unicode,
683+ # which is what we get when the env is retrieved over D-Bus as JSON
684+ if 'oauth' in env:
685+ env['oauth'] = dict(
686+ (k.encode('ascii'), v.encode('ascii'))
687+ for (k, v) in env['oauth'].iteritems()
688+ )
689+ return env
690
691=== added file 'dmedia/api.py'
692--- dmedia/api.py 1970-01-01 00:00:00 +0000
693+++ dmedia/api.py 2011-04-11 12:02:24 +0000
694@@ -0,0 +1,87 @@
695+# Authors:
696+# Jason Gerard DeRose <jderose@novacut.com>
697+#
698+# dmedia: distributed media library
699+# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com>
700+#
701+# This file is part of `dmedia`.
702+#
703+# `dmedia` is free software: you can redistribute it and/or modify it under the
704+# terms of the GNU Affero General Public License as published by the Free
705+# Software Foundation, either version 3 of the License, or (at your option) any
706+# later version.
707+#
708+# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY
709+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
710+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
711+# details.
712+#
713+# You should have received a copy of the GNU Affero General Public License along
714+# with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
715+
716+"""
717+Python convenience API for talking to dmedia components over DBus.
718+"""
719+
720+import json
721+
722+import dbus
723+
724+from dmedia.constants import BUS
725+
726+# On my system, this takes around 0.15 seconds, almost all because of
727+# >>> from desktopcouch.records.http import OAuthSession
728+#
729+#from dmedia.abstractcouch import load_env
730+
731+
732+def load_env(env_s):
733+ env = json.loads(env_s)
734+ # FIXME: hack to work-around for Python oauth not working with unicode,
735+ # which is what we get when the env is retrieved over D-Bus as JSON
736+ if 'oauth' in env:
737+ env['oauth'] = dict(
738+ (k.encode('ascii'), v.encode('ascii'))
739+ for (k, v) in env['oauth'].iteritems()
740+ )
741+ return env
742+
743+
744+class DMedia(object):
745+ """
746+ Talk to "org.freedesktop.DMedia".
747+ """
748+ def __init__(self, bus=BUS):
749+ self.bus = bus
750+ self.conn = dbus.SessionBus()
751+ self._proxy = None
752+
753+ @property
754+ def proxy(self):
755+ if self._proxy is None:
756+ self._proxy = self.conn.get_object(self.bus, '/')
757+ return self._proxy
758+
759+ def version(self):
760+ return self.proxy.Version()
761+
762+ def kill(self):
763+ self.proxy.Kill()
764+ self._proxy = None
765+
766+ def get_env(self, env_s=None):
767+ if not env_s:
768+ env_s = self.proxy.GetEnv()
769+ return load_env(env_s)
770+
771+ def get_auth_url(self):
772+ return self.proxy.GetAuthURL()
773+
774+ def has_app(self):
775+ return self.proxy.HasApp()
776+
777+
778+class DMediaImporter(object):
779+ """
780+ Talk to "org.freedesktop.DMediaImporter".
781+ """
782
783=== modified file 'dmedia/constants.py'
784--- dmedia/constants.py 2011-02-25 16:38:29 +0000
785+++ dmedia/constants.py 2011-04-11 12:02:24 +0000
786@@ -48,7 +48,9 @@
787
788 # D-Bus releated:
789 BUS = 'org.freedesktop.DMedia'
790-INTERFACE = BUS
791+IFACE = BUS
792+IMPORT_BUS = 'org.freedesktop.DMediaImporter'
793+IMPORT_IFACE = IMPORT_BUS
794 DC_BUS = 'org.desktopcouch.CouchDB'
795 DC_INTERFACE = DC_BUS
796
797
798=== added file 'dmedia/core.py'
799--- dmedia/core.py 1970-01-01 00:00:00 +0000
800+++ dmedia/core.py 2011-04-11 12:02:24 +0000
801@@ -0,0 +1,195 @@
802+# Authors:
803+# Jason Gerard DeRose <jderose@novacut.com>
804+#
805+# dmedia: distributed media library
806+# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com>
807+#
808+# This file is part of `dmedia`.
809+#
810+# `dmedia` is free software: you can redistribute it and/or modify it under the
811+# terms of the GNU Affero General Public License as published by the Free
812+# Software Foundation, either version 3 of the License, or (at your option) any
813+# later version.
814+#
815+# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY
816+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
817+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
818+# details.
819+#
820+# You should have received a copy of the GNU Affero General Public License along
821+# with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
822+
823+"""
824+Core dmedia entry-point/API - start here!
825+
826+For background, please see:
827+
828+ https://bugs.launchpad.net/dmedia/+bug/753260
829+
830+
831+Security note on /dmedia/_local/dmedia
832+======================================
833+
834+When `DMedia.init_filestores()` is called, it creates `FileStore` instances
835+based solely on information in the non-replicated /dmedia/_local/dmedia
836+document... despite the fact that the exact same information is also available
837+in the corresponding 'dmedia/store' documents.
838+
839+When it comes to deciding what files dmedia will read and write, it's prudent to
840+assume that replicated documents are untrustworthy.
841+
842+The dangerous approach would be to use a view to get all the 'dmedia/store'
843+documents with a machine_id that matches this machine_id, and initialize those `FileStore`. But the problem is that if an attacker gained control of just one
844+of your replicating peers or services, they could insert arbitrary
845+'dmedia/store' documents, and have dmedia happily initialize `FileStore` at
846+those mount points. And that would be "a bad thing".
847+
848+So although the corresponding 'dmedia/store' records are created (if they don't
849+already exists), they are completely ignored when it comes to deciding what
850+filestores and mount points are configured.
851+"""
852+
853+import logging
854+from copy import deepcopy
855+import os
856+from os import path
857+
858+from couchdb import ResourceNotFound
859+try:
860+ import desktopcouch
861+ from desktopcouch.application.platform import find_port
862+ from desktopcouch.application.local_files import get_oauth_tokens
863+except ImportError:
864+ desktopcouch = None
865+
866+try:
867+ from dmedia.webui.app import App
868+except ImportError:
869+ App = None
870+
871+from .constants import DBNAME
872+from .abstractcouch import get_server, get_db, load_env
873+from .schema import random_id, create_machine, create_store
874+from .views import init_views
875+
876+
877+log = logging.getLogger()
878+
879+
880+def get_env(dbname=DBNAME, no_dc=False):
881+ """
882+ Return default CouchDB environment.
883+
884+ This will return an appropriate environment based on whether desktopcouch is
885+ available. If you supply ``no_dc=True``, the environment for the default
886+ system wide CouchDB will be returned, even if desktopcouch is available.
887+
888+ For example:
889+
890+ >>> get_env(no_dc=True)
891+ {'url': 'http://localhost:5984/', 'dbname': 'dmedia', 'port': 5984}
892+ >>> get_env(dbname='foo', no_dc=True)
893+ {'url': 'http://localhost:5984/', 'dbname': 'foo', 'port': 5984}
894+
895+ Not a perfect solution, but works for now.
896+ """
897+ if desktopcouch is None or no_dc:
898+ return {
899+ 'dbname': dbname,
900+ 'port': 5984,
901+ 'url': 'http://localhost:5984/',
902+ }
903+ port = find_port()
904+ return {
905+ 'dbname': dbname,
906+ 'port': port,
907+ 'url': 'http://localhost:%d/' % port,
908+ 'oauth': get_oauth_tokens(),
909+ }
910+
911+
912+class LocalStores(object):
913+ def by_id(self, _id):
914+ pass
915+
916+ def by_path(self, parentdir):
917+ pass
918+
919+ def by_device(self, device):
920+ pass
921+
922+
923+class Core(object):
924+ def __init__(self, dbname=DBNAME, no_dc=False, env_s=None):
925+ if env_s:
926+ self.env = load_env(env_s)
927+ else:
928+ self.env = get_env(dbname, no_dc)
929+ self.home = path.abspath(os.environ['HOME'])
930+ if not path.isdir(self.home):
931+ raise ValueError('HOME is not a dir: {!}'.format(self.home))
932+ self.server = get_server(self.env)
933+ self.db = get_db(self.env, self.server)
934+ self._has_app = None
935+
936+ def bootstrap(self):
937+ (self.local, self.machine) = self.init_local()
938+ self.machine_id = self.machine['_id']
939+ self.env['machine_id'] = self.machine_id
940+ store = self.init_filestores()
941+ self.env['filestore'] = {'_id': store['_id'], 'path': store['path']}
942+ init_views(self.db)
943+
944+ def init_local(self):
945+ """
946+ Get the /dmedia/_local/dmedia document, creating it if needed.
947+ """
948+ local_id = '_local/dmedia'
949+ try:
950+ local = self.db[local_id]
951+ except ResourceNotFound:
952+ machine = create_machine()
953+ local = {
954+ '_id': local_id,
955+ 'machine': deepcopy(machine),
956+ 'filestores': {},
957+ }
958+ self.db.save(local)
959+ self.db.save(machine)
960+ else:
961+ try:
962+ machine = self.db[local['machine']['_id']]
963+ except ResourceNotFound:
964+ machine = deepcopy(local['machine'])
965+ self.db.save(machine)
966+ return (local, machine)
967+
968+ def init_filestores(self):
969+ if not self.local['filestores']:
970+ store = create_store(self.home, self.machine_id)
971+ self.local['filestores'][store['_id']] = deepcopy(store)
972+ self.local['default_filestore'] = store['_id']
973+ self.db.save(self.local)
974+ self.db.save(store)
975+ return self.local['filestores'][self.local['default_filestore']]
976+
977+ def init_app(self):
978+ if App is None:
979+ log.info('init_app(): `dmedia.webui.app` not available')
980+ return False
981+ log.info('init_app(): creating /dmedia/app document')
982+ doc = App().get_doc()
983+ _id = doc['_id']
984+ assert '_rev' not in doc
985+ try:
986+ old = self.db[_id]
987+ doc['_rev'] = old['_rev']
988+ self.db.save(doc)
989+ except ResourceNotFound:
990+ self.db.save(doc)
991+ return True
992+
993+ def has_app(self):
994+ if self._has_app is None:
995+ self._has_app = self.init_app()
996+ return self._has_app
997
998=== modified file 'dmedia/filestore.py'
999--- dmedia/filestore.py 2011-04-06 20:56:54 +0000
1000+++ dmedia/filestore.py 2011-04-11 12:02:24 +0000
1001@@ -164,7 +164,6 @@
1002 from threading import Thread
1003 from Queue import Queue
1004
1005-from .schema import create_store
1006 from .errors import AmbiguousPath, FileStoreTraversal
1007 from .errors import DuplicateFile, IntegrityError
1008 from .constants import LEAF_SIZE, TYPE_ERROR, EXT_PAT
1009@@ -559,7 +558,7 @@
1010 To create a `FileStore`, you give it the directory that will be its base on
1011 the filesystem:
1012
1013- >>> fs = FileStore('/home/jderose/.dmedia') #doctest: +SKIP
1014+ >>> fs = FileStore('/home/jderose') #doctest: +SKIP
1015 >>> fs.base #doctest: +SKIP
1016 '/home/jderose/.dmedia'
1017
1018@@ -567,7 +566,7 @@
1019
1020 >>> fs = FileStore()
1021 >>> fs.base #doctest: +ELLIPSIS
1022- '/tmp/store...'
1023+ '/tmp/.../.dmedia'
1024
1025 You can add files to the store using `FileStore.import_file()`:
1026
1027@@ -580,7 +579,7 @@
1028 path of the file using `FileStore.path()`:
1029
1030 >>> fs.path('HIGJPQWY4PI7G7IFOB2G4TKY6PMTJSI7', 'mov') #doctest: +ELLIPSIS
1031- '/tmp/store.../HI/GJPQWY4PI7G7IFOB2G4TKY6PMTJSI7.mov'
1032+ '/tmp/.../.dmedia/HI/GJPQWY4PI7G7IFOB2G4TKY6PMTJSI7.mov'
1033
1034 As the files are assumed to be read-only and unchanging, moving a file into
1035 its canonical location must be atomic. There are 2 scenarios that must be
1036@@ -601,35 +600,31 @@
1037 `fallocate()` function, which calls the Linux ``fallocate`` command.
1038 """
1039
1040- def __init__(self, base=None, machine_id=None):
1041- if base is None:
1042- base = tempfile.mkdtemp(prefix='store.')
1043- self.base = safe_path(base)
1044+ def __init__(self, parent=None, dotdir='.dmedia'):
1045+ if parent is None:
1046+ parent = tempfile.mkdtemp(prefix='store.')
1047+ self.parent = safe_path(parent)
1048+ if not path.isdir(self.parent):
1049+ raise ValueError('%s.parent not a directory: %r' %
1050+ (self.__class__.__name__, parent)
1051+ )
1052+ self.base = path.join(self.parent, dotdir)
1053 try:
1054- os.makedirs(self.base)
1055+ os.mkdir(self.base)
1056 except OSError:
1057 pass
1058 if not path.isdir(self.base):
1059 raise ValueError('%s.base not a directory: %r' %
1060 (self.__class__.__name__, self.base)
1061 )
1062-
1063- # FIXME: This is too high-level for FileStore, should instead be deault
1064- # with by the core API entry point as FileStore are first initialized
1065- self.record = path.join(self.base, 'store.json')
1066- try:
1067- fp = open(self.record, 'rb')
1068- doc = json.load(fp)
1069- except IOError:
1070- fp = open(self.record, 'wb')
1071- doc = create_store(self.base, machine_id)
1072- json.dump(doc, fp, sort_keys=True, indent=4)
1073- fp.close()
1074- self._doc = doc
1075- self._id = doc['_id']
1076+ if path.islink(self.base):
1077+ raise ValueError('{!r} is symlink to {!r}'.format(
1078+ self.base, os.readlink(self.base)
1079+ )
1080+ )
1081
1082 def __repr__(self):
1083- return '%s(%r)' % (self.__class__.__name__, self.base)
1084+ return '%s(%r)' % (self.__class__.__name__, self.parent)
1085
1086 ############################################
1087 # Methods to prevent path traversals attacks
1088@@ -656,7 +651,7 @@
1089
1090 >>> fs = FileStore()
1091 >>> fs.join('NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW') #doctest: +ELLIPSIS
1092- '/tmp/store.../NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW'
1093+ '/tmp/.../.dmedia/NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW'
1094
1095 However, a `FileStoreTraversal` is raised if *parts* cause a path
1096 traversal outside of the `FileStore` base directory:
1097@@ -664,14 +659,14 @@
1098 >>> fs.join('../ssh') #doctest: +ELLIPSIS
1099 Traceback (most recent call last):
1100 ...
1101- FileStoreTraversal: '/tmp/ssh' outside base '/tmp/store...'
1102+ FileStoreTraversal: '/tmp/.../ssh' outside base '/tmp/.../.dmedia'
1103
1104 Or Likewise if an absolute path is included in *parts*:
1105
1106 >>> fs.join('NW', '/etc', 'ssh') #doctest: +ELLIPSIS
1107 Traceback (most recent call last):
1108 ...
1109- FileStoreTraversal: '/etc/ssh' outside base '/tmp/store...'
1110+ FileStoreTraversal: '/etc/ssh' outside base '/tmp/.../.dmedia'
1111
1112 Also see `FileStore.create_parent()`.
1113 """
1114@@ -689,14 +684,14 @@
1115 >>> fs.create_parent('/foo/my.ogv') #doctest: +ELLIPSIS
1116 Traceback (most recent call last):
1117 ...
1118- FileStoreTraversal: '/foo/my.ogv' outside base '/tmp/store...'
1119+ FileStoreTraversal: '/foo/my.ogv' outside base '/tmp/.../.dmedia'
1120
1121 It also protects against malicious filenames like this:
1122
1123 >>> fs.create_parent('/foo/../bar/my.ogv') #doctest: +ELLIPSIS
1124 Traceback (most recent call last):
1125 ...
1126- FileStoreTraversal: '/bar/my.ogv' outside base '/tmp/store...'
1127+ FileStoreTraversal: '/bar/my.ogv' outside base '/tmp/.../.dmedia'
1128
1129 If doesn't already exists, the directory containing *filename* is
1130 created. Returns the directory containing *filename*.
1131@@ -747,12 +742,12 @@
1132
1133 >>> fs = FileStore()
1134 >>> fs.path('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW') #doctest: +ELLIPSIS
1135- '/tmp/store.../NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW'
1136+ '/tmp/.../.dmedia/NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW'
1137
1138 Or with a file extension:
1139
1140 >>> fs.path('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW', 'txt') #doctest: +ELLIPSIS
1141- '/tmp/store.../NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW.txt'
1142+ '/tmp/.../.dmedia/NW/BNVXVK5DQGIOW7MYR4K3KA5K22W7NW.txt'
1143
1144 If called with ``create=True``, the parent directory is created with
1145 `FileStore.create_parent()`.
1146@@ -854,12 +849,12 @@
1147
1148 >>> fs = FileStore()
1149 >>> fs.tmp('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW') #doctest: +ELLIPSIS
1150- '/tmp/store.../transfers/NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW'
1151+ '/tmp/.../.dmedia/transfers/NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW'
1152
1153 Or with a file extension:
1154
1155 >>> fs.tmp('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW', 'txt') #doctest: +ELLIPSIS
1156- '/tmp/store.../transfers/NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW.txt'
1157+ '/tmp/.../.dmedia/transfers/NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW.txt'
1158
1159 If called with ``create=True``, the parent directory is created with
1160 `FileStore.create_parent()`.
1161@@ -979,7 +974,7 @@
1162 >>> tmp_fp = open(fs.join('foo.mov'), 'wb')
1163 >>> chash = 'ZR765XWSF6S7JQHLUI4GCG5BHGPE252O'
1164 >>> fs.tmp_move(tmp_fp, chash, 'mov') #doctest: +ELLIPSIS
1165- '/tmp/store.../ZR/765XWSF6S7JQHLUI4GCG5BHGPE252O.mov'
1166+ '/tmp/.../.dmedia/ZR/765XWSF6S7JQHLUI4GCG5BHGPE252O.mov'
1167
1168 Note, however, that this method does *not* verify the content hash of
1169 the temporary file! This is by design as many operations will compute
1170@@ -1081,7 +1076,7 @@
1171 >>> fs = FileStore()
1172 >>> tmp = fs.tmp('TGX33XXWU3EVHEEY5J7NBOJGKBFXLEBK', 'mov', create=True)
1173 >>> tmp #doctest: +ELLIPSIS
1174- '/tmp/store.../transfers/TGX33XXWU3EVHEEY5J7NBOJGKBFXLEBK.mov'
1175+ '/tmp/.../.dmedia/transfers/TGX33XXWU3EVHEEY5J7NBOJGKBFXLEBK.mov'
1176
1177 Then the downloader will write to the temporary file as it's being
1178 downloaded:
1179@@ -1102,7 +1097,7 @@
1180
1181 >>> dst = fs.tmp_verify_move('TGX33XXWU3EVHEEY5J7NBOJGKBFXLEBK', 'mov')
1182 >>> dst #doctest: +ELLIPSIS
1183- '/tmp/store.../TG/X33XXWU3EVHEEY5J7NBOJGKBFXLEBK.mov'
1184+ '/tmp/.../.dmedia/TG/X33XXWU3EVHEEY5J7NBOJGKBFXLEBK.mov'
1185
1186 The return value is the absolute path of the canonical file.
1187
1188
1189=== modified file 'dmedia/gtkui/client.py'
1190--- dmedia/gtkui/client.py 2011-03-27 10:45:13 +0000
1191+++ dmedia/gtkui/client.py 2011-04-11 12:02:24 +0000
1192@@ -27,7 +27,7 @@
1193 import dbus.mainloop.glib
1194 from gi.repository import GObject
1195 from gi.repository.GObject import TYPE_PYOBJECT
1196-from dmedia.constants import BUS, INTERFACE, EXTENSIONS
1197+from dmedia.constants import IMPORT_BUS, IMPORT_IFACE, EXTENSIONS
1198
1199
1200 # We need mainloop integration to test signals:
1201@@ -122,7 +122,7 @@
1202
1203 def __init__(self, bus=None):
1204 super(Client, self).__init__()
1205- self._bus = (BUS if bus is None else bus)
1206+ self._bus = (IMPORT_BUS if bus is None else bus)
1207 self._conn = dbus.SessionBus()
1208 self._proxy = None
1209
1210@@ -137,27 +137,27 @@
1211 return self._proxy
1212
1213 def _call(self, name, *args):
1214- method = self.proxy.get_dbus_method(name, dbus_interface=INTERFACE)
1215+ method = self.proxy.get_dbus_method(name, dbus_interface=IMPORT_IFACE)
1216 return method(*args)
1217
1218 def _connect_signals(self):
1219 self.proxy.connect_to_signal(
1220- 'BatchStarted', self._on_BatchStarted, INTERFACE
1221- )
1222- self.proxy.connect_to_signal(
1223- 'BatchFinished', self._on_BatchFinished, INTERFACE
1224- )
1225- self.proxy.connect_to_signal(
1226- 'ImportStarted', self._on_ImportStarted, INTERFACE
1227- )
1228- self.proxy.connect_to_signal(
1229- 'ImportCount', self._on_ImportCount, INTERFACE
1230- )
1231- self.proxy.connect_to_signal(
1232- 'ImportProgress', self._on_ImportProgress, INTERFACE
1233- )
1234- self.proxy.connect_to_signal(
1235- 'ImportFinished', self._on_ImportFinished, INTERFACE
1236+ 'BatchStarted', self._on_BatchStarted, IMPORT_IFACE
1237+ )
1238+ self.proxy.connect_to_signal(
1239+ 'BatchFinished', self._on_BatchFinished, IMPORT_IFACE
1240+ )
1241+ self.proxy.connect_to_signal(
1242+ 'ImportStarted', self._on_ImportStarted, IMPORT_IFACE
1243+ )
1244+ self.proxy.connect_to_signal(
1245+ 'ImportCount', self._on_ImportCount, IMPORT_IFACE
1246+ )
1247+ self.proxy.connect_to_signal(
1248+ 'ImportProgress', self._on_ImportProgress, IMPORT_IFACE
1249+ )
1250+ self.proxy.connect_to_signal(
1251+ 'ImportFinished', self._on_ImportFinished, IMPORT_IFACE
1252 )
1253
1254 def _on_BatchStarted(self, batch_id):
1255
1256=== modified file 'dmedia/gtkui/service.py'
1257--- dmedia/gtkui/service.py 2011-03-28 12:38:29 +0000
1258+++ dmedia/gtkui/service.py 2011-04-11 12:02:24 +0000
1259@@ -22,7 +22,11 @@
1260 # with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
1261
1262 """
1263-Makes dmedia functionality avaible over D-Bus.
1264+D-Bus service implementing Pro File Import UX.
1265+
1266+For details, please see:
1267+
1268+ https://wiki.ubuntu.com/AyatanaDmediaLovefest
1269 """
1270
1271 from os import path
1272@@ -35,9 +39,8 @@
1273 import dbus.mainloop.glib
1274
1275 from dmedia import __version__
1276-from dmedia.constants import BUS, INTERFACE, DBNAME, EXT_MAP
1277+from dmedia.constants import IMPORT_BUS, IMPORT_IFACE, EXT_MAP
1278 from dmedia.importer import ImportManager
1279-from dmedia.metastore import MetaStore
1280
1281 from .util import NotifyManager, Timer, import_started, batch_finished
1282
1283@@ -71,11 +74,12 @@
1284 'ImportFinished',
1285 ])
1286
1287- def __init__(self, env, killfunc=None):
1288+ def __init__(self, env, bus, killfunc):
1289+ assert callable(killfunc)
1290 self._env = env
1291+ self._bus = bus
1292 self._killfunc = killfunc
1293- self._bus = env.get('bus', BUS)
1294- self._dbname = env.get('dbname', DBNAME)
1295+ self._bus = bus
1296 self._no_gui = env.get('no_gui', False)
1297 log.info('Starting service on %r', self._bus)
1298 self._conn = dbus.SessionBus()
1299@@ -120,21 +124,12 @@
1300 self._indicator.set_menu(self._menu)
1301 self._indicator.set_status(AppIndicator.IndicatorStatus.ACTIVE)
1302
1303- self._metastore = None
1304 self._manager = None
1305
1306 @property
1307- def metastore(self):
1308- if self._metastore is None:
1309- self._metastore = MetaStore(self._env)
1310- return self._metastore
1311-
1312- @property
1313 def manager(self):
1314 if self._manager is None:
1315- self._manager = ImportManager(
1316- self.metastore.get_env(), self._on_signal
1317- )
1318+ self._manager = ImportManager(self._env, self._on_signal)
1319 self._manager.start()
1320 return self._manager
1321
1322@@ -164,7 +159,7 @@
1323 except Exception:
1324 log.exception('Could not open dmedia database in Futon')
1325
1326- @dbus.service.signal(INTERFACE, signature='s')
1327+ @dbus.service.signal(IMPORT_IFACE, signature='s')
1328 def BatchStarted(self, batch_id):
1329 """
1330 Fired at transition from idle to at least one active import.
1331@@ -181,7 +176,7 @@
1332 self._indicator.set_menu(self._menu)
1333 self._timer.start()
1334
1335- @dbus.service.signal(INTERFACE, signature='sa{sx}')
1336+ @dbus.service.signal(IMPORT_IFACE, signature='sa{sx}')
1337 def BatchFinished(self, batch_id, stats):
1338 """
1339 Fired at transition from at least one active import to idle.
1340@@ -201,7 +196,7 @@
1341 self._timer.stop()
1342 self._current.hide()
1343
1344- @dbus.service.signal(INTERFACE, signature='ss')
1345+ @dbus.service.signal(IMPORT_IFACE, signature='ss')
1346 def ImportStarted(self, base, import_id):
1347 """
1348 Fired when card is inserted.
1349@@ -219,19 +214,19 @@
1350 # via FireWire or USB
1351 self._notify.replace(summary, body, 'notification-device-usb')
1352
1353- @dbus.service.signal(INTERFACE, signature='ssx')
1354+ @dbus.service.signal(IMPORT_IFACE, signature='ssx')
1355 def ImportCount(self, base, import_id, total):
1356 pass
1357
1358- @dbus.service.signal(INTERFACE, signature='ssiia{ss}')
1359+ @dbus.service.signal(IMPORT_IFACE, signature='ssiia{ss}')
1360 def ImportProgress(self, base, import_id, completed, total, info):
1361 pass
1362
1363- @dbus.service.signal(INTERFACE, signature='ssa{sx}')
1364+ @dbus.service.signal(IMPORT_IFACE, signature='ssa{sx}')
1365 def ImportFinished(self, base, import_id, stats):
1366 pass
1367
1368- @dbus.service.method(INTERFACE, in_signature='', out_signature='')
1369+ @dbus.service.method(IMPORT_IFACE, in_signature='', out_signature='')
1370 def Kill(self):
1371 """
1372 Kill the dmedia service process.
1373@@ -243,14 +238,14 @@
1374 log.info('Calling killfunc()')
1375 self._killfunc()
1376
1377- @dbus.service.method(INTERFACE, in_signature='', out_signature='s')
1378+ @dbus.service.method(IMPORT_IFACE, in_signature='', out_signature='s')
1379 def Version(self):
1380 """
1381 Return dmedia version.
1382 """
1383 return __version__
1384
1385- @dbus.service.method(INTERFACE, in_signature='as', out_signature='as')
1386+ @dbus.service.method(IMPORT_IFACE, in_signature='as', out_signature='as')
1387 def GetExtensions(self, types):
1388 """
1389 Get a list of extensions based on broad categories in *types*.
1390@@ -267,7 +262,7 @@
1391 extensions.update(EXT_MAP[key])
1392 return sorted(extensions)
1393
1394- @dbus.service.method(INTERFACE, in_signature='sb', out_signature='s')
1395+ @dbus.service.method(IMPORT_IFACE, in_signature='sb', out_signature='s')
1396 def StartImport(self, base, extract):
1397 """
1398 Start import of card mounted at *base*.
1399@@ -289,7 +284,7 @@
1400 return 'started'
1401 return 'already_running'
1402
1403- @dbus.service.method(INTERFACE, in_signature='s', out_signature='s')
1404+ @dbus.service.method(IMPORT_IFACE, in_signature='s', out_signature='s')
1405 def StopImport(self, base):
1406 """
1407 In running, stop the import of directory or file at *base*.
1408@@ -298,7 +293,7 @@
1409 return 'stopped'
1410 return 'not_running'
1411
1412- @dbus.service.method(INTERFACE, in_signature='', out_signature='as')
1413+ @dbus.service.method(IMPORT_IFACE, in_signature='', out_signature='as')
1414 def ListImports(self):
1415 """
1416 Return list of currently running imports.
1417
1418=== modified file 'dmedia/gtkui/tests/test_client.py'
1419--- dmedia/gtkui/tests/test_client.py 2011-03-27 10:45:13 +0000
1420+++ dmedia/gtkui/tests/test_client.py 2011-04-11 12:02:24 +0000
1421@@ -27,6 +27,7 @@
1422 from os import path
1423 from subprocess import Popen
1424 import time
1425+import json
1426
1427 import dbus
1428 from dbus.proxies import ProxyObject
1429@@ -44,7 +45,7 @@
1430
1431 tree = path.dirname(path.dirname(path.abspath(dmedia.__file__)))
1432 assert path.isfile(path.join(tree, 'setup.py'))
1433-script = path.join(tree, 'dmedia-service')
1434+script = path.join(tree, 'dmedia-importer-service')
1435 assert path.isfile(script)
1436
1437
1438@@ -73,7 +74,7 @@
1439 self.handlers[name] = callback
1440
1441
1442-class test_Client(CouchCase):
1443+class TestClient(CouchCase):
1444 klass = client.Client
1445
1446 def setUp(self):
1447@@ -87,17 +88,17 @@
1448 How do people usually unit test dbus services? This works, but not sure
1449 if there is a better idiom in common use. --jderose
1450 """
1451- super(test_Client, self).setUp()
1452+ super(TestClient, self).setUp()
1453 self.bus = random_bus()
1454 cmd = [script, '--no-gui',
1455- '--dbname', self.dbname,
1456 '--bus', self.bus,
1457+ '--env', json.dumps(self.env),
1458 ]
1459 self.service = Popen(cmd)
1460 time.sleep(1) # Give dmedia-service time to start
1461
1462 def tearDown(self):
1463- super(test_Client, self).tearDown()
1464+ super(TestClient, self).tearDown()
1465 try:
1466 self.service.terminate()
1467 self.service.wait()
1468@@ -112,7 +113,7 @@
1469 def test_init(self):
1470 # Test with no bus
1471 inst = self.klass()
1472- self.assertEqual(inst._bus, 'org.freedesktop.DMedia')
1473+ self.assertEqual(inst._bus, 'org.freedesktop.DMediaImporter')
1474 self.assertTrue(isinstance(inst._conn, dbus.SessionBus))
1475 self.assertTrue(inst._proxy is None)
1476
1477@@ -243,19 +244,19 @@
1478 self.assertEqual(
1479 signals.messages[3],
1480 ('import_progress', inst, base, import_id, 1, 3,
1481- dict(action='imported', src=src1, _id=mov_hash)
1482+ dict(action='imported', src=src1)
1483 )
1484 )
1485 self.assertEqual(
1486 signals.messages[4],
1487 ('import_progress', inst, base, import_id, 2, 3,
1488- dict(action='imported', src=src2, _id=thm_hash)
1489+ dict(action='imported', src=src2)
1490 )
1491 )
1492 self.assertEqual(
1493 signals.messages[5],
1494 ('import_progress', inst, base, import_id, 3, 3,
1495- dict(action='skipped', src=dup1, _id=mov_hash)
1496+ dict(action='skipped', src=dup1)
1497 )
1498 )
1499 self.assertEqual(
1500
1501=== removed file 'dmedia/gtkui/tests/test_service.py'
1502--- dmedia/gtkui/tests/test_service.py 2011-03-27 09:56:34 +0000
1503+++ dmedia/gtkui/tests/test_service.py 1970-01-01 00:00:00 +0000
1504@@ -1,53 +0,0 @@
1505-# Authors:
1506-# Jason Gerard DeRose <jderose@novacut.com>
1507-#
1508-# dmedia: distributed media library
1509-# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com>
1510-#
1511-# This file is part of `dmedia`.
1512-#
1513-# `dmedia` is free software: you can redistribute it and/or modify it under the
1514-# terms of the GNU Affero General Public License as published by the Free
1515-# Software Foundation, either version 3 of the License, or (at your option) any
1516-# later version.
1517-#
1518-# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY
1519-# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
1520-# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
1521-# details.
1522-#
1523-# You should have received a copy of the GNU Affero General Public License along
1524-# with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
1525-
1526-"""
1527-Unit tests for `dmedia.metastore` module.
1528-"""
1529-
1530-from unittest import TestCase
1531-
1532-from dmedia.tests.helpers import TempDir, random_bus
1533-from dmedia.tests.couch import CouchCase
1534-from dmedia import importer
1535-from dmedia.gtkui import service
1536-
1537-
1538-class test_DMedia(CouchCase):
1539- klass = service.DMedia
1540-
1541- def test_init(self):
1542- bus = random_bus()
1543- self.env['bus'] = bus
1544- self.env['no_gui'] = True
1545- def kill():
1546- pass
1547- inst = self.klass(self.env, killfunc=kill)
1548- self.assertTrue(inst._killfunc is kill)
1549- self.assertTrue(inst._bus is bus)
1550- self.assertTrue(inst._dbname is self.dbname)
1551- self.assertTrue(inst._no_gui)
1552- self.assertEqual(inst._manager, None)
1553-
1554- m = inst.manager
1555- self.assertTrue(inst._manager is m)
1556- self.assertTrue(isinstance(m, importer.ImportManager))
1557- self.assertEqual(m._callback, inst._on_signal)
1558
1559=== modified file 'dmedia/importer.py'
1560--- dmedia/importer.py 2011-04-08 20:59:58 +0000
1561+++ dmedia/importer.py 2011-04-11 12:02:24 +0000
1562@@ -40,8 +40,6 @@
1563 )
1564 from .filestore import FileStore, quick_id, safe_open, safe_ext, pack_leaves
1565 from .extractor import merge_metadata
1566-from .abstractcouch import get_env, get_couchdb_server, get_dmedia_db
1567-
1568
1569 mimetypes.init()
1570 DOTDIR = '.dmedia'
1571@@ -165,15 +163,8 @@
1572 def __init__(self, env, q, key, args):
1573 super(ImportWorker, self).__init__(env, q, key, args)
1574 (self.base, self.extract) = args
1575- self.home = path.abspath(os.environ['HOME'])
1576- self.filestore = FileStore(
1577- path.join(self.home, DOTDIR),
1578- self.env.get('machine_id')
1579- )
1580- try:
1581- self.db.save(self.filestore._doc)
1582- except couchdb.ResourceConflict:
1583- pass
1584+ self.filestore = FileStore(self.env['filestore']['path'])
1585+ self.filestore_id = self.env['filestore']['_id']
1586
1587 self.filetuples = None
1588 self._processed = []
1589@@ -269,8 +260,8 @@
1590
1591 try:
1592 doc = self.db[chash]
1593- if self.filestore._id not in doc['stored']:
1594- doc['stored'][self.filestore._id] = {
1595+ if self.filestore_id not in doc['stored']:
1596+ doc['stored'][self.filestore_id] = {
1597 'copies': 1,
1598 'time': time.time(),
1599 }
1600@@ -279,7 +270,7 @@
1601 except couchdb.ResourceNotFound as e:
1602 pass
1603
1604- doc = create_file(stat.st_size, leaves, self.filestore._id,
1605+ doc = create_file(stat.st_size, leaves, self.filestore_id,
1606 copies=1, ext=ext
1607 )
1608 assert doc['_id'] == chash
1609
1610=== modified file 'dmedia/schema.py'
1611--- dmedia/schema.py 2011-04-07 03:03:07 +0000
1612+++ dmedia/schema.py 2011-04-11 12:02:24 +0000
1613@@ -315,6 +315,8 @@
1614 from base64 import b32encode, b32decode, b64encode
1615 import re
1616 import time
1617+import socket
1618+import platform
1619
1620 from .constants import TYPE_ERROR, EXT_PAT
1621
1622@@ -1284,7 +1286,7 @@
1623 'bytes': file_size,
1624 'ext': ext,
1625 'origin': origin,
1626- 'stored': {
1627+ 'stored': {
1628 store: {
1629 'copies': copies,
1630 'time': ts,
1631@@ -1293,7 +1295,21 @@
1632 }
1633
1634
1635-def create_store(base, machine_id, copies=1):
1636+def create_machine():
1637+ """
1638+ Create a 'dmedia/machine' document.
1639+ """
1640+ return {
1641+ '_id': random_id(),
1642+ 'ver': 0,
1643+ 'type': 'dmedia/machine',
1644+ 'time': time.time(),
1645+ 'hostname': socket.gethostname(),
1646+ 'distribution': list(platform.linux_distribution()),
1647+ }
1648+
1649+
1650+def create_store(parentdir, machine_id, copies=1):
1651 """
1652 Create a 'dmedia/store' document.
1653 """
1654@@ -1304,7 +1320,7 @@
1655 'time': time.time(),
1656 'plugin': 'filestore',
1657 'copies': copies,
1658- 'path': base,
1659+ 'path': parentdir,
1660 'machine_id': machine_id,
1661 }
1662
1663
1664=== added directory 'dmedia/service'
1665=== added file 'dmedia/service/__init__.py'
1666--- dmedia/service/__init__.py 1970-01-01 00:00:00 +0000
1667+++ dmedia/service/__init__.py 2011-04-11 12:02:24 +0000
1668@@ -0,0 +1,94 @@
1669+# Authors:
1670+# Jason Gerard DeRose <jderose@novacut.com>
1671+#
1672+# dmedia: distributed media library
1673+# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com>
1674+#
1675+# This file is part of `dmedia`.
1676+#
1677+# `dmedia` is free software: you can redistribute it and/or modify it under the
1678+# terms of the GNU Affero General Public License as published by the Free
1679+# Software Foundation, either version 3 of the License, or (at your option) any
1680+# later version.
1681+#
1682+# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY
1683+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
1684+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
1685+# details.
1686+#
1687+# You should have received a copy of the GNU Affero General Public License along
1688+# with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
1689+
1690+"""
1691+Core dmedia DBus service at org.freedesktop.DMedia.
1692+"""
1693+
1694+import logging
1695+import json
1696+import time
1697+
1698+import gnomekeyring
1699+import dbus
1700+import dbus.service
1701+
1702+from dmedia import __version__
1703+from dmedia.constants import IFACE
1704+from dmedia.core import Core
1705+
1706+
1707+log = logging.getLogger()
1708+
1709+
1710+class DMedia(dbus.service.Object):
1711+ def __init__(self, couchargs, bus, killfunc, start=None):
1712+ self._bus = bus
1713+ self._killfunc = killfunc
1714+ log.info('Starting dmedia core service on %r', bus)
1715+ self._conn = dbus.SessionBus()
1716+ super(DMedia, self).__init__(self._conn, object_path='/')
1717+ self._busname = dbus.service.BusName(bus, self._conn)
1718+ self._core = Core(*couchargs)
1719+ self._core.bootstrap()
1720+ self._env_s = json.dumps(self._core.env)
1721+ if start is not None:
1722+ log.info('Started service in %.3f seconds', time.time() - start)
1723+
1724+ @dbus.service.method(IFACE, in_signature='', out_signature='s')
1725+ def Version(self):
1726+ """
1727+ Return dmedia version.
1728+ """
1729+ return __version__
1730+
1731+ @dbus.service.method(IFACE, in_signature='', out_signature='')
1732+ def Kill(self):
1733+ """
1734+ Kill the `dmedia-service` process.
1735+ """
1736+ log.info('Killing dmedia core service on %r', self._bus)
1737+ self._killfunc()
1738+
1739+ @dbus.service.method(IFACE, in_signature='', out_signature='s')
1740+ def GetEnv(self):
1741+ """
1742+ Return dmedia env as JSON string.
1743+ """
1744+ return self._env_s
1745+
1746+ @dbus.service.method(IFACE, in_signature='', out_signature='s')
1747+ def GetAuthURL(self):
1748+ """
1749+ Get URL with basic auth user and password.
1750+ """
1751+ data = gnomekeyring.find_items_sync(
1752+ gnomekeyring.ITEM_GENERIC_SECRET,
1753+ {'desktopcouch': 'basic'}
1754+ )
1755+ (user, password) = data[0].secret.split(':')
1756+ return 'http://{user}:{password}@localhost:{port}/'.format(
1757+ user=user, password=password, port=self._core.env['port']
1758+ )
1759+
1760+ @dbus.service.method(IFACE, in_signature='', out_signature='b')
1761+ def HasApp(self):
1762+ return self._core.has_app()
1763
1764=== modified file 'dmedia/tests/couch.py'
1765--- dmedia/tests/couch.py 2011-03-27 09:56:34 +0000
1766+++ dmedia/tests/couch.py 2011-04-11 12:02:24 +0000
1767@@ -27,7 +27,8 @@
1768
1769 import couchdb
1770
1771-from dmedia.abstractcouch import get_env, get_couchdb_server
1772+from dmedia.core import get_env
1773+from dmedia.abstractcouch import get_server
1774 from dmedia.schema import random_id
1775
1776 from .helpers import TempHome
1777@@ -48,15 +49,16 @@
1778
1779 def setUp(self):
1780 self.home = TempHome()
1781- self.dbname = 'dmedia_test'
1782+ self.dbname = 'test_dmedia'
1783 self.env = get_env(self.dbname)
1784- server = get_couchdb_server(self.env)
1785+ server = get_server(self.env)
1786 try:
1787 del server[self.dbname]
1788 except couchdb.ResourceNotFound:
1789 pass
1790 self.machine_id = random_id()
1791 self.env['machine_id'] = self.machine_id
1792+ self.env['filestore'] = {'_id': random_id(), 'path': self.home.path}
1793
1794 def tearDown(self):
1795 self.home = None
1796
1797=== modified file 'dmedia/tests/test_abstractcouch.py'
1798--- dmedia/tests/test_abstractcouch.py 2011-02-21 10:55:29 +0000
1799+++ dmedia/tests/test_abstractcouch.py 2011-04-11 12:02:24 +0000
1800@@ -24,8 +24,10 @@
1801 """
1802
1803 from unittest import TestCase
1804+import json
1805
1806 import couchdb
1807+from couchdb import ResourceNotFound
1808 from desktopcouch.application.platform import find_port
1809 from desktopcouch.application.local_files import get_oauth_tokens
1810 from desktopcouch.records.http import OAuthSession
1811@@ -34,29 +36,34 @@
1812
1813 from .helpers import raises
1814
1815+
1816+def dc_env(dbname='test_dmedia'):
1817+ """
1818+ Create desktopcouch environment.
1819+ """
1820+ port = find_port()
1821+ return {
1822+ 'dbname': dbname,
1823+ 'port': port,
1824+ 'url': 'http://localhost:%d/' % port,
1825+ 'oauth': get_oauth_tokens(),
1826+ }
1827+
1828+
1829+def json_env(dbname='test_dmedia'):
1830+ """
1831+ For testing with dc env that has gone though json.dumps() + json.loads().
1832+ """
1833+ return json.loads(json.dumps(dc_env(dbname)))
1834+
1835+
1836 class test_functions(TestCase):
1837 def tearDown(self):
1838 if abstractcouch.OAuthSession is None:
1839 abstractcouch.OAuthSession = OAuthSession
1840
1841- def dc_env(self):
1842- """
1843- Create an *env* for desktopcouch.
1844- """
1845- port = find_port()
1846- return {
1847- 'port': port,
1848- 'url': 'http://localhost:%d/' % port,
1849- 'oauth': get_oauth_tokens(),
1850- }
1851-
1852- def test_get_couchdb_server(self):
1853- f = abstractcouch.get_couchdb_server
1854-
1855- # Test with empty env
1856- s = f({})
1857- self.assertTrue(isinstance(s, couchdb.Server))
1858- self.assertEqual(repr(s), "<Server 'http://localhost:5984/'>")
1859+ def test_get_server(self):
1860+ f = abstractcouch.get_server
1861
1862 # Test with only url
1863 s = f({'url': 'http://localhost:5984/'})
1864@@ -64,7 +71,7 @@
1865 self.assertEqual(repr(s), "<Server 'http://localhost:5984/'>")
1866
1867 # Test with desktopcouch
1868- env = self.dc_env()
1869+ env = dc_env()
1870 s = f(env)
1871 self.assertTrue(isinstance(s, couchdb.Server))
1872 self.assertEqual(
1873@@ -81,127 +88,49 @@
1874 )
1875
1876 # Test when OAuthSession is not imported, oauth not provided
1877- s = f({})
1878- self.assertTrue(isinstance(s, couchdb.Server))
1879- self.assertEqual(repr(s), "<Server 'http://localhost:5984/'>")
1880- s = f({'url': 'http://localhost:5984/'})
1881- self.assertTrue(isinstance(s, couchdb.Server))
1882- self.assertEqual(repr(s), "<Server 'http://localhost:5984/'>")
1883-
1884- def test_get_dmedia_db(self):
1885- f = abstractcouch.get_dmedia_db
1886-
1887- # Test when server is not provided
1888- env = self.dc_env()
1889-
1890- assert 'dmedia' not in env
1891- d = f(env)
1892- self.assertTrue(isinstance(d, couchdb.Database))
1893- self.assertEqual(repr(d), "<Database 'dmedia'>")
1894- self.assertEqual(d.info()['db_name'], 'dmedia')
1895-
1896- env['dbname'] = None
1897- d = f(env)
1898- self.assertTrue(isinstance(d, couchdb.Database))
1899- self.assertEqual(repr(d), "<Database 'dmedia'>")
1900- self.assertEqual(d.info()['db_name'], 'dmedia')
1901-
1902- env['dbname'] = 'dmedia'
1903- d = f(env)
1904- self.assertTrue(isinstance(d, couchdb.Database))
1905- self.assertEqual(repr(d), "<Database 'dmedia'>")
1906- self.assertEqual(d.info()['db_name'], 'dmedia')
1907-
1908- env['dbname'] = 'dmedia_test'
1909- d = f(env)
1910- self.assertTrue(isinstance(d, couchdb.Database))
1911- self.assertEqual(repr(d), "<Database 'dmedia_test'>")
1912- self.assertEqual(d.info()['db_name'], 'dmedia_test')
1913-
1914-
1915- # Test when server *is* provided
1916- env = self.dc_env()
1917- server = abstractcouch.get_couchdb_server(env)
1918-
1919- assert 'dmedia' not in env
1920- d = f(env, server)
1921- self.assertTrue(isinstance(d, couchdb.Database))
1922- self.assertEqual(repr(d), "<Database 'dmedia'>")
1923- self.assertEqual(d.info()['db_name'], 'dmedia')
1924-
1925- env['dbname'] = None
1926- d = f(env, server)
1927- self.assertTrue(isinstance(d, couchdb.Database))
1928- self.assertEqual(repr(d), "<Database 'dmedia'>")
1929- self.assertEqual(d.info()['db_name'], 'dmedia')
1930-
1931- env['dbname', server] = 'dmedia'
1932- d = f(env)
1933- self.assertTrue(isinstance(d, couchdb.Database))
1934- self.assertEqual(repr(d), "<Database 'dmedia'>")
1935- self.assertEqual(d.info()['db_name'], 'dmedia')
1936-
1937- env['dbname'] = 'dmedia_test'
1938- d = f(env, server)
1939- self.assertTrue(isinstance(d, couchdb.Database))
1940- self.assertEqual(repr(d), "<Database 'dmedia_test'>")
1941- self.assertEqual(d.info()['db_name'], 'dmedia_test')
1942-
1943-
1944- # Test when server=None is explicitly provided
1945- env = self.dc_env()
1946-
1947- assert 'dmedia' not in env
1948- d = f(env, server=None)
1949- self.assertTrue(isinstance(d, couchdb.Database))
1950- self.assertEqual(repr(d), "<Database 'dmedia'>")
1951- self.assertEqual(d.info()['db_name'], 'dmedia')
1952-
1953- env['dbname'] = None
1954- d = f(env, server=None)
1955- self.assertTrue(isinstance(d, couchdb.Database))
1956- self.assertEqual(repr(d), "<Database 'dmedia'>")
1957- self.assertEqual(d.info()['db_name'], 'dmedia')
1958-
1959- env['dbname'] = 'dmedia'
1960- d = f(env, server=None)
1961- self.assertTrue(isinstance(d, couchdb.Database))
1962- self.assertEqual(repr(d), "<Database 'dmedia'>")
1963- self.assertEqual(d.info()['db_name'], 'dmedia')
1964-
1965- env['dbname'] = 'dmedia_test'
1966- d = f(env, server=None)
1967- self.assertTrue(isinstance(d, couchdb.Database))
1968- self.assertEqual(repr(d), "<Database 'dmedia_test'>")
1969- self.assertEqual(d.info()['db_name'], 'dmedia_test')
1970-
1971- def test_get_env(self):
1972- f = abstractcouch.get_env
1973- port = find_port()
1974- url = 'http://localhost:%d/' % port
1975- oauth = get_oauth_tokens()
1976-
1977- # Test when OAuthSession is available
1978- self.assertEqual(
1979- f(),
1980- {'port': port, 'url': url, 'oauth': oauth, 'dbname': None}
1981- )
1982- self.assertEqual(
1983- f(dbname=None),
1984- {'port': port, 'url': url, 'oauth': oauth, 'dbname': None}
1985- )
1986- self.assertEqual(
1987- f(dbname='dmedia'),
1988- {'port': port, 'url': url, 'oauth': oauth, 'dbname': 'dmedia'}
1989- )
1990- self.assertEqual(
1991- f(dbname='dmedia_test'),
1992- {'port': port, 'url': url, 'oauth': oauth, 'dbname': 'dmedia_test'}
1993- )
1994-
1995- # Test when OAuthSession is *not* available
1996- abstractcouch.OAuthSession = None
1997- self.assertEqual(f(), {'dbname': None})
1998- self.assertEqual(f(dbname=None), {'dbname': None})
1999- self.assertEqual(f(dbname='dmedia'), {'dbname': 'dmedia'})
2000- self.assertEqual(f(dbname='dmedia_test'), {'dbname': 'dmedia_test'})
2001+ s = f({'url': 'http://localhost:666/'})
2002+ self.assertTrue(isinstance(s, couchdb.Server))
2003+ self.assertEqual(repr(s), "<Server 'http://localhost:666/'>")
2004+
2005+
2006+ def test_get_db(self):
2007+ f = abstractcouch.get_db
2008+ env = dc_env('test_dmedia')
2009+ server = abstractcouch.get_server(env)
2010+
2011+ # Make sure database doesn't exist:
2012+ try:
2013+ server.delete('test_dmedia')
2014+ except ResourceNotFound:
2015+ pass
2016+
2017+ # Test when db does not exist, server not provided
2018+ self.assertNotIn('test_dmedia', server)
2019+ db = f(env)
2020+ self.assertTrue(isinstance(db, couchdb.Database))
2021+ self.assertEqual(repr(db), "<Database 'test_dmedia'>")
2022+ self.assertEqual(db.info()['db_name'], 'test_dmedia')
2023+ self.assertIn('test_dmedia', server)
2024+
2025+ # Test when db exists, server not provided
2026+ db = f(env)
2027+ self.assertTrue(isinstance(db, couchdb.Database))
2028+ self.assertEqual(repr(db), "<Database 'test_dmedia'>")
2029+ self.assertEqual(db.info()['db_name'], 'test_dmedia')
2030+ self.assertIn('test_dmedia', server)
2031+
2032+ # Test when db does not exist, server *is* provided
2033+ server.delete('test_dmedia')
2034+ self.assertNotIn('test_dmedia', server)
2035+ db = f(env, server=server)
2036+ self.assertTrue(isinstance(db, couchdb.Database))
2037+ self.assertEqual(repr(db), "<Database 'test_dmedia'>")
2038+ self.assertEqual(db.info()['db_name'], 'test_dmedia')
2039+ self.assertIn('test_dmedia', server)
2040+
2041+ # Test when db exists, server *is* provided
2042+ db = f(env, server=server)
2043+ self.assertTrue(isinstance(db, couchdb.Database))
2044+ self.assertEqual(repr(db), "<Database 'test_dmedia'>")
2045+ self.assertEqual(db.info()['db_name'], 'test_dmedia')
2046+ self.assertIn('test_dmedia', server)
2047
2048=== added file 'dmedia/tests/test_api.py'
2049--- dmedia/tests/test_api.py 1970-01-01 00:00:00 +0000
2050+++ dmedia/tests/test_api.py 2011-04-11 12:02:24 +0000
2051@@ -0,0 +1,125 @@
2052+# Authors:
2053+# Jason Gerard DeRose <jderose@novacut.com>
2054+#
2055+# dmedia: distributed media library
2056+# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com>
2057+#
2058+# This file is part of `dmedia`.
2059+#
2060+# `dmedia` is free software: you can redistribute it and/or modify it under the
2061+# terms of the GNU Affero General Public License as published by the Free
2062+# Software Foundation, either version 3 of the License, or (at your option) any
2063+# later version.
2064+#
2065+# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY
2066+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
2067+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
2068+# details.
2069+#
2070+# You should have received a copy of the GNU Affero General Public License along
2071+# with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
2072+
2073+"""
2074+Unit tests for `dmedia.api` module.
2075+"""
2076+
2077+import os
2078+from os import path
2079+from subprocess import Popen
2080+import json
2081+import time
2082+
2083+import gnomekeyring
2084+
2085+import dmedia
2086+from dmedia.abstractcouch import get_db
2087+from dmedia import api
2088+
2089+from .couch import CouchCase
2090+from .helpers import random_bus
2091+
2092+
2093+tree = path.dirname(path.dirname(path.abspath(dmedia.__file__)))
2094+assert path.isfile(path.join(tree, 'setup.py'))
2095+script = path.join(tree, 'dmedia-service')
2096+assert path.isfile(script)
2097+
2098+
2099+def get_auth():
2100+ data = gnomekeyring.find_items_sync(
2101+ gnomekeyring.ITEM_GENERIC_SECRET,
2102+ {'desktopcouch': 'basic'}
2103+ )
2104+ (user, password) = data[0].secret.split(':')
2105+ return (user, password)
2106+
2107+
2108+class TestDMedia(CouchCase):
2109+ klass = api.DMedia
2110+
2111+ def setUp(self):
2112+ """
2113+ Launch dmedia dbus service using a random bus name.
2114+
2115+ This will launch dmedia-service with a random bus name like this:
2116+
2117+ dmedia-service --bus org.test3ISHAWZVSWVN5I5S.DMedia
2118+
2119+ How do people usually unit test dbus services? This works, but not sure
2120+ if there is a better idiom in common use. --jderose
2121+ """
2122+ super(TestDMedia, self).setUp()
2123+ self.bus = random_bus()
2124+ cmd = [script,
2125+ '--bus', self.bus,
2126+ '--env', json.dumps(self.env),
2127+ ]
2128+ self.service = Popen(cmd)
2129+ time.sleep(1) # Give dmedia-service time to start
2130+
2131+ def tearDown(self):
2132+ super(TestDMedia, self).tearDown()
2133+ try:
2134+ self.service.terminate()
2135+ self.service.wait()
2136+ except OSError:
2137+ pass
2138+ finally:
2139+ self.service = None
2140+
2141+ def test_all(self):
2142+ inst = self.klass(self.bus)
2143+
2144+ # DMedia.Version()
2145+ self.assertEqual(inst.version(), dmedia.__version__)
2146+
2147+ # DMedia.GetEnv()
2148+ env = inst.get_env()
2149+ self.assertEqual(env['oauth'], self.env['oauth'])
2150+ self.assertEqual(env['port'], self.env['port'])
2151+ self.assertEqual(env['url'], self.env['url'])
2152+ self.assertEqual(env['dbname'], self.env['dbname'])
2153+
2154+ # DMedia.GetAuthURL()
2155+ (user, password) = get_auth()
2156+ self.assertEqual(
2157+ inst.get_auth_url(),
2158+ 'http://{user}:{password}@localhost:{port}/'.format(
2159+ user=user, password=password, port=self.env['port']
2160+ )
2161+ )
2162+
2163+ # DMedia.HasApp()
2164+ db = get_db(self.env)
2165+ self.assertNotIn('app', db)
2166+ self.assertTrue(inst.has_app())
2167+ self.assertTrue(db['app']['_rev'].startswith('1-'))
2168+ self.assertTrue(inst.has_app())
2169+ self.assertTrue(db['app']['_rev'].startswith('1-'))
2170+
2171+ # DMedia.Kill()
2172+ self.assertIsNone(self.service.poll(), None)
2173+ inst.kill()
2174+ self.assertTrue(inst._proxy is None)
2175+ time.sleep(1) # Give dmedia-service time to shutdown
2176+ self.assertEqual(self.service.poll(), 0)
2177
2178=== added file 'dmedia/tests/test_core.py'
2179--- dmedia/tests/test_core.py 1970-01-01 00:00:00 +0000
2180+++ dmedia/tests/test_core.py 2011-04-11 12:02:24 +0000
2181@@ -0,0 +1,336 @@
2182+# Authors:
2183+# Jason Gerard DeRose <jderose@novacut.com>
2184+#
2185+# dmedia: distributed media library
2186+# Copyright (C) 2011 Jason Gerard DeRose <jderose@novacut.com>
2187+#
2188+# This file is part of `dmedia`.
2189+#
2190+# `dmedia` is free software: you can redistribute it and/or modify it under the
2191+# terms of the GNU Affero General Public License as published by the Free
2192+# Software Foundation, either version 3 of the License, or (at your option) any
2193+# later version.
2194+#
2195+# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY
2196+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
2197+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
2198+# details.
2199+#
2200+# You should have received a copy of the GNU Affero General Public License along
2201+# with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
2202+
2203+"""
2204+Unit tests for `dmedia.core` module.
2205+"""
2206+
2207+from unittest import TestCase
2208+import json
2209+
2210+import couchdb
2211+import desktopcouch
2212+from desktopcouch.application.platform import find_port
2213+from desktopcouch.application.local_files import get_oauth_tokens
2214+
2215+from dmedia.webui.app import App
2216+from dmedia.schema import random_id
2217+from dmedia import core
2218+
2219+from .couch import CouchCase
2220+
2221+
2222+def dc_env(dbname):
2223+ """
2224+ Create desktopcouch environment.
2225+ """
2226+ port = find_port()
2227+ return {
2228+ 'dbname': dbname,
2229+ 'port': port,
2230+ 'url': 'http://localhost:%d/' % port,
2231+ 'oauth': get_oauth_tokens(),
2232+ }
2233+
2234+
2235+class TestFunctions(TestCase):
2236+ def tearDown(self):
2237+ if core.desktopcouch is None:
2238+ core.desktopcouch = desktopcouch
2239+
2240+ def test_get_env(self):
2241+ f = core.get_env
2242+
2243+ # Test when desktopcouch is available
2244+ self.assertEqual(f(), dc_env('dmedia'))
2245+ self.assertEqual(f('foo'), dc_env('foo'))
2246+ self.assertEqual(f(dbname='bar'), dc_env('bar'))
2247+
2248+ # Test when desktopcouch is available but no_dc=True
2249+ self.assertEqual(
2250+ f(no_dc=True),
2251+ {
2252+ 'dbname': 'dmedia',
2253+ 'port': 5984,
2254+ 'url': 'http://localhost:5984/',
2255+ }
2256+ )
2257+ self.assertEqual(
2258+ f(dbname='foo', no_dc=True),
2259+ {
2260+ 'dbname': 'foo',
2261+ 'port': 5984,
2262+ 'url': 'http://localhost:5984/',
2263+ }
2264+ )
2265+ self.assertEqual(
2266+ f('bar', True),
2267+ {
2268+ 'dbname': 'bar',
2269+ 'port': 5984,
2270+ 'url': 'http://localhost:5984/',
2271+ }
2272+ )
2273+
2274+ # Test when desktopcouch is *not* available
2275+ core.desktopcouch = None
2276+ self.assertEqual(
2277+ f(),
2278+ {
2279+ 'dbname': 'dmedia',
2280+ 'port': 5984,
2281+ 'url': 'http://localhost:5984/',
2282+ }
2283+ )
2284+ self.assertEqual(
2285+ f('foo'),
2286+ {
2287+ 'dbname': 'foo',
2288+ 'port': 5984,
2289+ 'url': 'http://localhost:5984/',
2290+ }
2291+ )
2292+ self.assertEqual(
2293+ f(dbname='bar'),
2294+ {
2295+ 'dbname': 'bar',
2296+ 'port': 5984,
2297+ 'url': 'http://localhost:5984/',
2298+ }
2299+ )
2300+
2301+
2302+class TestCore(CouchCase):
2303+ klass = core.Core
2304+
2305+ def tearDown(self):
2306+ super(TestCore, self).tearDown()
2307+ if core.App is None:
2308+ core.App = App
2309+
2310+ def test_init(self):
2311+ inst = self.klass(self.dbname)
2312+ self.assertNotEqual(inst.env, self.env)
2313+ self.assertEqual(
2314+ set(inst.env),
2315+ set(['port', 'url', 'dbname', 'oauth'])
2316+ )
2317+ self.assertEqual(inst.env['dbname'], self.dbname)
2318+ self.assertEqual(inst.env['port'], self.env['port'])
2319+ self.assertEqual(inst.env['url'], self.env['url'])
2320+ self.assertEqual(inst.env['oauth'], self.env['oauth'])
2321+ self.assertEqual(inst.home, self.home.path)
2322+ self.assertIsInstance(inst.db, couchdb.Database)
2323+
2324+ inst = self.klass(env_s=json.dumps(self.env))
2325+ self.assertEqual(inst.env, self.env)
2326+
2327+ def test_bootstrap(self):
2328+ inst = self.klass(self.dbname)
2329+ self.assertNotIn('machine_id', inst.env)
2330+ self.assertIsNone(inst.bootstrap())
2331+ self.assertEqual(inst.env['machine_id'], inst.machine_id)
2332+
2333+ def test_init_local(self):
2334+ inst = self.klass(self.dbname)
2335+
2336+ # Test when _local/dmedia doesn't exist:
2337+ (local, machine) = inst.init_local()
2338+
2339+ self.assertIsInstance(local, dict)
2340+ self.assertEqual(
2341+ set(local),
2342+ set([
2343+ '_id',
2344+ '_rev',
2345+ 'machine',
2346+ 'filestores',
2347+ ])
2348+ )
2349+ self.assertEqual(local['filestores'], {})
2350+ self.assertEqual(local, inst.db['_local/dmedia'])
2351+
2352+ self.assertIsInstance(machine, dict)
2353+ self.assertEqual(
2354+ set(machine),
2355+ set([
2356+ '_id',
2357+ '_rev',
2358+ 'ver',
2359+ 'type',
2360+ 'time',
2361+ 'hostname',
2362+ 'distribution',
2363+ ])
2364+ )
2365+ self.assertEqual(machine, inst.db[local['machine']['_id']])
2366+
2367+ # Test when _local/machine exists but 'dmedia/machine' doc doesn't:
2368+ inst.db.delete(machine)
2369+ (local2, machine2) = inst.init_local()
2370+ self.assertEqual(local2, local)
2371+ self.assertTrue(machine2['_rev'].startswith('3-'))
2372+ self.assertNotEqual(machine2['_rev'], machine['_rev'])
2373+ d = dict(machine2)
2374+ d.pop('_rev')
2375+ self.assertEqual(d, local['machine'])
2376+
2377+ # Test when both _local/dmedia and 'dmedia/machine' doc exist:
2378+ local3 = {
2379+ '_id': '_local/dmedia',
2380+ '_rev': local2['_rev'],
2381+ 'machine': {
2382+ '_id': 'foobar',
2383+ 'hello': 'world',
2384+ }
2385+ }
2386+ inst.db.save(local3)
2387+ machine3 = {
2388+ '_id': 'foobar',
2389+ '_rev': machine2['_rev'],
2390+ 'hello': 'naughty nurse',
2391+ }
2392+ inst.db.save(machine3)
2393+ (local4, machine4) = inst.init_local()
2394+ self.assertEqual(local4, local3)
2395+ self.assertEqual(machine4, machine3)
2396+
2397+ def test_init_filestores(self):
2398+ inst = self.klass(self.dbname)
2399+ (inst.local, inst.machine) = inst.init_local()
2400+ inst.machine_id = inst.machine['_id']
2401+ inst.env['machine_id'] = inst.machine_id
2402+
2403+ self.assertEqual(inst.local['filestores'], {})
2404+ self.assertNotIn('default_filestore', inst.local)
2405+ lstore = inst.init_filestores()
2406+ self.assertEqual(inst.local, inst.db['_local/dmedia'])
2407+ self.assertEqual(len(inst.local['filestores']), 1)
2408+ _id = inst.local['default_filestore']
2409+ self.assertEqual(inst.local['filestores'][_id], lstore)
2410+ self.assertEqual(
2411+ set(lstore),
2412+ set([
2413+ '_id',
2414+ 'ver',
2415+ 'type',
2416+ 'time',
2417+ 'plugin',
2418+ 'copies',
2419+ 'path',
2420+ 'machine_id',
2421+ ])
2422+ )
2423+ self.assertEqual(lstore['ver'], 0)
2424+ self.assertEqual(lstore['type'], 'dmedia/store')
2425+ self.assertEqual(lstore['plugin'], 'filestore')
2426+ self.assertEqual(lstore['copies'], 1)
2427+ self.assertEqual(lstore['path'], self.home.path)
2428+ self.assertEqual(lstore['machine_id'], inst.machine_id)
2429+
2430+ store = inst.db[_id]
2431+ self.assertTrue(store['_rev'].startswith('1-'))
2432+ store.pop('_rev')
2433+ self.assertEqual(store, lstore)
2434+
2435+ # Try again when docs already exist:
2436+ self.assertEqual(inst.init_filestores(), lstore)
2437+
2438+ def test_init_app(self):
2439+ inst = self.klass(self.dbname)
2440+
2441+ # App is available
2442+ self.assertNotIn('app', inst.db)
2443+ self.assertIs(inst.init_app(), True)
2444+ self.assertIn('app', inst.db)
2445+ self.assertIs(inst.init_app(), True)
2446+
2447+ # App is not available
2448+ core.App = None
2449+ self.assertIs(inst.init_app(), False)
2450+
2451+ # App is available again (make sure there is no state)
2452+ core.App = App
2453+ self.assertIs(inst.init_app(), True)
2454+ self.assertIs(inst.init_app(), True)
2455+
2456+ def test_has_app(self):
2457+ class Sub(self.klass):
2458+ _calls = 0
2459+
2460+ def init_app(self):
2461+ self._calls += 1
2462+ return 'A' * self._calls
2463+
2464+ inst = Sub(self.dbname)
2465+ self.assertIsNone(inst._has_app)
2466+
2467+ inst._has_app = 'foo'
2468+ self.assertEqual(inst.has_app(), 'foo')
2469+ self.assertEqual(inst._calls, 0)
2470+
2471+ inst._has_app = None
2472+ self.assertEqual(inst.has_app(), 'A')
2473+ self.assertEqual(inst._calls, 1)
2474+ self.assertEqual(inst._has_app, 'A')
2475+
2476+ self.assertEqual(inst.has_app(), 'A')
2477+ self.assertEqual(inst._calls, 1)
2478+ self.assertEqual(inst._has_app, 'A')
2479+
2480+ inst._has_app = None
2481+ self.assertEqual(inst.has_app(), 'AA')
2482+ self.assertEqual(inst._calls, 2)
2483+ self.assertEqual(inst._has_app, 'AA')
2484+
2485+ self.assertEqual(inst.has_app(), 'AA')
2486+ self.assertEqual(inst._calls, 2)
2487+ self.assertEqual(inst._has_app, 'AA')
2488+
2489+
2490+ # Test the real thing, no App
2491+ core.App = None
2492+ inst = self.klass(self.dbname)
2493+ self.assertIs(inst.has_app(), False)
2494+ self.assertIs(inst._has_app, False)
2495+ self.assertNotIn('app', inst.db)
2496+
2497+
2498+ # Test the real thing, App available
2499+ core.App = App
2500+ inst = self.klass(self.dbname)
2501+
2502+ self.assertNotIn('app', inst.db)
2503+ self.assertIs(inst.has_app(), True)
2504+ self.assertIs(inst._has_app, True)
2505+ rev = inst.db['app']['_rev']
2506+ self.assertTrue(rev.startswith('1-'))
2507+
2508+ self.assertIs(inst.has_app(), True)
2509+ self.assertIs(inst._has_app, True)
2510+ self.assertEqual(inst.db['app']['_rev'], rev)
2511+
2512+ inst._has_app = None
2513+ self.assertIs(inst.has_app(), True)
2514+ self.assertIs(inst._has_app, True)
2515+ rev2 = inst.db['app']['_rev']
2516+ self.assertNotEqual(rev2, rev)
2517+ self.assertTrue(rev2.startswith('2-'))
2518
2519=== modified file 'dmedia/tests/test_downloader.py'
2520--- dmedia/tests/test_downloader.py 2011-02-22 14:07:47 +0000
2521+++ dmedia/tests/test_downloader.py 2011-04-11 12:02:24 +0000
2522@@ -175,8 +175,8 @@
2523 tmp = TempDir()
2524 fs = FileStore(tmp.path)
2525 inst = self.klass('', fs, mov_hash, 'mov')
2526- d = tmp.join('transfers')
2527- f = tmp.join('transfers', mov_hash + '.mov')
2528+ d = tmp.join('.dmedia', 'transfers')
2529+ f = tmp.join('.dmedia', 'transfers', mov_hash + '.mov')
2530 self.assertFalse(path.exists(d))
2531 self.assertFalse(path.exists(f))
2532 self.assertEqual(inst.get_tmp(), f)
2533@@ -187,8 +187,8 @@
2534 tmp = TempDir()
2535 fs = FileStore(tmp.path)
2536 inst = self.klass('', fs, mov_hash)
2537- d = tmp.join('transfers')
2538- f = tmp.join('transfers', mov_hash)
2539+ d = tmp.join('.dmedia', 'transfers')
2540+ f = tmp.join('.dmedia', 'transfers', mov_hash)
2541 self.assertFalse(path.exists(d))
2542 self.assertFalse(path.exists(f))
2543 self.assertEqual(inst.get_tmp(), f)
2544@@ -200,10 +200,10 @@
2545 fs = FileStore(tmp.path)
2546 inst = self.klass('', fs, mov_hash, 'mov')
2547
2548- src_d = tmp.join('transfers')
2549- src = tmp.join('transfers', mov_hash + '.mov')
2550- dst_d = tmp.join(mov_hash[:2])
2551- dst = tmp.join(mov_hash[:2], mov_hash[2:] + '.mov')
2552+ src_d = tmp.join('.dmedia', 'transfers')
2553+ src = tmp.join('.dmedia', 'transfers', mov_hash + '.mov')
2554+ dst_d = tmp.join('.dmedia', mov_hash[:2])
2555+ dst = tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov')
2556
2557 # Test when transfers/ dir doesn't exist:
2558 e = raises(IOError, inst.finalize)
2559
2560=== modified file 'dmedia/tests/test_filestore.py'
2561--- dmedia/tests/test_filestore.py 2011-02-27 03:47:55 +0000
2562+++ dmedia/tests/test_filestore.py 2011-04-11 12:02:24 +0000
2563@@ -39,7 +39,7 @@
2564 from dmedia.errors import AmbiguousPath, FileStoreTraversal
2565 from dmedia.errors import DuplicateFile, IntegrityError
2566 from dmedia.filestore import HashList
2567-from dmedia import filestore, constants, schema
2568+from dmedia import filestore, constants
2569 from dmedia.constants import TYPE_ERROR, EXT_PAT, LEAF_SIZE
2570
2571
2572@@ -60,7 +60,6 @@
2573 # Test with normalized absolute path:
2574 self.assertEqual(f('/home/jderose/.dmedia'), '/home/jderose/.dmedia')
2575
2576-
2577 def test_safe_open(self):
2578 f = filestore.safe_open
2579 tmp = TempDir()
2580@@ -501,43 +500,66 @@
2581 self.assertEqual(e.pathname, '/foo/bar/../../root')
2582 self.assertEqual(e.abspath, '/root')
2583
2584- # Test when base is a file
2585+ # Test when parent does not exist
2586+ tmp = TempDir()
2587+ parent = tmp.join('foo')
2588+ e = raises(ValueError, self.klass, parent)
2589+ self.assertEqual(
2590+ str(e),
2591+ 'FileStore.parent not a directory: %r' % parent
2592+ )
2593+
2594+ # Test when parent is a file
2595+ tmp = TempDir()
2596+ parent = tmp.touch('foo')
2597+ e = raises(ValueError, self.klass, parent)
2598+ self.assertEqual(
2599+ str(e),
2600+ 'FileStore.parent not a directory: %r' % parent
2601+ )
2602+
2603+ # Test when parent=None
2604+ inst = self.klass()
2605+ self.assertTrue(path.isdir(inst.parent))
2606+ self.assertTrue(inst.parent.startswith('/tmp/store.'))
2607+ base = path.join(inst.parent, '.dmedia')
2608+ self.assertTrue(path.isdir(base))
2609+
2610+ # Test when base does not exist
2611+ tmp = TempDir()
2612+ base = tmp.join('.dmedia')
2613+ inst = self.klass(tmp.path)
2614+ self.assertEqual(inst.parent, tmp.path)
2615+ self.assertEqual(inst.base, base)
2616+ self.assertTrue(path.isdir(inst.base))
2617+
2618+ # Test when base exists and is a directory
2619+ tmp = TempDir()
2620+ base = tmp.makedirs('.dmedia')
2621+ inst = self.klass(tmp.path)
2622+ self.assertEqual(inst.parent, tmp.path)
2623+ self.assertEqual(inst.base, base)
2624+ self.assertTrue(path.isdir(inst.base))
2625+
2626+ # Test when base exists and is a file
2627 tmp = TempDir()
2628 base = tmp.touch('.dmedia')
2629- e = raises(ValueError, self.klass, base)
2630+ e = raises(ValueError, self.klass, tmp.path)
2631 self.assertEqual(
2632 str(e),
2633 'FileStore.base not a directory: %r' % base
2634 )
2635
2636- # Test when base does not exist
2637+ # Test when base exists and is a symlink to a dir
2638 tmp = TempDir()
2639+ d = tmp.makedirs('foo')
2640 base = tmp.join('.dmedia')
2641- record = tmp.join('.dmedia', 'store.json')
2642- inst = self.klass(base)
2643- self.assertEqual(inst.base, base)
2644- self.assertTrue(path.isdir(inst.base))
2645- self.assertEqual(inst.record, record)
2646- self.assertTrue(path.isfile(record))
2647- store_s = open(record, 'rb').read()
2648- doc = json.loads(store_s)
2649- self.assertEqual(schema.check_dmedia_store(doc), None)
2650- self.assertEqual(inst._doc, doc)
2651- self.assertEqual(inst._id, doc['_id'])
2652-
2653- # Test when base exists and is a directory
2654- inst = self.klass(base)
2655- self.assertEqual(inst.base, base)
2656- self.assertTrue(path.isdir(inst.base))
2657- self.assertEqual(inst.record, record)
2658- self.assertTrue(path.isfile(record))
2659- self.assertEqual(open(record, 'rb').read(), store_s)
2660-
2661- # Test when base=None
2662- inst = self.klass()
2663- self.assertTrue(path.isdir(inst.base))
2664- self.assertTrue(inst.base.startswith('/tmp/store.'))
2665- self.assertEqual(inst.record, path.join(inst.base, 'store.json'))
2666+ os.symlink(d, base)
2667+ e = raises(ValueError, self.klass, tmp.path)
2668+ self.assertEqual(
2669+ str(e),
2670+ '{!r} is symlink to {!r}'.format(base, d)
2671+ )
2672
2673 def test_repr(self):
2674 tmp = TempDir()
2675@@ -551,29 +573,31 @@
2676 # Methods to prevent path traversals attacks
2677 def test_check_path(self):
2678 tmp = TempDir()
2679- base = tmp.join('foo', 'bar')
2680- inst = self.klass(base)
2681+ parent = tmp.makedirs('foo')
2682+ base = tmp.join('foo', '.dmedia')
2683+ inst = self.klass(parent)
2684
2685- bad = tmp.join('foo', 'barNone', 'stuff')
2686+ bad = tmp.join('foo', '.dmedia2', 'stuff')
2687 e = raises(FileStoreTraversal, inst.check_path, bad)
2688 self.assertEqual(e.pathname, bad)
2689 self.assertEqual(e.abspath, bad)
2690 self.assertEqual(e.base, base)
2691
2692- bad = tmp.join('foo', 'bar', '..', 'barNone')
2693+ bad = tmp.join('foo', '.dmedia', '..', '.dmedia2')
2694 assert '..' in bad
2695 e = raises(FileStoreTraversal, inst.check_path, bad)
2696 self.assertEqual(e.pathname, bad)
2697- self.assertEqual(e.abspath, tmp.join('foo', 'barNone'))
2698+ self.assertEqual(e.abspath, tmp.join('foo', '.dmedia2'))
2699 self.assertEqual(e.base, base)
2700
2701- good = tmp.join('foo', 'bar', 'stuff')
2702+ good = tmp.join('foo', '.dmedia', 'stuff')
2703 self.assertEqual(inst.check_path(good), good)
2704
2705 def test_join(self):
2706 tmp = TempDir()
2707- base = tmp.join('foo', 'bar')
2708- inst = self.klass(base)
2709+ parent = tmp.makedirs('foo')
2710+ base = tmp.join('foo', '.dmedia')
2711+ inst = self.klass(parent)
2712
2713 # Test with an absolute path in parts:
2714 e = raises(FileStoreTraversal, inst.join, 'dmedia', '/root')
2715@@ -585,31 +609,32 @@
2716 e = raises(FileStoreTraversal, inst.join, 'NW', '..', '..', '.ssh')
2717 self.assertEqual(
2718 e.pathname,
2719- tmp.join('foo', 'bar', 'NW', '..', '..', '.ssh')
2720+ tmp.join('foo', '.dmedia', 'NW', '..', '..', '.ssh')
2721 )
2722 self.assertEqual(e.abspath, tmp.join('foo', '.ssh'))
2723 self.assertEqual(e.base, base)
2724
2725 # Test for former security issue! See:
2726 # https://bugs.launchpad.net/dmedia/+bug/708663
2727- e = raises(FileStoreTraversal, inst.join, '..', 'barNone', 'stuff')
2728+ e = raises(FileStoreTraversal, inst.join, '..', '.dmedia2', 'stuff')
2729 self.assertEqual(
2730 e.pathname,
2731- tmp.join('foo', 'bar', '..', 'barNone', 'stuff')
2732+ tmp.join('foo', '.dmedia', '..', '.dmedia2', 'stuff')
2733 )
2734- self.assertEqual(e.abspath, tmp.join('foo', 'barNone', 'stuff'))
2735+ self.assertEqual(e.abspath, tmp.join('foo', '.dmedia2', 'stuff'))
2736 self.assertEqual(e.base, base)
2737
2738 # Test with some correct parts:
2739 self.assertEqual(
2740 inst.join('NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW'),
2741- tmp.join('foo', 'bar', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2742+ tmp.join('foo', '.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2743 )
2744
2745 def test_create_parent(self):
2746 tmp = TempDir()
2747 tmp2 = TempDir()
2748 inst = self.klass(tmp.path)
2749+ base = tmp.join('.dmedia')
2750
2751 # Test with a normpath but outside of base:
2752 f = tmp2.join('foo', 'bar')
2753@@ -619,7 +644,7 @@
2754 e = raises(FileStoreTraversal, inst.create_parent, f)
2755 self.assertEqual(e.pathname, f)
2756 self.assertEqual(e.abspath, f)
2757- self.assertEqual(e.base, tmp.path)
2758+ self.assertEqual(e.base, base)
2759 self.assertFalse(path.exists(f))
2760 self.assertFalse(path.exists(d))
2761
2762@@ -632,13 +657,13 @@
2763 e = raises(FileStoreTraversal, inst.create_parent, f)
2764 self.assertEqual(e.pathname, f)
2765 self.assertEqual(e.abspath, tmp2.join('baz', 'f'))
2766- self.assertEqual(e.base, tmp.path)
2767+ self.assertEqual(e.base, base)
2768 self.assertFalse(path.exists(f))
2769 self.assertFalse(path.exists(d))
2770
2771 # Test with some correct parts:
2772- f = tmp.join('NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2773- d = tmp.join('NW')
2774+ f = tmp.join('.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2775+ d = tmp.join('.dmedia', 'NW')
2776 self.assertFalse(path.exists(f))
2777 self.assertFalse(path.exists(d))
2778 self.assertEqual(inst.create_parent(f), d)
2779@@ -649,8 +674,8 @@
2780 self.assertTrue(path.isdir(d))
2781
2782 # Confirm that it's using os.makedirs(), not os.mkdir()
2783- f = tmp.join('OM', 'LU', 'WE', 'IP')
2784- d = tmp.join('OM', 'LU', 'WE')
2785+ f = tmp.join('.dmedia', 'OM', 'LU', 'WE', 'IP')
2786+ d = tmp.join('.dmedia', 'OM', 'LU', 'WE')
2787 self.assertFalse(path.exists(f))
2788 self.assertFalse(path.exists(d))
2789 self.assertEqual(inst.create_parent(f), d)
2790@@ -661,18 +686,19 @@
2791 self.assertTrue(path.isdir(d))
2792
2793 # Test with 1-deep:
2794- f = tmp.join('woot')
2795+ f = tmp.join('.dmedia', 'woot')
2796 self.assertFalse(path.exists(f))
2797- self.assertEqual(inst.create_parent(f), tmp.path)
2798+ self.assertEqual(inst.create_parent(f), base)
2799 self.assertFalse(path.exists(f))
2800
2801 # Test for former security issue! See:
2802 # https://bugs.launchpad.net/dmedia/+bug/708663
2803 tmp = TempDir()
2804- base = tmp.join('foo', 'bar')
2805- bad = tmp.join('foo', 'barNone', 'stuff')
2806- baddir = tmp.join('foo', 'barNone')
2807- inst = self.klass(base)
2808+ parent = tmp.makedirs('foo')
2809+ base = tmp.join('foo', '.dmedia')
2810+ bad = tmp.join('foo', '.dmedia2', 'stuff')
2811+ baddir = tmp.join('foo', '.dmedia2')
2812+ inst = self.klass(parent)
2813 e = raises(FileStoreTraversal, inst.create_parent, bad)
2814 self.assertEqual(e.pathname, bad)
2815 self.assertEqual(e.abspath, bad)
2816@@ -717,16 +743,17 @@
2817
2818 def test_path(self):
2819 tmp = TempDir()
2820- base = tmp.join('foo', 'bar')
2821- inst = self.klass(base)
2822+ parent = tmp.makedirs('foo')
2823+ base = tmp.join('foo', '.dmedia')
2824+ inst = self.klass(parent)
2825
2826 self.assertEqual(
2827 inst.path('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW'),
2828- tmp.join('foo', 'bar', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2829+ tmp.join('foo', '.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2830 )
2831 self.assertEqual(
2832 inst.path('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW', ext='ogv'),
2833- tmp.join('foo', 'bar', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW.ogv')
2834+ tmp.join('foo', '.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW.ogv')
2835 )
2836
2837 # Test to make sure hashes are getting checked with safe_b32():
2838@@ -755,8 +782,8 @@
2839 tmp = TempDir()
2840 inst = self.klass(tmp.path)
2841
2842- f = tmp.join('NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2843- d = tmp.join('NW')
2844+ f = tmp.join('.dmedia', 'NW', 'BNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2845+ d = tmp.join('.dmedia', 'NW')
2846 self.assertFalse(path.exists(f))
2847 self.assertFalse(path.exists(d))
2848 self.assertEqual(
2849@@ -780,7 +807,7 @@
2850 self.assertFalse(inst.exists(mov_hash, 'mov'))
2851
2852 # File exists
2853- tmp.touch(mov_hash[:2], mov_hash[2:] + '.mov')
2854+ tmp.touch('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov')
2855 self.assertTrue(inst.exists(mov_hash, 'mov'))
2856
2857 def test_open(self):
2858@@ -834,7 +861,7 @@
2859 self.assertEqual(e.errno, 2)
2860
2861 # File exists
2862- f = tmp.touch(mov_hash[:2], mov_hash[2:] + '.mov')
2863+ f = tmp.touch('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov')
2864 self.assertTrue(path.isfile(f))
2865 inst.remove(mov_hash, 'mov')
2866 self.assertFalse(path.exists(f))
2867@@ -880,11 +907,13 @@
2868
2869 self.assertEqual(
2870 inst.tmp('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW'),
2871- tmp.join('transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2872+ tmp.join('.dmedia', 'transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2873 )
2874 self.assertEqual(
2875 inst.tmp('NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW', ext='ogv'),
2876- tmp.join('transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW.ogv')
2877+ tmp.join(
2878+ '.dmedia', 'transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW.ogv'
2879+ )
2880 )
2881
2882 # Test to make sure hashes are getting checked with safe_b32():
2883@@ -913,8 +942,8 @@
2884 tmp = TempDir()
2885 inst = self.klass(tmp.path)
2886
2887- f = tmp.join('transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2888- d = tmp.join('transfers')
2889+ f = tmp.join('.dmedia', 'transfers', 'NWBNVXVK5DQGIOW7MYR4K3KA5K22W7NW')
2890+ d = tmp.join('.dmedia', 'transfers')
2891 self.assertFalse(path.exists(f))
2892 self.assertFalse(path.exists(d))
2893 self.assertEqual(
2894@@ -935,7 +964,7 @@
2895
2896 tmp = TempDir()
2897 inst = self.klass(tmp.path)
2898- filename = tmp.join('transfers', chash)
2899+ filename = tmp.join('.dmedia', 'transfers', chash)
2900
2901 # Test when file dosen't yet exist
2902 fp = inst.allocate_for_transfer(2311, chash)
2903@@ -965,7 +994,7 @@
2904
2905 tmp = TempDir()
2906 inst = self.klass(tmp.path)
2907- filename = tmp.join('transfers', chash + '.mov')
2908+ filename = tmp.join('.dmedia', 'transfers', chash + '.mov')
2909
2910 # Test when file dosen't yet exist
2911 fp = inst.allocate_for_transfer(2311, chash, ext='mov')
2912@@ -992,7 +1021,7 @@
2913
2914 def test_allocate_for_import(self):
2915 tmp = TempDir()
2916- imports = tmp.join('imports')
2917+ imports = tmp.join('.dmedia', 'imports')
2918
2919 inst = self.klass(tmp.path)
2920 self.assertFalse(path.isdir(imports))
2921@@ -1020,7 +1049,7 @@
2922
2923 def test_allocate_for_write(self):
2924 tmp = TempDir()
2925- writes = tmp.join('writes')
2926+ writes = tmp.join('.dmedia', 'writes')
2927
2928 inst = self.klass(tmp.path)
2929 self.assertFalse(path.isdir(writes))
2930@@ -1052,7 +1081,7 @@
2931 base = tmp.join('.dmedia')
2932 dst_d = tmp.join('.dmedia', mov_hash[:2])
2933 dst = tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov')
2934- inst = self.klass(base)
2935+ inst = self.klass(tmp.path)
2936
2937 # Test with wrong tmp_fp type
2938 e = raises(TypeError, inst.tmp_move, 17, mov_hash, 'mov')
2939@@ -1157,10 +1186,10 @@
2940 tmp = TempDir()
2941 inst = self.klass(tmp.path)
2942
2943- src_d = tmp.join('transfers')
2944- src = tmp.join('transfers', mov_hash + '.mov')
2945- dst_d = tmp.join(mov_hash[:2])
2946- dst = tmp.join(mov_hash[:2], mov_hash[2:] + '.mov')
2947+ src_d = tmp.join('.dmedia', 'transfers')
2948+ src = tmp.join('.dmedia', 'transfers', mov_hash + '.mov')
2949+ dst_d = tmp.join('.dmedia', mov_hash[:2])
2950+ dst = tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov')
2951
2952 # Test when transfers/ dir doesn't exist:
2953 e = raises(IOError, inst.tmp_verify_move, mov_hash, 'mov')
2954@@ -1235,7 +1264,7 @@
2955 dst = tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov')
2956 shutil.copy(sample_mov, src)
2957
2958- inst = self.klass(base)
2959+ inst = self.klass(tmp.path)
2960 self.assertTrue(path.isfile(src))
2961 self.assertTrue(path.isdir(base))
2962 self.assertFalse(path.exists(dst))
2963
2964=== modified file 'dmedia/tests/test_importer.py'
2965--- dmedia/tests/test_importer.py 2011-04-06 12:35:49 +0000
2966+++ dmedia/tests/test_importer.py 2011-04-11 12:02:24 +0000
2967@@ -40,7 +40,7 @@
2968 from dmedia.filestore import FileStore
2969 from dmedia.schema import random_id
2970 from dmedia import importer, schema
2971-from dmedia.abstractcouch import get_env, get_dmedia_db
2972+from dmedia.abstractcouch import get_db
2973 from .helpers import TempDir, TempHome, raises
2974 from .helpers import DummyQueue, DummyCallback, prep_import_source
2975 from .helpers import sample_mov, sample_thm
2976@@ -203,8 +203,8 @@
2977 self.assertTrue(isinstance(inst.server, couchdb.Server))
2978 self.assertTrue(isinstance(inst.db, couchdb.Database))
2979
2980- self.assertEqual(inst.home, self.home.path)
2981 self.assertTrue(isinstance(inst.filestore, FileStore))
2982+ self.assertEqual(inst.filestore.parent, self.home.path)
2983 self.assertEqual(inst.filestore.base, self.home.join('.dmedia'))
2984
2985 # Test with extract = False
2986@@ -302,7 +302,7 @@
2987 self.assertTrue(inst.doc is None)
2988 _id = inst.start()
2989 self.assertEqual(len(_id), 24)
2990- db = get_dmedia_db(self.env)
2991+ db = get_db(self.env)
2992 self.assertEqual(inst.doc, db[_id])
2993 self.assertEqual(
2994 set(inst.doc),
2995@@ -459,7 +459,7 @@
2996 old['stored'] = {rid: {'copies': 2, 'time': 1234567890}}
2997 inst.db.save(old)
2998 (action, doc) = inst._import_file(src2)
2999- fid = inst.filestore._id
3000+ fid = inst.filestore_id
3001 self.assertEqual(action, 'skipped')
3002 self.assertEqual(set(doc['stored']), set([rid, fid]))
3003 t = doc['stored'][fid]['time']
3004
3005=== modified file 'dmedia/tests/test_transcoder.py'
3006--- dmedia/tests/test_transcoder.py 2011-02-27 13:29:05 +0000
3007+++ dmedia/tests/test_transcoder.py 2011-04-11 12:02:24 +0000
3008@@ -224,13 +224,13 @@
3009 self.assertEqual(inst.src_fp.mode, 'rb')
3010 self.assertEqual(
3011 inst.src_fp.name,
3012- self.tmp.join(mov_hash[:2], mov_hash[2:] + '.mov')
3013+ self.tmp.join('.dmedia', mov_hash[:2], mov_hash[2:] + '.mov')
3014 )
3015
3016 self.assertTrue(isinstance(inst.dst_fp, file))
3017 self.assertEqual(inst.dst_fp.mode, 'r+b')
3018 self.assertTrue(
3019- inst.dst_fp.name.startswith(self.tmp.join('writes'))
3020+ inst.dst_fp.name.startswith(self.tmp.join('.dmedia', 'writes'))
3021 )
3022 self.assertTrue(inst.dst_fp.name.endswith('.ogv'))
3023
3024
3025=== renamed file 'dmedia/tests/test_metastore.py' => 'dmedia/tests/test_views.py'
3026--- dmedia/tests/test_metastore.py 2011-02-20 19:18:43 +0000
3027+++ dmedia/tests/test_views.py 2011-04-11 12:02:24 +0000
3028@@ -2,7 +2,7 @@
3029 # Jason Gerard DeRose <jderose@novacut.com>
3030 #
3031 # dmedia: distributed media library
3032-# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com>
3033+# Copyright (C) 2010, 2011 Jason Gerard DeRose <jderose@novacut.com>
3034 #
3035 # This file is part of `dmedia`.
3036 #
3037@@ -20,150 +20,79 @@
3038 # with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
3039
3040 """
3041-Unit tests for `dmedia.metastore` module.
3042+Unit tests for `dmedia.views` module.
3043 """
3044
3045 from unittest import TestCase
3046-import os
3047-import shutil
3048-import socket
3049-import platform
3050-
3051-import couchdb
3052-
3053-from dmedia import metastore
3054-from .helpers import TempDir, TempHome
3055+
3056+from dmedia.abstractcouch import get_db
3057+from dmedia import views
3058+
3059 from .couch import CouchCase
3060
3061
3062 class test_functions(TestCase):
3063 def test_build_design_doc(self):
3064- f = metastore.build_design_doc
3065- views = (
3066+ f = views.build_design_doc
3067+ views_ = (
3068 ('bytes', 'foo', '_sum'),
3069 ('mtime', 'bar', None),
3070 )
3071- self.assertEqual(f('file', views),
3072- (
3073- '_design/file',
3074- {
3075- '_id': '_design/file',
3076- 'language': 'javascript',
3077- 'views': {
3078- 'bytes': {
3079- 'map': 'foo',
3080- 'reduce': '_sum',
3081- },
3082- 'mtime': {
3083- 'map': 'bar',
3084- },
3085- }
3086+ self.assertEqual(f('file', views_),
3087+ {
3088+ '_id': '_design/file',
3089+ 'language': 'javascript',
3090+ 'views': {
3091+ 'bytes': {
3092+ 'map': 'foo',
3093+ 'reduce': '_sum',
3094+ },
3095+ 'mtime': {
3096+ 'map': 'bar',
3097+ },
3098 }
3099- )
3100- )
3101-
3102- def test_create_machine(self):
3103- f = metastore.create_machine
3104- doc = f()
3105- self.assertTrue(isinstance(doc, dict))
3106- self.assertEqual(
3107- set(doc),
3108- set([
3109- '_id',
3110- 'machine_id',
3111- 'type',
3112- 'time',
3113- 'hostname',
3114- 'distribution',
3115- ])
3116- )
3117- self.assertEqual(doc['type'], 'dmedia/machine')
3118- self.assertEqual(doc['_id'], '_local/machine')
3119- self.assertEqual(doc['hostname'], socket.gethostname())
3120- self.assertEqual(doc['distribution'], platform.linux_distribution())
3121-
3122-
3123-class test_MetaStore(CouchCase):
3124- klass = metastore.MetaStore
3125-
3126- def new(self):
3127- return self.klass(self.env)
3128-
3129- def test_init(self):
3130- inst = self.new()
3131- self.assertEqual(inst.env, self.env)
3132- self.assertTrue(isinstance(inst.server, couchdb.Server))
3133- self.assertTrue(isinstance(inst.db, couchdb.Database))
3134-
3135- def update(self):
3136- inst = self.new()
3137- '_local/app'
3138- inst.update(dict(_id=_id, foo='bar'))
3139- old = inst.db[_id]
3140- inst.update(dict(_id=_id, foo='bar'))
3141- self.assertEqual(inst.db[_id]['_rev'], old['_rev'])
3142- inst.update(dict(_id=_id, foo='baz'))
3143- self.assertNotEqual(inst.db[_id]['_rev'], old['_rev'])
3144-
3145- def test_create_machine(self):
3146- inst = self.new()
3147- self.assertFalse('_local/machine' in inst.db)
3148- _id = inst.create_machine()
3149- self.assertTrue('_local/machine' in inst.db)
3150- self.assertTrue(_id in inst.db)
3151- loc = inst.db['_local/machine']
3152- doc = inst.db[_id]
3153- self.assertEqual(set(loc), set(doc))
3154- self.assertEqual(loc['machine_id'], doc['machine_id'])
3155- self.assertEqual(loc['time'], doc['time'])
3156-
3157- self.assertEqual(inst._machine_id, None)
3158- self.assertEqual(inst.machine_id, _id)
3159- self.assertEqual(inst._machine_id, _id)
3160-
3161- def test_total_bytes(self):
3162- inst = self.new()
3163- self.assertEqual(inst.total_bytes(), 0)
3164- total = 0
3165- for exp in xrange(20, 31):
3166- size = 2 ** exp + 1
3167- total += size
3168- inst.db.create({'bytes': size, 'type': 'dmedia/file'})
3169- self.assertEqual(inst.total_bytes(), total)
3170-
3171- def test_extensions(self):
3172- inst = self.new()
3173- self.assertEqual(list(inst.extensions()), [])
3174- for i in xrange(17):
3175- inst.db.create({'ext': 'mov', 'type': 'dmedia/file'})
3176- inst.db.create({'ext': 'jpg', 'type': 'dmedia/file'})
3177- inst.db.create({'ext': 'cr2', 'type': 'dmedia/file'})
3178- self.assertEqual(
3179- list(inst.extensions()),
3180- [
3181- ('cr2', 17),
3182- ('jpg', 17),
3183- ('mov', 17),
3184- ]
3185- )
3186- for i in xrange(27):
3187- inst.db.create({'ext': 'mov', 'type': 'dmedia/file'})
3188- inst.db.create({'ext': 'jpg', 'type': 'dmedia/file'})
3189- self.assertEqual(
3190- list(inst.extensions()),
3191- [
3192- ('cr2', 17),
3193- ('jpg', 44),
3194- ('mov', 44),
3195- ]
3196- )
3197- for i in xrange(25):
3198- inst.db.create({'ext': 'mov', 'type': 'dmedia/file'})
3199- self.assertEqual(
3200- list(inst.extensions()),
3201- [
3202- ('cr2', 17),
3203- ('jpg', 44),
3204- ('mov', 69),
3205- ]
3206- )
3207+ }
3208+ )
3209+
3210+
3211+class TestCouchFunctions(CouchCase):
3212+ def test_update_design_doc(self):
3213+ f = views.update_design_doc
3214+ db = get_db(self.env)
3215+
3216+ # Test when design doesn't exist:
3217+ doc = views.build_design_doc('user',
3218+ [('video', views.user_video, None)]
3219+ )
3220+ self.assertEqual(f(db, doc), 'new')
3221+ self.assertTrue(db['_design/user']['_rev'].startswith('1-'))
3222+
3223+ # Test when design is same:
3224+ doc = views.build_design_doc('user',
3225+ [('video', views.user_video, None)]
3226+ )
3227+ self.assertEqual(f(db, doc), 'same')
3228+ self.assertTrue(db['_design/user']['_rev'].startswith('1-'))
3229+
3230+ # Test when design is changed:
3231+ doc = views.build_design_doc('user',
3232+ [('video', views.user_audio, None)]
3233+ )
3234+ self.assertEqual(f(db, doc), 'changed')
3235+ self.assertTrue(db['_design/user']['_rev'].startswith('2-'))
3236+
3237+ # Again test when design is same:
3238+ doc = views.build_design_doc('user',
3239+ [('video', views.user_audio, None)]
3240+ )
3241+ self.assertEqual(f(db, doc), 'same')
3242+ self.assertTrue(db['_design/user']['_rev'].startswith('2-'))
3243+
3244+ def test_init_views(self):
3245+ db = get_db(self.env)
3246+ views.init_views(db)
3247+ for (name, views_) in views.designs:
3248+ doc = views.build_design_doc(name, views_)
3249+ saved = db[doc['_id']]
3250+ doc['_rev'] = saved['_rev']
3251+ self.assertEqual(saved, doc)
3252
3253=== renamed file 'dmedia/metastore.py' => 'dmedia/views.py'
3254--- dmedia/metastore.py 2011-04-06 18:27:00 +0000
3255+++ dmedia/views.py 2011-04-11 12:02:24 +0000
3256@@ -2,7 +2,7 @@
3257 # Jason Gerard DeRose <jderose@novacut.com>
3258 #
3259 # dmedia: distributed media library
3260-# Copyright (C) 2010 Jason Gerard DeRose <jderose@novacut.com>
3261+# Copyright (C) 2010, 2011 Jason Gerard DeRose <jderose@novacut.com>
3262 #
3263 # This file is part of `dmedia`.
3264 #
3265@@ -20,19 +20,15 @@
3266 # with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
3267
3268 """
3269-Store meta-data in desktop-couch.
3270+Defines the dmedia CouchDB views.
3271 """
3272
3273-from os import path
3274-import time
3275-import socket
3276-import platform
3277-
3278-import gnomekeyring
3279-from couchdb import ResourceNotFound, ResourceConflict
3280-
3281-from .abstractcouch import get_couchdb_server, get_dmedia_db
3282-from .schema import random_id
3283+import logging
3284+
3285+from couchdb import ResourceNotFound
3286+
3287+
3288+log = logging.getLogger()
3289
3290
3291 _sum = '_sum'
3292@@ -189,156 +185,74 @@
3293 """
3294
3295
3296+designs = (
3297+ ('type', (
3298+ ('type', type_type, _count),
3299+ )),
3300+
3301+ ('batch', (
3302+ ('time', batch_time, None),
3303+ )),
3304+
3305+ ('import', (
3306+ ('time', import_time, None),
3307+ )),
3308+
3309+ ('file', (
3310+ ('stored', file_stored, _sum),
3311+ ('import_id', file_import_id, None),
3312+ ('bytes', file_bytes, _sum),
3313+ ('ext', file_ext, _count),
3314+ ('mime', file_mime, _count),
3315+ ('mtime', file_mtime, None),
3316+ )),
3317+
3318+ ('user', (
3319+ ('copies', user_copies, None),
3320+ ('media', user_media, _count),
3321+ ('tags', user_tags, _count),
3322+ ('all', user_all, None),
3323+ ('video', user_video, None),
3324+ ('image', user_image, None),
3325+ ('audio', user_audio, None),
3326+ )),
3327+)
3328+
3329+
3330+def iter_views(views):
3331+ for (name, map_, reduce_) in views:
3332+ if reduce_ is None:
3333+ yield (name, {'map': map_.strip()})
3334+ else:
3335+ yield (name, {'map': map_.strip(), 'reduce': reduce_.strip()})
3336+
3337+
3338 def build_design_doc(design, views):
3339- _id = '_design/' + design
3340- d = {}
3341- for (view, map_, reduce_) in views:
3342- d[view] = {'map': map_.strip()}
3343- if reduce_ is not None:
3344- d[view]['reduce'] = reduce_.strip()
3345 doc = {
3346- '_id': _id,
3347+ '_id': '_design/' + design,
3348 'language': 'javascript',
3349- 'views': d,
3350- }
3351- return (_id, doc)
3352-
3353-
3354-def create_machine():
3355- # FIXME: this '_local/machine' business probably isn't a very good approach.
3356- # Plus this doesn't directly conform with schema.check_dmedia()
3357- return {
3358- '_id': '_local/machine',
3359- 'machine_id': random_id(),
3360- 'type': 'dmedia/machine',
3361- 'time': time.time(),
3362- 'hostname': socket.gethostname(),
3363- 'distribution': platform.linux_distribution(),
3364- }
3365-
3366-
3367-def docs_equal(doc, old):
3368- if old is None:
3369- return False
3370- doc['_rev'] = old['_rev']
3371- doc_att = doc.get('_attachments', {})
3372- old_att = old.get('_attachments', {})
3373- for key in old_att:
3374- if key in doc_att:
3375- doc_att[key]['revpos'] = old_att[key]['revpos']
3376- return doc == old
3377-
3378-
3379-
3380-class MetaStore(object):
3381- designs = (
3382- ('type', (
3383- ('type', type_type, _count),
3384- )),
3385-
3386- ('batch', (
3387- ('time', batch_time, None),
3388- )),
3389-
3390- ('import', (
3391- ('time', import_time, None),
3392- )),
3393-
3394- ('file', (
3395- ('stored', file_stored, _sum),
3396- ('import_id', file_import_id, None),
3397- ('bytes', file_bytes, _sum),
3398- ('ext', file_ext, _count),
3399- ('mime', file_mime, _count),
3400- ('mtime', file_mtime, None),
3401- )),
3402-
3403- ('user', (
3404- ('copies', user_copies, None),
3405- ('media', user_media, _count),
3406- ('tags', user_tags, _count),
3407- ('all', user_all, None),
3408- ('video', user_video, None),
3409- ('image', user_image, None),
3410- ('audio', user_audio, None),
3411- )),
3412- )
3413-
3414- def __init__(self, env):
3415- self.env = env
3416- self.server = get_couchdb_server(env)
3417- self.db = get_dmedia_db(env, self.server)
3418- self.create_views()
3419- self._machine_id = None
3420-
3421- def get_env(self):
3422- env = dict(self.env)
3423- env['machine_id'] = self.machine_id
3424- return env
3425-
3426- def get_basic_auth(self):
3427- data = gnomekeyring.find_items_sync(
3428- gnomekeyring.ITEM_GENERIC_SECRET,
3429- {'desktopcouch': 'basic'}
3430- )
3431- (user, password) = data[0].secret.split(':')
3432- return (user, password)
3433-
3434- def get_port(self):
3435- return self.env.get('port')
3436-
3437- def get_uri(self):
3438- return 'http://localhost:%s' % self.get_port()
3439-
3440- def get_auth_uri(self):
3441- (user, password) = self.get_basic_auth()
3442- return 'http://%s:%s@localhost:%s' % (
3443- user, password, self.get_port()
3444- )
3445-
3446- def create_machine(self):
3447- try:
3448- loc = self.db['_local/machine']
3449- except ResourceNotFound:
3450- loc = self.sync(create_machine())
3451- doc = dict(loc)
3452- doc['_id'] = doc['machine_id']
3453- try:
3454- self.db[doc['_id']] = doc
3455- except ResourceConflict:
3456- pass
3457- return loc['machine_id']
3458-
3459- @property
3460- def machine_id(self):
3461- if self._machine_id is None:
3462- self._machine_id = self.create_machine()
3463- return self._machine_id
3464-
3465- def update(self, doc):
3466- """
3467- Create *doc* if it doesn't exists, update doc only if different.
3468- """
3469- _id = doc['_id']
3470- old = self.db.get(_id, attachments=True)
3471- if not docs_equal(doc, old):
3472- self.db[_id] = doc
3473-
3474- def sync(self, doc):
3475- _id = doc['_id']
3476- self.db[_id] = doc
3477- return self.db[_id]
3478-
3479- def create_views(self):
3480- for (name, views) in self.designs:
3481- (_id, doc) = build_design_doc(name, views)
3482- self.update(doc)
3483-
3484- def total_bytes(self):
3485- for row in self.db.view('_design/file/_view/bytes'):
3486- return row.value
3487- return 0
3488-
3489- def extensions(self):
3490- for row in self.db.view('_design/file/_view/ext', group=True):
3491- yield (row.key, row.value)
3492+ 'views': dict(iter_views(views)),
3493+ }
3494+ return doc
3495+
3496+
3497+def update_design_doc(db, doc):
3498+ assert '_rev' not in doc
3499+ try:
3500+ old = db[doc['_id']]
3501+ doc['_rev'] = old['_rev']
3502+ if doc != old:
3503+ db.save(doc)
3504+ return 'changed'
3505+ else:
3506+ return 'same'
3507+ except ResourceNotFound:
3508+ db.save(doc)
3509+ return 'new'
3510+
3511+
3512+def init_views(db):
3513+ log.info('Initializing views in %r', db)
3514+ for (name, views) in designs:
3515+ doc = build_design_doc(name, views)
3516+ update_design_doc(db, doc)
3517
3518=== modified file 'dmedia/workers.py'
3519--- dmedia/workers.py 2011-02-20 18:54:27 +0000
3520+++ dmedia/workers.py 2011-04-11 12:02:24 +0000
3521@@ -30,7 +30,7 @@
3522 import logging
3523
3524 from .constants import TYPE_ERROR
3525-from .abstractcouch import get_couchdb_server, get_dmedia_db
3526+from .abstractcouch import get_server, get_db
3527
3528
3529 log = logging.getLogger()
3530@@ -101,7 +101,7 @@
3531 inst = klass(env, q, key, args)
3532 inst.run()
3533 except Exception as e:
3534- log.exception('exception in procces %d, worker=%r', pid)
3535+ log.exception('exception in procces %d, worker=%r', pid, worker)
3536 q.put(dict(
3537 signal='error',
3538 args=(key, exception_name(e), str(e)),
3539@@ -159,8 +159,8 @@
3540 class CouchWorker(Worker):
3541 def __init__(self, env, q, key, args):
3542 super(CouchWorker, self).__init__(env, q, key, args)
3543- self.server = get_couchdb_server(env)
3544- self.db = get_dmedia_db(env, self.server)
3545+ self.server = get_server(env)
3546+ self.db = get_db(env, self.server)
3547
3548
3549 class Manager(object):
3550@@ -272,5 +272,5 @@
3551 class CouchManager(Manager):
3552 def __init__(self, env, callback=None):
3553 super(CouchManager, self).__init__(env, callback)
3554- self.server = get_couchdb_server(env)
3555- self.db = get_dmedia_db(env, self.server)
3556+ self.server = get_server(env)
3557+ self.db = get_db(env, self.server)
3558
3559=== modified file 'setup.py'
3560--- setup.py 2011-03-27 14:12:36 +0000
3561+++ setup.py 2011-04-11 12:02:24 +0000
3562@@ -133,31 +133,54 @@
3563 license='AGPLv3+',
3564 cmdclass={'test': Test},
3565
3566- scripts=['dmedia-cli', 'dmedia-import', 'dmedia-gtk'],
3567- packages=['dmedia', 'dmedia.webui', 'dmedia.gtkui'],
3568- package_data={'dmedia.webui': ['data/*']},
3569-
3570+ scripts=[
3571+ 'dmedia-cli',
3572+ 'dmedia-import',
3573+ 'dmedia-gtk',
3574+ ],
3575+ packages=[
3576+ 'dmedia',
3577+ 'dmedia.service',
3578+ 'dmedia.webui',
3579+ 'dmedia.gtkui',
3580+ ],
3581+ package_data={
3582+ 'dmedia.webui': ['data/*']
3583+ },
3584 data_files=[
3585- ('share/man/man1', ['data/dmedia-cli.1']),
3586- ('share/applications', ['data/dmedia-import.desktop']),
3587- #^ this enables Nautilus to use dmedia-import as a handler for
3588- #media devices such as cameras. `sudo update-desktop-database`
3589- #may need to run for this to show up in the Nautilus
3590- #media handling preferences.
3591- ('share/pixmaps', ['data/dmedia.svg']),
3592+ ('share/man/man1',
3593+ ['share/dmedia-cli.1']
3594+ ),
3595+ ('share/applications',
3596+ ['share/dmedia-import.desktop']
3597+ ),
3598+ ('share/pixmaps',
3599+ ['share/dmedia.svg']
3600+ ),
3601 ('share/pixmaps/dmedia',
3602 [
3603- 'data/indicator-rendermenu.svg',
3604- 'data/indicator-rendermenu-att.svg',
3605- ]
3606- ),
3607- ('share/icons/hicolor/scalable/status/',
3608- [
3609- 'data/indicator-rendermenu.svg',
3610- 'data/indicator-rendermenu-att.svg',
3611- ]
3612- ), #enables status icons to be referenced by icon name
3613- ('share/dbus-1/services', ['data/org.freedesktop.DMedia.service']),
3614- ('lib/dmedia', ['dmedia-service', 'dummy-client']),
3615+ 'share/indicator-rendermenu.svg',
3616+ 'share/indicator-rendermenu-att.svg',
3617+ ]
3618+ ),
3619+ ('share/icons/hicolor/scalable/status',
3620+ [
3621+ 'share/indicator-rendermenu.svg',
3622+ 'share/indicator-rendermenu-att.svg',
3623+ ]
3624+ ),
3625+ ('share/dbus-1/services',
3626+ [
3627+ 'share/org.freedesktop.DMedia.service',
3628+ 'share/org.freedesktop.DMediaImporter.service',
3629+ ]
3630+ ),
3631+ ('lib/dmedia',
3632+ [
3633+ 'dmedia-service',
3634+ 'dmedia-importer-service',
3635+ 'dummy-client',
3636+ ]
3637+ ),
3638 ],
3639 )
3640
3641=== renamed directory 'data' => 'share'
3642=== added file 'share/org.freedesktop.DMedia.service'
3643--- share/org.freedesktop.DMedia.service 1970-01-01 00:00:00 +0000
3644+++ share/org.freedesktop.DMedia.service 2011-04-11 12:02:24 +0000
3645@@ -0,0 +1,3 @@
3646+[D-BUS Service]
3647+Name=org.freedesktop.DMedia
3648+Exec=/usr/lib/dmedia/dmedia-service
3649
3650=== renamed file 'data/org.freedesktop.DMedia.service' => 'share/org.freedesktop.DMediaImporter.service'
3651--- data/org.freedesktop.DMedia.service 2011-02-01 06:00:48 +0000
3652+++ share/org.freedesktop.DMediaImporter.service 2011-04-11 12:02:24 +0000
3653@@ -1,3 +1,3 @@
3654 [D-BUS Service]
3655-Name=org.freedesktop.DMedia
3656-Exec=/usr/lib/dmedia/dmedia-service
3657+Name=org.freedesktop.DMediaImporter
3658+Exec=/usr/lib/dmedia/dmedia-importer-service

Subscribers

People subscribed via source and target branches