Dmedia

Merge lp:~jderose/dmedia/v1-on into lp:dmedia

  1. v1-on
  2. Merge into trunk
Proposed by Jason Gerard DeRose
Status: Merged
Merged at revision: 626
Proposed branch: lp:~jderose/dmedia/v1-on
Merge into: lp:dmedia
Diff against target: 1472 lines (+263/-207)
25 files modified
debian/control (+7/-7)
dmedia-service (+1/-1)
dmedia-v0-v1-upgrade (+76/-0)
dmedia/core.py (+3/-2)
dmedia/gtk/ui/client.html (+2/-2)
dmedia/identity.py (+15/-15)
dmedia/jobs.py (+2/-2)
dmedia/schema.py (+34/-32)
dmedia/server.py (+3/-3)
dmedia/service/avahi.py (+2/-2)
dmedia/service/tests/test_avahi.py (+2/-1)
dmedia/service/tests/test_background.py (+4/-4)
dmedia/startup.py (+14/-8)
dmedia/tests/base.py (+7/-7)
dmedia/tests/test_core.py (+1/-1)
dmedia/tests/test_extractor.py (+2/-2)
dmedia/tests/test_identity.py (+41/-41)
dmedia/tests/test_local.py (+1/-1)
dmedia/tests/test_metastore.py (+3/-3)
dmedia/tests/test_schema.py (+34/-34)
dmedia/tests/test_server.py (+3/-3)
dmedia/tests/test_startup.py (+4/-4)
reference.py (+0/-30)
test-client-server.py (+1/-1)
ui/dmedia.js (+1/-1)
To merge this branch: bzr merge lp:~jderose/dmedia/v1-on
Reviewer Review Type Date Requested Status
xzcvczx (community) Approve
dmedia Dev Pending
Review via email: mp+161339@code.launchpad.net

Description of the change

For background, see this bug:

  https://bugs.launchpad.net/dmedia/+bug/1174035

Changes include:

 * Main Dmedia DB is now 'dmedia-1' instead of 'dmedia-0'

 * Project DBs are now 'dmedia-1-<PROJECT_ID>' instead of 'dmedia-0-<PROJECT_ID>'

 * Switched document ID encoding from Base32 to Dbase32 encoding

 * As there is no way to migrate the SSL certificates to Dbase32 encoding, the user will have to go through the first run dialog and peer their machines again; so that the old machine and user config doesn't conflict, they are now saved in 'machine-1.json' and 'user-1.json' instead of 'machine.json' and 'user.json'

 * Challenge-Response protocol now uses Dbase32 encoding for the secret

 * debian/control: bumped min component versions to >= 13.05

 * Removed outdated 'reference.py' file

Note when you run dmedia-service on a machine where you're already using Dmedia, it will leave your dmedia-0 and dmedia-0-* databases untouched, but there isn't a migration tool yet. That will come shortly. This change will cause no dataloss, but you wont be able to use your library with 13.05 till the migration tool is done.

To post a comment you must log in.
lp:~jderose/dmedia/v1-on updated
640. By Jason Gerard DeRose

Quick doodle on dmedia-v0-v1-upgrade script that so far just builds the v0=>v1 file ID mapping

641. By Jason Gerard DeRose

Tweak to dmedia-v0-v1-upgrade

642. By Jason Gerard DeRose

One more tweak to dmedia-v0-v1-upgrade

Revision history for this message
xzcvczx (xzcvczx) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'debian/control'
2--- debian/control 2013-03-30 02:52:04 +0000
3+++ debian/control 2013-04-28 21:19:24 +0000
4@@ -5,9 +5,9 @@
5 Build-Depends: debhelper (>= 9),
6 python3-all (>= 3.3),
7 python3.3 (>= 3.3.0-11ubuntu2),
8- python3-filestore (>= 13.03),
9- python3-microfiber (>= 13.04),
10- python3-usercouch (>= 13.03),
11+ python3-filestore (>= 13.05),
12+ python3-microfiber (>= 13.05),
13+ python3-usercouch (>= 13.05),
14 Standards-Version: 3.9.3
15 X-Python3-Version: >= 3.3
16 Homepage: https://launchpad.net/dmedia
17@@ -15,10 +15,10 @@
18 Package: dmedia
19 Architecture: all
20 Depends: ${python3:Depends}, ${misc:Depends},
21- python3-filestore (>= 13.03),
22- python3-microfiber (>= 13.04),
23- python3-userwebkit (>= 13.03),
24- python3-usercouch (>= 13.03),
25+ python3-filestore (>= 13.05),
26+ python3-microfiber (>= 13.05),
27+ python3-userwebkit (>= 13.05),
28+ python3-usercouch (>= 13.05),
29 bzr,
30 python3-dbus,
31 python3-gi,
32
33=== modified file 'dmedia-service'
34--- dmedia-service 2013-04-26 04:20:26 +0000
35+++ dmedia-service 2013-04-28 21:19:24 +0000
36@@ -510,7 +510,7 @@
37 if not self.UpdateProject(project_id):
38 self.pending_update = project_id
39 self.Snapshot(schema.project_db_name(project_id))
40- self.Snapshot('dmedia-0')
41+ self.Snapshot(schema.DB_NAME)
42
43 @dbus.service.method(IFACE, in_signature='s', out_signature='')
44 def SetAutoFormat(self, value):
45
46=== added file 'dmedia-v0-v1-upgrade'
47--- dmedia-v0-v1-upgrade 1970-01-01 00:00:00 +0000
48+++ dmedia-v0-v1-upgrade 2013-04-28 21:19:24 +0000
49@@ -0,0 +1,76 @@
50+#!/usr/bin/python3
51+
52+# dmedia: distributed media library
53+# Copyright (C) 2013 Novacut Inc
54+#
55+# This file is part of `dmedia`.
56+#
57+# `dmedia` is free software: you can redistribute it and/or modify it under
58+# the terms of the GNU Affero General Public License as published by the Free
59+# Software Foundation, either version 3 of the License, or (at your option) any
60+# later version.
61+#
62+# `dmedia` is distributed in the hope that it will be useful, but WITHOUT ANY
63+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
64+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
65+# details.
66+#
67+# You should have received a copy of the GNU Affero General Public License along
68+# with `dmedia`. If not, see <http://www.gnu.org/licenses/>.
69+#
70+# Authors:
71+# Jason Gerard DeRose <jderose@novacut.com>
72+
73+import logging
74+
75+from filestore import Migration
76+from microfiber import Database, dmedia_env, encode_attachment, Attachment
77+
78+from dmedia.util import get_db
79+from dmedia.metastore import MetaStore, BufferedSave
80+
81+
82+logging.basicConfig(level=logging.DEBUG)
83+log = logging.getLogger()
84+
85+
86+def create_mapping_doc(v0_id, v1_ch):
87+ v1_leaf_hashes = Attachment('application/octet-stream', v1_ch.leaf_hashes)
88+ return {
89+ '_id': v0_id,
90+ '_attachments': {
91+ 'v1_leaf_hashes': encode_attachment(v1_leaf_hashes),
92+ },
93+ 'bytes': v1_ch.file_size,
94+ 'v1_id': v1_ch.id,
95+ }
96+
97+
98+db = get_db(dmedia_env())
99+ms = MetaStore(db)
100+
101+mdb = db.database('migrate-0-to-1')
102+mdb.ensure()
103+buf = BufferedSave(mdb)
104+
105+
106+try:
107+ local_stores = ms.get_local_stores()
108+ for fs in local_stores:
109+ if not fs.needs_migration:
110+ log.info('FileStore %s at %r does not need migration', fs.id, fs.parentdir)
111+ continue
112+ migration = Migration(fs)
113+ for (v0_id, v1_id, v1_ch) in migration:
114+ if v1_ch is not None:
115+ doc = create_mapping_doc(v0_id, v1_ch)
116+ log.info(v0_id)
117+ buf.save(doc)
118+ buf.flush()
119+except Exception as e:
120+ buf.flush()
121+ raise e
122+
123+
124+print(buf.count, buf.conflicts)
125+
126
127=== modified file 'dmedia/core.py'
128--- dmedia/core.py 2013-04-26 02:43:09 +0000
129+++ dmedia/core.py 2013-04-28 21:19:24 +0000
130@@ -41,7 +41,7 @@
131 from copy import deepcopy
132 from base64 import b64encode
133
134-from dbase32.rfc3548 import isb32
135+from dbase32 import isdb32
136 from microfiber import Server, Database, NotFound, Conflict, BulkConflict, id_slice_iter
137 from filestore import FileStore, check_root_hash, check_id, DOTNAME, FileNotFound
138
139@@ -320,7 +320,7 @@
140
141
142 def is_file_id(_id):
143- return isb32(_id) and len(_id) == 48
144+ return isdb32(_id) and len(_id) == 48
145
146
147 def clean_file_id(_id):
148@@ -401,6 +401,7 @@
149 self.restart_vigilance()
150
151 def _add_filestore(self, fs, doc):
152+ assert isdb32(fs.id)
153 self.stores.add(fs)
154 try:
155 fs.purge_tmp()
156
157=== modified file 'dmedia/gtk/ui/client.html'
158--- dmedia/gtk/ui/client.html 2012-10-24 10:09:24 +0000
159+++ dmedia/gtk/ui/client.html 2013-04-28 21:19:24 +0000
160@@ -8,7 +8,7 @@
161
162 "use strict";
163
164-var B32ALPHABET = '234567ABCDEFGHIJKLMNOPQRSTUVWXYZ';
165+var DB32ALPHABET = '3456789ABCDEFGHIJKLMNOPQRSTUVWXY';
166
167 var UI = {
168 on_load: function() {
169@@ -39,7 +39,7 @@
170 var b32, i;
171 for (i=0; i<orig.length; i++) {
172 b32 = orig[i];
173- if (B32ALPHABET.indexOf(b32) >= 0 ) {
174+ if (DB32ALPHABET.indexOf(b32) >= 0 ) {
175 value += b32;
176 }
177 }
178
179=== modified file 'dmedia/identity.py'
180--- dmedia/identity.py 2013-02-22 01:29:15 +0000
181+++ dmedia/identity.py 2013-04-28 21:19:24 +0000
182@@ -32,8 +32,8 @@
183 from subprocess import check_call, check_output
184 import logging
185
186-from skein import skein512
187-from dbase32.rfc3548 import b32enc, b32dec, random_id
188+from _skein import skein512
189+from dbase32 import db32enc, db32dec, random_id
190
191
192 # Skein personalization strings
193@@ -102,14 +102,14 @@
194 For example:
195
196 >>> hash_pubkey(b'The PEM encoded public key')
197- 'JTHPOXBZKRMAUGDKXDR74ZRVVSKYUMTHZNQUOSUNTQ2IO4UD'
198+ 'CMAIHQ4SDKF3N96DQ6KYVSKOOLDRNFMASGJNHLNGMJTBHVN6'
199
200 """
201 skein = skein512(pubkey_data,
202 digest_bits=240,
203 pers=PERS_PUBKEY,
204 )
205- return b32enc(skein.digest())
206+ return db32enc(skein.digest())
207
208
209 def compute_response(secret, challenge, nonce, challenger_hash, responder_hash):
210@@ -135,7 +135,7 @@
211 )
212 skein.update(challenger_hash)
213 skein.update(responder_hash)
214- return b32enc(skein.digest())
215+ return db32enc(skein.digest())
216
217
218 def compute_csr_mac(secret, csr_data, remote_hash, local_hash):
219@@ -156,7 +156,7 @@
220 key=secret,
221 key_id=(remote_hash + local_hash),
222 )
223- return b32enc(skein.digest())
224+ return db32enc(skein.digest())
225
226
227 def compute_cert_mac(secret, cert_data, remote_hash, local_hash):
228@@ -177,7 +177,7 @@
229 key=secret,
230 key_id=(remote_hash + local_hash),
231 )
232- return b32enc(skein.digest())
233+ return db32enc(skein.digest())
234
235
236 class ChallengeResponse:
237@@ -188,41 +188,41 @@
238 def __init__(self, _id, peer_id):
239 self.id = _id
240 self.peer_id = peer_id
241- self.local_hash = b32dec(_id)
242- self.remote_hash = b32dec(peer_id)
243+ self.local_hash = db32dec(_id)
244+ self.remote_hash = db32dec(peer_id)
245 assert len(self.local_hash) == 30
246 assert len(self.remote_hash) == 30
247
248 def get_secret(self):
249 # 40-bit secret (8 characters when base32 encoded)
250 self.secret = os.urandom(5)
251- return b32enc(self.secret)
252+ return db32enc(self.secret)
253
254 def set_secret(self, secret):
255 assert len(secret) == 8
256- self.secret = b32dec(secret)
257+ self.secret = db32dec(secret)
258 assert len(self.secret) == 5
259
260 def get_challenge(self):
261 self.challenge = os.urandom(20)
262- return b32enc(self.challenge)
263+ return db32enc(self.challenge)
264
265 def create_response(self, challenge):
266 nonce = os.urandom(20)
267 response = compute_response(
268 self.secret,
269- b32dec(challenge),
270+ db32dec(challenge),
271 nonce,
272 self.remote_hash,
273 self.local_hash
274 )
275- return (b32enc(nonce), response)
276+ return (db32enc(nonce), response)
277
278 def check_response(self, nonce, response):
279 expected = compute_response(
280 self.secret,
281 self.challenge,
282- b32dec(nonce),
283+ db32dec(nonce),
284 self.local_hash,
285 self.remote_hash
286 )
287
288=== modified file 'dmedia/jobs.py'
289--- dmedia/jobs.py 2012-08-19 04:01:31 +0000
290+++ dmedia/jobs.py 2013-04-28 21:19:24 +0000
291@@ -44,13 +44,13 @@
292
293 >>> from dmedia.schema import check_job
294 >>> doc = {
295-... '_id': 'H6VVCPDJZ7CSFG4V6EEYCPPD',
296+... '_id': 'H6VVCPDJY7CSFG4V6EEYCPPD',
297 ... 'type': 'dmedia/job',
298 ... 'time': 1234567890,
299 ... 'status': 'waiting',
300 ... 'worker': 'novacut-renderer',
301 ... 'files': [
302-... 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR2BKILJOA3CP7QZW',
303+... 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR3BKILJOA3CP7QYW',
304 ... ],
305 ... 'job': {
306 ... 'Dmedia': 'ignores everything in job',
307
308=== modified file 'dmedia/schema.py'
309--- dmedia/schema.py 2013-02-21 15:00:58 +0000
310+++ dmedia/schema.py 2013-04-28 21:19:24 +0000
311@@ -33,14 +33,14 @@
312 For example:
313
314 >>> good = {
315-... '_id': 'NZXXMYLDOV2F6ZTUO5PWM5DX',
316+... '_id': 'NYXXMYLDOV3F6YTUO5PWM5DX',
317 ... 'type': 'dmedia/foo',
318 ... 'time': 1234567890,
319 ... }
320 ...
321 >>> check_dmedia(good) # Returns None
322 >>> bad = {
323-... '_id': 'NZXXMYLDOV2F6ZTUO5PWM5DX',
324+... '_id': 'NYXXMYLDOV3F6YTUO5PWM5DX',
325 ... 'kind': 'dmedia/foo', # Changed!
326 ... 'time': 1234567890,
327 ... }
328@@ -247,14 +247,13 @@
329 import socket
330 import os
331
332-from dbase32 import RANDOM_B32LEN
333-from dbase32.rfc3548 import isb32, random_id
334+from dbase32 import isdb32, random_id, RANDOM_B32LEN
335 from filestore import DIGEST_B32LEN, TYPE_ERROR
336 from microfiber import encode_attachment, Attachment
337
338
339 # schema-compatibility version:
340-VER = 0
341+VER = 1
342
343 # versioned primary database name:
344 DB_NAME = 'dmedia-{}'.format(VER)
345@@ -263,7 +262,7 @@
346 EXT_PAT = '^[a-z0-9]+(\.[a-z0-9]+)?$'
347
348 # Pattern to match a project DB name
349-PROJECT_DB_PAT = '^dmedia-0-([234567abcdefghijklmnopqrstuvwxyz]{24})$'
350+PROJECT_DB_PAT = '^dmedia-1-([3456789abcdefghijklmnopqrstuvwxy]{24})$'
351
352
353 # Some private helper functions that don't directly define any schema.
354@@ -556,9 +555,9 @@
355 '{}: length of ID ({}) not multiple of 8: {!r}'.format(
356 label, len(value), value)
357 )
358- if not isb32(value):
359+ if not isdb32(value):
360 raise ValueError(
361- '{}: ID not subset of B32ALPHABET: {!r}'.format(
362+ '{}: ID not subset of DB32ALPHABET: {!r}'.format(
363 label, value)
364 )
365
366@@ -572,7 +571,7 @@
367 >>> _random_id('1OTXJHVEXTKNXZHCMHDVF276', "doc['_id']")
368 Traceback (most recent call last):
369 ...
370- ValueError: doc['_id']: random ID not subset of B32ALPHABET: '1OTXJHVEXTKNXZHCMHDVF276'
371+ ValueError: doc['_id']: random ID not subset of DB32ALPHABET: '1OTXJHVEXTKNXZHCMHDVF276'
372
373 """
374 if not isinstance(value, str):
375@@ -584,9 +583,9 @@
376 '{}: random ID must be {} characters, got {}: {!r}'.format(
377 label, RANDOM_B32LEN, len(value), value)
378 )
379- if not isb32(value):
380+ if not isdb32(value):
381 raise ValueError(
382- '{}: random ID not subset of B32ALPHABET: {!r}'.format(
383+ '{}: random ID not subset of DB32ALPHABET: {!r}'.format(
384 label, value)
385 )
386
387@@ -612,9 +611,9 @@
388 '{}: intrinsic ID must be {} characters, got {}: {!r}'.format(
389 label, DIGEST_B32LEN, len(value), value)
390 )
391- if not isb32(value):
392+ if not isdb32(value):
393 raise ValueError(
394- '{}: intrinsic ID not subset of B32ALPHABET: {!r}'.format(
395+ '{}: intrinsic ID not subset of DB32ALPHABET: {!r}'.format(
396 label, value)
397 )
398
399@@ -629,7 +628,7 @@
400 For example, a conforming value:
401
402 >>> doc = {
403- ... '_id': 'NZXXMYLDOV2F6ZTUO5PWM5DX',
404+ ... '_id': 'NYXXMYLDOV3F6YTUO5PWM5DX',
405 ... 'type': 'dmedia/foo',
406 ... 'time': 1234567890,
407 ... }
408@@ -656,7 +655,7 @@
409 For example, a conforming value:
410
411 >>> doc = {
412- ... '_id': 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR2BKILJOA3CP7QZW',
413+ ... '_id': 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR3BKILJOA3CP7QYW',
414 ... '_attachments': {
415 ... 'leaf_hashes': {
416 ... 'data': 'v7t381LIyKsBCUYhkGreXx2qKTyyMfMD2eHWWp/L',
417@@ -794,7 +793,7 @@
418 For example, a conforming value:
419
420 >>> doc = {
421- ... '_id': 'NZXXMYLDOV2F6ZTUO5PWM5DX',
422+ ... '_id': 'NYXXMYLDOV3F6YTUO5PWM5DX',
423 ... 'type': 'dmedia/store',
424 ... 'time': 1234567890,
425 ... 'plugin': 'filestore',
426@@ -964,10 +963,10 @@
427 For example, a conforming value:
428
429 >>> doc = {
430- ... '_id': 'HB6YSCKAY27KIWUTWKGKCTNI',
431+ ... '_id': 'DMODRYHY4ELVOU7333CUI6TI',
432 ... 'type': 'dmedia/project',
433 ... 'time': 1234567890,
434- ... 'db_name': 'dmedia-0-hb6ysckay27kiwutwkgkctni',
435+ ... 'db_name': 'dmedia-1-dmodryhy4elvou7333cui6ti',
436 ... 'title': 'UDS-P',
437 ... 'count': 42,
438 ... 'bytes': 22020096000,
439@@ -1001,11 +1000,12 @@
440
441 For example:
442
443- >>> project_db_name('HB6YSCKAY27KIWUTWKGKCTNI')
444- 'dmedia-0-hb6ysckay27kiwutwkgkctni'
445+ >>> project_db_name('DMODRYHY4ELVOU7333CUI6TI')
446+ 'dmedia-1-dmodryhy4elvou7333cui6ti'
447
448 Also see `get_project_id()`.
449 """
450+ assert isdb32(_id)
451 return '-'.join(['dmedia', str(VER), _id.lower()])
452
453
454@@ -1015,20 +1015,22 @@
455
456 For example:
457
458- >>> get_project_id('dmedia-0-hb6ysckay27kiwutwkgkctni')
459- 'HB6YSCKAY27KIWUTWKGKCTNI'
460+ >>> get_project_id('dmedia-1-dmodryhy4elvou7333cui6ti')
461+ 'DMODRYHY4ELVOU7333CUI6TI'
462
463 If *db_name* doesn't match the expected naming convention, ``None`` is
464 returned:
465
466- >>> get_project_id('dmedia-hb6ysckay27kiwutwkgkctni') is None
467+ >>> get_project_id('dmedia-dmodryhy4elvou7333cui6ti') is None
468 True
469
470 Also see `project_db_name()`.
471 """
472 match = re.match(PROJECT_DB_PAT, db_name)
473 if match:
474- return match.group(1).upper()
475+ _id = match.group(1).upper()
476+ assert isdb32(_id)
477+ return _id
478
479
480 def create_project(title=''):
481@@ -1066,13 +1068,13 @@
482 For example, a conforming value:
483
484 >>> doc = {
485- ... '_id': 'H6VVCPDJZ7CSFG4V6EEYCPPD',
486+ ... '_id': 'H6VVCPDJY7CSFG4V6EEYCPPD',
487 ... 'type': 'dmedia/job',
488 ... 'time': 1234567890,
489 ... 'status': 'waiting',
490 ... 'worker': 'novacut-renderer',
491 ... 'files': [
492- ... 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR2BKILJOA3CP7QZW',
493+ ... 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR3BKILJOA3CP7QYW',
494 ... ],
495 ... 'job': {
496 ... 'Dmedia': 'ignores everything in job',
497@@ -1084,18 +1086,18 @@
498 Once a job is running, it will be updated like this:
499
500 >>> doc = {
501- ... '_id': 'H6VVCPDJZ7CSFG4V6EEYCPPD',
502+ ... '_id': 'H6VVCPDJY7CSFG4V6EEYCPPD',
503 ... 'type': 'dmedia/job',
504 ... 'time': 1234567890,
505 ... 'status': 'executing',
506 ... 'worker': 'novacut-renderer',
507 ... 'files': [
508- ... 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR2BKILJOA3CP7QZW',
509+ ... 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR3BKILJOA3CP7QYW',
510 ... ],
511 ... 'job': {
512 ... 'Dmedia': 'ignores everything in job',
513 ... },
514- ... 'machine_id': '2VQ27USKA4U776JIEP7WQJEH',
515+ ... 'machine_id': '3VQ37USKA4U776JIEP7WQJEH',
516 ... 'time_start': 1240000000,
517 ... }
518 ...
519@@ -1104,18 +1106,18 @@
520 Finally, when a job in finished, it will be updated like this:
521
522 >>> doc = {
523- ... '_id': 'H6VVCPDJZ7CSFG4V6EEYCPPD',
524+ ... '_id': 'H6VVCPDJY7CSFG4V6EEYCPPD',
525 ... 'type': 'dmedia/job',
526 ... 'time': 1234567890,
527 ... 'status': 'completed',
528 ... 'worker': 'novacut-renderer',
529 ... 'files': [
530- ... 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR2BKILJOA3CP7QZW',
531+ ... 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR3BKILJOA3CP7QYW',
532 ... ],
533 ... 'job': {
534 ... 'Dmedia': 'ignores everything in job',
535 ... },
536- ... 'machine_id': '2VQ27USKA4U776JIEP7WQJEH',
537+ ... 'machine_id': '3VQ37USKA4U776JIEP7WQJEH',
538 ... 'time_start': 1240000000,
539 ... 'time_end': 1240000000,
540 ... 'result': {
541
542=== modified file 'dmedia/server.py'
543--- dmedia/server.py 2013-04-25 04:12:24 +0000
544+++ dmedia/server.py 2013-04-28 21:19:24 +0000
545@@ -33,7 +33,7 @@
546
547 from filestore import DIGEST_B32LEN, LEAF_SIZE
548 from microfiber import dumps, basic_auth_header, CouchBase, dumps
549-from dbase32.rfc3548 import isb32
550+from dbase32 import isdb32
551
552 import dmedia
553 from dmedia import __version__
554@@ -72,7 +72,7 @@
555 if len(parts) > 3:
556 raise BadRequest('too many slashes in request path')
557 _id = parts[0]
558- if not (len(_id) == DIGEST_B32LEN and isb32(_id)):
559+ if not (len(_id) == DIGEST_B32LEN and isdb32(_id)):
560 raise BadRequest('badly formed dmedia ID')
561 try:
562 start = (int(parts[1]) if len(parts) > 1 else 0)
563@@ -279,7 +279,7 @@
564 if method not in ('GET', 'HEAD'):
565 raise WSGIError('405 Method Not Allowed')
566 _id = shift_path_info(environ)
567- if not isb32(_id):
568+ if not isdb32(_id):
569 raise WSGIError('400 Bad File ID')
570 if len(_id) != DIGEST_B32LEN:
571 raise WSGIError('400 Bad File ID Length')
572
573=== modified file 'dmedia/service/avahi.py'
574--- dmedia/service/avahi.py 2013-02-22 20:48:37 +0000
575+++ dmedia/service/avahi.py 2013-04-28 21:19:24 +0000
576@@ -60,7 +60,7 @@
577 self.port = port
578 self.ssl_config = ssl_config
579 ctx = Context(env)
580- self.db = Database('dmedia-0', ctx=ctx)
581+ self.db = Database('dmedia-1', ctx=ctx)
582 self.server = Server(ctx=ctx)
583 self.replications = {}
584 try:
585@@ -198,7 +198,7 @@
586
587 def get_names(self):
588 for name in self.server.get('_all_dbs'):
589- if name.startswith('dmedia-0') or name.startswith('novacut-0'):
590+ if name.startswith('dmedia-1') or name.startswith('novacut-1'):
591 yield name
592
593 def replication_worker(self, cancel, start):
594
595=== modified file 'dmedia/service/tests/test_avahi.py'
596--- dmedia/service/tests/test_avahi.py 2012-10-16 02:48:19 +0000
597+++ dmedia/service/tests/test_avahi.py 2013-04-28 21:19:24 +0000
598@@ -30,6 +30,7 @@
599
600 from dmedia.tests.couch import CouchCase
601 from dmedia.service import avahi
602+from dmedia.util import get_db
603
604
605 random = SystemRandom()
606@@ -41,7 +42,7 @@
607
608 class TestAvahi(CouchCase):
609 def test_init(self):
610- db = microfiber.Database('dmedia-0', self.env)
611+ db = get_db(self.env)
612 self.assertTrue(db.ensure())
613 port = random_port()
614 ssl_config = 'the SSL config'
615
616=== modified file 'dmedia/service/tests/test_background.py'
617--- dmedia/service/tests/test_background.py 2013-04-21 19:47:37 +0000
618+++ dmedia/service/tests/test_background.py 2013-04-28 21:19:24 +0000
619@@ -30,7 +30,7 @@
620
621 from microfiber import Database
622 from usercouch.misc import TempCouch
623-from dbase32.rfc3548 import random_id
624+from dbase32 import random_id
625
626 from dmedia.tests.base import TempDir, random_file_id
627 from dmedia.tests.couch import CouchCase
628@@ -73,7 +73,7 @@
629
630 class TestLazyAccess(TestCase):
631 def test_init(self):
632- db = Database('dmedia-0')
633+ db = Database('dmedia-1')
634 inst = background.LazyAccess(db)
635 self.assertIs(inst.db, db)
636 self.assertIsInstance(inst.delay, int)
637@@ -89,7 +89,7 @@
638 self.assertIsNone(inst.timeout_id)
639
640 def test_access(self):
641- db = Database('dmedia-0')
642+ db = Database('dmedia-1')
643 inst = background.LazyAccess(db)
644
645 # So it's easier to test without requiring a mainloop:
646@@ -130,7 +130,7 @@
647
648 def test_flush(self):
649 couch = TempCouch()
650- db = Database('dmedia-0', couch.bootstrap())
651+ db = Database('dmedia-1', couch.bootstrap())
652 db.ensure()
653 inst = background.LazyAccess(db)
654
655
656=== modified file 'dmedia/startup.py'
657--- dmedia/startup.py 2012-10-24 23:23:30 +0000
658+++ dmedia/startup.py 2013-04-28 21:19:24 +0000
659@@ -37,6 +37,10 @@
660 from .identity import PKI
661
662
663+MACHINE = 'machine-1'
664+USER = 'user-1'
665+
666+
667 log = logging.getLogger()
668
669
670@@ -56,6 +60,8 @@
671 separators=(',',': '),
672 indent=4,
673 )
674+ #fp.flush()
675+ #os.fsync(fp.fileno())
676 fp.close()
677 os.rename(tmp, filename)
678
679@@ -84,8 +90,8 @@
680 def __init__(self, basedir):
681 super().__init__(basedir)
682 self.pki = PKI(self.paths.ssl)
683- self.machine = self.load_config('machine')
684- self.user = self.load_config('user')
685+ self.machine = self.load_config(MACHINE)
686+ self.user = self.load_config(USER)
687 self.mthread = None
688
689 def load_config(self, name):
690@@ -105,8 +111,8 @@
691 log.info('... machine_id: %s', machine_id)
692 self.pki.create_ca(machine_id)
693 doc = create_doc(machine_id, 'dmedia/machine')
694- self.save_config('machine', doc)
695- self.machine = self.load_config('machine')
696+ self.save_config(MACHINE, doc)
697+ self.machine = self.load_config(MACHINE)
698 return machine_id
699
700 def create_machine_if_needed(self):
701@@ -140,15 +146,15 @@
702 self.pki.create_csr(self.machine['_id'])
703 self.pki.issue_cert(self.machine['_id'], user_id)
704 doc = create_doc(user_id, 'dmedia/user')
705- self.save_config('user', doc)
706- self.user = self.load_config('user')
707+ self.save_config(USER, doc)
708+ self.user = self.load_config(USER)
709 return user_id
710
711 def set_user(self, user_id):
712 log.info('... user_id: %s', user_id)
713 doc = create_doc(user_id, 'dmedia/user')
714- self.save_config('user', doc)
715- self.user = self.load_config('user')
716+ self.save_config(USER, doc)
717+ self.user = self.load_config(USER)
718
719 def load_pki(self):
720 if self.user is None:
721
722=== modified file 'dmedia/tests/base.py'
723--- dmedia/tests/base.py 2013-04-13 03:36:33 +0000
724+++ dmedia/tests/base.py 2013-04-28 21:19:24 +0000
725@@ -36,7 +36,7 @@
726 import filestore
727 from filestore import File, Leaf, ContentHash, Batch, Hasher, LEAF_SIZE
728 from filestore import scandir
729-from dbase32.rfc3548 import random_id
730+from dbase32 import random_id
731
732
733 MAX_SIZE = LEAF_SIZE * 3
734@@ -68,18 +68,18 @@
735 mov = path.join(datadir, 'MVI_5751.MOV')
736 thm = path.join(datadir, 'MVI_5751.THM')
737 mov_ch = ContentHash(
738- 'SM3GS4DUDVXOEU2DTTTWU5HKNRK777IWNSI5UQ4ZWNQGRXAN',
739+ '4UHYMLYR8SD7XV8HJ3M9959R83OF4KIAFDBTLLT4YV9ALB5N',
740 20202333,
741 b64decode(b''.join([
742- b'Ps9ZlZ5RALOGrqUbXJYJDFJaLClkGKkv4gYu2cWn',
743- b'dse+QPUQFn9Q6FBkhhX0hjDGHOyMnFtGdAgRY1Gc',
744- b'XzjjVS002vjsMkVKb4/+E7qmeGfHsBFFbYV127ux'
745+ b'uD18L29E3bVC+mxTj3cTRCpsq27DF4stuA5YsCxC',
746+ b'mtL4/ynnXHIcvPw5fuwpZJWZUMrY9LhKlc7Slfx6',
747+ b'JO6hEEGmxtKmxE0U7Nbi4r+o35Ao80KzPurTXsto',
748 ]))
749 )
750 thm_ch = ContentHash(
751- 'MXPCFNUNPDAWHQWC5QNTPP2U5OF2J267QQVALXX6B5TRJKJB',
752+ 'PW7537TPDOR78CYVL4NIASYTUVTEXJN5RUKYV5N7QNMLNBCT',
753 27328,
754- b64decode(b'RwtCvXTjDrah3O23qNkobCGNF6hq7HYIB4TRx2Dh'),
755+ b64decode(b'zz8pyu1XxbfRwkGdkayShOA/hTCPZ85pRkSy+j30'),
756 )
757
758 def setUp(self):
759
760=== modified file 'dmedia/tests/test_core.py'
761--- dmedia/tests/test_core.py 2013-04-22 06:15:43 +0000
762+++ dmedia/tests/test_core.py 2013-04-28 21:19:24 +0000
763@@ -34,7 +34,7 @@
764 import multiprocessing
765
766 import microfiber
767-from dbase32.rfc3548 import random_id
768+from dbase32 import random_id
769 import filestore
770
771 from dmedia.local import LocalStores
772
773=== modified file 'dmedia/tests/test_extractor.py'
774--- dmedia/tests/test_extractor.py 2013-02-22 05:27:39 +0000
775+++ dmedia/tests/test_extractor.py 2013-04-28 21:19:24 +0000
776@@ -544,7 +544,7 @@
777 'ctime': 1287520994.68,
778 'meta': {
779 'bar': value2,
780- 'canon_thm': 'MXPCFNUNPDAWHQWC5QNTPP2U5OF2J267QQVALXX6B5TRJKJB',
781+ 'canon_thm': 'PW7537TPDOR78CYVL4NIASYTUVTEXJN5RUKYV5N7QNMLNBCT',
782 'aperture': 11.0,
783 'shutter': '1/100',
784 'iso': 100,
785@@ -591,7 +591,7 @@
786 'ctime': 1287520994.68,
787 'meta': {
788 'bar': value2,
789- 'canon_thm': 'MXPCFNUNPDAWHQWC5QNTPP2U5OF2J267QQVALXX6B5TRJKJB',
790+ 'canon_thm': 'PW7537TPDOR78CYVL4NIASYTUVTEXJN5RUKYV5N7QNMLNBCT',
791 'aperture': 11.0,
792 'shutter': '1/100',
793 'iso': 100,
794
795=== modified file 'dmedia/tests/test_identity.py'
796--- dmedia/tests/test_identity.py 2013-02-22 01:29:15 +0000
797+++ dmedia/tests/test_identity.py 2013-04-28 21:19:24 +0000
798@@ -29,7 +29,7 @@
799 import subprocess
800 import socket
801
802-from dbase32.rfc3548 import b32enc, b32dec, random_id
803+from dbase32 import db32enc, db32dec, random_id
804 from skein import skein512
805
806 from .base import TempDir
807@@ -47,7 +47,7 @@
808 pers=b'20120918 jderose@novacut.com dmedia/pubkey',
809 )
810 self.assertEqual(
811- b32dec(_id),
812+ db32dec(_id),
813 skein.digest()
814 )
815
816@@ -75,7 +75,7 @@
817 nonce=(challenge + nonce),
818 )
819 self.assertEqual(
820- b32dec(response),
821+ db32dec(response),
822 skein.digest()
823 )
824
825@@ -129,7 +129,7 @@
826 key_id=(remote_hash + local_hash),
827 )
828 self.assertEqual(
829- b32dec(mac),
830+ db32dec(mac),
831 skein.digest()
832 )
833
834@@ -177,7 +177,7 @@
835 key_id=(remote_hash + local_hash),
836 )
837 self.assertEqual(
838- b32dec(mac),
839+ db32dec(mac),
840 skein.digest()
841 )
842
843@@ -218,8 +218,8 @@
844 inst = identity.ChallengeResponse(id1, id2)
845 self.assertIs(inst.id, id1)
846 self.assertIs(inst.peer_id, id2)
847- self.assertEqual(inst.local_hash, identity.b32dec(id1))
848- self.assertEqual(inst.remote_hash, identity.b32dec(id2))
849+ self.assertEqual(inst.local_hash, identity.db32dec(id1))
850+ self.assertEqual(inst.remote_hash, identity.db32dec(id2))
851
852 def test_get_secret(self):
853 id1 = random_id(30)
854@@ -228,12 +228,12 @@
855 s1 = inst.get_secret()
856 self.assertIsInstance(s1, str)
857 self.assertEqual(len(s1), 8)
858- self.assertEqual(identity.b32dec(s1), inst.secret)
859+ self.assertEqual(identity.db32dec(s1), inst.secret)
860 s2 = inst.get_secret()
861 self.assertNotEqual(s1, s2)
862 self.assertIsInstance(s2, str)
863 self.assertEqual(len(s2), 8)
864- self.assertEqual(identity.b32dec(s2), inst.secret)
865+ self.assertEqual(identity.db32dec(s2), inst.secret)
866
867 def test_set_secret(self):
868 id1 = random_id(30)
869@@ -241,10 +241,10 @@
870 inst = identity.ChallengeResponse(id1, id2)
871 s1 = random_id(5)
872 self.assertIsNone(inst.set_secret(s1))
873- self.assertEqual(identity.b32enc(inst.secret), s1)
874+ self.assertEqual(identity.db32enc(inst.secret), s1)
875 s2 = random_id(5)
876 self.assertIsNone(inst.set_secret(s2))
877- self.assertEqual(identity.b32enc(inst.secret), s2)
878+ self.assertEqual(identity.db32enc(inst.secret), s2)
879
880 def test_get_challenge(self):
881 id1 = random_id(30)
882@@ -253,19 +253,19 @@
883 c1 = inst.get_challenge()
884 self.assertIsInstance(c1, str)
885 self.assertEqual(len(c1), 32)
886- self.assertEqual(identity.b32dec(c1), inst.challenge)
887+ self.assertEqual(identity.db32dec(c1), inst.challenge)
888 c2 = inst.get_challenge()
889 self.assertNotEqual(c1, c2)
890 self.assertIsInstance(c2, str)
891 self.assertEqual(len(c2), 32)
892- self.assertEqual(identity.b32dec(c2), inst.challenge)
893+ self.assertEqual(identity.db32dec(c2), inst.challenge)
894
895 def test_create_response(self):
896 id1 = random_id(30)
897 id2 = random_id(30)
898 inst = identity.ChallengeResponse(id1, id2)
899- local_hash = b32dec(id1)
900- remote_hash = b32dec(id2)
901+ local_hash = db32dec(id1)
902+ remote_hash = db32dec(id2)
903 secret1 = random_id(5)
904 challenge1 = random_id(20)
905 inst.set_secret(secret1)
906@@ -276,7 +276,7 @@
907 self.assertEqual(len(response1), 56)
908 self.assertEqual(response1,
909 identity.compute_response(
910- b32dec(secret1), b32dec(challenge1), b32dec(nonce1),
911+ db32dec(secret1), db32dec(challenge1), db32dec(nonce1),
912 remote_hash, local_hash
913 )
914 )
915@@ -291,7 +291,7 @@
916 self.assertEqual(len(response2), 56)
917 self.assertEqual(response2,
918 identity.compute_response(
919- b32dec(secret1), b32dec(challenge1), b32dec(nonce2),
920+ db32dec(secret1), db32dec(challenge1), db32dec(nonce2),
921 remote_hash, local_hash
922 )
923 )
924@@ -310,7 +310,7 @@
925 self.assertEqual(len(response3), 56)
926 self.assertEqual(response3,
927 identity.compute_response(
928- b32dec(secret2), b32dec(challenge1), b32dec(nonce3),
929+ db32dec(secret2), db32dec(challenge1), db32dec(nonce3),
930 remote_hash, local_hash
931 )
932 )
933@@ -330,7 +330,7 @@
934 self.assertEqual(len(response4), 56)
935 self.assertEqual(response4,
936 identity.compute_response(
937- b32dec(secret2), b32dec(challenge2), b32dec(nonce4),
938+ db32dec(secret2), db32dec(challenge2), db32dec(nonce4),
939 remote_hash, local_hash
940 )
941 )
942@@ -339,20 +339,20 @@
943 id1 = random_id(30)
944 id2 = random_id(30)
945 inst = identity.ChallengeResponse(id1, id2)
946- local_hash = b32dec(id1)
947- remote_hash = b32dec(id2)
948+ local_hash = db32dec(id1)
949+ remote_hash = db32dec(id2)
950 secret = inst.get_secret()
951 challenge = inst.get_challenge()
952 nonce = random_id(20)
953 response = identity.compute_response(
954- b32dec(secret), b32dec(challenge), b32dec(nonce),
955+ db32dec(secret), db32dec(challenge), db32dec(nonce),
956 local_hash, remote_hash
957 )
958 self.assertIsNone(inst.check_response(nonce, response))
959
960 # Test with (local, remote) order flipped
961 bad = identity.compute_response(
962- b32dec(secret), b32dec(challenge), b32dec(nonce),
963+ db32dec(secret), db32dec(challenge), db32dec(nonce),
964 remote_hash, local_hash
965 )
966 with self.assertRaises(identity.WrongResponse) as cm:
967@@ -361,13 +361,13 @@
968 self.assertEqual(cm.exception.got, bad)
969 self.assertFalse(hasattr(inst, 'secret'))
970 self.assertFalse(hasattr(inst, 'challenge'))
971- inst.secret = b32dec(secret)
972- inst.challenge = b32dec(challenge)
973+ inst.secret = db32dec(secret)
974+ inst.challenge = db32dec(challenge)
975
976 # Test with wrong secret
977 for i in range(100):
978 bad = identity.compute_response(
979- os.urandom(5), b32dec(challenge), b32dec(nonce),
980+ os.urandom(5), db32dec(challenge), db32dec(nonce),
981 local_hash, remote_hash
982 )
983 with self.assertRaises(identity.WrongResponse) as cm:
984@@ -376,13 +376,13 @@
985 self.assertEqual(cm.exception.got, bad)
986 self.assertFalse(hasattr(inst, 'secret'))
987 self.assertFalse(hasattr(inst, 'challenge'))
988- inst.secret = b32dec(secret)
989- inst.challenge = b32dec(challenge)
990+ inst.secret = db32dec(secret)
991+ inst.challenge = db32dec(challenge)
992
993 # Test with wrong challenge
994 for i in range(100):
995 bad = identity.compute_response(
996- b32dec(secret), os.urandom(20), b32dec(nonce),
997+ db32dec(secret), os.urandom(20), db32dec(nonce),
998 local_hash, remote_hash
999 )
1000 with self.assertRaises(identity.WrongResponse) as cm:
1001@@ -391,13 +391,13 @@
1002 self.assertEqual(cm.exception.got, bad)
1003 self.assertFalse(hasattr(inst, 'secret'))
1004 self.assertFalse(hasattr(inst, 'challenge'))
1005- inst.secret = b32dec(secret)
1006- inst.challenge = b32dec(challenge)
1007+ inst.secret = db32dec(secret)
1008+ inst.challenge = db32dec(challenge)
1009
1010 # Test with wrong nonce
1011 for i in range(100):
1012 bad = identity.compute_response(
1013- b32dec(secret), b32dec(challenge), os.urandom(20),
1014+ db32dec(secret), db32dec(challenge), os.urandom(20),
1015 local_hash, remote_hash
1016 )
1017 with self.assertRaises(identity.WrongResponse) as cm:
1018@@ -406,13 +406,13 @@
1019 self.assertEqual(cm.exception.got, bad)
1020 self.assertFalse(hasattr(inst, 'secret'))
1021 self.assertFalse(hasattr(inst, 'challenge'))
1022- inst.secret = b32dec(secret)
1023- inst.challenge = b32dec(challenge)
1024+ inst.secret = db32dec(secret)
1025+ inst.challenge = db32dec(challenge)
1026
1027 # Test with wrong local_hash
1028 for i in range(100):
1029 bad = identity.compute_response(
1030- b32dec(secret), b32dec(challenge), b32dec(nonce),
1031+ db32dec(secret), db32dec(challenge), db32dec(nonce),
1032 os.urandom(30), remote_hash
1033 )
1034 with self.assertRaises(identity.WrongResponse) as cm:
1035@@ -421,13 +421,13 @@
1036 self.assertEqual(cm.exception.got, bad)
1037 self.assertFalse(hasattr(inst, 'secret'))
1038 self.assertFalse(hasattr(inst, 'challenge'))
1039- inst.secret = b32dec(secret)
1040- inst.challenge = b32dec(challenge)
1041+ inst.secret = db32dec(secret)
1042+ inst.challenge = db32dec(challenge)
1043
1044 # Test with wrong remote_hash
1045 for i in range(100):
1046 bad = identity.compute_response(
1047- b32dec(secret), b32dec(challenge), b32dec(nonce),
1048+ db32dec(secret), db32dec(challenge), db32dec(nonce),
1049 local_hash, os.urandom(30)
1050 )
1051 with self.assertRaises(identity.WrongResponse) as cm:
1052@@ -436,14 +436,14 @@
1053 self.assertEqual(cm.exception.got, bad)
1054 self.assertFalse(hasattr(inst, 'secret'))
1055 self.assertFalse(hasattr(inst, 'challenge'))
1056- inst.secret = b32dec(secret)
1057- inst.challenge = b32dec(challenge)
1058+ inst.secret = db32dec(secret)
1059+ inst.challenge = db32dec(challenge)
1060
1061 # Test with more nonce, used as expected:
1062 for i in range(100):
1063 newnonce = random_id(20)
1064 good = identity.compute_response(
1065- b32dec(secret), b32dec(challenge), b32dec(newnonce),
1066+ db32dec(secret), db32dec(challenge), db32dec(newnonce),
1067 local_hash, remote_hash
1068 )
1069 self.assertNotEqual(good, response)
1070
1071=== modified file 'dmedia/tests/test_local.py'
1072--- dmedia/tests/test_local.py 2013-04-13 03:36:33 +0000
1073+++ dmedia/tests/test_local.py 2013-04-28 21:19:24 +0000
1074@@ -29,7 +29,7 @@
1075
1076 import filestore
1077 from filestore import FileStore, DIGEST_B32LEN, DIGEST_BYTES
1078-from dbase32.rfc3548 import random_id
1079+from dbase32 import random_id
1080
1081 from .base import TempDir
1082 from .couch import CouchCase
1083
1084=== modified file 'dmedia/tests/test_metastore.py'
1085--- dmedia/tests/test_metastore.py 2013-04-22 01:23:52 +0000
1086+++ dmedia/tests/test_metastore.py 2013-04-28 21:19:24 +0000
1087@@ -33,7 +33,7 @@
1088
1089 from filestore import FileStore, DIGEST_BYTES
1090 from filestore.misc import TempFileStore
1091-from dbase32.rfc3548 import random_id
1092+from dbase32 import random_id
1093 import microfiber
1094 from microfiber import dumps, Conflict
1095
1096@@ -2232,7 +2232,7 @@
1097 self.assertEqual(fs1.import_file(open(file.name, 'rb')), ch)
1098 self.assertEqual(fs2.import_file(open(file.name, 'rb')), ch)
1099
1100- # Test when file doc isn't in dmedia-0
1101+ # Test when file doc isn't in dmedia-1
1102 with self.assertRaises(microfiber.NotFound) as cm:
1103 ms.remove(fs1, ch.id)
1104 with self.assertRaises(microfiber.NotFound) as cm:
1105@@ -2279,7 +2279,7 @@
1106 fs = util.init_filestore(tmp.dir)[0]
1107 (file, ch) = tmp.random_file()
1108
1109- # Test when file doc isn't in dmedia-0
1110+ # Test when file doc isn't in dmedia-1
1111 with self.assertRaises(microfiber.NotFound) as cm:
1112 ms.remove(fs, ch.id)
1113
1114
1115=== modified file 'dmedia/tests/test_schema.py'
1116--- dmedia/tests/test_schema.py 2013-04-13 03:36:33 +0000
1117+++ dmedia/tests/test_schema.py 2013-04-28 21:19:24 +0000
1118@@ -31,7 +31,7 @@
1119 import time
1120
1121 from filestore import TYPE_ERROR, DIGEST_BYTES, ContentHash
1122-from dbase32.rfc3548 import random_id
1123+from dbase32 import random_id
1124
1125 from .base import TempDir
1126
1127@@ -43,7 +43,7 @@
1128 f = schema.check_dmedia
1129
1130 bad = [
1131- ('_id', 'MZZG2ZDSOQVSW2TEMVZG643F'),
1132+ ('_id', 'MYYG3YDSOQVSW3TEMVYG643F'),
1133 ('type', 'dmedia/foo'),
1134 ('time', 1234567890),
1135 ]
1136@@ -55,7 +55,7 @@
1137 )
1138
1139 good = {
1140- '_id': 'MZZG2ZDSOQVSW2TEMVZG643F',
1141+ '_id': 'MYYG3YDSOQVSW3TEMVYG643F',
1142 'type': 'dmedia/foo',
1143 'time': 1234567890,
1144 'foo': 'bar',
1145@@ -80,7 +80,7 @@
1146 f(bad)
1147 self.assertEqual(
1148 str(cm.exception),
1149- "doc['_id']: ID not subset of B32ALPHABET: 'MZZG2ZDS0QVSW2TEMVZG643F'"
1150+ "doc['_id']: ID not subset of DB32ALPHABET: 'MZZG2ZDS0QVSW2TEMVZG643F'"
1151 )
1152
1153 # Check with bad "_id" length:
1154@@ -147,7 +147,7 @@
1155 def test_check_file(self):
1156 # Test with good doc:
1157 good = {
1158- '_id': 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR2BKILJOA3CP7QZW',
1159+ '_id': 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR3BKILJOA3CP7QYW',
1160 '_attachments': {
1161 'leaf_hashes': {
1162 'data': 'v7t381LIyKsBCUYhkGreXx2qKTyyMfMD2eHWWp/L',
1163@@ -160,7 +160,7 @@
1164 'bytes': 20202333,
1165 'origin': 'user',
1166 'stored': {
1167- 'MZZG2ZDSOQVSW2TEMVZG643F': {
1168+ 'MYYG3YDSOQVSW3TEMVYG643F': {
1169 'copies': 2,
1170 'mtime': 1234567890,
1171 },
1172@@ -316,110 +316,110 @@
1173
1174 # Missing "copies":
1175 bad = deepcopy(good)
1176- del bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['copies']
1177+ del bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['copies']
1178 with self.assertRaises(ValueError) as cm:
1179 schema.check_file(bad)
1180 self.assertEqual(
1181 str(cm.exception),
1182- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['copies'] does not exist"
1183+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['copies'] does not exist"
1184 )
1185
1186 # Bad "copies" type:
1187 bad = deepcopy(good)
1188- bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['copies'] = 1.5
1189+ bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['copies'] = 1.5
1190 with self.assertRaises(TypeError) as cm:
1191 schema.check_file(bad)
1192 self.assertEqual(
1193 str(cm.exception),
1194 TYPE_ERROR.format(
1195- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['copies']", int, float, 1.5
1196+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['copies']", int, float, 1.5
1197 )
1198 )
1199
1200 # Bad "copies" value:
1201 bad = deepcopy(good)
1202- bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['copies'] = -1
1203+ bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['copies'] = -1
1204 with self.assertRaises(ValueError) as cm:
1205 schema.check_file(bad)
1206 self.assertEqual(
1207 str(cm.exception),
1208- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['copies'] must be >= 0; got -1"
1209+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['copies'] must be >= 0; got -1"
1210 )
1211
1212 # Missing "mtime":
1213 bad = deepcopy(good)
1214- del bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['mtime']
1215+ del bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['mtime']
1216 with self.assertRaises(ValueError) as cm:
1217 schema.check_file(bad)
1218 self.assertEqual(
1219 str(cm.exception),
1220- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['mtime'] does not exist"
1221+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['mtime'] does not exist"
1222 )
1223
1224 # Bad "mtime" type:
1225 bad = deepcopy(good)
1226- bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['mtime'] = 123.45
1227+ bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['mtime'] = 123.45
1228 with self.assertRaises(TypeError) as cm:
1229 schema.check_file(bad)
1230 self.assertEqual(
1231 str(cm.exception),
1232 TYPE_ERROR.format(
1233- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['mtime']", int, float, 123.45
1234+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['mtime']", int, float, 123.45
1235 )
1236 )
1237
1238 # Bad "mtime" value:
1239 bad = deepcopy(good)
1240- bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['mtime'] = -1
1241+ bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['mtime'] = -1
1242 with self.assertRaises(ValueError) as cm:
1243 schema.check_file(bad)
1244 self.assertEqual(
1245 str(cm.exception),
1246- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['mtime'] must be >= 0; got -1"
1247+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['mtime'] must be >= 0; got -1"
1248 )
1249
1250 # Bad "verified" type:
1251 bad = deepcopy(good)
1252- bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['verified'] = 1234.5
1253+ bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['verified'] = 1234.5
1254 with self.assertRaises(TypeError) as cm:
1255 schema.check_file(bad)
1256 self.assertEqual(
1257 str(cm.exception),
1258 TYPE_ERROR.format(
1259- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['verified']", int, float, 1234.5
1260+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['verified']", int, float, 1234.5
1261 )
1262 )
1263
1264 # Bad "verified" value:
1265 bad = deepcopy(good)
1266- bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['verified'] = -1
1267+ bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['verified'] = -1
1268 with self.assertRaises(ValueError) as cm:
1269 schema.check_file(bad)
1270 self.assertEqual(
1271 str(cm.exception),
1272- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['verified'] must be >= 0; got -1"
1273+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['verified'] must be >= 0; got -1"
1274 )
1275
1276 # Bad "pinned" type:
1277 bad = deepcopy(good)
1278- bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['pinned'] = 1
1279+ bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['pinned'] = 1
1280 with self.assertRaises(TypeError) as cm:
1281 schema.check_file(bad)
1282 self.assertEqual(
1283 str(cm.exception),
1284 TYPE_ERROR.format(
1285- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['pinned']", bool, int, 1
1286+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['pinned']", bool, int, 1
1287 )
1288 )
1289
1290 # Bad "pinned" value:
1291 bad = deepcopy(good)
1292- bad['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['pinned'] = False
1293+ bad['stored']['MYYG3YDSOQVSW3TEMVYG643F']['pinned'] = False
1294 with self.assertRaises(ValueError) as cm:
1295 schema.check_file(bad)
1296 self.assertEqual(
1297 str(cm.exception),
1298- "doc['stored']['MZZG2ZDSOQVSW2TEMVZG643F']['pinned'] must equal True; got False"
1299+ "doc['stored']['MYYG3YDSOQVSW3TEMVYG643F']['pinned'] must equal True; got False"
1300 )
1301
1302 ##################################################
1303@@ -765,23 +765,23 @@
1304 def test_project_db_name(self):
1305 self.assertEqual(
1306 schema.project_db_name('AAAAAAAAAAAAAAAAAAAAAAAA'),
1307- 'dmedia-0-aaaaaaaaaaaaaaaaaaaaaaaa',
1308+ 'dmedia-1-aaaaaaaaaaaaaaaaaaaaaaaa',
1309 )
1310 _id = random_id()
1311 self.assertEqual(
1312 schema.project_db_name(_id),
1313- 'dmedia-0-{}'.format(_id.lower())
1314+ 'dmedia-1-{}'.format(_id.lower())
1315 )
1316
1317 def test_get_project_id(self):
1318 self.assertEqual(
1319- schema.get_project_id('dmedia-0-aaaaaaaaaaaaaaaaaaaaaaaa'),
1320+ schema.get_project_id('dmedia-1-aaaaaaaaaaaaaaaaaaaaaaaa'),
1321 'AAAAAAAAAAAAAAAAAAAAAAAA'
1322 )
1323- self.assertIsNone(schema.get_project_id('dmedia-0'))
1324- self.assertIsNone(schema.get_project_id('novacut-0'))
1325+ self.assertIsNone(schema.get_project_id('dmedia-1'))
1326+ self.assertIsNone(schema.get_project_id('novacut-1'))
1327 self.assertIsNone(
1328- schema.get_project_id('novacut-0-aaaaaaaaaaaaaaaaaaaaaaaa')
1329+ schema.get_project_id('novacut-1-aaaaaaaaaaaaaaaaaaaaaaaa')
1330 )
1331 # Make sure we can round-trip with project_db_name():
1332 for i in range(1000):
1333@@ -800,13 +800,13 @@
1334
1335 def test_check_job(self):
1336 good = {
1337- '_id': 'H6VVCPDJZ7CSFG4V6EEYCPPD',
1338+ '_id': 'H6VVCPDJY7CSFG4V6EEYCPPD',
1339 'type': 'dmedia/job',
1340 'time': 1234567890,
1341 'status': 'waiting',
1342 'worker': 'novacut-renderer',
1343 'files': [
1344- 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR2BKILJOA3CP7QZW',
1345+ 'ROHNRBKS6T4YETP5JHEGQ3OLSBDBWRCKR3BKILJOA3CP7QYW',
1346 ],
1347 'job': {
1348 'Dmedia': 'ignores everything in job',
1349
1350=== modified file 'dmedia/tests/test_server.py'
1351--- dmedia/tests/test_server.py 2013-04-25 03:27:37 +0000
1352+++ dmedia/tests/test_server.py 2013-04-28 21:19:24 +0000
1353@@ -32,7 +32,7 @@
1354 from base64 import b64encode
1355 from queue import Queue
1356
1357-from dbase32.rfc3548 import b32enc, b32dec, random_id
1358+from dbase32 import db32enc, db32dec, random_id
1359 from usercouch.misc import TempCouch
1360 from filestore import DIGEST_B32LEN, DIGEST_BYTES
1361 import microfiber
1362@@ -548,7 +548,7 @@
1363 }
1364 sr = StartResponse()
1365 ret = app(environ, sr)
1366- data = dumps({'challenge': b32enc(cr.challenge)}).encode('utf-8')
1367+ data = dumps({'challenge': db32enc(cr.challenge)}).encode('utf-8')
1368 self.assertEqual(ret, [data])
1369 self.assertEqual(sr.status, '200 OK')
1370 self.assertEqual(sr.headers,
1371@@ -775,7 +775,7 @@
1372 self.assertEqual(app.state, 'gave_challenge')
1373 self.assertIsInstance(obj, dict)
1374 self.assertEqual(set(obj), set(['challenge']))
1375- self.assertEqual(local.challenge, b32dec(obj['challenge']))
1376+ self.assertEqual(local.challenge, db32dec(obj['challenge']))
1377 with self.assertRaises(microfiber.BadRequest) as cm:
1378 client.get('challenge')
1379 self.assertEqual(
1380
1381=== modified file 'dmedia/tests/test_startup.py'
1382--- dmedia/tests/test_startup.py 2012-10-24 23:23:30 +0000
1383+++ dmedia/tests/test_startup.py 2013-04-28 21:19:24 +0000
1384@@ -89,8 +89,8 @@
1385
1386 machine_id = random_id()
1387 user_id = random_id()
1388- json.dump({'_id': machine_id}, open(tmp.join('machine.json'), 'w'))
1389- json.dump({'_id': user_id}, open(tmp.join('user.json'), 'w'))
1390+ json.dump({'_id': machine_id}, open(tmp.join('machine-1.json'), 'w'))
1391+ json.dump({'_id': user_id}, open(tmp.join('user-1.json'), 'w'))
1392 inst = startup.DmediaCouch(tmp.dir)
1393 self.assertEqual(inst.basedir, tmp.dir)
1394 self.assertIsInstance(inst.pki, PKI)
1395@@ -140,7 +140,7 @@
1396 self.assertEqual(len(machine_id), 48)
1397 self.assertIsInstance(inst.machine, dict)
1398 self.assertEqual(inst.machine['_id'], machine_id)
1399- self.assertEqual(inst.load_config('machine'), inst.machine)
1400+ self.assertEqual(inst.load_config('machine-1'), inst.machine)
1401
1402 with self.assertRaises(Exception) as cm:
1403 inst.create_machine()
1404@@ -178,7 +178,7 @@
1405 self.assertEqual(len(user_id), 48)
1406 self.assertIsInstance(inst.user, dict)
1407 self.assertEqual(inst.user['_id'], user_id)
1408- self.assertEqual(inst.load_config('user'), inst.user)
1409+ self.assertEqual(inst.load_config('user-1'), inst.user)
1410
1411 with self.assertRaises(Exception) as cm:
1412 inst.create_user()
1413
1414=== removed file 'reference.py'
1415--- reference.py 2012-12-11 19:45:07 +0000
1416+++ reference.py 1970-01-01 00:00:00 +0000
1417@@ -1,30 +0,0 @@
1418-#!/usr/bin/python3
1419-
1420-from os import path
1421-import sys
1422-import time
1423-
1424-from microfiber import Database, dmedia_env, NotFound, Conflict
1425-from dmedia.util import init_filestore
1426-from dmedia import schema
1427-from dmedia.metastore import create_stored
1428-
1429-parentdir = path.abspath(sys.argv[1])
1430-fs = init_filestore(parentdir)[0]
1431-db = Database('dmedia-0', dmedia_env())
1432-
1433-for st in fs:
1434- try:
1435- db.get(st.id)
1436- print('exists:', st.id)
1437- continue
1438- except NotFound:
1439- pass
1440- ch = fs.verify(st.id)
1441- stored = create_stored(st.id, fs)
1442- doc = schema.create_file(time.time(), ch, stored)
1443- try:
1444- db.save(doc)
1445- print(st.id)
1446- except Conflict:
1447- print('conflict', st.id)
1448
1449=== modified file 'test-client-server.py'
1450--- test-client-server.py 2013-04-16 11:25:40 +0000
1451+++ test-client-server.py 2013-04-28 21:19:24 +0000
1452@@ -7,7 +7,7 @@
1453
1454 import dbus
1455 from filestore.misc import TempFileStore
1456-from dbase32.rfc3548 import random_id
1457+from dbase32 import random_id
1458
1459 import dmedia
1460 from dmedia.util import get_db
1461
1462=== modified file 'ui/dmedia.js'
1463--- ui/dmedia.js 2013-01-27 15:21:35 +0000
1464+++ ui/dmedia.js 2013-04-28 21:19:24 +0000
1465@@ -1,6 +1,6 @@
1466 "use strict";
1467
1468-var db = new couch.Database('dmedia-0');
1469+var db = new couch.Database('dmedia-1');
1470
1471 var UI = {
1472 tabinit: {},

Subscribers

People subscribed via source and target branches