Merge lp:~jderose/dmedia/bittorrent into lp:dmedia

[Jason Gerard DeRose]

* Adds FileStore.finalize_transfer() to capture an important pattern some downloaders will need (in particular the bittorrent downloader)

* Adds basic but working dmedia.downloader.TorrentDownloader that uses python-libtorrent... this is what the bug is all about

* Adds test-torrent.py script which will download a 44MB video file.

You run the test script like this:

$ ./test-torrent.py

It will create a FileStore in ~/.dmedia_test/, and will download this torrent:

http://novacut.s3.amazonaws.com/37GDNHANX7RCBMBGTYLSIK7TMTUQSKDS.mov?torrent

The test-torrent.py script will be removed in the near future.

Note that TorrentDownloader and Downloader (http) are deliberately crude... I'm adding several rough download/upload backends so I can get a feel for what abstract API captures them well. We want a download Manager that can dispatches download Workers generically without needing to understand specifics of download backend.

Next I'm going to add rough S3 upload/download. And then I will design the abstract API, make download Manager and download Worker all shiny.

[lp:~jderose/dmedia/bittorrent]

168. By Jason Gerard DeRose on 2011-02-11

Fixed outdated comment

[Jason Gerard DeRose (jderose)]

Ah, note that you'll need to install python-libtorrent to test this stuff. This should install everything you need:

sudo apt-get install python-desktopcouch-records totem libimage-exiftool-perl python-webkit gir1.2-webkit-1.0 python-paste python-notify python-appindicator python-xdg python-libtorrent

[Martin Owens (doctormo)]

It looks good, needs a bit of documentation. But otherwise a good patch.

review: Approve

[Paul Hummer (rockstar)]

<rockstar> jderose, any reason you're using relative imports?
<jderose> because i'm partial to them... i like that it makes it perefectly clear which modules are internal to package in question
<jderose> although i know python community seems divided on this
<rockstar> Isn't that what PEP 8's module ordering is for?
<rockstar> I think PEP 8 is kinda the standard among the community.
<jderose> yes, but unless you know all the modules/packages being used, it can be difficult to quickly spot where imports outside python stdlib start, where imports internal to package start
<rockstar> There's a blank line there, right?
<rockstar> import stdlib
<rockstar>
<rockstar> import thirdparty
<rockstar>
<rockstar> import dmedia
<jderose> ah, mabye i haven't ready pep 8 recently enough
<rockstar> I don't think it's PEP 8 specifically, but another PEP in the spirit of PEP 8.
<jderose> hmm, might be one i haven't read
<rockstar> jderose, also, my eyes twitch every time I see a function or class without a docstring. :)
<jderose> rockstar: in TorrentDownloader? that one i made note about... i'm feeling out what common/abstract API needs to be for download backends, so i didn't spend time with docstrings as API will change in next day or too
<jderose> which i know is an excuse, but things are moving oh so quickly
<jderose> and of course there are a lot of missing docstrings... sometimes i wait till i feel like the API is mature enough, cocepts are clear enough to merit detailed explanation
<rockstar> Maybe a docstring that says "XXX: Put some docstrings here."
<jderose> see, another fancy sounding excuse! ^^^ :P
<jderose> true, i should do that... good idea
<rockstar> I'm willing to bet you'll get a better feeling of API clarity when you add docstrings.
<rockstar> If you can't explain it, of course it's not clear. :)
<jderose> sometimes... although sometimes i've found that if i write detailed docstrings too quickly, there is a certain friction to changing API... you protect work in docstrings at expense of improving API
<rockstar> Hm, maybe that's a workflow difference between you and I.
<jderose> i'm all about programing as sculpture... whittle away, refactor, take a step back
<jderose> rockstar: could be you have a better attention span than i do :)
<rockstar> Although if anyone writes against the dmedia libs, they deserve whatever they get.
<rockstar> jderose, why are your test case classes starting with test_ ?
<jderose> i've made it pretty clear that things are in very heavy flux still... a long term quality api is more important that trying to make only backword compatible too early on, IMHO
<jderose> rockstar: ah, habit from using nosetests for so long
<rockstar> jderose, nose finds Test*
* jderose is really getting grilled by rockstar... honored to merit such a review :P
<rockstar> jderose, "grilled" isn't the word I'd want to use. :)
<jderose> rockstar: i know it's not PEP8 standard, but i like visual clarity of test_FooBar better than TestFooBar... helps eye more quiclky zero in on class name, IMHO
<jderose> i meant grilled playfully... appreciate the attention to detail
<jderose> it was like 30 seconds tell you said "PEP 8"... so you're a g...

review: Needs Fixing

[Jason Gerard DeRose (jderose)]

As discussed with rockstar, I'm going to fix the import best practices in a separate merge, so I'm approving this.

doctormo and rockstar, thanks for the reviews!

review: Approve

=== modified file 'debian/control'
--- debian/control	2011-02-01 23:53:30 +0000
+++ debian/control	2011-02-11 17:26:30 +0000
@@ -19,7 +19,8 @@
  python-paste,
  python-notify,
  python-appindicator,
- python-xdg
+ python-xdg,
+ python-libtorrent
 Provides: python-dmedia
 Replaces: python-dmedia (<< ${source:Version})
 Conflicts: python-dmedia (<< ${source:Version})

=== modified file 'dmedia/downloader.py'
--- dmedia/downloader.py	2011-01-21 12:42:47 +0000
+++ dmedia/downloader.py	2011-02-11 17:26:30 +0000
@@ -23,14 +23,17 @@
 Download files in chunks using HTTP Range requests.
 """
 
+from os import path
 from base64 import b32encode
 from urlparse import urlparse
 from httplib import HTTPConnection, HTTPSConnection
 import logging
+import time
 from . import __version__
-from .constants import CHUNK_SIZE
+from .constants import CHUNK_SIZE, TYPE_ERROR
 from .errors import DownloadFailure
-from .filestore import HashList, HASH
+from .filestore import FileStore, HashList, HASH
+import libtorrent
 
 USER_AGENT = 'dmedia %s' % __version__
 log = logging.getLogger()
@@ -155,3 +158,50 @@
     def run(self):
         for (i, chash) in enumerate(self.leaves):
             self.process_leaf(i, chash)
+
+
+class TorrentDownloader(object):
+    def __init__(self, torrent, fs, chash, ext=None):
+        if not isinstance(fs, FileStore):
+            raise TypeError(
+                TYPE_ERROR % ('fs', FileStore, type(fs), fs)
+            )
+        self.torrent = torrent
+        self.fs = fs
+        self.chash = chash
+        self.ext = ext
+
+    def get_tmp(self):
+        tmp = self.fs.temp(self.chash, self.ext, create=True)
+        log.debug('Writting file to %r', tmp)
+        return tmp
+
+    def finalize(self):
+        dst = self.fs.finalize_transfer(self.chash, self.ext)
+        log.debug('Canonical name is %r', dst)
+        return dst
+
+    def run(self):
+        log.info('Downloading torrent %r %r', self.chash, self.ext)
+        tmp = self.get_tmp()
+        session = libtorrent.session()
+        session.listen_on(6881, 6891)
+
+        info = libtorrent.torrent_info(
+            libtorrent.bdecode(self.torrent)
+        )
+
+        torrent = session.add_torrent({
+            'ti': info,
+            'save_path': path.dirname(tmp),
+        })
+
+        while not torrent.is_seed():
+            s = torrent.status()
+            log.debug('Downloaded %d%%', s.progress * 100)
+            time.sleep(2)
+
+        session.remove_torrent(torrent)
+        time.sleep(1)
+
+        return self.finalize()

=== modified file 'dmedia/errors.py'
--- dmedia/errors.py	2011-01-27 16:54:48 +0000
+++ dmedia/errors.py	2011-02-11 17:26:30 +0000
@@ -64,3 +64,7 @@
 
 class DownloadFailure(DmediaError):
     _format = 'leaf %(leaf)d expected %(expected)r; got %(got)r'
+
+
+class IntegrityError(DmediaError):
+    _format = 'got chash %(got)r; expected %(expected)r for %(filename)r'

=== modified file 'dmedia/filestore.py'
--- dmedia/filestore.py	2011-02-07 05:46:11 +0000
+++ dmedia/filestore.py	2011-02-11 17:26:30 +0000
@@ -49,7 +49,8 @@
 from threading import Thread
 from Queue import Queue
 from .schema import create_store
-from .errors import AmbiguousPath, DuplicateFile, FileStoreTraversal
+from .errors import AmbiguousPath, FileStoreTraversal
+from .errors import DuplicateFile, IntegrityError
 from .constants import LEAF_SIZE, TRANSFERS_DIR, IMPORTS_DIR, TYPE_ERROR, EXT_PAT
 
 B32LENGTH = 32  # Length of base32-encoded hash
@@ -619,6 +620,76 @@
         fallocate(size, filename)
         return open(filename, 'r+b')
 
+    def finalize_transfer(self, chash, ext=None):
+        """
+        Move canonically named temporary file to its final canonical location.
+
+        This method will check the content hash of the canonically-named
+        temporary file with content hash *chash* and extension *ext*.  If the
+        content hash is correct, it will do an ``os.fchmod()`` to set read-only
+        permissions, and then rename the file into its canonical location.
+
+        If the content hash is incorrect, `IntegrityError` is raised.  If the
+        canonical file already exists, `DuplicateFile` is raised.  Lastly, if
+        the temporary does not exist, ``IOError`` is raised.
+
+        This method will typically be used with the BitTorrent downloader or
+        similar, in which case the content hash will be known prior to
+        downloading.  The downloader will first determine the canonical
+        temporary file name, like this:
+
+        >>> fs = FileStore()
+        >>> tmp = fs.temp('ZR765XWSF6S7JQHLUI4GCG5BHGPE252O', 'mov', create=True)
+        >>> tmp  #doctest: +ELLIPSIS
+        '/tmp/store.../transfers/ZR765XWSF6S7JQHLUI4GCG5BHGPE252O.mov'
+
+
+        Then the downloader will write to the temporary file as it's being
+        downloaded:
+
+        >>> from dmedia.tests import sample_mov  # Sample .MOV file
+        >>> src_fp = open(sample_mov, 'rb')
+        >>> tmp_fp = open(tmp, 'wb')
+        >>> while True:
+        ...     chunk = src_fp.read(2**20)  # Read in 1MiB chunks
+        ...     if not chunk:
+        ...         break
+        ...     tmp_fp.write(chunk)
+        ...
+        >>> tmp_fp.close()
+
+
+        Finally, the downloader will move the temporary file into its canonical
+        location:
+
+        >>> dst = fs.finalize_transfer('ZR765XWSF6S7JQHLUI4GCG5BHGPE252O', 'mov')
+        >>> dst  #doctest: +ELLIPSIS
+        '/tmp/store.../ZR/765XWSF6S7JQHLUI4GCG5BHGPE252O.mov'
+
+
+        Note above that this method returns the full path of the canonically
+        named file.
+        """
+        # Open temporary file and check content hash:
+        tmp = self.temp(chash, ext)
+        tmp_fp = open(tmp, 'rb')
+        h = HashList(tmp_fp)
+        got = h.run()
+        if got != chash:
+            raise IntegrityError(got=got, expected=chash, filename=tmp_fp.name)
+
+        # Get canonical name, check for duplicate:
+        dst = self.path(chash, ext, create=True)
+        if path.exists(dst):
+            raise DuplicateFile(chash=chash, src=tmp_fp.name, dst=dst)
+
+        # Set file to read-only and rename into canonical location
+        os.fchmod(tmp_fp.fileno(), 0o444)
+        os.rename(tmp_fp.name, dst)
+
+        # Return canonical filename:
+        return dst
+
     def import_file(self, src_fp, ext=None):
         """
         Atomically copy open file *src_fp* into this file store.

=== modified file 'dmedia/tests/test_downloader.py'
--- dmedia/tests/test_downloader.py	2011-01-21 12:11:33 +0000
+++ dmedia/tests/test_downloader.py	2011-02-11 17:26:30 +0000
@@ -23,11 +23,17 @@
 Unit tests for `dmedia.downloader` module.
 """
 
+import os
+from os import path
 from unittest import TestCase
 from hashlib import sha1
 from base64 import b32encode
 from .helpers import raises, TempDir
-from dmedia.errors import DownloadFailure
+from .helpers import sample_mov, sample_thm
+from .helpers import mov_hash, thm_hash
+from dmedia.constants import TYPE_ERROR, LEAF_SIZE
+from dmedia.errors import DownloadFailure, DuplicateFile, IntegrityError
+from dmedia.filestore import FileStore, HashList
 from dmedia import downloader
 
 
@@ -137,3 +143,130 @@
         inst = Example(a, b, b)
         self.assertEqual(inst.process_leaf(7, a_hash), a)
         self.assertEqual(inst.dst_fp._chunk, a)
+
+
+class test_TorrentDownloader(TestCase):
+    klass = downloader.TorrentDownloader
+
+    def test_init(self):
+        tmp = TempDir()
+        fs = FileStore(tmp.path)
+
+        e = raises(TypeError, self.klass, '', 17, mov_hash)
+        self.assertEqual(
+            str(e),
+            TYPE_ERROR % ('fs', FileStore, int, 17)
+        )
+
+        inst = self.klass('', fs, mov_hash)
+        self.assertEqual(inst.torrent, '')
+        self.assertTrue(inst.fs is fs)
+        self.assertEqual(inst.chash, mov_hash)
+        self.assertEqual(inst.ext, None)
+
+        inst = self.klass('', fs, mov_hash, ext='mov')
+        self.assertEqual(inst.torrent, '')
+        self.assertTrue(inst.fs is fs)
+        self.assertEqual(inst.chash, mov_hash)
+        self.assertEqual(inst.ext, 'mov')
+
+    def test_get_tmp(self):
+        # Test with ext='mov'
+        tmp = TempDir()
+        fs = FileStore(tmp.path)
+        inst = self.klass('', fs, mov_hash, 'mov')
+        d = tmp.join('transfers')
+        f = tmp.join('transfers', mov_hash + '.mov')
+        self.assertFalse(path.exists(d))
+        self.assertFalse(path.exists(f))
+        self.assertEqual(inst.get_tmp(), f)
+        self.assertTrue(path.isdir(d))
+        self.assertFalse(path.exists(f))
+
+        # Test with ext=None
+        tmp = TempDir()
+        fs = FileStore(tmp.path)
+        inst = self.klass('', fs, mov_hash)
+        d = tmp.join('transfers')
+        f = tmp.join('transfers', mov_hash)
+        self.assertFalse(path.exists(d))
+        self.assertFalse(path.exists(f))
+        self.assertEqual(inst.get_tmp(), f)
+        self.assertTrue(path.isdir(d))
+        self.assertFalse(path.exists(f))
+
+    def test_finalize(self):
+        tmp = TempDir()
+        fs = FileStore(tmp.path)
+        inst = self.klass('', fs, mov_hash, 'mov')
+
+        src_d = tmp.join('transfers')
+        src = tmp.join('transfers', mov_hash + '.mov')
+        dst_d = tmp.join(mov_hash[:2])
+        dst = tmp.join(mov_hash[:2], mov_hash[2:] + '.mov')
+
+        # Test when transfers/ dir doesn't exist:
+        e = raises(IOError, inst.finalize)
+        self.assertFalse(path.exists(src_d))
+        self.assertFalse(path.exists(dst_d))
+        self.assertFalse(path.exists(dst))
+
+        # Test when transfers/ exists but file does not:
+        self.assertEqual(fs.temp(mov_hash, 'mov', create=True), src)
+        self.assertTrue(path.isdir(src_d))
+        e = raises(IOError, inst.finalize)
+        self.assertFalse(path.exists(src))
+        self.assertFalse(path.exists(dst_d))
+        self.assertFalse(path.exists(dst))
+
+        # Test when file has wrong content hash and wrong size:
+        open(src, 'wb').write(open(sample_thm, 'rb').read())
+        e = raises(IntegrityError, inst.finalize)
+        self.assertEqual(e.got, thm_hash)
+        self.assertEqual(e.expected, mov_hash)
+        self.assertEqual(e.filename, src)
+        self.assertFalse(path.exists(dst_d))
+        self.assertFalse(path.exists(dst))
+
+        # Test when file has wrong content hash and *correct* size:
+        fp1 = open(sample_mov, 'rb')
+        fp2 = open(src, 'wb')
+        while True:
+            chunk = fp1.read(LEAF_SIZE)
+            if not chunk:
+                break
+            fp2.write(chunk)
+        fp1.close()
+
+        # Now change final byte at end file:
+        fp2.seek(-1, os.SEEK_END)
+        fp2.write('A')
+        fp2.close()
+        self.assertEqual(path.getsize(sample_mov), path.getsize(src))
+
+        e = raises(IntegrityError, inst.finalize)
+        self.assertEqual(e.got, 'UECTT7A7EIHZ2SGGBMMO5WTTSVU4SUWM')
+        self.assertEqual(e.expected, mov_hash)
+        self.assertEqual(e.filename, src)
+        self.assertFalse(path.exists(dst_d))
+        self.assertFalse(path.exists(dst))
+
+        # Test with correct content hash:
+        fp1 = open(sample_mov, 'rb')
+        fp2 = open(src, 'wb')
+        while True:
+            chunk = fp1.read(LEAF_SIZE)
+            if not chunk:
+                break
+            fp2.write(chunk)
+        fp1.close()
+        fp2.close()
+        self.assertEqual(inst.finalize(), dst)
+        self.assertTrue(path.isdir(src_d))
+        self.assertFalse(path.exists(src))
+        self.assertTrue(path.isdir(dst_d))
+        self.assertTrue(path.isfile(dst))
+
+        # Check content hash of file in canonical location
+        fp = open(dst, 'rb')
+        self.assertEqual(HashList(fp).run(), mov_hash)

=== modified file 'dmedia/tests/test_filestore.py'
--- dmedia/tests/test_filestore.py	2011-02-07 05:46:11 +0000
+++ dmedia/tests/test_filestore.py	2011-02-11 17:26:30 +0000
@@ -34,10 +34,12 @@
 from .helpers import TempDir, TempHome, raises
 from .helpers import sample_mov, sample_thm
 from .helpers import mov_hash, mov_leaves, mov_qid
-from dmedia.errors import AmbiguousPath, FileStoreTraversal, DuplicateFile
+from .helpers import thm_hash, thm_leaves, thm_qid
+from dmedia.errors import AmbiguousPath, FileStoreTraversal
+from dmedia.errors import DuplicateFile, IntegrityError
 from dmedia.filestore import HashList
 from dmedia import filestore, constants, schema
-from dmedia.constants import TYPE_ERROR, EXT_PAT
+from dmedia.constants import TYPE_ERROR, EXT_PAT, LEAF_SIZE
 
 
 class test_functions(TestCase):
@@ -892,6 +894,81 @@
         self.assertEqual(path.dirname(fp.name), imports)
         self.assertTrue(fp.name.endswith('.mov'))
 
+    def test_finalize_transfer(self):
+        tmp = TempDir()
+        inst = self.klass(tmp.path)
+
+        src_d = tmp.join('transfers')
+        src = tmp.join('transfers', mov_hash + '.mov')
+        dst_d = tmp.join(mov_hash[:2])
+        dst = tmp.join(mov_hash[:2], mov_hash[2:] + '.mov')
+
+        # Test when transfers/ dir doesn't exist:
+        e = raises(IOError, inst.finalize_transfer, mov_hash, 'mov')
+        self.assertFalse(path.exists(src_d))
+        self.assertFalse(path.exists(dst_d))
+        self.assertFalse(path.exists(dst))
+
+        # Test when transfers/ exists but file does not:
+        self.assertEqual(inst.temp(mov_hash, 'mov', create=True), src)
+        self.assertTrue(path.isdir(src_d))
+        e = raises(IOError, inst.finalize_transfer, mov_hash, 'mov')
+        self.assertFalse(path.exists(src))
+        self.assertFalse(path.exists(dst_d))
+        self.assertFalse(path.exists(dst))
+
+        # Test when file has wrong content hash and wrong size:
+        open(src, 'wb').write(open(sample_thm, 'rb').read())
+        e = raises(IntegrityError, inst.finalize_transfer, mov_hash, 'mov')
+        self.assertEqual(e.got, thm_hash)
+        self.assertEqual(e.expected, mov_hash)
+        self.assertEqual(e.filename, src)
+        self.assertFalse(path.exists(dst_d))
+        self.assertFalse(path.exists(dst))
+
+        # Test when file has wrong content hash and *correct* size:
+        fp1 = open(sample_mov, 'rb')
+        fp2 = open(src, 'wb')
+        while True:
+            chunk = fp1.read(LEAF_SIZE)
+            if not chunk:
+                break
+            fp2.write(chunk)
+        fp1.close()
+
+        # Now change final byte at end file:
+        fp2.seek(-1, os.SEEK_END)
+        fp2.write('A')
+        fp2.close()
+        self.assertEqual(path.getsize(sample_mov), path.getsize(src))
+
+        e = raises(IntegrityError, inst.finalize_transfer, mov_hash, 'mov')
+        self.assertEqual(e.got, 'UECTT7A7EIHZ2SGGBMMO5WTTSVU4SUWM')
+        self.assertEqual(e.expected, mov_hash)
+        self.assertEqual(e.filename, src)
+        self.assertFalse(path.exists(dst_d))
+        self.assertFalse(path.exists(dst))
+
+        # Test with correct content hash:
+        fp1 = open(sample_mov, 'rb')
+        fp2 = open(src, 'wb')
+        while True:
+            chunk = fp1.read(LEAF_SIZE)
+            if not chunk:
+                break
+            fp2.write(chunk)
+        fp1.close()
+        fp2.close()
+        self.assertEqual(inst.finalize_transfer(mov_hash, 'mov'), dst)
+        self.assertTrue(path.isdir(src_d))
+        self.assertFalse(path.exists(src))
+        self.assertTrue(path.isdir(dst_d))
+        self.assertTrue(path.isfile(dst))
+
+        # Check content hash of file in canonical location
+        fp = open(dst, 'rb')
+        self.assertEqual(HashList(fp).run(), mov_hash)
+
     def test_import_file(self):
         tmp = TempDir()
         src = tmp.join('movie.mov')

=== added file 'test-torrent.py'
--- test-torrent.py	1970-01-01 00:00:00 +0000
+++ test-torrent.py	2011-02-11 17:26:30 +0000
@@ -0,0 +1,120 @@
+#!/usr/bin/env python
+
+"""
+Tests downloading torrent from:
+
+http://novacut.s3.amazonaws.com/37GDNHANX7RCBMBGTYLSIK7TMTUQSKDS.mov?torrent
+"""
+
+from __future__ import print_function
+
+import sys
+import os
+from os import path
+import time
+import logging
+from base64 import b32decode, b64decode
+from dmedia.downloader import TorrentDownloader
+from dmedia.filestore import FileStore
+
+logging.basicConfig(
+    level=logging.DEBUG,
+    format='\t'.join(['%(levelname)s', '%(message)s'])
+)
+
+# Known size, top hash, and leaf hashes for test video:
+size = 44188757
+
+chash = '37GDNHANX7RCBMBGTYLSIK7TMTUQSKDS'
+
+leaves_b32 = [
+    'TDCPJHYQVEVCTMLIKEQITVMJKSUIETHD',
+    'UAG5HQCLH6PGA4RAYXDEFRNCSZDTMLXU',
+    '2DGBOSUSSDG5OKASXNQTJG3MANGL4H2U',
+    'D4TMBNAQOOFMIWB2ATT2GR7Y262EATVM',
+    'SEYVEQPAVCROXPYZWIXN4YZRHOZV2MWV',
+    'B2I7VCLIVV4LBSRTGSRQNNXDDPWCKNLA',
+]
+
+leaves = [b32decode(l) for l in leaves_b32]
+
+
+# The .torrent file, base64 encoded:
+tdata = """
+ZDg6YW5ub3VuY2U0MjpodHRwOi8vdHJhY2tlci5hbWF6b25hd3MuY29tOjY5NjkvYW5ub3VuY2Ux
+Mzphbm5vdW5jZS1saXN0bGw0MjpodHRwOi8vdHJhY2tlci5hbWF6b25hd3MuY29tOjY5NjkvYW5u
+b3VuY2VlZTQ6aW5mb2Q2Omxlbmd0aGk0NDE4ODc1N2U0Om5hbWUzNjozN0dETkhBTlg3UkNCTUJH
+VFlMU0lLN1RNVFVRU0tEUy5tb3YxMjpwaWVjZSBsZW5ndGhpMjYyMTQ0ZTY6cGllY2VzMzM4MDqr
+KimqIY/Y6lI6kGX+koXSVvVYgXilfN8qnUAk31PJttEN8RXgagd3dLD41PXZjf5KXt84gp1i8Cgh
+pfPOKRi3wBOGwNoq2JCjiHiW4wEobgaePx3alOno8Xl/phRuRoUoK/FgPJ6fm5JRulnrh5lonsmJ
+iShyE+B9N5Kax2lFN5t6B1W+6y51ROzBDep9E8MaBobFyINvHkRsDY0mNgpTpwdtqAiZTOzsj9L4
+YE966dNMTBVXKWC8i5pSEuRN72Bf7wXF9DoOvBaNk1vMpzKzd+GGDRpKYoZcROuyUbEzYMuI+HgE
+5CvZGZZH01yVfelfotaMK1ogSShZAserPHEGLEfNwBMn51wSrLGrYiGN770n4O3tLSpmaZ1ZGaE5
+Ot2nZ+ZmSJmcZWF5Uu6e07QONhziX3SYn58rvqON7fJ/mz2GiO+9ARMfv/pc6ElbaVIAEHJy2Yej
+5uYIRtaaeraRG/KFgzvuxidwtePSyZzoM+1j/BaTDig+76N1hrei6R8/oUkU+rtn2MjYwaJtH+y5
+/iLp418vHyjqTUsFNGGBh24nJT7LcXhakfkkjmDmCKiZ6SDVbKW47EfpR8lCnwnHC4KXz7l/yddv
+m2Oc6fuUAX11DEffcSclEtPx0/VEtih1uYgoovq6ef4Eg09PwRmEQ1mfy99yCk62WSEefeYWDHq0
+WO7ih+3mPTaimIMp6lClIf3DRszxe2bta3C/D+J4E+NtAsSSzkKn1MK5w+SFxj3ooQmrl1jaYv+I
+QzBeVWb5KRDpzwuIr7LwaeV4/ClqGi1p4c+juohEFoLsrttB0xN9dRZcOtGVwMXqDrz4ukAtla7l
+y+A93K/X60idwWteUlDDiZ8nK//1wmnaS5i1dDhEU+s+3M6JPB/w0pDjqoRFcuc99om0+7GS3E0V
+NH+/yazeA9il4sJ5qpl8frN1SCevhoGQqg604YEQwyZpuqy8vjIwbRDwPr4ly56F75wcxpPOaWZd
+xRqf5Sp53mDSTV3lEg7Jkm+H8n1o+fjVR5dnqoiYmQuepriBnJKcMLZx1ENnD04b1QD04oM4A/hX
+FSjHjO2/fR4Q/uQXxKliv5WSdvvXSkTyC09eSXjR7OTLG7PHEkBNTlLkBOGDZR7DzUClCIHozS5V
+qSSy/qTdmcLFwsU/LRAMMTUfb9H/JpN0olIO47A5MQvVWO3t2bh/FWcgr+yNODBKBPP/W4YSsjKa
+bJ2HBfRhePuiij/DfEGga6qfyHSmOM2lX9u58JW6IlWfTxWMNfaC4p7KZA7T3rnO7QOpRqss3lGy
+W6XWoJcCGF1jt6NztG67aRBAEv6n60KMKjEq6I5b8xTnHYaLjqlYxRr0ayzq/dJV+fev8W2gArfH
+cRLv6VhGBmyaMBCxxinHjs6IaQj+Grn0SH5bWHKKPwV/pB+aFykce845bwzpNpIlOanWfiNo5lRC
+wlBNx/XPMMERs3rOohkshp35bsYd+Y4tbuk/sDSJdcU9kq6lAMzc8WTIltmJ3kShD1YFGGlu9jJN
+h3KNoUUYLHPvCLx4yN7JzNr15X+J8mUh07sSLgprmTbw4oj2WgZZujpx54I8j7ynYM6w4sqshmya
+FLLzkMnv2HaJtG1kwtzGX+S2Pn6kp/JH0I5JU/lAM61SOprOyBV4SeJHCrEmL8BPHZzPLj3mF84S
+gEMijEK0pdk3dvmy/9JlmwiUqC8g+0POAfsZirubaG1c55HQ1hKdSrZOZZbyzdGfwFDndOP3JXFY
+miQLAH90mmd5JKR9+LrRv4nYH0HC092uf4smKzcFUYHpy0eJd3/UJKU64uE3z4X3JeZcSaboESMu
+6j7nzbk/ZEvQKZ/R/LJ93c90Hk8GMB3OenwZai0LMSMmCwlxGyvAJUuheZ+1jD8+uH7Ryz1znadg
+L8U5sTXkFKscH7ctlTKQUSyFWECb1szevd+4nOqpEfty55GoN1VOBsj7uLe0TQFSqryUEqSbhBUK
+5ZUlt/5orrDTyw4q4zxMXV3t+swhiJ3/xh1USwpELpdP5BsExmhpW7p9WScj17GqojfzHEgMrS90
+591i8/CiRs/XRFpn1j4AeUFRam7XU0f5o5QpjSwQo9/Icf40xU+DYwaCRHYnM1gvtsBJjRWBGShC
+38wFJ80VQJDq1jjYeFL99SY6n33OqBZPW+/EX0LU7pzepxWgvl1RnsaoLjvQzbFdj+akBgQWtrJd
+W5JCKH3qNq0Y9f1hRhUXAWLE0UMW1IlCDwaNnPYm/kKyoilYoxi4pj9xWGEaI+mOhrRZbvBXIzMp
+LKIiPx/VWg/2B3fWw4VWmAWQN66J26j8imTUAO74Uk9iUbU4htax0VU5pCWguU6peqCfXfOrCfDb
++CIQP/2OzU52R+k9VzPTMdH+WfXqXD/whq0azSm8davU3zI/XQ0A6ASEw7jdwj8dW0jTQCPMWGRD
+lngE1sW8a6Gf37PtjGLrCfllnrYQmiJeDom23VbubxjGFi4lsUikXakh8yQauEN0Omx1D3FO86sn
+u45XAq6OO9Ifh68R5D6VtpSv7Wi7JVibICqWqfxMunbf5NOuET7Nk4yPIwzKHr1+JFxY6B3CeZP6
+K8KBYmamGcFTgfsnvPS+4sodae66hPJUSDE/Hjsn/fvNLqOoqIu+3jDcL7EgSNaC0iVQuG0M/sOR
++Rj1Kzg04sHJz0QjdmH9940aR/VgssLGhTS2GQcdCZeabRX0rZY3k7zfXSMP+I9N0jQpV/2fI+oD
+GdPwv1CcQWn+0e00sU7UeY5Qqap0M9/GNR8C4w2TjINHqbIAPOxCVIQo+aeqpbe6Es/CJkBpxFj7
+U6TJf3yMCWy0vhoYetWD0gBAab6stDwnQeRNPUjqN96qhLUFqyZgPNuWvQPpjXYrJS0xSFN59LVV
+s8FhEU+lagNpRgPaUsqMdoxx9x7l3COesxU5Hl7znezKL+YY6TBJQIbCnCjAWNmv6t8Jj92rON+M
+/TER3Q7Pzhy9TJFY1Mq6sjEslx+6V1uhYEG0amQ3cZ217ffK4i3Y4kFrFtRfEYCvCiXogTqVz4yG
+3JmsyJrokSpyRDmP6d2EXHybFUGi5PGgh1+jgHuHK65beaez/Ldd8WXOfxxjCukQ2/ZAaycoTGYL
+qv2FJab3ce10JQFZgxKGtK3OaIfprSGP1yqpfWX2/WwBv3g3j00gW4b/a8UpOeS0L4usuf6HrK00
+TUArAegiFfikLUZqTcDlp0T6X0XIZ1xgZ/NKUFY8Ooi+YRwSzDmblBLuYVArpjldSjeuilWPI1Dn
+xmSm7BgYrI7pMG2NYlBMGvntMFFK6705OD6d4a2UKg9Tzt4LU8eah2XWYnpjAO+730bmjFr5rHy7
++KVdaSkGfjJ4G8zjKa3qwapFtGj5lbczHtpXlYW1YiRfDPdgr7IGSQ9yfAlEwJIAhi17OIPck4y3
+9dQei5XdPDrpzO6E7bWtA/5TJkOjeKAOgiO5m9L7sssdKBqPOmrljXHmZTQc0RbUFXtteLrNq7uD
+af8tDdlsWJSlG/qxhNSFHz7YcKxw9huM5pyFbTRmiIVVFq3rQ38zAES5RjSAlSs0Bw/zhUNUvmmq
+Z99T7JDcRzU2rfTNOZ7zno0xXhnCGAGoeFclIw+TqFVW2mLSl2WUrkdhE5YiYDq0sRT64ajlg3tF
+rhvVK0AQuWNHm5LzF4gJq/eksf3iQyDVDCZlXQIoGnSzwDqA4dKHU/vm7yv7lmxbfeiKGlGNiK8r
+PrIcNRMRFcjry/FbDEJlLvHLGdFIVvGxbXA/4R6bfj32fSLNWqq14Vps53r8qmDIdTfmQMSmREu2
+N/0HwBtxKlF0pVtU68wuxubqceZF5Y3d4K/R+8pGDWz5j6Fa6qBb3Bqoa9TrVkQhuoN7XGTOKE/u
+NlX4Y7gROs6g/pe37U+eTTRIkiGgcGFQSZOktTCkAlpzCgQeIgWfQf3GiKDuy6x9ISHqnqROboHA
+JBTbsz6stH3+57ihMiWJ3uOTcivqrJ8PpjKb0ISxdR+XMw3baXklnN9b6fIXVw2BlfqORJAwRjz5
+i51/MICGnBdIs24CAb/SJQgezzspKHURdCo9Cr7wgH5S/9NheuP3ChjnjhkcIq6iIBRgMx6cQo/u
+xl3m6OmpZ6qyy6JLpoNJECjUW4MQB/uXpGE8xZEbPddYewOAFGRP4pub0FTBJx7tpMyE8VxGV31A
+529Iw0KqKFK1efvtMQyCd/BTkxqjHIk5ahfEcKkIv06fjdXCRngISP4LKjOcZYeb/Ltb58pIAA12
+2xEAiTe74jzv4jxw2mBAAEBy6JZCCQhvfXRadNoJrWs0W4WDsKIXUtfN2tqPIE1KGgnYj6GRVi4g
+IQAiJ41Gb6sQUl7iyQbJ6gEOP0wo1vnIH7udeN3AMND+rJlT9sEuNJPwHRjjMm7ThgOumOjv6m5w
+9Nfq2yu22LozPsNqDhbmrRSS5IhvtmYdll5iEC+EFt6L8Jpf4CtsHVmQhnX0ePk5JA7/EzYSoaId
+oo1fdItR4cLxIFVRK+YFGjEyOngtYW16LWJ1Y2tldDc6bm92YWN1dDk6eC1hbXota2V5MzY6MzdH
+RE5IQU5YN1JDQk1CR1RZTFNJSzdUTVRVUVNLRFMubW92ZWU=
+"""
+
+
+# Create a FileStore in ~/.dmedia_test/
+home = path.abspath(os.environ['HOME'])
+base = path.join(home, '.dmedia_test')
+fs = FileStore(base)
+
+t = TorrentDownloader(b64decode(tdata), fs, chash, 'mov')
+t.run()
+
+assert path.getsize(fs.path(chash, 'mov')) == size