Merge lp:~rye/duplicity/mediafire into lp:~duplicity-team/duplicity/0.7-series

[Roman Yepishev]

Backend for https://www.mediafire.com

Requires https://pypi.python.org/pypi/mediafire/ installed.

To test:

1. Create an account at https://www.mediafire.com
2. Run:

pip install mediafire

export FTP_PASSWORD=your-mediafire-password

duplicity /path/to/dir mf://your-email%<email address hidden>/backup-directory

[edso (ed.so)]

hey Roman,

looks good! any specific reason why you didn't use the existing user/passphrase API in from backend.py ?

i really see no need to add an additional set of backend specific env vars.

..ede/duply.net

review: Needs Information

[lp:~rye/duplicity/mediafire]

1180. By Roman Yepishev on 2016-02-10

Use built-in username/password support in backends.py.

Backend is no longer reading environment variables.

MEDIAFIRE_EMAIL ->
   mf://duplicity%<email address hidden>/some_path

MEDIAFIRE_PASSWORD ->
   provided via command line via get_password() or
   FTP_PASSWORD env var or
   mf://duplicity%40example.com:some%<email address hidden>/some_path

[Roman Yepishev (rye)]

Nope, there was no reason not to use the built-in support for the username/password retrieval.

Updated code to remove backend-specific env vars.

[edso (ed.so)]

thanks Roman!

review: Approve

=== modified file 'bin/duplicity.1'
--- bin/duplicity.1	2016-02-05 09:58:57 +0000
+++ bin/duplicity.1	2016-02-10 17:21:56 +0000
@@ -1205,6 +1205,16 @@
 .B "A NOTE ON MULTI BACKEND"
 below.
 .RE
+.PP
+.BR "MediaFire"
+.PP
+.RS
+mf://user[:password]@mediafire.com/some_dir
+.PP
+See also
+.B "A NOTE ON MEDIAFIRE BACKEND"
+below.
+.RE
 
 .SH TIME FORMATS
 duplicity uses time strings in two places.  Firstly, many of the files
@@ -1871,6 +1881,22 @@
 .B SWIFT_AUTHVERSION
 is unspecified, it will default to version 1.
 
+.SH A NOTE ON MEDIAFIRE BACKEND
+This backend requires
+.B mediafire
+python library to be installed on the system. See
+.BR REQUIREMENTS .
+
+Use URL escaping for username (and password, if provided via command line):
+
+.PP
+.RS
+mf://[e-mail]/some_folder
+.PP
+.RE
+
+The destination folder will be created for you if it does not exist.
+
 .SH A NOTE ON SYMMETRIC ENCRYPTION AND SIGNING
 Signing and symmetrically encrypt at the same time with the gpg binary on the
 command line, as used within duplicity, is a specifically challenging issue.
@@ -2075,6 +2101,10 @@
 .B Python kerberos module
 for kerberos authentication
 - https://github.com/02strich/pykerberos
+.TP
+.BR "MediaFire backend"
+.B MediaFire Python Open SDK
+- https://pypi.python.org/pypi/mediafire/
 
 .SH AUTHOR
 .TP

=== added file 'duplicity/backends/mediafirebackend.py'
--- duplicity/backends/mediafirebackend.py	1970-01-01 00:00:00 +0000
+++ duplicity/backends/mediafirebackend.py	2016-02-10 17:21:56 +0000
@@ -0,0 +1,138 @@
+# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
+#
+# Copyright 2016 Roman Yepishev <[e-mail]>
+#
+# This file is part of duplicity.
+#
+# Duplicity is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.
+#
+# Duplicity is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with duplicity; if not, write to the Free Software Foundation,
+# Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+"""MediaFire Duplicity Backend"""
+
+import os
+
+import duplicity.backend
+
+from duplicity.errors import BackendException
+
+DUPLICITY_APP_ID = '45593'
+
+
+class MediafireBackend(duplicity.backend.Backend):
+    """Use this backend when saving to MediaFire
+
+    URLs look like mf:/root/folder.
+    """
+    def __init__(self, parsed_url):
+        try:
+            import mediafire.client
+        except ImportError:
+            raise BackendException("This backend requires "
+                                   "the mediafire library")
+
+        duplicity.backend.Backend.__init__(self, parsed_url)
+
+        mediafire_email = parsed_url.username
+        mediafire_password = self.get_password()
+
+        self._file_res = mediafire.client.File
+        self._folder_res = mediafire.client.Folder
+        self._downloaderror_exc = mediafire.client.DownloadError
+        self._notfound_exc = mediafire.client.ResourceNotFoundError
+
+        self.client = mediafire.client.MediaFireClient()
+        self.client.login(app_id=DUPLICITY_APP_ID,
+                          email=mediafire_email,
+                          password=mediafire_password)
+
+        # //username:password@host/path/to/folder -> path/to/folder
+        uri = 'mf:///' + parsed_url.path.split('/', 3)[3]
+
+        # Create folder if it does not exist and make sure it is private
+        # See MediaFire Account Settings /Security and Privacy / Share Link
+        # to set "Inherit from parent folder"
+        try:
+            folder = self.client.get_resource_by_uri(uri)
+            if not isinstance(folder, self._folder_res):
+                raise BackendException("target_url already exists "
+                                       "and is not a folder")
+        except mediafire.client.ResourceNotFoundError:
+            # force folder to be private
+            folder = self.client.create_folder(uri, recursive=True)
+            self.client.update_folder_metadata(uri, privacy='private')
+
+        self.folder = folder
+
+    def _put(self, source_path, remote_filename=None):
+        """Upload file"""
+        # Use source file name if remote one is not defined
+        if remote_filename is None:
+            remote_filename = os.path.basename(source_path.name)
+
+        uri = self._build_uri(remote_filename)
+
+        with self.client.upload_session():
+            self.client.upload_file(source_path.open('rb'), uri)
+
+    def _get(self, filename, local_path):
+        """Download file"""
+        uri = self._build_uri(filename)
+        try:
+            self.client.download_file(uri, local_path.open('wb'))
+        except self._downloaderror_exc as ex:
+            raise BackendException(ex)
+
+    def _list(self):
+        """List files in backup directory"""
+        uri = self._build_uri()
+        filenames = []
+        for item in self.client.get_folder_contents_iter(uri):
+            if not isinstance(item, self._file_res):
+                continue
+
+            filenames.append(item['filename'].encode('utf-8'))
+
+        return filenames
+
+    def _delete(self, filename):
+        """Delete single file"""
+        uri = self._build_uri(filename)
+        self.client.delete_file(uri)
+
+    def _delete_list(self, filename_list):
+        """Delete list of files"""
+        for filename in filename_list:
+            self._delete(filename)
+
+    def _query(self, filename):
+        """Stat the remote file"""
+        uri = self._build_uri(filename)
+
+        try:
+            resource = self.client.get_resource_by_uri(uri)
+            size = int(resource['size'])
+        except self._notfound_exc:
+            size = -1
+
+        return {'size': size}
+
+    def _build_uri(self, filename=None):
+        """Build relative URI"""
+        return (
+            'mf:' + self.folder["folderkey"] +
+            ('/' + filename if filename else '')
+        )
+
+
+duplicity.backend.register_backend("mf", MediafireBackend)

=== modified file 'duplicity/commandline.py'
--- duplicity/commandline.py	2016-02-02 12:20:14 +0000
+++ duplicity/commandline.py	2016-02-10 17:21:56 +0000
@@ -918,6 +918,7 @@
   onedrive://%(some_dir)s
   azure://%(container_name)s
   b2://%(account_id)s[:%(application_key)s]@%(bucket_name)s/[%(some_dir)s/]
+  mf://%(user)s[:%(password)s]@%(other_host)s/%(some_dir)s
 
 """ % dict