Merge lp:~kartoch/duplicity/hubic into lp:~duplicity-team/duplicity/0.7-series

[Kartoch]

Hubic support

Add a new backend for the support of Hubic, a french cloud provider offering cheap space storage.

The code adds a pre-authentication step to the swift backend, requesting some web services to obtain pre-authentication URL and token.

Usage:
- url is "hubic://default"
- user / password is deliver to duplicity via export variables HUBIC_USERNAME and HUBIC_PASSWORD

At the present time there is few limitations:
- the URL MUST BE "hubic://default", thus there is no way to place backup in a sub-directory
- token are valid 8 hours. If the backup takes more time, the user needs to relaunch duplicity to continue it. This is a limitation of the swift code to relaunch an authentication step in case of "not authorized" error during transfer.

[lp:~kartoch/duplicity/hubic]

998. By Kartoch on 2014-08-18

Add try/catch to handle error during hubic authentication

[Kartoch (kartoch)]

It seems Hubic will change auth service soon for OAuth and will drop web services soon, so my patches will be probably useless...

[Kenneth Loafman (kenneth-loafman)]

Does this mean that the merge request can be 'rejected' (that seems to be the only option LP gives us)?

[edso (ed.so)]

this also misses the necessary documentation in the man page (sections requirements, url format etc..).

..ede

[Kartoch (kartoch)]

yeah reject it: web service is deprecated and will be remove at any time.

Unmerged revisions

998. By Kartoch on 2014-08-18

Add try/catch to handle error during hubic authentication

997. By Kartoch on 2014-08-17

Refactoring code to inherite from swiftbackend

996. By Kartoch on 2014-08-17

Preliminary support for HUBIC storage

- add a new backend dedicated to hubic
- copy/paste from swift backend to test it (refactroing later)

=== added file 'duplicity/backends/hubicbackend.py'
--- duplicity/backends/hubicbackend.py	1970-01-01 00:00:00 +0000
+++ duplicity/backends/hubicbackend.py	2014-08-18 06:26:27 +0000
@@ -0,0 +1,104 @@
+import base64
+import json
+import os
+
+import duplicity.backend
+from duplicity.backends.swiftbackend import SwiftBackend
+from duplicity import log
+from duplicity import util
+from duplicity.errors import BackendException
+
+try:
+    import requests
+except ImportError:
+    raise BackendException("This backend requires the python-requests library.")
+
+
+class HubicBackend(SwiftBackend):
+
+    """
+    Backend for Hubic
+    """
+
+    OVH_SESSION_HANDLER = 'https://ws.ovh.com/sessionHandler/r4/rest.dispatcher/'
+    HUBIC_WS = 'https://ws.ovh.com/hubic/r5/rest.dispatcher/'
+
+    def __init__(self, parsed_url):
+
+        if 'HUBIC_USERNAME' not in os.environ:
+            raise BackendException('HUBIC_USERNAME environment variable '
+                                   'not set.')
+        login = os.environ['HUBIC_USERNAME']
+
+        if 'HUBIC_PASSWORD' not in os.environ:
+            raise BackendException('HUBIC_PASSWORD environment variable '
+                                   'not set.')
+        password = os.environ['HUBIC_PASSWORD']
+
+        if 'SWIFT_PREAUTHURL' in os.environ or \
+           'SWIFT_PREAUTHTOKEN' in os.environ or \
+           'SWIFT_USERNAME' in os.environ or \
+           'SWIFT_PASSWORD' in os.environ or \
+           'SWIFT_AUTHURL' in os.environ:
+            raise BackendException('SWIFT_* environment variables '
+                                   'must not be set.')
+
+        (url, token) = self._authenticate(login, password)
+
+        os.environ['SWIFT_PREAUTHURL'] = url
+        os.environ['SWIFT_PREAUTHTOKEN'] = token
+
+        super(HubicBackend, self).__init__(parsed_url)
+
+    def _authenticate(self, login, password):
+
+        try:
+            # get anonymous session id
+            r = requests.get(self.OVH_SESSION_HANDLER + 'getAnonymousSession')
+            anonymous_session_id = r.json()['answer']['session']['id']
+
+            log.Debug("HUBIC: hubic anonymous session id:" + anonymous_session_id)
+
+            params = {'sessionId': anonymous_session_id,
+                      'email': login}
+            payload = {'params': json.dumps(params)}
+
+            # get hubic id
+            r = requests.get(self.HUBIC_WS + 'getHubics', params=payload)
+            hubics = r.json()
+            hubics_id = hubics['answer'][0]['id']
+            log.Debug("HUBIC: hubics id:" + hubics_id)
+
+            # get authenticated session id
+            params = {'login': hubics['answer'][0]['nic'],
+                      'password': password,
+                      'context': 'hubic'}
+            payload = {'params': json.dumps(params)}
+            r = requests.get(self.OVH_SESSION_HANDLER + 'login', params=payload)
+            self.session_id = r.json()['answer']['session']['id']
+            log.Debug("HUBIC: authenticated hubics id:" + self.session_id)
+
+            # get storage URL and authorization token
+            params = {'sessionId': self.session_id,
+                      'hubicId': hubics_id}
+            payload = {'params': json.dumps(params)}
+
+            r = requests.get(self.HUBIC_WS + 'getHubic', params=payload)
+            storage_url = base64.b64decode(r.json()['answer']['credentials']['username'])
+            log.Debug("HUBIC: storage url:" + str(storage_url))
+            auth_token = r.json()['answer']['credentials']['secret']
+            log.Debug("HUBIC: auth token:" + auth_token)
+        except Exception as e:
+            log.FatalError("Authentication to hubic failed: %s %s"
+                           % (e.__class__.__name__, util.uexc(e)),
+                           log.ErrorCode.backend_error)
+
+        return storage_url, auth_token
+
+    def _close(self):
+        log.Debug("HUBIC: logout")
+        params = {'sessionId': self.session_id}
+        payload = {'params': json.dumps(params)}
+        requests.get(self.OVH_SESSION_HANDLER + 'logout', params=payload)
+
+duplicity.backend.register_backend("hubic", HubicBackend)

=== modified file 'duplicity/commandline.py'
--- duplicity/commandline.py	2014-06-28 14:48:21 +0000
+++ duplicity/commandline.py	2014-08-18 06:26:27 +0000
@@ -848,6 +848,7 @@
   ftp://%(user)s[:%(password)s]@%(other_host)s[:%(port)s]/%(some_dir)s
   ftps://%(user)s[:%(password)s]@%(other_host)s[:%(port)s]/%(some_dir)s
   hsi://%(user)s[:%(password)s]@%(other_host)s[:%(port)s]/%(some_dir)s
+  hubic://default
   imap://%(user)s[:%(password)s]@%(other_host)s[:%(port)s]/%(some_dir)s
   rsync://%(user)s[:%(password)s]@%(other_host)s[:%(port)s]::/%(module)s/%(some_dir)s
   rsync://%(user)s[:%(password)s]@%(other_host)s[:%(port)s]/%(relative_path)s