Merge lp:~mterry/duplicity/backend-unification into lp:duplicity/0.6

[Michael Terry]

Reorganize and simplify backend code. Specifically:

- Formalize the expected API between backends and duplicity. See the new file duplicity/backends/README for the instructions I've given authors.

- Add some tests for our backend wrapper class as well as some tests for individual backends. For several backends that have some commands do all the heavy lifting (hsi, tahoe, ftp), I've added fake little mock commands so that we can test them locally. This doesn't truly test our integration with those commands, but at least lets us test the backend glue code itself.

- Removed a lot of duplicate and unused code which backends were using (or not using). This branch drops 700 lines of code (~20%) in duplicity/backends!

- Simplified expectations of backends. Our wrapper code now does all the retrying, and all the exception handling. Backends can 'fire and forget' trusting our wrappers to give the user a reasonable error message. Obviously, backends can also add more details and make nicer error messages. But they don't *have* to.

- Separate out the backend classes from our wrapper class. Now there is no possibility of namespace collision. All our API methods use one underscore. Anything else (zero or two underscores) are for the backend class's use.

- Added the concept of a 'backend prefix' which is used by par2 and gio backends to provide generic support for "schema+" in urls -- like par2+ or gio+. I've since marked the '--gio' flag as deprecated, in favor of 'gio+'. Now you can even nest such backends like par2+gio+file://blah/blah.

- The switch to control which cloudfiles backend had a typo. I fixed this, but I'm not sure I should have? If we haven't had complaints, maybe we can just drop the old backend.

- I manually tested all the backends we have (except hsi and tahoe -- but those are simple wrappers around commands and I did test those via mocks per above). I also added a bunch more manual backend tests to ./testing/manual/backendtest.py, which can now be run like the above to test all the files you have configured in config.py or you can pass it a URL which it will use for testing (useful for backend authors).

[edso (ed.so)]

A+ for effort ;) .. we should probably make that a 0.7 release, stating that there might be some hickups (bugs) due to major code revision.
two more comments below..

On 28.04.2014 04:55, Michael Terry wrote:
SNIP
>
>
> === modified file 'bin/duplicity'
> --- bin/duplicity 2014-04-19 19:54:54 +0000
> +++ bin/duplicity 2014-04-28 02:49:55 +0000
> @@ -289,8 +289,6 @@
>
> def validate_block(orig_size, dest_filename):
> info = backend.query_info([dest_filename])[dest_filename]
> - if 'size' not in info:
> - return # backend didn't know how to query size

this could raise a key not found error. can you really guaratee the key will be there?

> size = info['size']
> if size is None:
> return # error querying file
>
> === modified file 'duplicity/backend.py'
> --- duplicity/backend.py 2014-04-25 23:53:46 +0000
> +++ duplicity/backend.py 2014-04-28 02:49:55 +0000
> @@ -24,6 +24,7 @@
SNIP
>
> # These URL schemes have a backend with a notion of an RFC "network location".
> # The 'file' and 's3+http' schemes should not be in this list.
> @@ -69,7 +74,6 @@
> uses_netloc = ['ftp',
> 'ftps',
> 'hsi',
> - 'rsync',
> 's3',
> 'scp', 'ssh', 'sftp',
> 'webdav', 'webdavs',

any reason why you removed rsync here?

..ede

[Kenneth Loafman (kenneth-loafman)]

ede, I don't see a problem with a missing key.

As to 0.7 -- I'm thinking that some of the latest changes, not bug fixes,
should be removed from 0.6 and applied to 0.7. We could make the next
release, 0.6.24, the last of the 0.6 series. We probably need to start an
email thread to discuss this, but I think it's time.

On Mon, Apr 28, 2014 at 4:51 AM, edso <email address hidden> wrote:

> A+ for effort ;) .. we should probably make that a 0.7 release, stating
> that there might be some hickups (bugs) due to major code revision.
> two more comments below..
>
> On 28.04.2014 04:55, Michael Terry wrote:
> SNIP
> >
> >
> > === modified file 'bin/duplicity'
> > --- bin/duplicity 2014-04-19 19:54:54 +0000
> > +++ bin/duplicity 2014-04-28 02:49:55 +0000
> > @@ -289,8 +289,6 @@
> >
> > def validate_block(orig_size, dest_filename):
> > info = backend.query_info([dest_filename])[dest_filename]
> > - if 'size' not in info:
> > - return # backend didn't know how to query size
>
> this could raise a key not found error. can you really guaratee the key
> will be there?
>
> > size = info['size']
> > if size is None:
> > return # error querying file
> >
> > === modified file 'duplicity/backend.py'
> > --- duplicity/backend.py 2014-04-25 23:53:46 +0000
> > +++ duplicity/backend.py 2014-04-28 02:49:55 +0000
> > @@ -24,6 +24,7 @@
> SNIP
> >
> > # These URL schemes have a backend with a notion of an RFC "network
> location".
> > # The 'file' and 's3+http' schemes should not be in this list.
> > @@ -69,7 +74,6 @@
> > uses_netloc = ['ftp',
> > 'ftps',
> > 'hsi',
> > - 'rsync',
> > 's3',
> > 'scp', 'ssh', 'sftp',
> > 'webdav', 'webdavs',
>
> any reason why you removed rsync here?
>
> ..ede
>
> --
>
> https://code.launchpad.net/~mterry/duplicity/backend-unification/+merge/216764
> Your team duplicity-team is requested to review the proposed merge of
> lp:~mterry/duplicity/backend-unification into lp:duplicity.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~duplicity-team
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~duplicity-team
> More help : https://help.launchpad.net/ListHelp
>

[edso (ed.so)]

On 28.04.2014 13:21, Kenneth Loafman wrote:
> ede, I don't see a problem with a missing key.

        size = info['size']
will throw an error if the dictionary misses they key. we had that earlier. but MT will probably know if he hacked the info routines safely enough to guarantee it's existence.

> As to 0.7 -- I'm thinking that some of the latest changes, not bug fixes,
> should be removed from 0.6 and applied to 0.7. We could make the next
> release, 0.6.24, the last of the 0.6 series. We probably need to start an
> email thread to discuss this, but I think it's time.

that's an alternative route to go, although i am not aware of major bugs since 0.6.23 ..ede

>
>
> On Mon, Apr 28, 2014 at 4:51 AM, edso <email address hidden> wrote:
>
>> A+ for effort ;) .. we should probably make that a 0.7 release, stating
>> that there might be some hickups (bugs) due to major code revision.
>> two more comments below..
>>
>> On 28.04.2014 04:55, Michael Terry wrote:
>> SNIP
>>>
>>>
>>> === modified file 'bin/duplicity'
>>> --- bin/duplicity 2014-04-19 19:54:54 +0000
>>> +++ bin/duplicity 2014-04-28 02:49:55 +0000
>>> @@ -289,8 +289,6 @@
>>>
>>> def validate_block(orig_size, dest_filename):
>>> info = backend.query_info([dest_filename])[dest_filename]
>>> - if 'size' not in info:
>>> - return # backend didn't know how to query size
>>
>> this could raise a key not found error. can you really guaratee the key
>> will be there?
>>
>>> size = info['size']
>>> if size is None:
>>> return # error querying file
>>>
>>> === modified file 'duplicity/backend.py'
>>> --- duplicity/backend.py 2014-04-25 23:53:46 +0000
>>> +++ duplicity/backend.py 2014-04-28 02:49:55 +0000
>>> @@ -24,6 +24,7 @@
>> SNIP
>>>
>>> # These URL schemes have a backend with a notion of an RFC "network
>> location".
>>> # The 'file' and 's3+http' schemes should not be in this list.
>>> @@ -69,7 +74,6 @@
>>> uses_netloc = ['ftp',
>>> 'ftps',
>>> 'hsi',
>>> - 'rsync',
>>> 's3',
>>> 'scp', 'ssh', 'sftp',
>>> 'webdav', 'webdavs',
>>
>> any reason why you removed rsync here?
>>
>> ..ede
>>
>> --
>>
>> https://code.launchpad.net/~mterry/duplicity/backend-unification/+merge/216764
>> Your team duplicity-team is requested to review the proposed merge of
>> lp:~mterry/duplicity/backend-unification into lp:duplicity.
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~duplicity-team
>> Post to : <email address hidden>
>> Unsubscribe : https://launchpad.net/~duplicity-team
>> More help : https://help.launchpad.net/ListHelp
>>
>

[Kenneth Loafman (kenneth-loafman)]

Sorry, was looking above the comment, not below. Not enough coffee!

There are a number of bugs/additions in the current CHANGELOG slated for 0.6.24. Looking at them again, this change is the trigger for 0.7 startup.

Mike, I really like tox. It's really easy to merge changes and retest.

[Michael Terry (mterry)]

> this could raise a key not found error. can you really guaratee the key will be there?

Yes, because now query_info calls the backend query function, then makes sure that 'size' exists for each filename passed. This was part of the "allow backends to be dumb" strategy -- if they skip a filename or don't provide 'size', the rest of duplicity doesn't have to care. Guarantees come from the wrapper class, not the backends.

> any reason why you removed rsync here?

Yeah, I changed the rsync backend to support relative/local urls. This was so we could test it as part of our normal automated tests (without needing a server). The list I removed 'rsync' from was a list of backends that require a hostname.

[edso (ed.so)]

On 28.04.2014 14:55, Michael Terry wrote:
>> this could raise a key not found error. can you really guaratee the key will be there?
>
> Yes, because now query_info calls the backend query function, then makes sure that 'size' exists for each filename passed. This was part of the "allow backends to be dumb" strategy -- if they skip a filename or don't provide 'size', the rest of duplicity doesn't have to care. Guarantees come from the wrapper class, not the backends.
>
>> any reason why you removed rsync here?
>
> Yeah, I changed the rsync backend to support relative/local urls. This was so we could test it as part of our normal automated tests (without needing a server). The list I removed 'rsync' from was a list of backends that require a hostname.
>

well done.. thx ede

[Kenneth Loafman (kenneth-loafman)]

Either of you see a show-stopper to keep 0.6.24 from going out as-is? We
have an impressive list of changes already.

I've resurrected the old 0.7 series, merged in the current 0.6, and will
merge this set of changes in later. Does everyone agree that this is a
good place to make the switch to 0.7? Do we want to make 0.7 the current
focus of development after the release of 0.6.24?

On Mon, Apr 28, 2014 at 9:15 AM, edso <email address hidden> wrote:

> On 28.04.2014 14:55, Michael Terry wrote:
> >> this could raise a key not found error. can you really guaratee the key
> will be there?
> >
> > Yes, because now query_info calls the backend query function, then makes
> sure that 'size' exists for each filename passed. This was part of the
> "allow backends to be dumb" strategy -- if they skip a filename or don't
> provide 'size', the rest of duplicity doesn't have to care. Guarantees
> come from the wrapper class, not the backends.
> >
> >> any reason why you removed rsync here?
> >
> > Yeah, I changed the rsync backend to support relative/local urls. This
> was so we could test it as part of our normal automated tests (without
> needing a server). The list I removed 'rsync' from was a list of backends
> that require a hostname.
> >
>
> well done.. thx ede
>
> --
>
> https://code.launchpad.net/~mterry/duplicity/backend-unification/+merge/216764
> You are subscribed to branch lp:duplicity.
>

[Michael Terry (mterry)]

Well, two separate decisions: release 0.6.24 or not and whether we want to go to 0.7.

But I don't have a reason to wait on releasing 0.6.24.

And as for 0.7, I'm fine with that too. Just a number. :)

But this branch itself feels like not a very momentous change. By which I mean, end users shouldn't care about this at all. Doesn't fix any big bugs (except maybe the --cf-backend argument) and doesn't add any new features.

The bump to require python 2.6 feels like a more natural jumping off point, but as edso noted, 0.6.23 already has 2.6isms in it. So not really a change there either.

So... :shrug:

[edso (ed.so)]

main concern for me is to have a "stable" version, to point users to if they stumble over a showstopper after Mike's reworks. that can be 0.6.24
i wonder if it'd make sense to also exclude all the other modifications Mike in his current run (since drop-pexpect) as they may also contain some side effects that may be better kept in 0.7dev.

considering the major backend differences of Mike's latest commit i agree to make a 0.7 the new devel focus.

..ede

On 28.04.2014 16:45, Kenneth Loafman wrote:
> Either of you see a show-stopper to keep 0.6.24 from going out as-is? We
> have an impressive list of changes already.
>
> I've resurrected the old 0.7 series, merged in the current 0.6, and will
> merge this set of changes in later. Does everyone agree that this is a
> good place to make the switch to 0.7? Do we want to make 0.7 the current
> focus of development after the release of 0.6.24?
>
>
> On Mon, Apr 28, 2014 at 9:15 AM, edso <email address hidden> wrote:
>
>> On 28.04.2014 14:55, Michael Terry wrote:
>>>> this could raise a key not found error. can you really guaratee the key
>> will be there?
>>>
>>> Yes, because now query_info calls the backend query function, then makes
>> sure that 'size' exists for each filename passed. This was part of the
>> "allow backends to be dumb" strategy -- if they skip a filename or don't
>> provide 'size', the rest of duplicity doesn't have to care. Guarantees
>> come from the wrapper class, not the backends.
>>>
>>>> any reason why you removed rsync here?
>>>
>>> Yeah, I changed the rsync backend to support relative/local urls. This
>> was so we could test it as part of our normal automated tests (without
>> needing a server). The list I removed 'rsync' from was a list of backends
>> that require a hostname.
>>>
>>
>> well done.. thx ede
>>
>> --
>>
>> https://code.launchpad.net/~mterry/duplicity/backend-unification/+merge/216764
>> You are subscribed to branch lp:duplicity.
>>
>

[edso (ed.so)]

On 28.04.2014 16:53, Michael Terry wrote:
> Well, two separate decisions: release 0.6.24 or not and whether we want to go to 0.7.
>
> But I don't have a reason to wait on releasing 0.6.24.
>
> And as for 0.7, I'm fine with that too. Just a number. :)
>
> But this branch itself feels like not a very momentous change. By which I mean, end users shouldn't care about this at all. Doesn't fix any big bugs (except maybe the --cf-backend argument) and doesn't add any new features.
>

it's just the sheer amount of modified code that worries me a bit. as i wrote in the other email

also,
1. old backend code e.g. released by somebody privately will not work anymore when dropped into duplicity/backends
2. true, officially switching to 2.6 might be another point, somebody might want to remove the 2.6'isms from the 0.6 branch if they really need duplicity on some oldold distro. supposing we also keep the 2.6 changes only in 0.7 branch.

..ede

[Kenneth Loafman (kenneth-loafman)]

Hmmm, this is getting complicated, but I see your points.

We can test against 2.4 and 2.5 using the answers here:
http://askubuntu.com/questions/125342/how-can-i-install-python-2-6-on-12-04(the
deadsnakes repositories, love the name). So, if we wanted, we could
move the modernizations over to 0.7 and fix 0.6.24 to work on older
Pythons.

That seems like a cleaner and more rational break to me. Thoughts?

On Mon, Apr 28, 2014 at 10:06 AM, edso <email address hidden> wrote:

> On 28.04.2014 16:53, Michael Terry wrote:
> > Well, two separate decisions: release 0.6.24 or not and whether we want
> to go to 0.7.
> >
> > But I don't have a reason to wait on releasing 0.6.24.
> >
> > And as for 0.7, I'm fine with that too. Just a number. :)
> >
> > But this branch itself feels like not a very momentous change. By which
> I mean, end users shouldn't care about this at all. Doesn't fix any big
> bugs (except maybe the --cf-backend argument) and doesn't add any new
> features.
> >
>
> it's just the sheer amount of modified code that worries me a bit. as i
> wrote in the other email
>
> also,
> 1. old backend code e.g. released by somebody privately will not work
> anymore when dropped into duplicity/backends
> 2. true, officially switching to 2.6 might be another point, somebody
> might want to remove the 2.6'isms from the 0.6 branch if they really need
> duplicity on some oldold distro. supposing we also keep the 2.6 changes
> only in 0.7 branch.
>
> ..ede
>
> --
>
> https://code.launchpad.net/~mterry/duplicity/backend-unification/+merge/216764
> You are subscribed to branch lp:duplicity.
>

[edso (ed.so)]

exactly my point, from a stability standpoint though :) ..ede

On 28.04.2014 17:18, Kenneth Loafman wrote:
> Hmmm, this is getting complicated, but I see your points.
>
> We can test against 2.4 and 2.5 using the answers here:
> http://askubuntu.com/questions/125342/how-can-i-install-python-2-6-on-12-04(the
> deadsnakes repositories, love the name). So, if we wanted, we could
> move the modernizations over to 0.7 and fix 0.6.24 to work on older
> Pythons.
>
> That seems like a cleaner and more rational break to me. Thoughts?
>
>
>
> On Mon, Apr 28, 2014 at 10:06 AM, edso <email address hidden> wrote:
>
>> On 28.04.2014 16:53, Michael Terry wrote:
>>> Well, two separate decisions: release 0.6.24 or not and whether we want
>> to go to 0.7.
>>>
>>> But I don't have a reason to wait on releasing 0.6.24.
>>>
>>> And as for 0.7, I'm fine with that too. Just a number. :)
>>>
>>> But this branch itself feels like not a very momentous change. By which
>> I mean, end users shouldn't care about this at all. Doesn't fix any big
>> bugs (except maybe the --cf-backend argument) and doesn't add any new
>> features.
>>>
>>
>> it's just the sheer amount of modified code that worries me a bit. as i
>> wrote in the other email
>>
>> also,
>> 1. old backend code e.g. released by somebody privately will not work
>> anymore when dropped into duplicity/backends
>> 2. true, officially switching to 2.6 might be another point, somebody
>> might want to remove the 2.6'isms from the 0.6 branch if they really need
>> duplicity on some oldold distro. supposing we also keep the 2.6 changes
>> only in 0.7 branch.
>>
>> ..ede
>>
>> --
>>
>> https://code.launchpad.net/~mterry/duplicity/backend-unification/+merge/216764
>> You are subscribed to branch lp:duplicity.
>>
>

[Michael Terry (mterry)]

> it's just the sheer amount of modified code that worries me a bit.
> as i wrote in the other email

Fair. I tried to thoroughly test, but I'm human. :)

> 1. old backend code e.g. released by somebody privately will not work
> anymore when dropped into duplicity/backends

This is a good point. I'm curious how many people do that; I can easily imagine it happening.

As for your comments, Ken:

> So, if we wanted, we could move the modernizations over to 0.7 and
> fix 0.6.24 to work on older Pythons.

As much as I hate giving any thoughts to py2.4/2.5, this is probably a nice thing to do. We clearly have interested users still on RHEL5, from my last survey on the mailing list.

If I have time in the next week, I can do this. Else, if you tackle this, note that you shouldn't do the testing on a recently-released distro. Because recent versions of tox/setuptools/virtualenv all dropped support for <2.6. Maybe try Ubuntu 12.04? I haven't tried an older distro myself yet, but ideally you'd be able to add "py24,py25" to the tox.ini file and just run tox to confirm that we work with older pythons. If that doesn't work, you'd have to go to an earlier release of your distro of choice.

[Kenneth Loafman (kenneth-loafman)]

Luckily, I'm still on 12.04. I'll test and see if 2.4/2.5 work.

On Mon, Apr 28, 2014 at 10:25 AM, Michael Terry <<email address hidden>
> wrote:

> > it's just the sheer amount of modified code that worries me a bit.
> > as i wrote in the other email
>
> Fair. I tried to thoroughly test, but I'm human. :)
>
> > 1. old backend code e.g. released by somebody privately will not work
> > anymore when dropped into duplicity/backends
>
> This is a good point. I'm curious how many people do that; I can easily
> imagine it happening.
>
> As for your comments, Ken:
>
> > So, if we wanted, we could move the modernizations over to 0.7 and
> > fix 0.6.24 to work on older Pythons.
>
> As much as I hate giving any thoughts to py2.4/2.5, this is probably a
> nice thing to do. We clearly have interested users still on RHEL5, from my
> last survey on the mailing list.
>
> If I have time in the next week, I can do this. Else, if you tackle this,
> note that you shouldn't do the testing on a recently-released distro.
> Because recent versions of tox/setuptools/virtualenv all dropped support
> for <2.6. Maybe try Ubuntu 12.04? I haven't tried an older distro myself
> yet, but ideally you'd be able to add "py24,py25" to the tox.ini file and
> just run tox to confirm that we work with older pythons. If that doesn't
> work, you'd have to go to an earlier release of your distro of choice.
> --
>
> https://code.launchpad.net/~mterry/duplicity/backend-unification/+merge/216764
> You are subscribed to branch lp:duplicity.
>

=== modified file 'bin/duplicity'
--- bin/duplicity	2014-04-19 19:54:54 +0000
+++ bin/duplicity	2014-04-28 02:49:55 +0000
@@ -289,8 +289,6 @@
 
     def validate_block(orig_size, dest_filename):
         info = backend.query_info([dest_filename])[dest_filename]
-        if 'size' not in info:
-            return # backend didn't know how to query size
         size = info['size']
         if size is None:
             return # error querying file

=== modified file 'duplicity/backend.py'
--- duplicity/backend.py	2014-04-25 23:53:46 +0000
+++ duplicity/backend.py	2014-04-28 02:49:55 +0000
@@ -24,6 +24,7 @@
 intended to be used by the backends themselves.
 """
 
+import errno
 import os
 import sys
 import socket
@@ -31,6 +32,7 @@
 import re
 import getpass
 import gettext
+import types
 import urllib
 import urlparse
 
@@ -38,11 +40,14 @@
 from duplicity import file_naming
 from duplicity import globals
 from duplicity import log
+from duplicity import path
 from duplicity import progress
+from duplicity import util
 
 from duplicity.util import exception_traceback
 
-from duplicity.errors import BackendException, FatalBackendError
+from duplicity.errors import BackendException
+from duplicity.errors import FatalBackendException
 from duplicity.errors import TemporaryLoadException
 from duplicity.errors import ConflictingScheme
 from duplicity.errors import InvalidBackendURL
@@ -54,8 +59,8 @@
 # todo: this should really NOT be done here
 socket.setdefaulttimeout(globals.timeout)
 
-_forced_backend = None
 _backends = {}
+_backend_prefixes = {}
 
 # These URL schemes have a backend with a notion of an RFC "network location".
 # The 'file' and 's3+http' schemes should not be in this list.
@@ -69,7 +74,6 @@
 uses_netloc = ['ftp',
                'ftps',
                'hsi',
-               'rsync',
                's3',
                'scp', 'ssh', 'sftp',
                'webdav', 'webdavs',
@@ -96,8 +100,6 @@
         if fn.endswith("backend.py"):
             fn = fn[:-3]
             imp = "duplicity.backends.%s" % (fn,)
-            # ignore gio as it is explicitly loaded in commandline.parse_cmdline_options()
-            if fn == "giobackend": continue
             try:
                 __import__(imp)
                 res = "Succeeded"
@@ -110,14 +112,6 @@
             continue
 
 
-def force_backend(backend):
-    """
-    Forces the use of a particular backend, regardless of schema
-    """
-    global _forced_backend
-    _forced_backend = backend
-
-
 def register_backend(scheme, backend_factory):
     """
     Register a given backend factory responsible for URL:s with the
@@ -144,6 +138,32 @@
     _backends[scheme] = backend_factory
 
 
+def register_backend_prefix(scheme, backend_factory):
+    """
+    Register a given backend factory responsible for URL:s with the
+    given scheme prefix.
+
+    The backend must be a callable which, when called with a URL as
+    the single parameter, returns an object implementing the backend
+    protocol (i.e., a subclass of Backend).
+
+    Typically the callable will be the Backend subclass itself.
+
+    This function is not thread-safe and is intended to be called
+    during module importation or start-up.
+    """
+    global _backend_prefixes
+
+    assert callable(backend_factory), "backend factory must be callable"
+
+    if scheme in _backend_prefixes:
+        raise ConflictingScheme("the prefix %s already has a backend "
+                                "associated with it"
+                                "" % (scheme,))
+
+    _backend_prefixes[scheme] = backend_factory
+
+
 def is_backend_url(url_string):
     """
     @return Whether the given string looks like a backend URL.
@@ -157,9 +177,9 @@
         return False
 
 
-def get_backend(url_string):
+def get_backend_object(url_string):
     """
-    Instantiate a backend suitable for the given URL, or return None
+    Find the right backend class instance for the given URL, or return None
     if the given string looks like a local path rather than a URL.
 
     Raise InvalidBackendURL if the URL is not a valid URL.
@@ -167,22 +187,44 @@
     if not is_backend_url(url_string):
         return None
 
+    global _backends, _backend_prefixes
+
     pu = ParsedUrl(url_string)
-
-    # Implicit local path
     assert pu.scheme, "should be a backend url according to is_backend_url"
 
-    global _backends, _forced_backend
-
-    if _forced_backend:
-        return _forced_backend(pu)
-    elif not pu.scheme in _backends:
-        raise UnsupportedBackendScheme(url_string)
-    else:
-        try:
-            return _backends[pu.scheme](pu)
-        except ImportError:
-            raise BackendException(_("Could not initialize backend: %s") % str(sys.exc_info()[1]))
+    factory = None
+
+    for prefix in _backend_prefixes:
+        if url_string.startswith(prefix + '+'):
+            factory = _backend_prefixes[prefix]
+            pu = ParsedUrl(url_string.lstrip(prefix + '+'))
+            break
+
+    if factory is None:
+        if not pu.scheme in _backends:
+            raise UnsupportedBackendScheme(url_string)
+        else:
+            factory = _backends[pu.scheme]
+
+    try:
+        return factory(pu)
+    except ImportError:
+        raise BackendException(_("Could not initialize backend: %s") % str(sys.exc_info()[1]))
+
+
+def get_backend(url_string):
+    """
+    Instantiate a backend suitable for the given URL, or return None
+    if the given string looks like a local path rather than a URL.
+
+    Raise InvalidBackendURL if the URL is not a valid URL.
+    """
+    if globals.use_gio:
+        url_string = 'gio+' + url_string
+    obj = get_backend_object(url_string)
+    if obj:
+        obj = BackendWrapper(obj)
+    return obj
 
 
 class ParsedUrl:
@@ -296,165 +338,74 @@
     # Replace the full network location with the stripped copy.
     return parsed_url.geturl().replace(parsed_url.netloc, straight_netloc, 1)
 
-
-# Decorator for backend operation functions to simplify writing one that
-# retries.  Make sure to add a keyword argument 'raise_errors' to your function
-# and if it is true, raise an exception on an error.  If false, fatal-log it.
-def retry(fn):
-    def iterate(*args):
-        for n in range(1, globals.num_retries):
-            try:
-                kwargs = {"raise_errors" : True}
-                return fn(*args, **kwargs)
-            except Exception as e:
-                log.Warn(_("Attempt %s failed: %s: %s")
-                         % (n, e.__class__.__name__, str(e)))
-                log.Debug(_("Backtrace of previous error: %s")
-                          % exception_traceback())
-                if isinstance(e, TemporaryLoadException):
-                    time.sleep(30) # wait longer before trying again
-                else:
-                    time.sleep(10) # wait a bit before trying again
-        # Now try one last time, but fatal-log instead of raising errors
-        kwargs = {"raise_errors" : False}
-        return fn(*args, **kwargs)
-    return iterate
-
-# same as above, a bit dumber and always dies fatally if last trial fails
-# hence no need for the raise_errors var ;), we really catch everything here
-# as we don't know what the underlying code comes up with and we really *do*
-# want to retry globals.num_retries times under all circumstances
-def retry_fatal(fn):
-    def _retry_fatal(self, *args):
-        try:
-            n = 0
-            for n in range(1, globals.num_retries):
+def _get_code_from_exception(backend, operation, e):
+    if isinstance(e, BackendException) and e.code != log.ErrorCode.backend_error:
+        return e.code
+    elif hasattr(backend, '_error_code'):
+        return backend._error_code(operation, e) or log.ErrorCode.backend_error
+    elif hasattr(e, 'errno'):
+        # A few backends return such errors (local, paramiko, etc)
+        if e.errno == errno.EACCES:
+            return log.ErrorCode.backend_permission_denied
+        elif e.errno == errno.ENOENT:
+            return log.ErrorCode.backend_not_found
+        elif e.errno == errno.ENOSPC:
+            return log.ErrorCode.backend_no_space
+    return log.ErrorCode.backend_error
+
+def retry(operation, fatal=True):
+    # Decorators with arguments introduce a new level of indirection.  So we
+    # have to return a decorator function (which itself returns a function!)
+    def outer_retry(fn):
+        def inner_retry(self, *args):
+            for n in range(1, globals.num_retries + 1):
                 try:
-                    self.retry_count = n
                     return fn(self, *args)
-                except FatalBackendError as e:
+                except FatalBackendException as e:
                     # die on fatal errors
                     raise e
                 except Exception as e:
                     # retry on anything else
-                    log.Warn(_("Attempt %s failed. %s: %s")
-                             % (n, e.__class__.__name__, str(e)))
                     log.Debug(_("Backtrace of previous error: %s")
                               % exception_traceback())
-                    time.sleep(10) # wait a bit before trying again
-        # final trial, die on exception
-            self.retry_count = n+1
-            return fn(self, *args)
-        except Exception as e:
-            log.Debug(_("Backtrace of previous error: %s")
-                        % exception_traceback())
-            log.FatalError(_("Giving up after %s attempts. %s: %s")
-                         % (self.retry_count, e.__class__.__name__, str(e)),
-                          log.ErrorCode.backend_error)
-        self.retry_count = 0
-
-    return _retry_fatal
+                    at_end = n == globals.num_retries
+                    code = _get_code_from_exception(self.backend, operation, e)
+                    if code == log.ErrorCode.backend_not_found:
+                        # If we tried to do something, but the file just isn't there,
+                        # no need to retry.
+                        at_end = True
+                    if at_end and fatal:
+                        def make_filename(f):
+                            if isinstance(f, path.ROPath):
+                                return util.escape(f.name)
+                            else:
+                                return util.escape(f)
+                        extra = ' '.join([operation] + [make_filename(x) for x in args if x])
+                        log.FatalError(_("Giving up after %s attempts. %s: %s")
+                                       % (n, e.__class__.__name__,
+                                          str(e)), code=code, extra=extra)
+                    else:
+                        log.Warn(_("Attempt %s failed. %s: %s")
+                                 % (n, e.__class__.__name__, str(e)))
+                    if not at_end:
+                        if isinstance(e, TemporaryLoadException):
+                            time.sleep(90) # wait longer before trying again
+                        else:
+                            time.sleep(30) # wait a bit before trying again
+                        if hasattr(self.backend, '_retry_cleanup'):
+                            self.backend._retry_cleanup()
+
+        return inner_retry
+    return outer_retry
+
 
 class Backend(object):
     """
-    Represents a generic duplicity backend, capable of storing and
-    retrieving files.
-
-    Concrete sub-classes are expected to implement:
-
-      - put
-      - get
-      - list
-      - delete
-      - close (if needed)
-
-    Optional:
-
-      - move
+    See README in backends directory for information on how to write a backend.
     """
-    
     def __init__(self, parsed_url):
         self.parsed_url = parsed_url
 
-    def put(self, source_path, remote_filename = None):
-        """
-        Transfer source_path (Path object) to remote_filename (string)
-
-        If remote_filename is None, get the filename from the last
-        path component of pathname.
-        """
-        raise NotImplementedError()
-
-    def move(self, source_path, remote_filename = None):
-        """
-        Move source_path (Path object) to remote_filename (string)
-
-        Same as put(), but unlinks source_path in the process.  This allows the
-        local backend to do this more efficiently using rename.
-        """
-        self.put(source_path, remote_filename)
-        source_path.delete()
-
-    def get(self, remote_filename, local_path):
-        """Retrieve remote_filename and place in local_path"""
-        raise NotImplementedError()
-
-    def list(self):
-        """
-        Return list of filenames (byte strings) present in backend
-        """
-        def tobytes(filename):
-            "Convert a (maybe unicode) filename to bytes"
-            if isinstance(filename, unicode):
-                # There shouldn't be any encoding errors for files we care
-                # about, since duplicity filenames are ascii.  But user files
-                # may be in the same directory.  So just replace characters.
-                return filename.encode(sys.getfilesystemencoding(), 'replace')
-            else:
-                return filename
-
-        if hasattr(self, '_list'):
-            # Make sure that duplicity internals only ever see byte strings
-            # for filenames, no matter what the backend thinks it is talking.
-            return [tobytes(x) for x in self._list()]
-        else:
-            raise NotImplementedError()
-
-    def delete(self, filename_list):
-        """
-        Delete each filename in filename_list, in order if possible.
-        """
-        raise NotImplementedError()
-
-    # Should never cause FatalError.
-    # Returns a dictionary of dictionaries.  The outer dictionary maps
-    # filenames to metadata dictionaries.  Supported metadata are:
-    #
-    # 'size': if >= 0, size of file
-    #         if -1, file is not found
-    #         if None, error querying file
-    #
-    # Returned dictionary is guaranteed to contain a metadata dictionary for
-    # each filename, but not all metadata are guaranteed to be present.
-    def query_info(self, filename_list, raise_errors=True):
-        """
-        Return metadata about each filename in filename_list
-        """
-        info = {}
-        if hasattr(self, '_query_list_info'):
-            info = self._query_list_info(filename_list)
-        elif hasattr(self, '_query_file_info'):
-            for filename in filename_list:
-                info[filename] = self._query_file_info(filename)
-
-        # Fill out any missing entries (may happen if backend has no support
-        # or its query_list support is lazy)
-        for filename in filename_list:
-            if filename not in info:
-                info[filename] = {}
-
-        return info
-
     """ use getpass by default, inherited backends may overwrite this behaviour """
     use_getpass = True
 
@@ -493,27 +444,7 @@
         else:
             return commandline
 
-    """
-    DEPRECATED:
-    run_command(_persist) - legacy wrappers for subprocess_popen(_persist)
-    """
-    def run_command(self, commandline):
-        return self.subprocess_popen(commandline)
-    def run_command_persist(self, commandline):
-        return self.subprocess_popen_persist(commandline)
-
-    """
-    DEPRECATED:
-    popen(_persist) - legacy wrappers for subprocess_popen(_persist)
-    """
-    def popen(self, commandline):
-        result, stdout, stderr = self.subprocess_popen(commandline)
-        return stdout
-    def popen_persist(self, commandline):
-        result, stdout, stderr = self.subprocess_popen_persist(commandline)
-        return stdout
-
-    def _subprocess_popen(self, commandline):
+    def __subprocess_popen(self, commandline):
         """
         For internal use.
         Execute the given command line, interpreted as a shell command.
@@ -525,6 +456,10 @@
 
         return p.returncode, stdout, stderr
 
+    """ a dictionary for breaking exceptions, syntax is
+        { 'command' : [ code1, code2 ], ... } see ftpbackend for an example """
+    popen_breaks = {}
+
     def subprocess_popen(self, commandline):
         """
         Execute the given command line with error check.
@@ -534,54 +469,179 @@
         """
         private = self.munge_password(commandline)
         log.Info(_("Reading results of '%s'") % private)
-        result, stdout, stderr = self._subprocess_popen(commandline)
+        result, stdout, stderr = self.__subprocess_popen(commandline)
         if result != 0:
-            raise BackendException("Error running '%s'" % private)
-        return result, stdout, stderr
-
-    """ a dictionary for persist breaking exceptions, syntax is
-        { 'command' : [ code1, code2 ], ... } see ftpbackend for an example """
-    popen_persist_breaks = {}
-
-    def subprocess_popen_persist(self, commandline):
-        """
-        Execute the given command line with error check.
-        Retries globals.num_retries times with 30s delay.
-        Returns int Exitcode, string StdOut, string StdErr
-
-        Raise a BackendException on failure.
-        """
-        private = self.munge_password(commandline)
-
-        for n in range(1, globals.num_retries+1):
-            # sleep before retry
-            if n > 1:
-                time.sleep(30)
-            log.Info(_("Reading results of '%s'") % private)
-            result, stdout, stderr = self._subprocess_popen(commandline)
-            if result == 0:
-                return result, stdout, stderr
-
             try:
                 m = re.search("^\s*([\S]+)", commandline)
                 cmd = m.group(1)
-                ignores = self.popen_persist_breaks[ cmd ]
+                ignores = self.popen_breaks[ cmd ]
                 ignores.index(result)
                 """ ignore a predefined set of error codes """
                 return 0, '', ''
             except (KeyError, ValueError):
-                pass
-
-            log.Warn(ngettext("Running '%s' failed with code %d (attempt #%d)",
-                              "Running '%s' failed with code %d (attempt #%d)", n) %
-                               (private, result, n))
-            if stdout or stderr:
-                    log.Warn(_("Error is:\n%s") % stderr + (stderr and stdout and "\n") + stdout)
-
-        log.Warn(ngettext("Giving up trying to execute '%s' after %d attempt",
-                          "Giving up trying to execute '%s' after %d attempts",
-                          globals.num_retries) % (private, globals.num_retries))
-        raise BackendException("Error running '%s'" % private)
+                raise BackendException("Error running '%s': returned %d, with output:\n%s" %
+                                       (private, result, stdout + '\n' + stderr))
+        return result, stdout, stderr
+
+
+class BackendWrapper(object):
+    """
+    Represents a generic duplicity backend, capable of storing and
+    retrieving files.
+    """
+    
+    def __init__(self, backend):
+        self.backend = backend
+
+    def __do_put(self, source_path, remote_filename):
+        if hasattr(self.backend, '_put'):
+            log.Info(_("Writing %s") % remote_filename)
+            self.backend._put(source_path, remote_filename)
+        else:
+            raise NotImplementedError()
+
+    @retry('put', fatal=True)
+    def put(self, source_path, remote_filename=None):
+        """
+        Transfer source_path (Path object) to remote_filename (string)
+
+        If remote_filename is None, get the filename from the last
+        path component of pathname.
+        """
+        if not remote_filename:
+            remote_filename = source_path.get_filename()
+        self.__do_put(source_path, remote_filename)
+
+    @retry('move', fatal=True)
+    def move(self, source_path, remote_filename=None):
+        """
+        Move source_path (Path object) to remote_filename (string)
+
+        Same as put(), but unlinks source_path in the process.  This allows the
+        local backend to do this more efficiently using rename.
+        """
+        if not remote_filename:
+            remote_filename = source_path.get_filename()
+        if hasattr(self.backend, '_move'):
+            if self.backend._move(source_path, remote_filename) is not False:
+                source_path.setdata()
+                return
+        self.__do_put(source_path, remote_filename)
+        source_path.delete()
+
+    @retry('get', fatal=True)
+    def get(self, remote_filename, local_path):
+        """Retrieve remote_filename and place in local_path"""
+        if hasattr(self.backend, '_get'):
+            self.backend._get(remote_filename, local_path)
+            if not local_path.exists():
+                raise BackendException(_("File %s not found locally after get "
+                                         "from backend") % util.ufn(local_path.name))
+            local_path.setdata()
+        else:
+            raise NotImplementedError()
+
+    @retry('list', fatal=True)
+    def list(self):
+        """
+        Return list of filenames (byte strings) present in backend
+        """
+        def tobytes(filename):
+            "Convert a (maybe unicode) filename to bytes"
+            if isinstance(filename, unicode):
+                # There shouldn't be any encoding errors for files we care
+                # about, since duplicity filenames are ascii.  But user files
+                # may be in the same directory.  So just replace characters.
+                return filename.encode(sys.getfilesystemencoding(), 'replace')
+            else:
+                return filename
+
+        if hasattr(self.backend, '_list'):
+            # Make sure that duplicity internals only ever see byte strings
+            # for filenames, no matter what the backend thinks it is talking.
+            return [tobytes(x) for x in self.backend._list()]
+        else:
+            raise NotImplementedError()
+
+    def delete(self, filename_list):
+        """
+        Delete each filename in filename_list, in order if possible.
+        """
+        assert type(filename_list) is not types.StringType
+        if hasattr(self.backend, '_delete_list'):
+            self._do_delete_list(filename_list)
+        elif hasattr(self.backend, '_delete'):
+            for filename in filename_list:
+                self._do_delete(filename)
+        else:
+            raise NotImplementedError()
+
+    @retry('delete', fatal=False)
+    def _do_delete_list(self, filename_list):
+        self.backend._delete_list(filename_list)
+
+    @retry('delete', fatal=False)
+    def _do_delete(self, filename):
+        self.backend._delete(filename)
+
+    # Should never cause FatalError.
+    # Returns a dictionary of dictionaries.  The outer dictionary maps
+    # filenames to metadata dictionaries.  Supported metadata are:
+    #
+    # 'size': if >= 0, size of file
+    #         if -1, file is not found
+    #         if None, error querying file
+    #
+    # Returned dictionary is guaranteed to contain a metadata dictionary for
+    # each filename, and all metadata are guaranteed to be present.
+    def query_info(self, filename_list):
+        """
+        Return metadata about each filename in filename_list
+        """
+        info = {}
+        if hasattr(self.backend, '_query_list'):
+            info = self._do_query_list(filename_list)
+            if info is None:
+                info = {}
+        elif hasattr(self.backend, '_query'):
+            for filename in filename_list:
+                info[filename] = self._do_query(filename)
+
+        # Fill out any missing entries (may happen if backend has no support
+        # or its query_list support is lazy)
+        for filename in filename_list:
+            if filename not in info or info[filename] is None:
+                info[filename] = {}
+            for metadata in ['size']:
+                info[filename].setdefault(metadata, None)
+
+        return info
+
+    @retry('query', fatal=False)
+    def _do_query_list(self, filename_list):
+        info = self.backend._query_list(filename_list)
+        if info is None:
+            info = {}
+        return info
+
+    @retry('query', fatal=False)
+    def _do_query(self, filename):
+        try:
+            return self.backend._query(filename)
+        except Exception as e:
+            code = _get_code_from_exception(self.backend, 'query', e)
+            if code == log.ErrorCode.backend_not_found:
+                return {'size': -1}
+            else:
+                raise e
+
+    def close(self):
+        """
+        Close the backend, releasing any resources held and
+        invalidating any file objects obtained from the backend.
+        """
+        if hasattr(self.backend, '_close'):
+            self.backend._close()
 
     def get_fileobj_read(self, filename, parseresults = None):
         """
@@ -598,37 +658,6 @@
         tdp.setdata()
         return tdp.filtered_open_with_delete("rb")
 
-    def get_fileobj_write(self, filename,
-                          parseresults = None,
-                          sizelist = None):
-        """
-        Return fileobj opened for writing, which will cause the file
-        to be written to the backend on close().
-
-        The file will be encoded as specified in parseresults (or as
-        read from the filename), and stored in a temp file until it
-        can be copied over and deleted.
-
-        If sizelist is not None, it should be set to an empty list.
-        The number of bytes will be inserted into the list.
-        """
-        if not parseresults:
-            parseresults = file_naming.parse(filename)
-            assert parseresults, u"Filename %s not correctly parsed" % util.ufn(filename)
-        tdp = dup_temp.new_tempduppath(parseresults)
-
-        def close_file_hook():
-            """This is called when returned fileobj is closed"""
-            self.put(tdp, filename)
-            if sizelist is not None:
-                tdp.setdata()
-                sizelist.append(tdp.getsize())
-            tdp.delete()
-
-        fh = dup_temp.FileobjHooked(tdp.filtered_open("wb"))
-        fh.addhook(close_file_hook)
-        return fh
-
     def get_data(self, filename, parseresults = None):
         """
         Retrieve a file from backend, process it, return contents.
@@ -637,18 +666,3 @@
         buf = fin.read()
         assert not fin.close()
         return buf
-
-    def put_data(self, buffer, filename, parseresults = None):
-        """
-        Put buffer into filename on backend after processing.
-        """
-        fout = self.get_fileobj_write(filename, parseresults)
-        fout.write(buffer)
-        assert not fout.close()
-
-    def close(self):
-        """
-        Close the backend, releasing any resources held and
-        invalidating any file objects obtained from the backend.
-        """
-        pass

=== added file 'duplicity/backends/README'
--- duplicity/backends/README	1970-01-01 00:00:00 +0000
+++ duplicity/backends/README	2014-04-28 02:49:55 +0000
@@ -0,0 +1,79 @@
+= How to write a backend, in five easy steps! =
+
+There are five main methods you want to implement:
+
+__init__  - Initial setup
+_get
+ - Get one file
+ - Retried if an exception is thrown
+_put
+ - Upload one file
+ - Retried if an exception is thrown
+_list
+ - List all files in the backend
+ - Return a list of filenames
+ - Retried if an exception is thrown
+_delete
+ - Delete one file
+ - Retried if an exception is thrown
+
+There are other methods you may optionally implement:
+
+_delete_list
+ - Delete list of files
+ - This is used in preference of _delete if defined
+ - Must gracefully handle individual file errors itself
+ - Retried if an exception is thrown
+_query
+ - Query metadata of one file
+ - Return a dict with a 'size' key, and a file size value (-1 for not found)
+ - Retried if an exception is thrown
+_query_list
+ - Query metadata of a list of files
+ - Return a dict of filenames mapping to a dict with a 'size' key,
+   and a file size value (-1 for not found)
+ - This is used in preference of _query if defined
+ - Must gracefully handle individual file errors itself
+ - Retried if an exception is thrown
+_retry_cleanup
+ - If the backend wants to do any bookkeeping or connection resetting inbetween
+   retries, do it here.
+_error_code
+ - Passed an exception thrown by your backend, return a log.ErrorCode that
+   corresponds to that exception
+_move
+ - If your backend can more optimally move a local file into its backend,
+   implement this.  If it's not implemented or returns False, _put will be
+   called instead (and duplicity will delete the source file after).
+ - Retried if an exception is thrown
+_close
+ - If your backend needs to clean up after itself, do that here.
+
+== Subclassing ==
+
+Always subclass from duplicity.backend.Backend
+
+== Registering ==
+
+You can register your class as a single backend like so:
+
+duplicity.backend.register_backend("foo", FooBackend)
+
+This will allow a URL like so: foo://hostname/path
+
+Or you can register your class as a meta backend like so:
+duplicity.backend.register_backend_prefix("bar", BarBackend)
+
+Which will allow a URL like so: bar+foo://hostname/path and your class will
+be passed the inner URL to either interpret how you like or create a new
+inner backend instance with duplicity.backend.get_backend_object(url).
+
+== Naming ==
+
+Any method that duplicity calls will start with one underscore.  Please use
+zero or two underscores in your method names to avoid conflicts.
+
+== Testing ==
+
+Use "./testing/manual/backendtest.py foo://hostname/path" to test your new
+backend.  It will load your backend from your current branch.

=== modified file 'duplicity/backends/_boto_multi.py'
--- duplicity/backends/_boto_multi.py	2014-04-17 21:54:04 +0000
+++ duplicity/backends/_boto_multi.py	2014-04-28 02:49:55 +0000
@@ -98,8 +98,8 @@
 
         self._pool = multiprocessing.Pool(processes=number_of_procs)
 
-    def close(self):
-        BotoSingleBackend.close(self)
+    def _close(self):
+        BotoSingleBackend._close(self)
         log.Debug("Closing pool")
         self._pool.terminate()
         self._pool.join()

=== modified file 'duplicity/backends/_boto_single.py'
--- duplicity/backends/_boto_single.py	2014-04-25 23:20:12 +0000
+++ duplicity/backends/_boto_single.py	2014-04-28 02:49:55 +0000
@@ -25,9 +25,7 @@
 import duplicity.backend
 from duplicity import globals
 from duplicity import log
-from duplicity.errors import * #@UnusedWildImport
-from duplicity.util import exception_traceback
-from duplicity.backend import retry
+from duplicity.errors import FatalBackendException, BackendException
 from duplicity import progress
 
 BOTO_MIN_VERSION = "2.1.1"
@@ -163,7 +161,7 @@
         self.resetConnection()
         self._listed_keys = {}
 
-    def close(self):
+    def _close(self):
         del self._listed_keys
         self._listed_keys = {}
         self.bucket = None
@@ -185,137 +183,69 @@
         self.conn = get_connection(self.scheme, self.parsed_url, self.storage_uri)
         self.bucket = self.conn.lookup(self.bucket_name)
 
-    def put(self, source_path, remote_filename=None):
+    def _retry_cleanup(self):
+        self.resetConnection()
+
+    def _put(self, source_path, remote_filename):
         from boto.s3.connection import Location
         if globals.s3_european_buckets:
             if not globals.s3_use_new_style:
-                log.FatalError("European bucket creation was requested, but not new-style "
-                               "bucket addressing (--s3-use-new-style)",
-                               log.ErrorCode.s3_bucket_not_style)
-        #Network glitch may prevent first few attempts of creating/looking up a bucket
-        for n in range(1, globals.num_retries+1):
-            if self.bucket:
-                break
-            if n > 1:
-                time.sleep(30)
-                self.resetConnection()
+                raise FatalBackendException("European bucket creation was requested, but not new-style "
+                                            "bucket addressing (--s3-use-new-style)",
+                                            code=log.ErrorCode.s3_bucket_not_style)
+
+        if self.bucket is None:
             try:
-                try:
-                    self.bucket = self.conn.get_bucket(self.bucket_name, validate=True)
-                except Exception as e:
-                    if "NoSuchBucket" in str(e):
-                        if globals.s3_european_buckets:
-                            self.bucket = self.conn.create_bucket(self.bucket_name,
-                                                                  location=Location.EU)
-                        else:
-                            self.bucket = self.conn.create_bucket(self.bucket_name)
+                self.bucket = self.conn.get_bucket(self.bucket_name, validate=True)
+            except Exception as e:
+                if "NoSuchBucket" in str(e):
+                    if globals.s3_european_buckets:
+                        self.bucket = self.conn.create_bucket(self.bucket_name,
+                                                              location=Location.EU)
                     else:
-                        raise e
-            except Exception as e:
-                log.Warn("Failed to create bucket (attempt #%d) '%s' failed (reason: %s: %s)"
-                         "" % (n, self.bucket_name,
-                               e.__class__.__name__,
-                               str(e)))
+                        self.bucket = self.conn.create_bucket(self.bucket_name)
+                else:
+                    raise
 
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
         key = self.bucket.new_key(self.key_prefix + remote_filename)
 
-        for n in range(1, globals.num_retries+1):
-            if n > 1:
-                # sleep before retry (new connection to a **hopeful** new host, so no need to wait so long)
-                time.sleep(10)
-
-            if globals.s3_use_rrs:
-                storage_class = 'REDUCED_REDUNDANCY'
-            else:
-                storage_class = 'STANDARD'
-            log.Info("Uploading %s/%s to %s Storage" % (self.straight_url, remote_filename, storage_class))
-            try:
-                if globals.s3_use_sse:
-                    headers = {
-                    'Content-Type': 'application/octet-stream',
-                    'x-amz-storage-class': storage_class,
-                    'x-amz-server-side-encryption': 'AES256'
-                }
-                else:
-                    headers = {
-                    'Content-Type': 'application/octet-stream',
-                    'x-amz-storage-class': storage_class
-                }
-                
-                upload_start = time.time()
-                self.upload(source_path.name, key, headers)
-                upload_end = time.time()
-                total_s = abs(upload_end-upload_start) or 1  # prevent a zero value!
-                rough_upload_speed = os.path.getsize(source_path.name)/total_s
-                self.resetConnection()
-                log.Debug("Uploaded %s/%s to %s Storage at roughly %f bytes/second" % (self.straight_url, remote_filename, storage_class, rough_upload_speed))
-                return
-            except Exception as e:
-                log.Warn("Upload '%s/%s' failed (attempt #%d, reason: %s: %s)"
-                         "" % (self.straight_url,
-                               remote_filename,
-                               n,
-                               e.__class__.__name__,
-                               str(e)))
-                log.Debug("Backtrace of previous error: %s" % (exception_traceback(),))
-                self.resetConnection()
-        log.Warn("Giving up trying to upload %s/%s after %d attempts" %
-                 (self.straight_url, remote_filename, globals.num_retries))
-        raise BackendException("Error uploading %s/%s" % (self.straight_url, remote_filename))
-
-    def get(self, remote_filename, local_path):
+        if globals.s3_use_rrs:
+            storage_class = 'REDUCED_REDUNDANCY'
+        else:
+            storage_class = 'STANDARD'
+        log.Info("Uploading %s/%s to %s Storage" % (self.straight_url, remote_filename, storage_class))
+        if globals.s3_use_sse:
+            headers = {
+            'Content-Type': 'application/octet-stream',
+            'x-amz-storage-class': storage_class,
+            'x-amz-server-side-encryption': 'AES256'
+        }
+        else:
+            headers = {
+            'Content-Type': 'application/octet-stream',
+            'x-amz-storage-class': storage_class
+        }
+        
+        upload_start = time.time()
+        self.upload(source_path.name, key, headers)
+        upload_end = time.time()
+        total_s = abs(upload_end-upload_start) or 1  # prevent a zero value!
+        rough_upload_speed = os.path.getsize(source_path.name)/total_s
+        log.Debug("Uploaded %s/%s to %s Storage at roughly %f bytes/second" % (self.straight_url, remote_filename, storage_class, rough_upload_speed))
+
+    def _get(self, remote_filename, local_path):
         key_name = self.key_prefix + remote_filename
         self.pre_process_download(remote_filename, wait=True)
         key = self._listed_keys[key_name]
-        for n in range(1, globals.num_retries+1):
-            if n > 1:
-                # sleep before retry (new connection to a **hopeful** new host, so no need to wait so long)
-                time.sleep(10)
-            log.Info("Downloading %s/%s" % (self.straight_url, remote_filename))
-            try:
-                self.resetConnection()
-                key.get_contents_to_filename(local_path.name)
-                local_path.setdata()
-                return
-            except Exception as e:
-                log.Warn("Download %s/%s failed (attempt #%d, reason: %s: %s)"
-                         "" % (self.straight_url,
-                               remote_filename,
-                               n,
-                               e.__class__.__name__,
-                               str(e)), 1)
-                log.Debug("Backtrace of previous error: %s" % (exception_traceback(),))
-
-        log.Warn("Giving up trying to download %s/%s after %d attempts" %
-                (self.straight_url, remote_filename, globals.num_retries))
-        raise BackendException("Error downloading %s/%s" % (self.straight_url, remote_filename))
+        self.resetConnection()
+        key.get_contents_to_filename(local_path.name)
 
     def _list(self):
         if not self.bucket:
             raise BackendException("No connection to backend")
-
-        for n in range(1, globals.num_retries+1):
-            if n > 1:
-                # sleep before retry
-                time.sleep(30)
-                self.resetConnection()
-            log.Info("Listing %s" % self.straight_url)
-            try:
-                return self._list_filenames_in_bucket()
-            except Exception as e:
-                log.Warn("List %s failed (attempt #%d, reason: %s: %s)"
-                         "" % (self.straight_url,
-                               n,
-                               e.__class__.__name__,
-                               str(e)), 1)
-                log.Debug("Backtrace of previous error: %s" % (exception_traceback(),))
-        log.Warn("Giving up trying to list %s after %d attempts" %
-                (self.straight_url, globals.num_retries))
-        raise BackendException("Error listng %s" % self.straight_url)
-
-    def _list_filenames_in_bucket(self):
+        return self.list_filenames_in_bucket()
+
+    def list_filenames_in_bucket(self):
         # We add a 'd' to the prefix to make sure it is not null (for boto) and
         # to optimize the listing of our filenames, which always begin with 'd'.
         # This will cause a failure in the regression tests as below:
@@ -336,76 +266,37 @@
                 pass
         return filename_list
 
-    def delete(self, filename_list):
-        for filename in filename_list:
-            self.bucket.delete_key(self.key_prefix + filename)
-            log.Debug("Deleted %s/%s" % (self.straight_url, filename))
+    def _delete(self, filename):
+        self.bucket.delete_key(self.key_prefix + filename)
 
-    @retry
-    def _query_file_info(self, filename, raise_errors=False):
-        try:
-            key = self.bucket.lookup(self.key_prefix + filename)
-            if key is None:
-                return {'size': -1}
-            return {'size': key.size}
-        except Exception as e:
-            log.Warn("Query %s/%s failed: %s"
-                     "" % (self.straight_url,
-                           filename,
-                           str(e)))
-            self.resetConnection()
-            if raise_errors:
-                raise e
-            else:
-                return {'size': None}
+    def _query(self, filename):
+        key = self.bucket.lookup(self.key_prefix + filename)
+        if key is None:
+            return {'size': -1}
+        return {'size': key.size}
 
     def upload(self, filename, key, headers):
-            key.set_contents_from_filename(filename, headers,
-                                           cb=progress.report_transfer,
-                                           num_cb=(max(2, 8 * globals.volsize / (1024 * 1024)))
-                                           )  # Max num of callbacks = 8 times x megabyte
-            key.close()
+        key.set_contents_from_filename(filename, headers,
+                                       cb=progress.report_transfer,
+                                       num_cb=(max(2, 8 * globals.volsize / (1024 * 1024)))
+                                       )  # Max num of callbacks = 8 times x megabyte
+        key.close()
 
-    def pre_process_download(self, files_to_download, wait=False):
+    def pre_process_download(self, remote_filename, wait=False):
         # Used primarily to move files in Glacier to S3
-        if isinstance(files_to_download, (bytes, str, unicode)):
-            files_to_download = [files_to_download]
+        key_name = self.key_prefix + remote_filename
+        if not self._listed_keys.get(key_name, False):
+            self._listed_keys[key_name] = list(self.bucket.list(key_name))[0]
+        key = self._listed_keys[key_name]
 
-        for remote_filename in files_to_download:
-            success = False
-            for n in range(1, globals.num_retries+1):
-                if n > 1:
-                    # sleep before retry (new connection to a **hopeful** new host, so no need to wait so long)
-                    time.sleep(10)
+        if key.storage_class == "GLACIER":
+            # We need to move the file out of glacier
+            if not self.bucket.get_key(key.key).ongoing_restore:
+                log.Info("File %s is in Glacier storage, restoring to S3" % remote_filename)
+                key.restore(days=1)  # Shouldn't need this again after 1 day
+            if wait:
+                log.Info("Waiting for file %s to restore from Glacier" % remote_filename)
+                while self.bucket.get_key(key.key).ongoing_restore:
+                    time.sleep(60)
                     self.resetConnection()
-                try:
-                    key_name = self.key_prefix + remote_filename
-                    if not self._listed_keys.get(key_name, False):
-                        self._listed_keys[key_name] = list(self.bucket.list(key_name))[0]
-                    key = self._listed_keys[key_name]
-
-                    if key.storage_class == "GLACIER":
-                        # We need to move the file out of glacier
-                        if not self.bucket.get_key(key.key).ongoing_restore:
-                            log.Info("File %s is in Glacier storage, restoring to S3" % remote_filename)
-                            key.restore(days=1)  # Shouldn't need this again after 1 day
-                        if wait:
-                            log.Info("Waiting for file %s to restore from Glacier" % remote_filename)
-                            while self.bucket.get_key(key.key).ongoing_restore:
-                                time.sleep(60)
-                                self.resetConnection()
-                            log.Info("File %s was successfully restored from Glacier" % remote_filename)
-                    success = True
-                    break
-                except Exception as e:
-                    log.Warn("Restoration from Glacier for file %s/%s failed (attempt #%d, reason: %s: %s)"
-                             "" % (self.straight_url,
-                                   remote_filename,
-                                   n,
-                                   e.__class__.__name__,
-                                   str(e)), 1)
-                    log.Debug("Backtrace of previous error: %s" % (exception_traceback(),))
-            if not success:
-                log.Warn("Giving up trying to restore %s/%s after %d attempts" %
-                        (self.straight_url, remote_filename, globals.num_retries))
-                raise BackendException("Error restoring %s/%s from Glacier to S3" % (self.straight_url, remote_filename))
+                log.Info("File %s was successfully restored from Glacier" % remote_filename)

=== modified file 'duplicity/backends/_cf_cloudfiles.py'
--- duplicity/backends/_cf_cloudfiles.py	2014-04-17 22:03:10 +0000
+++ duplicity/backends/_cf_cloudfiles.py	2014-04-28 02:49:55 +0000
@@ -19,14 +19,10 @@
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
 import os
-import time
 
 import duplicity.backend
-from duplicity import globals
 from duplicity import log
-from duplicity.errors import * #@UnusedWildImport
-from duplicity.util import exception_traceback
-from duplicity.backend import retry
+from duplicity.errors import BackendException
 
 class CloudFilesBackend(duplicity.backend.Backend):
     """
@@ -69,124 +65,37 @@
                            log.ErrorCode.connection_failed)
         self.container = conn.create_container(container)
 
-    def put(self, source_path, remote_filename = None):
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
-
-        for n in range(1, globals.num_retries+1):
-            log.Info("Uploading '%s/%s' " % (self.container, remote_filename))
-            try:
-                sobject = self.container.create_object(remote_filename)
-                sobject.load_from_filename(source_path.name)
-                return
-            except self.resp_exc as error:
-                log.Warn("Upload of '%s' failed (attempt %d): CloudFiles returned: %s %s"
-                         % (remote_filename, n, error.status, error.reason))
-            except Exception as e:
-                log.Warn("Upload of '%s' failed (attempt %s): %s: %s"
-                        % (remote_filename, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up uploading '%s' after %s attempts"
-                 % (remote_filename, globals.num_retries))
-        raise BackendException("Error uploading '%s'" % remote_filename)
-
-    def get(self, remote_filename, local_path):
-        for n in range(1, globals.num_retries+1):
-            log.Info("Downloading '%s/%s'" % (self.container, remote_filename))
-            try:
-                sobject = self.container.create_object(remote_filename)
-                f = open(local_path.name, 'w')
-                for chunk in sobject.stream():
-                    f.write(chunk)
-                local_path.setdata()
-                return
-            except self.resp_exc as resperr:
-                log.Warn("Download of '%s' failed (attempt %s): CloudFiles returned: %s %s"
-                         % (remote_filename, n, resperr.status, resperr.reason))
-            except Exception as e:
-                log.Warn("Download of '%s' failed (attempt %s): %s: %s"
-                         % (remote_filename, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up downloading '%s' after %s attempts"
-                 % (remote_filename, globals.num_retries))
-        raise BackendException("Error downloading '%s/%s'"
-                               % (self.container, remote_filename))
+    def _error_code(self, operation, e):
+        from cloudfiles.errors import NoSuchObject
+        if isinstance(e, NoSuchObject):
+            return log.ErrorCode.backend_not_found
+        elif isinstance(e, self.resp_exc):
+            if e.status == 404:
+                return log.ErrorCode.backend_not_found
+
+    def _put(self, source_path, remote_filename):
+        sobject = self.container.create_object(remote_filename)
+        sobject.load_from_filename(source_path.name)
+
+    def _get(self, remote_filename, local_path):
+        sobject = self.container.create_object(remote_filename)
+        with open(local_path.name, 'wb') as f:
+            for chunk in sobject.stream():
+                f.write(chunk)
 
     def _list(self):
-        for n in range(1, globals.num_retries+1):
-            log.Info("Listing '%s'" % (self.container))
-            try:
-                # Cloud Files will return a max of 10,000 objects.  We have
-                # to make multiple requests to get them all.
-                objs = self.container.list_objects()
-                keys = objs
-                while len(objs) == 10000:
-                    objs = self.container.list_objects(marker=keys[-1])
-                    keys += objs
-                return keys
-            except self.resp_exc as resperr:
-                log.Warn("Listing of '%s' failed (attempt %s): CloudFiles returned: %s %s"
-                         % (self.container, n, resperr.status, resperr.reason))
-            except Exception as e:
-                log.Warn("Listing of '%s' failed (attempt %s): %s: %s"
-                         % (self.container, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up listing of '%s' after %s attempts"
-                 % (self.container, globals.num_retries))
-        raise BackendException("Error listing '%s'"
-                               % (self.container))
-
-    def delete_one(self, remote_filename):
-        for n in range(1, globals.num_retries+1):
-            log.Info("Deleting '%s/%s'" % (self.container, remote_filename))
-            try:
-                self.container.delete_object(remote_filename)
-                return
-            except self.resp_exc as resperr:
-                if n > 1 and resperr.status == 404:
-                    # We failed on a timeout, but delete succeeded on the server
-                    log.Warn("Delete of '%s' missing after retry - must have succeded earler" % remote_filename )
-                    return
-                log.Warn("Delete of '%s' failed (attempt %s): CloudFiles returned: %s %s"
-                         % (remote_filename, n, resperr.status, resperr.reason))
-            except Exception as e:
-                log.Warn("Delete of '%s' failed (attempt %s): %s: %s"
-                         % (remote_filename, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up deleting '%s' after %s attempts"
-                 % (remote_filename, globals.num_retries))
-        raise BackendException("Error deleting '%s/%s'"
-                               % (self.container, remote_filename))
-
-    def delete(self, filename_list):
-        for file in filename_list:
-            self.delete_one(file)
-            log.Debug("Deleted '%s/%s'" % (self.container, file))
-
-    @retry
-    def _query_file_info(self, filename, raise_errors=False):
-        from cloudfiles.errors import NoSuchObject
-        try:
-            sobject = self.container.get_object(filename)
-            return {'size': sobject.size}
-        except NoSuchObject:
-            return {'size': -1}
-        except Exception as e:
-            log.Warn("Error querying '%s/%s': %s"
-                     "" % (self.container,
-                           filename,
-                           str(e)))
-            if raise_errors:
-                raise e
-            else:
-                return {'size': None}
-
-duplicity.backend.register_backend("cf+http", CloudFilesBackend)
+        # Cloud Files will return a max of 10,000 objects.  We have
+        # to make multiple requests to get them all.
+        objs = self.container.list_objects()
+        keys = objs
+        while len(objs) == 10000:
+            objs = self.container.list_objects(marker=keys[-1])
+            keys += objs
+        return keys
+
+    def _delete(self, filename):
+        self.container.delete_object(filename)
+
+    def _query(self, filename):
+        sobject = self.container.get_object(filename)
+        return {'size': sobject.size}

=== modified file 'duplicity/backends/_cf_pyrax.py'
--- duplicity/backends/_cf_pyrax.py	2014-04-17 22:03:10 +0000
+++ duplicity/backends/_cf_pyrax.py	2014-04-28 02:49:55 +0000
@@ -19,14 +19,11 @@
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
 import os
-import time
 
 import duplicity.backend
-from duplicity import globals
 from duplicity import log
-from duplicity.errors import *  # @UnusedWildImport
-from duplicity.util import exception_traceback
-from duplicity.backend import retry
+from duplicity.errors import BackendException
+
 
 class PyraxBackend(duplicity.backend.Backend):
     """
@@ -69,126 +66,39 @@
 
         self.client_exc = pyrax.exceptions.ClientException
         self.nso_exc = pyrax.exceptions.NoSuchObject
-        self.cloudfiles = pyrax.cloudfiles
         self.container = pyrax.cloudfiles.create_container(container)
 
-    def put(self, source_path, remote_filename = None):
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
-
-        for n in range(1, globals.num_retries + 1):
-            log.Info("Uploading '%s/%s' " % (self.container, remote_filename))
-            try:
-                self.container.upload_file(source_path.name, remote_filename)
-                return
-            except self.client_exc as error:
-                log.Warn("Upload of '%s' failed (attempt %d): pyrax returned: %s %s"
-                         % (remote_filename, n, error.__class__.__name__, error.message))
-            except Exception as e:
-                log.Warn("Upload of '%s' failed (attempt %s): %s: %s"
-                        % (remote_filename, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up uploading '%s' after %s attempts"
-                 % (remote_filename, globals.num_retries))
-        raise BackendException("Error uploading '%s'" % remote_filename)
-
-    def get(self, remote_filename, local_path):
-        for n in range(1, globals.num_retries + 1):
-            log.Info("Downloading '%s/%s'" % (self.container, remote_filename))
-            try:
-                sobject = self.container.get_object(remote_filename)
-                f = open(local_path.name, 'w')
-                f.write(sobject.get())
-                local_path.setdata()
-                return
-            except self.nso_exc:
-                return
-            except self.client_exc as resperr:
-                log.Warn("Download of '%s' failed (attempt %s): pyrax returned: %s %s"
-                         % (remote_filename, n, resperr.__class__.__name__, resperr.message))
-            except Exception as e:
-                log.Warn("Download of '%s' failed (attempt %s): %s: %s"
-                         % (remote_filename, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up downloading '%s' after %s attempts"
-                 % (remote_filename, globals.num_retries))
-        raise BackendException("Error downloading '%s/%s'"
-                               % (self.container, remote_filename))
+    def _error_code(self, operation, e):
+        if isinstance(e, self.nso_exc):
+            return log.ErrorCode.backend_not_found
+        elif isinstance(e, self.client_exc):
+            if e.status == 404:
+                return log.ErrorCode.backend_not_found
+        elif hasattr(e, 'http_status'):
+            if e.http_status == 404:
+                return log.ErrorCode.backend_not_found
+
+    def _put(self, source_path, remote_filename):
+        self.container.upload_file(source_path.name, remote_filename)
+
+    def _get(self, remote_filename, local_path):
+        sobject = self.container.get_object(remote_filename)
+        with open(local_path.name, 'wb') as f:
+            f.write(sobject.get())
 
     def _list(self):
-        for n in range(1, globals.num_retries + 1):
-            log.Info("Listing '%s'" % (self.container))
-            try:
-                # Cloud Files will return a max of 10,000 objects.  We have
-                # to make multiple requests to get them all.
-                objs = self.container.get_object_names()
-                keys = objs
-                while len(objs) == 10000:
-                    objs = self.container.get_object_names(marker = keys[-1])
-                    keys += objs
-                return keys
-            except self.client_exc as resperr:
-                log.Warn("Listing of '%s' failed (attempt %s): pyrax returned: %s %s"
-                         % (self.container, n, resperr.__class__.__name__, resperr.message))
-            except Exception as e:
-                log.Warn("Listing of '%s' failed (attempt %s): %s: %s"
-                         % (self.container, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up listing of '%s' after %s attempts"
-                 % (self.container, globals.num_retries))
-        raise BackendException("Error listing '%s'"
-                               % (self.container))
-
-    def delete_one(self, remote_filename):
-        for n in range(1, globals.num_retries + 1):
-            log.Info("Deleting '%s/%s'" % (self.container, remote_filename))
-            try:
-                self.container.delete_object(remote_filename)
-                return
-            except self.client_exc as resperr:
-                if n > 1 and resperr.status == 404:
-                    # We failed on a timeout, but delete succeeded on the server
-                    log.Warn("Delete of '%s' missing after retry - must have succeded earler" % remote_filename)
-                    return
-                log.Warn("Delete of '%s' failed (attempt %s): pyrax returned: %s %s"
-                         % (remote_filename, n, resperr.__class__.__name__, resperr.message))
-            except Exception as e:
-                log.Warn("Delete of '%s' failed (attempt %s): %s: %s"
-                         % (remote_filename, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up deleting '%s' after %s attempts"
-                 % (remote_filename, globals.num_retries))
-        raise BackendException("Error deleting '%s/%s'"
-                               % (self.container, remote_filename))
-
-    def delete(self, filename_list):
-        for file_ in filename_list:
-            self.delete_one(file_)
-            log.Debug("Deleted '%s/%s'" % (self.container, file_))
-
-    @retry
-    def _query_file_info(self, filename, raise_errors = False):
-        try:
-            sobject = self.container.get_object(filename)
-            return {'size': sobject.total_bytes}
-        except self.nso_exc:
-            return {'size': -1}
-        except Exception as e:
-            log.Warn("Error querying '%s/%s': %s"
-                     "" % (self.container,
-                           filename,
-                           str(e)))
-            if raise_errors:
-                raise e
-            else:
-                return {'size': None}
-
-duplicity.backend.register_backend("cf+http", PyraxBackend)
+        # Cloud Files will return a max of 10,000 objects.  We have
+        # to make multiple requests to get them all.
+        objs = self.container.get_object_names()
+        keys = objs
+        while len(objs) == 10000:
+            objs = self.container.get_object_names(marker = keys[-1])
+            keys += objs
+        return keys
+
+    def _delete(self, filename):
+        self.container.delete_object(filename)
+
+    def _query(self, filename):
+        sobject = self.container.get_object(filename)
+        return {'size': sobject.total_bytes}

=== modified file 'duplicity/backends/_ssh_paramiko.py'
--- duplicity/backends/_ssh_paramiko.py	2014-04-17 20:50:57 +0000
+++ duplicity/backends/_ssh_paramiko.py	2014-04-28 02:49:55 +0000
@@ -28,7 +28,6 @@
 import os
 import errno
 import sys
-import time
 import getpass
 import logging
 from binascii import hexlify
@@ -36,7 +35,7 @@
 import duplicity.backend
 from duplicity import globals
 from duplicity import log
-from duplicity.errors import *
+from duplicity.errors import BackendException
 
 read_blocksize=65635            # for doing scp retrievals, where we need to read ourselves
 
@@ -232,7 +231,6 @@
             except Exception as e:
                 raise BackendException("sftp negotiation failed: %s" % e)
 
-
             # move to the appropriate directory, possibly after creating it and its parents
             dirs = self.remote_dir.split(os.sep)
             if len(dirs) > 0:
@@ -257,157 +255,91 @@
                     except Exception as e:
                         raise BackendException("sftp chdir to %s failed: %s" % (self.sftp.normalize(".")+"/"+d,e))
 
-    def put(self, source_path, remote_filename = None):
-        """transfers a single file to the remote side.
-        In scp mode unavoidable quoting issues will make this fail if the remote directory or file name
-        contain single quotes."""
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
-        
-        for n in range(1, globals.num_retries+1):
-            if n > 1:
-                # sleep before retry
-                time.sleep(self.retry_delay)
-            try:
-                if (globals.use_scp):
-                    f=file(source_path.name,'rb')
-                    try:
-                        chan=self.client.get_transport().open_session()
-                        chan.settimeout(globals.timeout)
-                        chan.exec_command("scp -t '%s'" % self.remote_dir) # scp in sink mode uses the arg as base directory
-                    except Exception as e:
-                        raise BackendException("scp execution failed: %s" % e)
-                    # scp protocol: one 0x0 after startup, one after the Create meta, one after saving
-                    # if there's a problem: 0x1 or 0x02 and some error text
-                    response=chan.recv(1)
-                    if (response!="\0"):
-                        raise BackendException("scp remote error: %s" % chan.recv(-1))
-                    fstat=os.stat(source_path.name)
-                    chan.send('C%s %d %s\n' %(oct(fstat.st_mode)[-4:], fstat.st_size, remote_filename))
-                    response=chan.recv(1)
-                    if (response!="\0"):
-                        raise BackendException("scp remote error: %s" % chan.recv(-1))
-                    chan.sendall(f.read()+'\0')
-                    f.close()
-                    response=chan.recv(1)
-                    if (response!="\0"):
-                        raise BackendException("scp remote error: %s" % chan.recv(-1))
-                    chan.close()
-                    return
-                else:
-                    try:
-                        self.sftp.put(source_path.name,remote_filename)
-                        return
-                    except Exception as e:
-                        raise BackendException("sftp put of %s (as %s) failed: %s" % (source_path.name,remote_filename,e))
-            except Exception as e:
-                log.Warn("%s (Try %d of %d) Will retry in %d seconds." % (e,n,globals.num_retries,self.retry_delay))
-        raise BackendException("Giving up trying to upload '%s' after %d attempts" % (remote_filename,n))
-
-
-    def get(self, remote_filename, local_path):
-        """retrieves a single file from the remote side.
-        In scp mode unavoidable quoting issues will make this fail if the remote directory or file names
-        contain single quotes."""
-        
-        for n in range(1, globals.num_retries+1):
-            if n > 1:
-                # sleep before retry
-                time.sleep(self.retry_delay)
-            try:
-                if (globals.use_scp):
-                    try:
-                        chan=self.client.get_transport().open_session()
-                        chan.settimeout(globals.timeout)
-                        chan.exec_command("scp -f '%s/%s'" % (self.remote_dir,remote_filename))
-                    except Exception as e:
-                        raise BackendException("scp execution failed: %s" % e)
-
-                    chan.send('\0')     # overall ready indicator
-                    msg=chan.recv(-1)
-                    m=re.match(r"C([0-7]{4})\s+(\d+)\s+(\S.*)$",msg)
-                    if (m==None or m.group(3)!=remote_filename):
-                        raise BackendException("scp get %s failed: incorrect response '%s'" % (remote_filename,msg))
-                    chan.recv(1)        # dispose of the newline trailing the C message
-
-                    size=int(m.group(2))
-                    togo=size
-                    f=file(local_path.name,'wb')
-                    chan.send('\0')     # ready for data
-                    try:
-                        while togo>0:
-                            if togo>read_blocksize:
-                                blocksize = read_blocksize
-                            else:
-                                blocksize = togo
-                            buff=chan.recv(blocksize)
-                            f.write(buff)
-                            togo-=len(buff)
-                    except Exception as e:
-                        raise BackendException("scp get %s failed: %s" % (remote_filename,e))
-
-                    msg=chan.recv(1)    # check the final status
-                    if msg!='\0':
-                        raise BackendException("scp get %s failed: %s" % (remote_filename,chan.recv(-1)))
-                    f.close()
-                    chan.send('\0')     # send final done indicator
-                    chan.close()
-                    return
-                else:
-                    try:
-                        self.sftp.get(remote_filename,local_path.name)
-                        return
-                    except Exception as e:
-                        raise BackendException("sftp get of %s (to %s) failed: %s" % (remote_filename,local_path.name,e))
-                local_path.setdata()
-            except Exception as e:
-                log.Warn("%s (Try %d of %d) Will retry in %d seconds." % (e,n,globals.num_retries,self.retry_delay))
-        raise BackendException("Giving up trying to download '%s' after %d attempts" % (remote_filename,n))
+    def _put(self, source_path, remote_filename):
+        if globals.use_scp:
+            f=file(source_path.name,'rb')
+            try:
+                chan=self.client.get_transport().open_session()
+                chan.settimeout(globals.timeout)
+                chan.exec_command("scp -t '%s'" % self.remote_dir) # scp in sink mode uses the arg as base directory
+            except Exception as e:
+                raise BackendException("scp execution failed: %s" % e)
+            # scp protocol: one 0x0 after startup, one after the Create meta, one after saving
+            # if there's a problem: 0x1 or 0x02 and some error text
+            response=chan.recv(1)
+            if (response!="\0"):
+                raise BackendException("scp remote error: %s" % chan.recv(-1))
+            fstat=os.stat(source_path.name)
+            chan.send('C%s %d %s\n' %(oct(fstat.st_mode)[-4:], fstat.st_size, remote_filename))
+            response=chan.recv(1)
+            if (response!="\0"):
+                raise BackendException("scp remote error: %s" % chan.recv(-1))
+            chan.sendall(f.read()+'\0')
+            f.close()
+            response=chan.recv(1)
+            if (response!="\0"):
+                raise BackendException("scp remote error: %s" % chan.recv(-1))
+            chan.close()
+        else:
+            self.sftp.put(source_path.name,remote_filename)
+
+    def _get(self, remote_filename, local_path):
+        if globals.use_scp:
+            try:
+                chan=self.client.get_transport().open_session()
+                chan.settimeout(globals.timeout)
+                chan.exec_command("scp -f '%s/%s'" % (self.remote_dir,remote_filename))
+            except Exception as e:
+                raise BackendException("scp execution failed: %s" % e)
+
+            chan.send('\0')     # overall ready indicator
+            msg=chan.recv(-1)
+            m=re.match(r"C([0-7]{4})\s+(\d+)\s+(\S.*)$",msg)
+            if (m==None or m.group(3)!=remote_filename):
+                raise BackendException("scp get %s failed: incorrect response '%s'" % (remote_filename,msg))
+            chan.recv(1)        # dispose of the newline trailing the C message
+
+            size=int(m.group(2))
+            togo=size
+            f=file(local_path.name,'wb')
+            chan.send('\0')     # ready for data
+            try:
+                while togo>0:
+                    if togo>read_blocksize:
+                        blocksize = read_blocksize
+                    else:
+                        blocksize = togo
+                    buff=chan.recv(blocksize)
+                    f.write(buff)
+                    togo-=len(buff)
+            except Exception as e:
+                raise BackendException("scp get %s failed: %s" % (remote_filename,e))
+
+            msg=chan.recv(1)    # check the final status
+            if msg!='\0':
+                raise BackendException("scp get %s failed: %s" % (remote_filename,chan.recv(-1)))
+            f.close()
+            chan.send('\0')     # send final done indicator
+            chan.close()
+        else:
+            self.sftp.get(remote_filename,local_path.name)
 
     def _list(self):
-        """lists the contents of the one-and-only duplicity dir on the remote side.
-        In scp mode unavoidable quoting issues will make this fail if the directory name
-        contains single quotes."""
-        for n in range(1, globals.num_retries+1):
-            if n > 1:
-                # sleep before retry
-                time.sleep(self.retry_delay)
-            try:
-                if (globals.use_scp):
-                    output=self.runremote("ls -1 '%s'" % self.remote_dir,False,"scp dir listing ")
-                    return output.splitlines()
-                else:
-                    try:
-                        return self.sftp.listdir()
-                    except Exception as e:
-                        raise BackendException("sftp listing of %s failed: %s" % (self.sftp.getcwd(),e))
-            except Exception as e:
-                log.Warn("%s (Try %d of %d) Will retry in %d seconds." % (e,n,globals.num_retries,self.retry_delay))
-        raise BackendException("Giving up trying to list '%s' after %d attempts" % (self.remote_dir,n))
-
-    def delete(self, filename_list):
-        """deletes all files in the list on the remote side. In scp mode unavoidable quoting issues
-        will cause failures if filenames containing single quotes are encountered."""
-        for fn in filename_list:
-            # Try to delete each file several times before giving up completely.
-            for n in range(1, globals.num_retries+1):
-                try:
-                    if (globals.use_scp):
-                        self.runremote("rm '%s/%s'" % (self.remote_dir,fn),False,"scp rm ")
-                    else:
-                        try:
-                            self.sftp.remove(fn)
-                        except Exception as e:
-                            raise BackendException("sftp rm %s failed: %s" % (fn,e))
-
-                    # If we get here, we deleted this file successfully. Move on to the next one.
-                    break
-                except Exception as e:
-                    if n == globals.num_retries:
-                        log.FatalError(str(e), log.ErrorCode.backend_error)
-                    else:
-                        log.Warn("%s (Try %d of %d) Will retry in %d seconds." % (e,n,globals.num_retries,self.retry_delay))
-                        time.sleep(self.retry_delay)
+        # In scp mode unavoidable quoting issues will make this fail if the
+        # directory name contains single quotes.
+        if globals.use_scp:
+            output = self.runremote("ls -1 '%s'" % self.remote_dir, False, "scp dir listing ")
+            return output.splitlines()
+        else:
+            return self.sftp.listdir()
+
+    def _delete(self, filename):
+        # In scp mode unavoidable quoting issues will cause failures if
+        # filenames containing single quotes are encountered.
+        if globals.use_scp:
+            self.runremote("rm '%s/%s'" % (self.remote_dir, filename), False, "scp rm ")
+        else:
+            self.sftp.remove(filename)
 
     def runremote(self,cmd,ignoreexitcode=False,errorprefix=""):
         """small convenience function that opens a shell channel, runs remote command and returns
@@ -438,7 +370,3 @@
             raise BackendException("could not load '%s', maybe corrupt?" % (file))
         
         return sshconfig.lookup(host)
-
-duplicity.backend.register_backend("sftp", SSHParamikoBackend)
-duplicity.backend.register_backend("scp", SSHParamikoBackend)
-duplicity.backend.register_backend("ssh", SSHParamikoBackend)

=== modified file 'duplicity/backends/_ssh_pexpect.py'
--- duplicity/backends/_ssh_pexpect.py	2014-04-25 23:20:12 +0000
+++ duplicity/backends/_ssh_pexpect.py	2014-04-28 02:49:55 +0000
@@ -24,18 +24,20 @@
 # have the same syntax.  Also these strings will be executed by the
 # shell, so shouldn't have strange characters in them.
 
+from future_builtins import map
+
 import re
 import string
-import time
 import os
 
 import duplicity.backend
 from duplicity import globals
 from duplicity import log
-from duplicity.errors import * #@UnusedWildImport
+from duplicity.errors import BackendException
 
 class SSHPExpectBackend(duplicity.backend.Backend):
-    """This backend copies files using scp.  List not supported"""
+    """This backend copies files using scp.  List not supported.  Filenames
+       should not need any quoting or this will break."""
     def __init__(self, parsed_url):
         """scpBackend initializer"""
         duplicity.backend.Backend.__init__(self, parsed_url)
@@ -76,74 +78,67 @@
     def run_scp_command(self, commandline):
         """ Run an scp command, responding to password prompts """
         import pexpect
-        for n in range(1, globals.num_retries+1):
-            if n > 1:
-                # sleep before retry
-                time.sleep(self.retry_delay)
-            log.Info("Running '%s' (attempt #%d)" % (commandline, n))
-            child = pexpect.spawn(commandline, timeout = None)
-            if globals.ssh_askpass:
-                state = "authorizing"
-            else:
-                state = "copying"
-            while 1:
-                if state == "authorizing":
-                    match = child.expect([pexpect.EOF,
-                                          "(?i)timeout, server not responding",
-                                          "(?i)pass(word|phrase .*):",
-                                          "(?i)permission denied",
-                                          "authenticity"])
-                    log.Debug("State = %s, Before = '%s'" % (state, child.before.strip()))
-                    if match == 0:
-                        log.Warn("Failed to authenticate")
-                        break
-                    elif match == 1:
-                        log.Warn("Timeout waiting to authenticate")
-                        break
-                    elif match == 2:
-                        child.sendline(self.password)
-                        state = "copying"
-                    elif match == 3:
-                        log.Warn("Invalid SSH password")
-                        break
-                    elif match == 4:
-                        log.Warn("Remote host authentication failed (missing known_hosts entry?)")
-                        break
-                elif state == "copying":
-                    match = child.expect([pexpect.EOF,
-                                          "(?i)timeout, server not responding",
-                                          "stalled",
-                                          "authenticity",
-                                          "ETA"])
-                    log.Debug("State = %s, Before = '%s'" % (state, child.before.strip()))
-                    if match == 0:
-                        break
-                    elif match == 1:
-                        log.Warn("Timeout waiting for response")
-                        break
-                    elif match == 2:
-                        state = "stalled"
-                    elif match == 3:
-                        log.Warn("Remote host authentication failed (missing known_hosts entry?)")
-                        break
-                elif state == "stalled":
-                    match = child.expect([pexpect.EOF,
-                                          "(?i)timeout, server not responding",
-                                          "ETA"])
-                    log.Debug("State = %s, Before = '%s'" % (state, child.before.strip()))
-                    if match == 0:
-                        break
-                    elif match == 1:
-                        log.Warn("Stalled for too long, aborted copy")
-                        break
-                    elif match == 2:
-                        state = "copying"
-            child.close(force = True)
-            if child.exitstatus == 0:
-                return
-            log.Warn("Running '%s' failed (attempt #%d)" % (commandline, n))
-        log.Warn("Giving up trying to execute '%s' after %d attempts" % (commandline, globals.num_retries))
-        raise BackendException("Error running '%s'" % commandline)
+        log.Info("Running '%s'" % commandline)
+        child = pexpect.spawn(commandline, timeout = None)
+        if globals.ssh_askpass:
+            state = "authorizing"
+        else:
+            state = "copying"
+        while 1:
+            if state == "authorizing":
+                match = child.expect([pexpect.EOF,
+                                      "(?i)timeout, server not responding",
+                                      "(?i)pass(word|phrase .*):",
+                                      "(?i)permission denied",
+                                      "authenticity"])
+                log.Debug("State = %s, Before = '%s'" % (state, child.before.strip()))
+                if match == 0:
+                    log.Warn("Failed to authenticate")
+                    break
+                elif match == 1:
+                    log.Warn("Timeout waiting to authenticate")
+                    break
+                elif match == 2:
+                    child.sendline(self.password)
+                    state = "copying"
+                elif match == 3:
+                    log.Warn("Invalid SSH password")
+                    break
+                elif match == 4:
+                    log.Warn("Remote host authentication failed (missing known_hosts entry?)")
+                    break
+            elif state == "copying":
+                match = child.expect([pexpect.EOF,
+                                      "(?i)timeout, server not responding",
+                                      "stalled",
+                                      "authenticity",
+                                      "ETA"])
+                log.Debug("State = %s, Before = '%s'" % (state, child.before.strip()))
+                if match == 0:
+                    break
+                elif match == 1:
+                    log.Warn("Timeout waiting for response")
+                    break
+                elif match == 2:
+                    state = "stalled"
+                elif match == 3:
+                    log.Warn("Remote host authentication failed (missing known_hosts entry?)")
+                    break
+            elif state == "stalled":
+                match = child.expect([pexpect.EOF,
+                                      "(?i)timeout, server not responding",
+                                      "ETA"])
+                log.Debug("State = %s, Before = '%s'" % (state, child.before.strip()))
+                if match == 0:
+                    break
+                elif match == 1:
+                    log.Warn("Stalled for too long, aborted copy")
+                    break
+                elif match == 2:
+                    state = "copying"
+        child.close(force = True)
+        if child.exitstatus != 0:
+            raise BackendException("Error running '%s'" % commandline)
 
     def run_sftp_command(self, commandline, commands):
         """ Run an sftp command, responding to password prompts, passing commands from list """
@@ -160,76 +155,69 @@
                      "Couldn't delete file",
                      "open(.*): Failure"]
         max_response_len = max([len(p) for p in responses[1:]])
-        for n in range(1, globals.num_retries+1):
-            if n > 1:
-                # sleep before retry
-                time.sleep(self.retry_delay)
-            log.Info("Running '%s' (attempt #%d)" % (commandline, n))
-            child = pexpect.spawn(commandline, timeout = None, maxread=maxread)
-            cmdloc = 0
-            passprompt = 0
-            while 1:
-                msg = ""
-                match = child.expect(responses,
-                                     searchwindowsize=maxread+max_response_len)
-                log.Debug("State = sftp, Before = '%s'" % (child.before.strip()))
-                if match == 0:
-                    break
-                elif match == 1:
-                    msg = "Timeout waiting for response"
-                    break
-                if match == 2:
-                    if cmdloc < len(commands):
-                        command = commands[cmdloc]
-                        log.Info("sftp command: '%s'" % (command,))
-                        child.sendline(command)
-                        cmdloc += 1
-                    else:
-                        command = 'quit'
-                        child.sendline(command)
-                        res = child.before
-                elif match == 3:
-                    passprompt += 1
-                    child.sendline(self.password)
-                    if (passprompt>1):
-                        raise BackendException("Invalid SSH password.")
-                elif match == 4:
-                    if not child.before.strip().startswith("mkdir"):
-                        msg = "Permission denied"
-                        break
-                elif match == 5:
-                    msg = "Host key authenticity could not be verified (missing known_hosts entry?)"
-                    break
-                elif match == 6:
-                    if not child.before.strip().startswith("rm"):
-                        msg = "Remote file or directory does not exist in command='%s'" % (commandline,)
-                        break
-                elif match == 7:
-                    if not child.before.strip().startswith("Removing"):
-                        msg = "Could not delete file in command='%s'" % (commandline,)
-                        break;
-                elif match == 8:
+        log.Info("Running '%s'" % (commandline))
+        child = pexpect.spawn(commandline, timeout = None, maxread=maxread)
+        cmdloc = 0
+        passprompt = 0
+        while 1:
+            msg = ""
+            match = child.expect(responses,
+                                 searchwindowsize=maxread+max_response_len)
+            log.Debug("State = sftp, Before = '%s'" % (child.before.strip()))
+            if match == 0:
+                break
+            elif match == 1:
+                msg = "Timeout waiting for response"
+                break
+            if match == 2:
+                if cmdloc < len(commands):
+                    command = commands[cmdloc]
+                    log.Info("sftp command: '%s'" % (command,))
+                    child.sendline(command)
+                    cmdloc += 1
+                else:
+                    command = 'quit'
+                    child.sendline(command)
+                    res = child.before
+            elif match == 3:
+                passprompt += 1
+                child.sendline(self.password)
+                if (passprompt>1):
+                    raise BackendException("Invalid SSH password.")
+            elif match == 4:
+                if not child.before.strip().startswith("mkdir"):
+                    msg = "Permission denied"
+                    break
+            elif match == 5:
+                msg = "Host key authenticity could not be verified (missing known_hosts entry?)"
+                break
+            elif match == 6:
+                if not child.before.strip().startswith("rm"):
+                    msg = "Remote file or directory does not exist in command='%s'" % (commandline,)
+                    break
+            elif match == 7:
+                if not child.before.strip().startswith("Removing"):
                     msg = "Could not delete file in command='%s'" % (commandline,)
-                    break
-                elif match == 9:
-                    msg = "Could not open file in command='%s'" % (commandline,)
-                    break
-            child.close(force = True)
-            if child.exitstatus == 0:
-                return res
-            log.Warn("Running '%s' with commands:\n %s\n failed (attempt #%d): %s" % (commandline, "\n ".join(commands), n, msg))
-        raise BackendException("Giving up trying to execute '%s' with commands:\n %s\n after %d attempts" % (commandline, "\n ".join(commands), globals.num_retries))
+                    break;
+            elif match == 8:
+                msg = "Could not delete file in command='%s'" % (commandline,)
+                break
+            elif match == 9:
+                msg = "Could not open file in command='%s'" % (commandline,)
+                break
+        child.close(force = True)
+        if child.exitstatus == 0:
+            return res
+        else:
+            raise BackendException("Error running '%s': %s" % (commandline, msg))
 
-    def put(self, source_path, remote_filename = None):
+    def _put(self, source_path, remote_filename):
         if globals.use_scp:
-            self.put_scp(source_path, remote_filename = remote_filename)
+            self.put_scp(source_path, remote_filename)
         else:
-            self.put_sftp(source_path, remote_filename = remote_filename)
+            self.put_sftp(source_path, remote_filename)
 
-    def put_sftp(self, source_path, remote_filename = None):
-        """Use sftp to copy source_dir/filename to remote computer"""
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
+    def put_sftp(self, source_path, remote_filename):
         commands = ["put \"%s\" \"%s.%s.part\"" %
                     (source_path.name, self.remote_prefix, remote_filename),
                     "rename \"%s.%s.part\" \"%s%s\"" %
@@ -239,53 +227,36 @@
                                      self.host_string))
         self.run_sftp_command(commandline, commands)
 
-    def put_scp(self, source_path, remote_filename = None):
-        """Use scp to copy source_dir/filename to remote computer"""
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
+    def put_scp(self, source_path, remote_filename):
         commandline = "%s %s %s %s:%s%s" % \
             (self.scp_command, globals.ssh_options, source_path.name, self.host_string,
              self.remote_prefix, remote_filename)
         self.run_scp_command(commandline)
 
-    def get(self, remote_filename, local_path):
+    def _get(self, remote_filename, local_path):
         if globals.use_scp:
             self.get_scp(remote_filename, local_path)
         else:
             self.get_sftp(remote_filename, local_path)
 
     def get_sftp(self, remote_filename, local_path):
-        """Use sftp to get a remote file"""
         commands = ["get \"%s%s\" \"%s\"" %
                     (self.remote_prefix, remote_filename, local_path.name)]
         commandline = ("%s %s %s" % (self.sftp_command,
                                      globals.ssh_options,
                                      self.host_string))
         self.run_sftp_command(commandline, commands)
-        local_path.setdata()
-        if not local_path.exists():
-            raise BackendException("File %s not found locally after get "
-                                   "from backend" % local_path.name)
 
     def get_scp(self, remote_filename, local_path):
-        """Use scp to get a remote file"""
         commandline = "%s %s %s:%s%s %s" % \
             (self.scp_command, globals.ssh_options, self.host_string, self.remote_prefix,
              remote_filename, local_path.name)
         self.run_scp_command(commandline)
-        local_path.setdata()
-        if not local_path.exists():
-            raise BackendException("File %s not found locally after get "
-                                   "from backend" % local_path.name)
 
     def _list(self):
-        """
-        List files available for scp
-
-        Note that this command can get confused when dealing with
-        files with newlines in them, as the embedded newlines cannot
-        be distinguished from the file boundaries.
-        """
+        # Note that this command can get confused when dealing with
+        # files with newlines in them, as the embedded newlines cannot
+        # be distinguished from the file boundaries.
         dirs = self.remote_dir.split(os.sep)
         if len(dirs) > 0:
             if not dirs[0] :
@@ -304,16 +275,8 @@
 
         return [x for x in map(string.strip, l) if x]
 
-    def delete(self, filename_list):
-        """
-        Runs sftp rm to delete files.  Files must not require quoting.
-        """
+    def _delete(self, filename):
         commands = ["cd \"%s\"" % (self.remote_dir,)]
-        for fn in filename_list:
-            commands.append("rm \"%s\"" % fn)
+        commands.append("rm \"%s\"" % filename)
         commandline = ("%s %s %s" % (self.sftp_command, globals.ssh_options, self.host_string))
         self.run_sftp_command(commandline, commands)
-
-duplicity.backend.register_backend("ssh", SSHPExpectBackend)
-duplicity.backend.register_backend("scp", SSHPExpectBackend)
-duplicity.backend.register_backend("sftp", SSHPExpectBackend)

=== modified file 'duplicity/backends/botobackend.py'
--- duplicity/backends/botobackend.py	2014-04-17 21:54:04 +0000
+++ duplicity/backends/botobackend.py	2014-04-28 02:49:55 +0000
@@ -22,14 +22,12 @@
 
 import duplicity.backend
 from duplicity import globals
-from ._boto_multi import BotoBackend as BotoMultiUploadBackend
-from ._boto_single import BotoBackend as BotoSingleUploadBackend
 
 if globals.s3_use_multiprocessing:
-    duplicity.backend.register_backend("gs", BotoMultiUploadBackend)
-    duplicity.backend.register_backend("s3", BotoMultiUploadBackend)
-    duplicity.backend.register_backend("s3+http", BotoMultiUploadBackend)
+    from ._boto_multi import BotoBackend
 else:
-    duplicity.backend.register_backend("gs", BotoSingleUploadBackend)
-    duplicity.backend.register_backend("s3", BotoSingleUploadBackend)
-    duplicity.backend.register_backend("s3+http", BotoSingleUploadBackend)
+    from ._boto_single import BotoBackend
+
+duplicity.backend.register_backend("gs", BotoBackend)
+duplicity.backend.register_backend("s3", BotoBackend)
+duplicity.backend.register_backend("s3+http", BotoBackend)

=== modified file 'duplicity/backends/cfbackend.py'
--- duplicity/backends/cfbackend.py	2014-04-17 21:54:04 +0000
+++ duplicity/backends/cfbackend.py	2014-04-28 02:49:55 +0000
@@ -18,10 +18,13 @@
 # along with duplicity; if not, write to the Free Software Foundation,
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
+import duplicity.backend
 from duplicity import globals
 
 if (globals.cf_backend and
     globals.cf_backend.lower().strip() == 'pyrax'):
-    from . import _cf_pyrax
+    from ._cf_pyrax import PyraxBackend as CFBackend
 else:
-    from . import _cf_cloudfiles
+    from ._cf_cloudfiles import CloudFilesBackend as CFBackend
+
+duplicity.backend.register_backend("cf+http", CFBackend)

=== modified file 'duplicity/backends/dpbxbackend.py'
--- duplicity/backends/dpbxbackend.py	2014-04-17 21:49:37 +0000
+++ duplicity/backends/dpbxbackend.py	2014-04-28 02:49:55 +0000
@@ -32,14 +32,10 @@
 from functools import reduce
 
 import traceback, StringIO
-from exceptions import Exception
 
 import duplicity.backend
-from duplicity import globals
 from duplicity import log
-from duplicity.errors import *
-from duplicity import tempdir
-from duplicity.backend import retry_fatal
+from duplicity.errors import BackendException
 
 
 # This application key is registered in my name (jno at pisem dot net).
@@ -76,14 +72,14 @@
         def wrapper(self, *args):
             from dropbox import rest
             if login_required and not self.sess.is_linked():
-              log.FatalError("dpbx Cannot login: check your credentials",log.ErrorCode.dpbx_nologin)
+              raise BackendException("dpbx Cannot login: check your credentials", log.ErrorCode.dpbx_nologin)
               return
 
             try:
                 return f(self, *args)
             except TypeError as e:
                 log_exception(e)
-                log.FatalError('dpbx type error "%s"' % (e,), log.ErrorCode.backend_code_error)
+                raise BackendException('dpbx type error "%s"' % (e,))
             except rest.ErrorResponse as e:
                 msg = e.user_error_msg or str(e)
                 log.Error('dpbx error: %s' % (msg,), log.ErrorCode.backend_command_error)
@@ -165,25 +161,22 @@
           if not self.sess.is_linked(): # stil not logged in
             log.FatalError("dpbx Cannot login: check your credentials",log.ErrorCode.dpbx_nologin)
 
-    @retry_fatal
+    def _error_code(self, operation, e):
+        from dropbox import rest
+        if isinstance(e, rest.ErrorResponse):
+            if e.status == 404:
+                return log.ErrorCode.backend_not_found
+
     @command()
-    def put(self, source_path, remote_filename = None):
-        """Transfer source_path to remote_filename"""
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
-
+    def _put(self, source_path, remote_filename):
         remote_dir  = urllib.unquote(self.parsed_url.path.lstrip('/'))
         remote_path = os.path.join(remote_dir, remote_filename).rstrip()
-
         from_file = open(source_path.name, "rb")
-
         resp = self.api_client.put_file(remote_path, from_file)
         log.Debug( 'dpbx,put(%s,%s): %s'%(source_path.name, remote_path, resp))
 
-    @retry_fatal
     @command()
-    def get(self, remote_filename, local_path):
-        """Get remote filename, saving it to local_path"""
+    def _get(self, remote_filename, local_path):
         remote_path = os.path.join(urllib.unquote(self.parsed_url.path), remote_filename).rstrip()
 
         to_file = open( local_path.name, 'wb' )
@@ -196,10 +189,8 @@
 
         local_path.setdata()
 
-    @retry_fatal
     @command()
-    def _list(self,none=None):
-        """List files in directory"""
+    def _list(self):
         # Do a long listing to avoid connection reset
         remote_dir = urllib.unquote(self.parsed_url.path.lstrip('/')).rstrip()
         resp = self.api_client.metadata(remote_dir)
@@ -214,21 +205,15 @@
                 l.append(name.encode(encoding))
         return l
 
-    @retry_fatal
     @command()
-    def delete(self, filename_list):
-        """Delete files in filename_list"""
-        if not filename_list :
-          log.Debug('dpbx.delete(): no op')
-          return
+    def _delete(self, filename):
         remote_dir = urllib.unquote(self.parsed_url.path.lstrip('/')).rstrip()
-        for filename in filename_list:
-          remote_name = os.path.join( remote_dir, filename )
-          resp = self.api_client.file_delete( remote_name )
-          log.Debug('dpbx.delete(%s): %s'%(remote_name,resp))
+        remote_name = os.path.join( remote_dir, filename )
+        resp = self.api_client.file_delete( remote_name )
+        log.Debug('dpbx.delete(%s): %s'%(remote_name,resp))
 
     @command()
-    def close(self):
+    def _close(self):
       """close backend session? no! just "flush" the data"""
       info = self.api_client.account_info()
       log.Debug('dpbx.close():')

=== modified file 'duplicity/backends/ftpbackend.py'
--- duplicity/backends/ftpbackend.py	2014-04-25 23:20:12 +0000
+++ duplicity/backends/ftpbackend.py	2014-04-28 02:49:55 +0000
@@ -25,7 +25,6 @@
 import duplicity.backend
 from duplicity import globals
 from duplicity import log
-from duplicity.errors import * #@UnusedWildImport
 from duplicity import tempdir
 
 class FTPBackend(duplicity.backend.Backend):
@@ -65,7 +64,7 @@
         # This squelches the "file not found" result from ncftpls when
         # the ftp backend looks for a collection that does not exist.
         # version 3.2.2 has error code 5, 1280 is some legacy value
-        self.popen_persist_breaks[ 'ncftpls' ] = [ 5, 1280 ]
+        self.popen_breaks[ 'ncftpls' ] = [ 5, 1280 ]
 
         # Use an explicit directory name.
         if self.url_string[-1] != '/':
@@ -88,36 +87,28 @@
         if parsed_url.port != None and parsed_url.port != 21:
             self.flags += " -P '%s'" % (parsed_url.port)
 
-    def put(self, source_path, remote_filename = None):
-        """Transfer source_path to remote_filename"""
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
+    def _put(self, source_path, remote_filename):
         remote_path = os.path.join(urllib.unquote(self.parsed_url.path.lstrip('/')), remote_filename).rstrip()
         commandline = "ncftpput %s -m -V -C '%s' '%s'" % \
             (self.flags, source_path.name, remote_path)
-        self.run_command_persist(commandline)
+        self.subprocess_popen(commandline)
 
-    def get(self, remote_filename, local_path):
-        """Get remote filename, saving it to local_path"""
+    def _get(self, remote_filename, local_path):
         remote_path = os.path.join(urllib.unquote(self.parsed_url.path), remote_filename).rstrip()
         commandline = "ncftpget %s -V -C '%s' '%s' '%s'" % \
             (self.flags, self.parsed_url.hostname, remote_path.lstrip('/'), local_path.name)
-        self.run_command_persist(commandline)
-        local_path.setdata()
+        self.subprocess_popen(commandline)
 
     def _list(self):
-        """List files in directory"""
         # Do a long listing to avoid connection reset
         commandline = "ncftpls %s -l '%s'" % (self.flags, self.url_string)
-        l = self.popen_persist(commandline).split('\n')
+        _, l, _ = self.subprocess_popen(commandline)
         # Look for our files as the last element of a long list line
-        return [x.split()[-1] for x in l if x and not x.startswith("total ")]
+        return [x.split()[-1] for x in l.split('\n') if x and not x.startswith("total ")]
 
-    def delete(self, filename_list):
-        """Delete files in filename_list"""
-        for filename in filename_list:
-            commandline = "ncftpls %s -l -X 'DELE %s' '%s'" % \
-                (self.flags, filename, self.url_string)
-            self.popen_persist(commandline)
+    def _delete(self, filename):
+        commandline = "ncftpls %s -l -X 'DELE %s' '%s'" % \
+            (self.flags, filename, self.url_string)
+        self.subprocess_popen(commandline)
 
 duplicity.backend.register_backend("ftp", FTPBackend)

=== modified file 'duplicity/backends/ftpsbackend.py'
--- duplicity/backends/ftpsbackend.py	2014-04-25 23:20:12 +0000
+++ duplicity/backends/ftpsbackend.py	2014-04-28 02:49:55 +0000
@@ -28,7 +28,6 @@
 import duplicity.backend
 from duplicity import globals
 from duplicity import log
-from duplicity.errors import *
 from duplicity import tempdir
 
 class FTPSBackend(duplicity.backend.Backend):
@@ -85,42 +84,29 @@
             os.write(self.tempfile, "user %s %s\n" % (self.parsed_url.username, self.password))
         os.close(self.tempfile)
 
-        self.flags = "-f %s" % self.tempname
-
-    def put(self, source_path, remote_filename = None):
-        """Transfer source_path to remote_filename"""
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
+    def _put(self, source_path, remote_filename):
         remote_path = os.path.join(urllib.unquote(self.parsed_url.path.lstrip('/')), remote_filename).rstrip()
         commandline = "lftp -c 'source %s;put \'%s\' -o \'%s\''" % \
             (self.tempname, source_path.name, remote_path)
-        l = self.run_command_persist(commandline)
+        self.subprocess_popen(commandline)
 
-    def get(self, remote_filename, local_path):
-        """Get remote filename, saving it to local_path"""
+    def _get(self, remote_filename, local_path):
         remote_path = os.path.join(urllib.unquote(self.parsed_url.path), remote_filename).rstrip()
         commandline = "lftp -c 'source %s;get %s -o %s'" % \
             (self.tempname, remote_path.lstrip('/'), local_path.name)
-        self.run_command_persist(commandline)
-        local_path.setdata()
+        self.subprocess_popen(commandline)
 
     def _list(self):
-        """List files in directory"""
         # Do a long listing to avoid connection reset
         remote_dir = urllib.unquote(self.parsed_url.path.lstrip('/')).rstrip()
         commandline = "lftp -c 'source %s;ls \'%s\''" % (self.tempname, remote_dir)
-        l = self.popen_persist(commandline).split('\n')
+        _, l, _ = self.subprocess_popen(commandline)
         # Look for our files as the last element of a long list line
-        return [x.split()[-1] for x in l if x]
+        return [x.split()[-1] for x in l.split('\n') if x]
 
-    def delete(self, filename_list):
-        """Delete files in filename_list"""
-        filelist = ""
-        for filename in filename_list:
-            filelist += "\'%s\' " % filename
-        if filelist.rstrip():
-            remote_dir = urllib.unquote(self.parsed_url.path.lstrip('/')).rstrip()
-            commandline = "lftp -c 'source %s;cd \'%s\';rm %s'" % (self.tempname, remote_dir, filelist.rstrip())
-            self.popen_persist(commandline)
+    def _delete(self, filename):
+        remote_dir = urllib.unquote(self.parsed_url.path.lstrip('/')).rstrip()
+        commandline = "lftp -c 'source %s;cd \'%s\';rm \'%s\''" % (self.tempname, remote_dir, filename)
+        self.subprocess_popen(commandline)
 
 duplicity.backend.register_backend("ftps", FTPSBackend)

=== modified file 'duplicity/backends/gdocsbackend.py'
--- duplicity/backends/gdocsbackend.py	2014-04-17 20:50:57 +0000
+++ duplicity/backends/gdocsbackend.py	2014-04-28 02:49:55 +0000
@@ -23,9 +23,7 @@
 import urllib
 
 import duplicity.backend
-from duplicity.backend import retry
-from duplicity import log
-from duplicity.errors import * #@UnusedWildImport
+from duplicity.errors import BackendException
 
 
 class GDocsBackend(duplicity.backend.Backend):
@@ -53,14 +51,14 @@
         self.client = gdata.docs.client.DocsClient(source='duplicity $version')
         self.client.ssl = True
         self.client.http_client.debug = False
-        self.__authorize(parsed_url.username + '@' + parsed_url.hostname, self.get_password())
+        self._authorize(parsed_url.username + '@' + parsed_url.hostname, self.get_password())
 
         # Fetch destination folder entry (and crete hierarchy if required).
         folder_names = string.split(parsed_url.path[1:], '/')
         parent_folder = None
         parent_folder_id = GDocsBackend.ROOT_FOLDER_ID
         for folder_name in folder_names:
-            entries = self.__fetch_entries(parent_folder_id, 'folder', folder_name)
+            entries = self._fetch_entries(parent_folder_id, 'folder', folder_name)
             if entries is not None:
                 if len(entries) == 1:
                     parent_folder = entries[0]
@@ -77,106 +75,54 @@
                 raise BackendException("Error while fetching destination folder '%s'." % folder_name)
         self.folder = parent_folder
 
-    @retry
-    def put(self, source_path, remote_filename=None, raise_errors=False):
-        """Transfer source_path to remote_filename"""
-        # Default remote file name.
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
-
-        # Upload!
-        try:
-            # If remote file already exists in destination folder, remove it.
-            entries = self.__fetch_entries(self.folder.resource_id.text,
-                                           GDocsBackend.BACKUP_DOCUMENT_TYPE,
-                                           remote_filename)
-            for entry in entries:
-                self.client.delete(entry.get_edit_link().href + '?delete=true', force=True)
-
-            # Set uploader instance. Note that resumable uploads are required in order to
-            # enable uploads for all file types.
-            # (see http://googleappsdeveloper.blogspot.com/2011/05/upload-all-file-types-to-any-google.html)
-            file = source_path.open()
-            uploader = gdata.client.ResumableUploader(
-              self.client, file, GDocsBackend.BACKUP_DOCUMENT_TYPE, os.path.getsize(file.name),
-              chunk_size=gdata.client.ResumableUploader.DEFAULT_CHUNK_SIZE,
-              desired_class=gdata.docs.data.Resource)
-            if uploader:
-                # Chunked upload.
-                entry = gdata.docs.data.Resource(title=atom.data.Title(text=remote_filename))
-                uri = self.folder.get_resumable_create_media_link().href + '?convert=false'
-                entry = uploader.UploadFile(uri, entry=entry)
-                if not entry:
-                    self.__handle_error("Failed to upload file '%s' to remote folder '%s'"
-                                        % (source_path.get_filename(), self.folder.title.text), raise_errors)
-            else:
-                self.__handle_error("Failed to initialize upload of file '%s' to remote folder '%s'"
-                         % (source_path.get_filename(), self.folder.title.text), raise_errors)
-            assert not file.close()
-        except Exception as e:
-            self.__handle_error("Failed to upload file '%s' to remote folder '%s': %s"
-                                % (source_path.get_filename(), self.folder.title.text, str(e)), raise_errors)
-
-    @retry
-    def get(self, remote_filename, local_path, raise_errors=False):
-        """Get remote filename, saving it to local_path"""
-        try:
-            entries = self.__fetch_entries(self.folder.resource_id.text,
-                                           GDocsBackend.BACKUP_DOCUMENT_TYPE,
-                                           remote_filename)
-            if len(entries) == 1:
-                entry = entries[0]
-                self.client.DownloadResource(entry, local_path.name)
-                local_path.setdata()
-                return
-            else:
-                self.__handle_error("Failed to find file '%s' in remote folder '%s'"
-                                    % (remote_filename, self.folder.title.text), raise_errors)
-        except Exception as e:
-            self.__handle_error("Failed to download file '%s' in remote folder '%s': %s"
-                                 % (remote_filename, self.folder.title.text, str(e)), raise_errors)
-
-    @retry
-    def _list(self, raise_errors=False):
-        """List files in folder"""
-        try:
-            entries = self.__fetch_entries(self.folder.resource_id.text,
-                                           GDocsBackend.BACKUP_DOCUMENT_TYPE)
-            return [entry.title.text for entry in entries]
-        except Exception as e:
-            self.__handle_error("Failed to fetch list of files in remote folder '%s': %s"
-                                % (self.folder.title.text, str(e)), raise_errors)
-
-    @retry
-    def delete(self, filename_list, raise_errors=False):
-        """Delete files in filename_list"""
-        for filename in filename_list:
-            try:
-                entries = self.__fetch_entries(self.folder.resource_id.text,
-                                               GDocsBackend.BACKUP_DOCUMENT_TYPE,
-                                               filename)
-                if len(entries) > 0:
-                    success = True
-                    for entry in entries:
-                        if not self.client.delete(entry.get_edit_link().href + '?delete=true', force=True):
-                            success = False
-                    if not success:
-                        self.__handle_error("Failed to remove file '%s' in remote folder '%s'"
-                                            % (filename, self.folder.title.text), raise_errors)
-                else:
-                    log.Warn("Failed to fetch file '%s' in remote folder '%s'"
-                             % (filename, self.folder.title.text))
-            except Exception as e:
-                self.__handle_error("Failed to remove file '%s' in remote folder '%s': %s"
-                                    % (filename, self.folder.title.text, str(e)), raise_errors)
-
-    def __handle_error(self, message, raise_errors=True):
-        if raise_errors:
-            raise BackendException(message)
-        else:
-            log.FatalError(message, log.ErrorCode.backend_error)
-
-    def __authorize(self, email, password, captcha_token=None, captcha_response=None):
+    def _put(self, source_path, remote_filename):
+        self._delete(remote_filename)
+
+        # Set uploader instance. Note that resumable uploads are required in order to
+        # enable uploads for all file types.
+        # (see http://googleappsdeveloper.blogspot.com/2011/05/upload-all-file-types-to-any-google.html)
+        file = source_path.open()
+        uploader = gdata.client.ResumableUploader(
+          self.client, file, GDocsBackend.BACKUP_DOCUMENT_TYPE, os.path.getsize(file.name),
+          chunk_size=gdata.client.ResumableUploader.DEFAULT_CHUNK_SIZE,
+          desired_class=gdata.docs.data.Resource)
+        if uploader:
+            # Chunked upload.
+            entry = gdata.docs.data.Resource(title=atom.data.Title(text=remote_filename))
+            uri = self.folder.get_resumable_create_media_link().href + '?convert=false'
+            entry = uploader.UploadFile(uri, entry=entry)
+            if not entry:
+                raise BackendException("Failed to upload file '%s' to remote folder '%s'"
+                                       % (source_path.get_filename(), self.folder.title.text))
+        else:
+            raise BackendException("Failed to initialize upload of file '%s' to remote folder '%s'"
+                                   % (source_path.get_filename(), self.folder.title.text))
+        assert not file.close()
+
+    def _get(self, remote_filename, local_path):
+        entries = self._fetch_entries(self.folder.resource_id.text,
+                                      GDocsBackend.BACKUP_DOCUMENT_TYPE,
+                                      remote_filename)
+        if len(entries) == 1:
+            entry = entries[0]
+            self.client.DownloadResource(entry, local_path.name)
+        else:
+            raise BackendException("Failed to find file '%s' in remote folder '%s'"
+                                   % (remote_filename, self.folder.title.text))
+
+    def _list(self):
+        entries = self._fetch_entries(self.folder.resource_id.text,
+                                      GDocsBackend.BACKUP_DOCUMENT_TYPE)
+        return [entry.title.text for entry in entries]
+
+    def _delete(self, filename):
+        entries = self._fetch_entries(self.folder.resource_id.text,
+                                      GDocsBackend.BACKUP_DOCUMENT_TYPE,
+                                      filename)
+        for entry in entries:
+            self.client.delete(entry.get_edit_link().href + '?delete=true', force=True)
+
+    def _authorize(self, email, password, captcha_token=None, captcha_response=None):
         try:
             self.client.client_login(email,
                                      password,
@@ -189,17 +135,15 @@
             answer = None
             while not answer:
                 answer = raw_input('Answer to the challenge? ')
-            self.__authorize(email, password, challenge.captcha_token, answer)
+            self._authorize(email, password, challenge.captcha_token, answer)
         except gdata.client.BadAuthentication:
-            self.__handle_error('Invalid user credentials given. Be aware that accounts '
-                                'that use 2-step verification require creating an application specific '
-                                'access code for using this Duplicity backend. Follow the instrucction in '
-                                'http://www.google.com/support/accounts/bin/static.py?page=guide.cs&guide=1056283&topic=1056286 '
-                                'and create your application-specific password to run duplicity backups.')
-        except Exception as e:
-            self.__handle_error('Error while authenticating client: %s.' % str(e))
+            raise BackendException('Invalid user credentials given. Be aware that accounts '
+                                   'that use 2-step verification require creating an application specific '
+                                   'access code for using this Duplicity backend. Follow the instruction in '
+                                   'http://www.google.com/support/accounts/bin/static.py?page=guide.cs&guide=1056283&topic=1056286 '
+                                   'and create your application-specific password to run duplicity backups.')
 
-    def __fetch_entries(self, folder_id, type, title=None):
+    def _fetch_entries(self, folder_id, type, title=None):
         # Build URI.
         uri = '/feeds/default/private/full/%s/contents' % folder_id
         if type == 'folder':
@@ -211,34 +155,31 @@
         if title:
             uri += '&title=' + urllib.quote(title) + '&title-exact=true'
 
-        try:
-            # Fetch entries.
-            entries = self.client.get_all_resources(uri=uri)
-
-            # When filtering by entry title, API is returning (don't know why) documents in other
-            # folders (apart from folder_id) matching the title, so some extra filtering is required.
-            if title:
-                result = []
-                for entry in entries:
-                    resource_type = entry.get_resource_type()
-                    if (not type) \
-                       or (type == 'folder' and resource_type == 'folder') \
-                       or (type == GDocsBackend.BACKUP_DOCUMENT_TYPE and resource_type != 'folder'):
-
-                        if folder_id != GDocsBackend.ROOT_FOLDER_ID:
-                            for link in entry.in_collections():
-                                folder_entry = self.client.get_entry(link.href, None, None,
-                                                                     desired_class=gdata.docs.data.Resource)
-                                if folder_entry and (folder_entry.resource_id.text == folder_id):
-                                    result.append(entry)
-                        elif len(entry.in_collections()) == 0:
-                            result.append(entry)
-            else:
-                result = entries
-
-            # Done!
-            return result
-        except Exception as e:
-            self.__handle_error('Error while fetching remote entries: %s.' % str(e))
+        # Fetch entries.
+        entries = self.client.get_all_resources(uri=uri)
+
+        # When filtering by entry title, API is returning (don't know why) documents in other
+        # folders (apart from folder_id) matching the title, so some extra filtering is required.
+        if title:
+            result = []
+            for entry in entries:
+                resource_type = entry.get_resource_type()
+                if (not type) \
+                   or (type == 'folder' and resource_type == 'folder') \
+                   or (type == GDocsBackend.BACKUP_DOCUMENT_TYPE and resource_type != 'folder'):
+
+                    if folder_id != GDocsBackend.ROOT_FOLDER_ID:
+                        for link in entry.in_collections():
+                            folder_entry = self.client.get_entry(link.href, None, None,
+                                                                 desired_class=gdata.docs.data.Resource)
+                            if folder_entry and (folder_entry.resource_id.text == folder_id):
+                                result.append(entry)
+                    elif len(entry.in_collections()) == 0:
+                        result.append(entry)
+        else:
+            result = entries
+
+        # Done!
+        return result
 
 duplicity.backend.register_backend('gdocs', GDocsBackend)

=== modified file 'duplicity/backends/giobackend.py'
--- duplicity/backends/giobackend.py	2014-04-17 20:50:57 +0000
+++ duplicity/backends/giobackend.py	2014-04-28 02:49:55 +0000
@@ -19,18 +19,12 @@
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
 import os
-import types
 import subprocess
 import atexit
 import signal
-from gi.repository import Gio #@UnresolvedImport
-from gi.repository import GLib #@UnresolvedImport
 
 import duplicity.backend
-from duplicity.backend import retry
 from duplicity import log
-from duplicity import util
-from duplicity.errors import * #@UnusedWildImport
 
 def ensure_dbus():
     # GIO requires a dbus session bus which can start the gvfs daemons
@@ -46,36 +40,39 @@
                     atexit.register(os.kill, int(parts[1]), signal.SIGTERM)
                 os.environ[parts[0]] = parts[1]
 
-class DupMountOperation(Gio.MountOperation):
-    """A simple MountOperation that grabs the password from the environment
-       or the user.
-    """
-    def __init__(self, backend):
-        Gio.MountOperation.__init__(self)
-        self.backend = backend
-        self.connect('ask-password', self.ask_password_cb)
-        self.connect('ask-question', self.ask_question_cb)
-
-    def ask_password_cb(self, *args, **kwargs):
-        self.set_password(self.backend.get_password())
-        self.reply(Gio.MountOperationResult.HANDLED)
-
-    def ask_question_cb(self, *args, **kwargs):
-        # Obviously just always answering with the first choice is a naive
-        # approach.  But there's no easy way to allow for answering questions
-        # in duplicity's typical run-from-cron mode with environment variables.
-        # And only a couple gvfs backends ask questions: 'sftp' does about
-        # new hosts and 'afc' does if the device is locked.  0 should be a
-        # safe choice.
-        self.set_choice(0)
-        self.reply(Gio.MountOperationResult.HANDLED)
-
 class GIOBackend(duplicity.backend.Backend):
     """Use this backend when saving to a GIO URL.
        This is a bit of a meta-backend, in that it can handle multiple schemas.
        URLs look like schema://user@server/path.
     """
     def __init__(self, parsed_url):
+        from gi.repository import Gio #@UnresolvedImport
+        from gi.repository import GLib #@UnresolvedImport
+
+        class DupMountOperation(Gio.MountOperation):
+            """A simple MountOperation that grabs the password from the environment
+               or the user.
+            """
+            def __init__(self, backend):
+                Gio.MountOperation.__init__(self)
+                self.backend = backend
+                self.connect('ask-password', self.ask_password_cb)
+                self.connect('ask-question', self.ask_question_cb)
+
+            def ask_password_cb(self, *args, **kwargs):
+                self.set_password(self.backend.get_password())
+                self.reply(Gio.MountOperationResult.HANDLED)
+
+            def ask_question_cb(self, *args, **kwargs):
+                # Obviously just always answering with the first choice is a naive
+                # approach.  But there's no easy way to allow for answering questions
+                # in duplicity's typical run-from-cron mode with environment variables.
+                # And only a couple gvfs backends ask questions: 'sftp' does about
+                # new hosts and 'afc' does if the device is locked.  0 should be a
+                # safe choice.
+                self.set_choice(0)
+                self.reply(Gio.MountOperationResult.HANDLED)
+
         duplicity.backend.Backend.__init__(self, parsed_url)
 
         ensure_dbus()
@@ -86,8 +83,8 @@
         op = DupMountOperation(self)
         loop = GLib.MainLoop()
         self.remote_file.mount_enclosing_volume(Gio.MountMountFlags.NONE,
-                                                op, None, self.done_with_mount,
-                                                loop)
+                                                op, None,
+                                                self.__done_with_mount, loop)
         loop.run() # halt program until we're done mounting
 
         # Now make the directory if it doesn't exist
@@ -97,7 +94,9 @@
             if e.code != Gio.IOErrorEnum.EXISTS:
                 raise
 
-    def done_with_mount(self, fileobj, result, loop):
+    def __done_with_mount(self, fileobj, result, loop):
+        from gi.repository import Gio #@UnresolvedImport
+        from gi.repository import GLib #@UnresolvedImport
         try:
             fileobj.mount_enclosing_volume_finish(result)
         except GLib.GError as e:
@@ -107,97 +106,63 @@
                                % str(e), log.ErrorCode.connection_failed)
         loop.quit()
 
-    def handle_error(self, raise_error, e, op, file1=None, file2=None):
-        if raise_error:
-            raise e
-        code = log.ErrorCode.backend_error
+    def __copy_progress(self, *args, **kwargs):
+        pass
+
+    def __copy_file(self, source, target):
+        from gi.repository import Gio #@UnresolvedImport
+        source.copy(target,
+                    Gio.FileCopyFlags.OVERWRITE | Gio.FileCopyFlags.NOFOLLOW_SYMLINKS,
+                    None, self.__copy_progress, None)
+
+    def _error_code(self, operation, e):
+        from gi.repository import Gio #@UnresolvedImport
+        from gi.repository import GLib #@UnresolvedImport
         if isinstance(e, GLib.GError):
-            if e.code == Gio.IOErrorEnum.PERMISSION_DENIED:
-                code = log.ErrorCode.backend_permission_denied
+            if e.code == Gio.IOErrorEnum.FAILED and operation == 'delete':
+                # Sometimes delete will return a generic failure on a file not
+                # found (notably the FTP does that)
+                return log.ErrorCode.backend_not_found
+            elif e.code == Gio.IOErrorEnum.PERMISSION_DENIED:
+                return log.ErrorCode.backend_permission_denied
             elif e.code == Gio.IOErrorEnum.NOT_FOUND:
-                code = log.ErrorCode.backend_not_found
+                return log.ErrorCode.backend_not_found
             elif e.code == Gio.IOErrorEnum.NO_SPACE:
-                code = log.ErrorCode.backend_no_space
-        extra = ' '.join([util.escape(x) for x in [file1, file2] if x])
-        extra = ' '.join([op, extra])
-        log.FatalError(str(e), code, extra)
-
-    def copy_progress(self, *args, **kwargs):
-        pass
-
-    @retry
-    def copy_file(self, op, source, target, raise_errors=False):
-        log.Info(_("Writing %s") % target.get_parse_name())
-        try:
-            source.copy(target,
-                        Gio.FileCopyFlags.OVERWRITE | Gio.FileCopyFlags.NOFOLLOW_SYMLINKS,
-                        None, self.copy_progress, None)
-        except Exception as e:
-            self.handle_error(raise_errors, e, op, source.get_parse_name(),
-                              target.get_parse_name())
-
-    def put(self, source_path, remote_filename = None):
-        """Copy file to remote"""
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
+                return log.ErrorCode.backend_no_space
+
+    def _put(self, source_path, remote_filename):
+        from gi.repository import Gio #@UnresolvedImport
         source_file = Gio.File.new_for_path(source_path.name)
         target_file = self.remote_file.get_child(remote_filename)
-        self.copy_file('put', source_file, target_file)
+        self.__copy_file(source_file, target_file)
 
-    def get(self, filename, local_path):
-        """Get file and put in local_path (Path object)"""
+    def _get(self, filename, local_path):
+        from gi.repository import Gio #@UnresolvedImport
         source_file = self.remote_file.get_child(filename)
         target_file = Gio.File.new_for_path(local_path.name)
-        self.copy_file('get', source_file, target_file)
-        local_path.setdata()
+        self.__copy_file(source_file, target_file)
 
-    @retry
-    def _list(self, raise_errors=False):
-        """List files in that directory"""
+    def _list(self):
+        from gi.repository import Gio #@UnresolvedImport
         files = []
-        try:
-            enum = self.remote_file.enumerate_children(Gio.FILE_ATTRIBUTE_STANDARD_NAME,
-                                                       Gio.FileQueryInfoFlags.NOFOLLOW_SYMLINKS,
-                                                       None)
+        enum = self.remote_file.enumerate_children(Gio.FILE_ATTRIBUTE_STANDARD_NAME,
+                                                   Gio.FileQueryInfoFlags.NOFOLLOW_SYMLINKS,
+                                                   None)
+        info = enum.next_file(None)
+        while info:
+            files.append(info.get_name())
             info = enum.next_file(None)
-            while info:
-                files.append(info.get_name())
-                info = enum.next_file(None)
-        except Exception as e:
-            self.handle_error(raise_errors, e, 'list',
-                              self.remote_file.get_parse_name())
         return files
 
-    @retry
-    def delete(self, filename_list, raise_errors=False):
-        """Delete all files in filename list"""
-        assert type(filename_list) is not types.StringType
-        for filename in filename_list:
-            target_file = self.remote_file.get_child(filename)
-            try:
-                target_file.delete(None)
-            except Exception as e:
-                if isinstance(e, GLib.GError):
-                    if e.code == Gio.IOErrorEnum.NOT_FOUND:
-                        continue
-                self.handle_error(raise_errors, e, 'delete',
-                                  target_file.get_parse_name())
-                return
-
-    @retry
-    def _query_file_info(self, filename, raise_errors=False):
-        """Query attributes on filename"""
-        target_file = self.remote_file.get_child(filename)
-        attrs = Gio.FILE_ATTRIBUTE_STANDARD_SIZE
-        try:
-            info = target_file.query_info(attrs, Gio.FileQueryInfoFlags.NONE,
-                                          None)
-            return {'size': info.get_size()}
-        except Exception as e:
-            if isinstance(e, GLib.GError):
-                if e.code == Gio.IOErrorEnum.NOT_FOUND:
-                    return {'size': -1} # early exit, no need to retry
-            if raise_errors:
-                raise e
-            else:
-                return {'size': None}
+    def _delete(self, filename):
+        target_file = self.remote_file.get_child(filename)
+        target_file.delete(None)
+
+    def _query(self, filename):
+        from gi.repository import Gio #@UnresolvedImport
+        target_file = self.remote_file.get_child(filename)
+        info = target_file.query_info(Gio.FILE_ATTRIBUTE_STANDARD_SIZE,
+                                      Gio.FileQueryInfoFlags.NONE, None)
+        return {'size': info.get_size()}
+
+duplicity.backend.register_backend_prefix('gio', GIOBackend)

=== modified file 'duplicity/backends/hsibackend.py'
--- duplicity/backends/hsibackend.py	2014-04-25 23:20:12 +0000
+++ duplicity/backends/hsibackend.py	2014-04-28 02:49:55 +0000
@@ -20,9 +20,7 @@
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
 import os
-
 import duplicity.backend
-from duplicity.errors import * #@UnusedWildImport
 
 hsi_command = "hsi"
 class HSIBackend(duplicity.backend.Backend):
@@ -35,35 +33,23 @@
         else:
             self.remote_prefix = ""
 
-    def put(self, source_path, remote_filename = None):
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
+    def _put(self, source_path, remote_filename):
         commandline = '%s "put %s : %s%s"' % (hsi_command,source_path.name,self.remote_prefix,remote_filename)
-        try:
-            self.run_command(commandline)
-        except Exception:
-            print commandline
+        self.subprocess_popen(commandline)
 
-    def get(self, remote_filename, local_path):
+    def _get(self, remote_filename, local_path):
         commandline = '%s "get %s : %s%s"' % (hsi_command, local_path.name, self.remote_prefix, remote_filename)
-        self.run_command(commandline)
-        local_path.setdata()
-        if not local_path.exists():
-            raise BackendException("File %s not found" % local_path.name)
+        self.subprocess_popen(commandline)
 
-    def list(self):
+    def _list(self):
         commandline = '%s "ls -l %s"' % (hsi_command, self.remote_dir)
         l = os.popen3(commandline)[2].readlines()[3:]
         for i in range(0,len(l)):
             l[i] = l[i].split()[-1]
         return [x for x in l if x]
 
-    def delete(self, filename_list):
-        assert len(filename_list) > 0
-        for fn in filename_list:
-            commandline = '%s "rm %s%s"' % (hsi_command, self.remote_prefix, fn)
-            self.run_command(commandline)
+    def _delete(self, filename):
+        commandline = '%s "rm %s%s"' % (hsi_command, self.remote_prefix, filename)
+        self.subprocess_popen(commandline)
 
 duplicity.backend.register_backend("hsi", HSIBackend)
-
-

=== modified file 'duplicity/backends/imapbackend.py'
--- duplicity/backends/imapbackend.py	2014-04-17 22:03:10 +0000
+++ duplicity/backends/imapbackend.py	2014-04-28 02:49:55 +0000
@@ -44,7 +44,7 @@
                   (self.__class__.__name__, parsed_url.scheme, parsed_url.hostname, parsed_url.username))
 
         #  Store url for reconnection on error
-        self._url = parsed_url
+        self.url = parsed_url
 
         #  Set the username
         if ( parsed_url.username is None ):
@@ -61,12 +61,12 @@
         else:
             password = parsed_url.password
 
-        self._username = username
-        self._password = password
-        self._resetConnection()
+        self.username = username
+        self.password = password
+        self.resetConnection()
 
-    def _resetConnection(self):
-        parsed_url = self._url
+    def resetConnection(self):
+        parsed_url = self.url
         try:
             imap_server = os.environ['IMAP_SERVER']
         except KeyError:
@@ -74,32 +74,32 @@
 
         #  Try to close the connection cleanly
         try:
-            self._conn.close()
+            self.conn.close()
         except Exception:
             pass
 
         if (parsed_url.scheme == "imap"):
             cl = imaplib.IMAP4
-            self._conn = cl(imap_server, 143)
+            self.conn = cl(imap_server, 143)
         elif (parsed_url.scheme == "imaps"):
             cl = imaplib.IMAP4_SSL
-            self._conn = cl(imap_server, 993)
+            self.conn = cl(imap_server, 993)
 
         log.Debug("Type of imap class: %s" % (cl.__name__))
         self.remote_dir = re.sub(r'^/', r'', parsed_url.path, 1)
 
         #  Login
         if (not(globals.imap_full_address)):
-            self._conn.login(self._username, self._password)
-            self._conn.select(globals.imap_mailbox)
+            self.conn.login(self.username, self.password)
+            self.conn.select(globals.imap_mailbox)
             log.Info("IMAP connected")
         else:
-            self._conn.login(self._username + "@" + parsed_url.hostname, self._password)
-            self._conn.select(globals.imap_mailbox)
+            self.conn.login(self.username + "@" + parsed_url.hostname, self.password)
+            self.conn.select(globals.imap_mailbox)
             log.Info("IMAP connected")
 
 
-    def _prepareBody(self,f,rname):
+    def prepareBody(self,f,rname):
         mp = email.MIMEMultipart.MIMEMultipart()
 
         # I am going to use the remote_dir as the From address so that
@@ -117,9 +117,7 @@
 
         return mp.as_string()
 
-    def put(self, source_path, remote_filename = None):
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
+    def _put(self, source_path, remote_filename):
         f=source_path.open("rb")
         allowedTimeout = globals.timeout
         if (allowedTimeout == 0):
@@ -127,12 +125,12 @@
             allowedTimeout = 2880
         while allowedTimeout > 0:
             try:
-                self._conn.select(remote_filename)
-                body=self._prepareBody(f,remote_filename)
+                self.conn.select(remote_filename)
+                body=self.prepareBody(f,remote_filename)
                 # If we don't select the IMAP folder before
                 # append, the message goes into the INBOX.
-                self._conn.select(globals.imap_mailbox)
-                self._conn.append(globals.imap_mailbox, None, None, body)
+                self.conn.select(globals.imap_mailbox)
+                self.conn.append(globals.imap_mailbox, None, None, body)
                 break
             except (imaplib.IMAP4.abort, socket.error, socket.sslerror):
                 allowedTimeout -= 1
@@ -140,7 +138,7 @@
                 time.sleep(30)
                 while allowedTimeout > 0:
                     try:
-                        self._resetConnection()
+                        self.resetConnection()
                         break
                     except (imaplib.IMAP4.abort, socket.error, socket.sslerror):
                         allowedTimeout -= 1
@@ -149,15 +147,15 @@
 
         log.Info("IMAP mail with '%s' subject stored" % remote_filename)
 
-    def get(self, remote_filename, local_path):
+    def _get(self, remote_filename, local_path):
         allowedTimeout = globals.timeout
         if (allowedTimeout == 0):
             # Allow a total timeout of 1 day
             allowedTimeout = 2880
         while allowedTimeout > 0:
             try:
-                self._conn.select(globals.imap_mailbox)
-                (result,list) = self._conn.search(None, 'Subject', remote_filename)
+                self.conn.select(globals.imap_mailbox)
+                (result,list) = self.conn.search(None, 'Subject', remote_filename)
                 if result != "OK":
                     raise Exception(list[0])
 
@@ -165,7 +163,7 @@
                 if list[0] == '':
                     raise Exception("no mail with subject %s")
 
-                (result,list) = self._conn.fetch(list[0],"(RFC822)")
+                (result,list) = self.conn.fetch(list[0],"(RFC822)")
 
                 if result != "OK":
                     raise Exception(list[0])
@@ -185,7 +183,7 @@
                 time.sleep(30)
                 while allowedTimeout > 0:
                     try:
-                        self._resetConnection()
+                        self.resetConnection()
                         break
                     except (imaplib.IMAP4.abort, socket.error, socket.sslerror):
                         allowedTimeout -= 1
@@ -199,7 +197,7 @@
 
     def _list(self):
         ret = []
-        (result,list) = self._conn.select(globals.imap_mailbox)
+        (result,list) = self.conn.select(globals.imap_mailbox)
         if result != "OK":
             raise BackendException(list[0])
 
@@ -207,14 +205,14 @@
         # address
 
         # Search returns an error if you haven't selected an IMAP folder.
-        (result,list) = self._conn.search(None, 'FROM', self.remote_dir)
+        (result,list) = self.conn.search(None, 'FROM', self.remote_dir)
         if result!="OK":
             raise Exception(list[0])
         if list[0]=='':
             return ret
         nums=list[0].split(" ")
         set="%s:%s"%(nums[0],nums[-1])
-        (result,list) = self._conn.fetch(set,"(BODY[HEADER])")
+        (result,list) = self.conn.fetch(set,"(BODY[HEADER])")
         if result!="OK":
             raise Exception(list[0])
 
@@ -232,34 +230,32 @@
                     log.Info("IMAP LIST: %s %s" % (subj,header_from))
         return ret
 
-    def _imapf(self,fun,*args):
+    def imapf(self,fun,*args):
         (ret,list)=fun(*args)
         if ret != "OK":
             raise Exception(list[0])
         return list
 
-    def _delete_single_mail(self,i):
-        self._imapf(self._conn.store,i,"+FLAGS",'\\DELETED')
-
-    def _expunge(self):
-        list=self._imapf(self._conn.expunge)
-
-    def delete(self, filename_list):
-        assert len(filename_list) > 0
+    def delete_single_mail(self,i):
+        self.imapf(self.conn.store,i,"+FLAGS",'\\DELETED')
+
+    def expunge(self):
+        list=self.imapf(self.conn.expunge)
+
+    def _delete_list(self, filename_list):
         for filename in filename_list:
-            list = self._imapf(self._conn.search,None,"(SUBJECT %s)"%filename)
+            list = self.imapf(self.conn.search,None,"(SUBJECT %s)"%filename)
             list = list[0].split()
-            if len(list)==0 or list[0]=="":raise Exception("no such mail with subject '%s'"%filename)
-            self._delete_single_mail(list[0])
-            log.Notice("marked %s to be deleted" % filename)
-        self._expunge()
-        log.Notice("IMAP expunged %s files" % len(list))
+            if len(list) > 0 and list[0] != "":
+                self.delete_single_mail(list[0])
+                log.Notice("marked %s to be deleted" % filename)
+        self.expunge()
+        log.Notice("IMAP expunged %s files" % len(filename_list))
 
-    def close(self):
-        self._conn.select(globals.imap_mailbox)
-        self._conn.close()
-        self._conn.logout()
+    def _close(self):
+        self.conn.select(globals.imap_mailbox)
+        self.conn.close()
+        self.conn.logout()
 
 duplicity.backend.register_backend("imap", ImapBackend);
 duplicity.backend.register_backend("imaps", ImapBackend);
-

=== modified file 'duplicity/backends/localbackend.py'
--- duplicity/backends/localbackend.py	2014-04-17 20:50:57 +0000
+++ duplicity/backends/localbackend.py	2014-04-28 02:49:55 +0000
@@ -20,14 +20,11 @@
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
 import os
-import types
-import errno
 
 import duplicity.backend
 from duplicity import log
 from duplicity import path
-from duplicity import util
-from duplicity.errors import * #@UnusedWildImport
+from duplicity.errors import BackendException
 
 
 class LocalBackend(duplicity.backend.Backend):
@@ -43,90 +40,37 @@
         if not parsed_url.path.startswith('//'):
             raise BackendException("Bad file:// path syntax.")
         self.remote_pathdir = path.Path(parsed_url.path[2:])
-
-    def handle_error(self, e, op, file1 = None, file2 = None):
-        code = log.ErrorCode.backend_error
-        if hasattr(e, 'errno'):
-            if e.errno == errno.EACCES:
-                code = log.ErrorCode.backend_permission_denied
-            elif e.errno == errno.ENOENT:
-                code = log.ErrorCode.backend_not_found
-            elif e.errno == errno.ENOSPC:
-                code = log.ErrorCode.backend_no_space
-        extra = ' '.join([util.escape(x) for x in [file1, file2] if x])
-        extra = ' '.join([op, extra])
-        if op != 'delete' and op != 'query':
-            log.FatalError(str(e), code, extra)
-        else:
-            log.Warn(str(e), code, extra)
-
-    def move(self, source_path, remote_filename = None):
-        self.put(source_path, remote_filename, rename_instead = True)
-
-    def put(self, source_path, remote_filename = None, rename_instead = False):
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
-        target_path = self.remote_pathdir.append(remote_filename)
-        log.Info("Writing %s" % target_path.name)
-        """Try renaming first (if allowed to), copying if doesn't work"""
-        if rename_instead:
-            try:
-                source_path.rename(target_path)
-            except OSError:
-                pass
-            except Exception as e:
-                self.handle_error(e, 'put', source_path.name, target_path.name)
-            else:
-                return
-        try:
-            target_path.writefileobj(source_path.open("rb"))
-        except Exception as e:
-            self.handle_error(e, 'put', source_path.name, target_path.name)
-
-        """If we get here, renaming failed previously"""
-        if rename_instead:
-            """We need to simulate its behaviour"""
-            source_path.delete()
-
-    def get(self, filename, local_path):
-        """Get file and put in local_path (Path object)"""
+        try:
+            os.makedirs(self.remote_pathdir.base)
+        except Exception:
+            pass
+
+    def _move(self, source_path, remote_filename):
+        target_path = self.remote_pathdir.append(remote_filename)
+        try:
+            source_path.rename(target_path)
+            return True
+        except OSError:
+            return False
+
+    def _put(self, source_path, remote_filename):
+        target_path = self.remote_pathdir.append(remote_filename)
+        target_path.writefileobj(source_path.open("rb"))
+
+    def _get(self, filename, local_path):
         source_path = self.remote_pathdir.append(filename)
-        try:
-            local_path.writefileobj(source_path.open("rb"))
-        except Exception as e:
-            self.handle_error(e, 'get', source_path.name, local_path.name)
+        local_path.writefileobj(source_path.open("rb"))
 
     def _list(self):
-        """List files in that directory"""
-        try:
-                os.makedirs(self.remote_pathdir.base)
-        except Exception:
-                pass
-        try:
-            return self.remote_pathdir.listdir()
-        except Exception as e:
-            self.handle_error(e, 'list', self.remote_pathdir.name)
-
-    def delete(self, filename_list):
-        """Delete all files in filename list"""
-        assert type(filename_list) is not types.StringType
-        for filename in filename_list:
-            try:
-                self.remote_pathdir.append(filename).delete()
-            except Exception as e:
-                self.handle_error(e, 'delete', self.remote_pathdir.append(filename).name)
-
-    def _query_file_info(self, filename):
-        """Query attributes on filename"""
-        try:
-            target_file = self.remote_pathdir.append(filename)
-            if not os.path.exists(target_file.name):
-                return {'size': -1}
-            target_file.setdata()
-            size = target_file.getsize()
-            return {'size': size}
-        except Exception as e:
-            self.handle_error(e, 'query', target_file.name)
-            return {'size': None}
+        return self.remote_pathdir.listdir()
+
+    def _delete(self, filename):
+        self.remote_pathdir.append(filename).delete()
+
+    def _query(self, filename):
+        target_file = self.remote_pathdir.append(filename)
+        target_file.setdata()
+        size = target_file.getsize() if target_file.exists() else -1
+        return {'size': size}
 
 duplicity.backend.register_backend("file", LocalBackend)

=== modified file 'duplicity/backends/megabackend.py'
--- duplicity/backends/megabackend.py	2014-04-17 20:50:57 +0000
+++ duplicity/backends/megabackend.py	2014-04-28 02:49:55 +0000
@@ -22,9 +22,8 @@
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
 import duplicity.backend
-from duplicity.backend import retry
 from duplicity import log
-from duplicity.errors import * #@UnusedWildImport
+from duplicity.errors import BackendException
 
 
 class MegaBackend(duplicity.backend.Backend):
@@ -63,113 +62,64 @@
 
         self.folder = parent_folder
 
-    @retry
-    def put(self, source_path, remote_filename=None, raise_errors=False):
-        """Transfer source_path to remote_filename"""
-        # Default remote file name.
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
-
-        try:
-            # If remote file already exists in destination folder, remove it.
-            files = self.client.get_files()
-            entries = self.__filter_entries(files, self.folder, remote_filename, 'file')
-
-            for entry in entries:
-                self.client.delete(entry)
-
-            self.client.upload(source_path.get_canonical(), self.folder, dest_filename=remote_filename)
-
-        except Exception as e:
-            self.__handle_error("Failed to upload file '%s' to remote folder '%s': %s"
-                                % (source_path.get_canonical(), self.__get_node_name(self.folder), str(e)), raise_errors)
-
-    @retry
-    def get(self, remote_filename, local_path, raise_errors=False):
-        """Get remote filename, saving it to local_path"""
-        try:
-            files = self.client.get_files()
-            entries = self.__filter_entries(files, self.folder, remote_filename, 'file')
-
-            if len(entries):
-                # get first matching remote file
-                entry = entries.keys()[0]
-                self.client.download((entry, entries[entry]), dest_filename=local_path.name)
-                local_path.setdata()
-                return
-            else:
-                self.__handle_error("Failed to find file '%s' in remote folder '%s'"
-                                    % (remote_filename, self.__get_node_name(self.folder)), raise_errors)
-        except Exception as e:
-            self.__handle_error("Failed to download file '%s' in remote folder '%s': %s"
-                                 % (remote_filename, self.__get_node_name(self.folder), str(e)), raise_errors)
-
-    @retry
-    def _list(self, raise_errors=False):
-        """List files in folder"""
-        try:
-            entries = self.client.get_files_in_node(self.folder)
-            return [ self.client.get_name_from_file({entry:entries[entry]}) for entry in entries]
-        except Exception as e:
-            self.__handle_error("Failed to fetch list of files in remote folder '%s': %s"
-                                % (self.__get_node_name(self.folder), str(e)), raise_errors)
-
-    @retry
-    def delete(self, filename_list, raise_errors=False):
-        """Delete files in filename_list"""
-        files = self.client.get_files()
-        for filename in filename_list:
-            entries = self.__filter_entries(files, self.folder, filename)
-            try:
-                if len(entries) > 0:
-                    for entry in entries:
-                        if self.client.destroy(entry):
-                            self.__handle_error("Failed to remove file '%s' in remote folder '%s'"
-                                % (filename, self.__get_node_name(self.folder)), raise_errors)
-                else:
-                    log.Warn("Failed to fetch file '%s' in remote folder '%s'"
-                             % (filename, self.__get_node_name(self.folder)))
-            except Exception as e:
-                self.__handle_error("Failed to remove file '%s' in remote folder '%s': %s"
-                                    % (filename, self.__get_node_name(self.folder), str(e)), raise_errors)
+    def _put(self, source_path, remote_filename):
+        try:
+            self._delete(remote_filename)
+        except Exception:
+            pass
+        self.client.upload(source_path.get_canonical(), self.folder, dest_filename=remote_filename)
+
+    def _get(self, remote_filename, local_path):
+        files = self.client.get_files()
+        entries = self.__filter_entries(files, self.folder, remote_filename, 'file')
+        if len(entries):
+            # get first matching remote file
+            entry = entries.keys()[0]
+            self.client.download((entry, entries[entry]), dest_filename=local_path.name)
+        else:
+            raise BackendException("Failed to find file '%s' in remote folder '%s'"
+                                   % (remote_filename, self.__get_node_name(self.folder)),
+                                   code=log.ErrorCode.backend_not_found)
+
+    def _list(self):
+        entries = self.client.get_files_in_node(self.folder)
+        return [self.client.get_name_from_file({entry:entries[entry]}) for entry in entries]
+
+    def _delete(self, filename):
+        files = self.client.get_files()
+        entries = self.__filter_entries(files, self.folder, filename, 'file')
+        if len(entries):
+            self.client.destroy(entries.keys()[0])
+        else:
+            raise BackendException("Failed to find file '%s' in remote folder '%s'"
+                                   % (filename, self.__get_node_name(self.folder)),
+                                   code=log.ErrorCode.backend_not_found)
 
     def __get_node_name(self, handle):
         """get node name from public handle"""
         files = self.client.get_files()
         return self.client.get_name_from_file({handle:files[handle]})
-        
-    def __handle_error(self, message, raise_errors=True):
-        if raise_errors:
-            raise BackendException(message)
-        else:
-            log.FatalError(message, log.ErrorCode.backend_error)
 
     def __authorize(self, email, password):
-        try:
-            self.client.login(email, password)
-        except Exception as e:
-            self.__handle_error('Error while authenticating client: %s.' % str(e))
+        self.client.login(email, password)
 
     def __filter_entries(self, entries, parent_id=None, title=None, type=None):
         result = {}
         type_map = { 'folder': 1, 'file': 0 }
 
-        try:
-            for k, v in entries.items():
-                try:
-                    if parent_id != None:
-                        assert(v['p'] == parent_id)
-                    if title != None:
-                        assert(v['a']['n'] == title)
-                    if type != None:
-                        assert(v['t'] == type_map[type])
-                except AssertionError:
-                    continue
-
-                result.update({k:v})
-
-            return result
-        except Exception as e:
-            self.__handle_error('Error while fetching remote entries: %s.' % str(e))
+        for k, v in entries.items():
+            try:
+                if parent_id != None:
+                    assert(v['p'] == parent_id)
+                if title != None:
+                    assert(v['a']['n'] == title)
+                if type != None:
+                    assert(v['t'] == type_map[type])
+            except AssertionError:
+                continue
+
+            result.update({k:v})
+
+        return result
 
 duplicity.backend.register_backend('mega', MegaBackend)

=== renamed file 'duplicity/backends/~par2wrapperbackend.py' => 'duplicity/backends/par2backend.py'
--- duplicity/backends/~par2wrapperbackend.py	2014-04-17 19:53:30 +0000
+++ duplicity/backends/par2backend.py	2014-04-28 02:49:55 +0000
@@ -16,14 +16,16 @@
 # along with duplicity; if not, write to the Free Software Foundation,
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
+from future_builtins import filter
+
 import os
 import re
 from duplicity import backend
-from duplicity.errors import UnsupportedBackendScheme, BackendException
+from duplicity.errors import BackendException
 from duplicity import log
 from duplicity import globals
 
-class Par2WrapperBackend(backend.Backend):
+class Par2Backend(backend.Backend):
     """This backend wrap around other backends and create Par2 recovery files
     before the file and the Par2 files are transfered with the wrapped backend.
     
@@ -37,13 +39,15 @@
         except AttributeError:
             self.redundancy = 10
 
-        try:
-            url_string = self.parsed_url.url_string.lstrip('par2+')
-            self.wrapped_backend = backend.get_backend(url_string)
-        except:
-            raise UnsupportedBackendScheme(self.parsed_url.url_string)
-
-    def put(self, source_path, remote_filename = None):
+        self.wrapped_backend = backend.get_backend_object(parsed_url.url_string)
+
+        for attr in ['_get', '_put', '_list', '_delete', '_delete_list',
+                     '_query', '_query_list', '_retry_cleanup', '_error_code',
+                     '_move', '_close']:
+            if hasattr(self.wrapped_backend, attr):
+                setattr(self, attr, getattr(self, attr[1:]))
+
+    def transfer(self, method, source_path, remote_filename):
         """create Par2 files and transfer the given file and the Par2 files
         with the wrapped backend.
         
@@ -52,13 +56,14 @@
         the soure_path with remote_filename into this. 
         """
         import pexpect
-        if remote_filename is None:
-            remote_filename = source_path.get_filename()
 
         par2temp = source_path.get_temp_in_same_dir()
         par2temp.mkdir()
         source_symlink = par2temp.append(remote_filename)
-        os.symlink(source_path.get_canonical(), source_symlink.get_canonical())
+        source_target = source_path.get_canonical()
+        if not os.path.isabs(source_target):
+            source_target = os.path.join(os.getcwd(), source_target)
+        os.symlink(source_target, source_symlink.get_canonical())
         source_symlink.setdata()
 
         log.Info("Create Par2 recovery files")
@@ -70,16 +75,17 @@
             for file in par2temp.listdir():
                 files_to_transfer.append(par2temp.append(file))
 
-        ret = self.wrapped_backend.put(source_path, remote_filename)
+        method(source_path, remote_filename)
         for file in files_to_transfer:
-            self.wrapped_backend.put(file, file.get_filename())
+            method(file, file.get_filename())
 
         par2temp.deltree()
-        return ret
-
-    def move(self, source_path, remote_filename = None):
-        self.put(source_path, remote_filename)
-        source_path.delete()
+
+    def put(self, local, remote):
+        self.transfer(self.wrapped_backend._put, local, remote)
+
+    def move(self, local, remote):
+        self.transfer(self.wrapped_backend._move, local, remote)
 
     def get(self, remote_filename, local_path):
         """transfer remote_filename and the related .par2 file into
@@ -94,22 +100,23 @@
         par2temp.mkdir()
         local_path_temp = par2temp.append(remote_filename)
 
-        ret = self.wrapped_backend.get(remote_filename, local_path_temp)
+        self.wrapped_backend._get(remote_filename, local_path_temp)
 
         try:
             par2file = par2temp.append(remote_filename + '.par2')
-            self.wrapped_backend.get(par2file.get_filename(), par2file)
+            self.wrapped_backend._get(par2file.get_filename(), par2file)
 
             par2verify = 'par2 v -q -q %s %s' % (par2file.get_canonical(), local_path_temp.get_canonical())
             out, returncode = pexpect.run(par2verify, -1, True)
 
             if returncode:
                 log.Warn("File is corrupt. Try to repair %s" % remote_filename)
-                par2volumes = self.list(re.compile(r'%s\.vol[\d+]*\.par2' % remote_filename))
+                par2volumes = filter(re.compile((r'%s\.vol[\d+]*\.par2' % remote_filename).match,
+                                     self.wrapped_backend._list()))
 
                 for filename in par2volumes:
                     file = par2temp.append(filename)
-                    self.wrapped_backend.get(filename, file)
+                    self.wrapped_backend._get(filename, file)
 
                 par2repair = 'par2 r -q -q %s %s' % (par2file.get_canonical(), local_path_temp.get_canonical())
                 out, returncode = pexpect.run(par2repair, -1, True)
@@ -124,25 +131,23 @@
         finally:
             local_path_temp.rename(local_path)
             par2temp.deltree()
-        return ret
 
-    def list(self, filter = re.compile(r'(?!.*\.par2$)')):
-        """default filter all files that ends with ".par"
-        filter can be a re.compile instance or False for all remote files
+    def delete(self, filename):
+        """delete given filename and its .par2 files
         """
-        list = self.wrapped_backend.list()
-        if not filter:
-            return list
-        filtered_list = []
-        for item in list:
-            if filter.match(item):
-                filtered_list.append(item)
-        return filtered_list
-
-    def delete(self, filename_list):
+        self.wrapped_backend._delete(filename)
+
+        remote_list = self.list()
+        filename_list = [filename]
+        c =  re.compile(r'%s(?:\.vol[\d+]*)?\.par2' % filename)
+        for remote_filename in remote_list:
+            if c.match(remote_filename):
+                self.wrapped_backend._delete(remote_filename)
+
+    def delete_list(self, filename_list):
         """delete given filename_list and all .par2 files that belong to them
         """
-        remote_list = self.list(False)
+        remote_list = self.list()
 
         for filename in filename_list[:]:
             c =  re.compile(r'%s(?:\.vol[\d+]*)?\.par2' % filename)
@@ -150,46 +155,25 @@
                 if c.match(remote_filename):
                     filename_list.append(remote_filename)
 
-        return self.wrapped_backend.delete(filename_list)
-
-    """just return the output of coresponding wrapped backend
-    for all other functions
-    """
-    def query_info(self, filename_list, raise_errors=True):
-        return self.wrapped_backend.query_info(filename_list, raise_errors)
-
-    def get_password(self):
-        return self.wrapped_backend.get_password()
-
-    def munge_password(self, commandline):
-        return self.wrapped_backend.munge_password(commandline)
-
-    def run_command(self, commandline):
-        return self.wrapped_backend.run_command(commandline)
-    def run_command_persist(self, commandline):
-        return self.wrapped_backend.run_command_persist(commandline)
-
-    def popen(self, commandline):
-        return self.wrapped_backend.popen(commandline)
-    def popen_persist(self, commandline):
-        return self.wrapped_backend.popen_persist(commandline)
-
-    def _subprocess_popen(self, commandline):
-        return self.wrapped_backend._subprocess_popen(commandline)
-
-    def subprocess_popen(self, commandline):
-        return self.wrapped_backend.subprocess_popen(commandline)
-
-    def subprocess_popen_persist(self, commandline):
-        return self.wrapped_backend.subprocess_popen_persist(commandline)
+        return self.wrapped_backend._delete_list(filename_list)
+
+
+    def list(self):
+        return self.wrapped_backend._list()
+
+    def retry_cleanup(self):
+        self.wrapped_backend._retry_cleanup()
+
+    def error_code(self, operation, e):
+        return self.wrapped_backend._error_code(operation, e)
+
+    def query(self, filename):
+        return self.wrapped_backend._query(filename)
+
+    def query_list(self, filename_list):
+        return self.wrapped_backend._query(filename_list)
 
     def close(self):
-        return self.wrapped_backend.close()
-
-"""register this backend with leading "par2+" for all already known backends
-
-files must be sorted in duplicity.backend.import_backends to catch
-all supported backends
-"""
-for item in backend._backends.keys():
-    backend.register_backend('par2+' + item, Par2WrapperBackend)
+        self.wrapped_backend._close()
+
+backend.register_backend_prefix('par2', Par2Backend)

=== modified file 'duplicity/backends/rsyncbackend.py'
--- duplicity/backends/rsyncbackend.py	2014-04-25 23:20:12 +0000
+++ duplicity/backends/rsyncbackend.py	2014-04-28 02:49:55 +0000
@@ -23,7 +23,7 @@
 import tempfile
 
 import duplicity.backend
-from duplicity.errors import * #@UnusedWildImport
+from duplicity.errors import InvalidBackendURL
 from duplicity import globals, tempdir, util
 
 class RsyncBackend(duplicity.backend.Backend):
@@ -58,12 +58,13 @@
             if port:
                 port = " --port=%s" % port
         else:
+            host_string = host + ":" if host else ""
             if parsed_url.path.startswith("//"):
                 # its an absolute path
-                self.url_string = "%s:/%s" % (host, parsed_url.path.lstrip('/'))
+                self.url_string = "%s/%s" % (host_string, parsed_url.path.lstrip('/'))
             else:
                 # its a relative path
-                self.url_string = "%s:%s" % (host, parsed_url.path.lstrip('/'))
+                self.url_string = "%s%s" % (host_string, parsed_url.path.lstrip('/'))
             if parsed_url.port:
                 port = " -p %s" % parsed_url.port
         # add trailing slash if missing
@@ -105,29 +106,17 @@
         raise InvalidBackendURL("Could not determine rsync path: %s"
                                     "" % self.munge_password( url ) )
 
-    def run_command(self, commandline):
-        result, stdout, stderr = self.subprocess_popen_persist(commandline)
-        return result, stdout
-
-    def put(self, source_path, remote_filename = None):
-        """Use rsync to copy source_dir/filename to remote computer"""
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
+    def _put(self, source_path, remote_filename):
         remote_path = os.path.join(self.url_string, remote_filename)
         commandline = "%s %s %s" % (self.cmd, source_path.name, remote_path)
-        self.run_command(commandline)
+        self.subprocess_popen(commandline)
 
-    def get(self, remote_filename, local_path):
-        """Use rsync to get a remote file"""
+    def _get(self, remote_filename, local_path):
         remote_path = os.path.join (self.url_string, remote_filename)
         commandline = "%s %s %s" % (self.cmd, remote_path, local_path.name)
-        self.run_command(commandline)
-        local_path.setdata()
-        if not local_path.exists():
-            raise BackendException("File %s not found" % local_path.name)
+        self.subprocess_popen(commandline)
 
-    def list(self):
-        """List files"""
+    def _list(self):
         def split (str):
             line = str.split ()
             if len (line) > 4 and line[4] != '.':
@@ -135,20 +124,17 @@
             else:
                 return None
         commandline = "%s %s" % (self.cmd, self.url_string)
-        result, stdout = self.run_command(commandline)
+        result, stdout, stderr = self.subprocess_popen(commandline)
         return [x for x in map (split, stdout.split('\n')) if x]
 
-    def delete(self, filename_list):
-        """Delete files."""
+    def _delete_list(self, filename_list):
         delete_list = filename_list
         dont_delete_list = []
-        for file in self.list ():
+        for file in self._list ():
             if file in delete_list:
                 delete_list.remove (file)
             else:
                 dont_delete_list.append (file)
-        if len (delete_list) > 0:
-            raise BackendException("Files %s not found" % str (delete_list))
 
         dir = tempfile.mkdtemp()
         exclude, exclude_name = tempdir.default().mkstemp_file()
@@ -162,7 +148,7 @@
         exclude.close()
         commandline = ("%s --recursive --delete --exclude-from=%s %s/ %s" %
                                    (self.cmd, exclude_name, dir, self.url_string))
-        self.run_command(commandline)
+        self.subprocess_popen(commandline)
         for file in to_delete:
             util.ignore_missing(os.unlink, file)
         os.rmdir (dir)

=== modified file 'duplicity/backends/sshbackend.py'
--- duplicity/backends/sshbackend.py	2014-04-17 21:54:04 +0000
+++ duplicity/backends/sshbackend.py	2014-04-28 02:49:55 +0000
@@ -18,6 +18,7 @@
 # along with duplicity; if not, write to the Free Software Foundation,
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
+import duplicity.backend
 from duplicity import globals, log
 
 def warn_option(option, optionvar):
@@ -26,11 +27,15 @@
 
 if (globals.ssh_backend and
     globals.ssh_backend.lower().strip() == 'pexpect'):
-    from . import _ssh_pexpect
+    from ._ssh_pexpect import SSHPExpectBackend as SSHBackend
 else:
     # take user by the hand to prevent typo driven bug reports
     if globals.ssh_backend.lower().strip() != 'paramiko':
         log.Warn(_("Warning: Selected ssh backend '%s' is neither 'paramiko nor 'pexpect'. Will use default paramiko instead.") % globals.ssh_backend)
     warn_option("--scp-command", globals.scp_command)
     warn_option("--sftp-command", globals.sftp_command)
-    from . import _ssh_paramiko
+    from ._ssh_paramiko import SSHParamikoBackend as SSHBackend
+
+duplicity.backend.register_backend("sftp", SSHBackend)
+duplicity.backend.register_backend("scp", SSHBackend)
+duplicity.backend.register_backend("ssh", SSHBackend)

=== modified file 'duplicity/backends/swiftbackend.py'
--- duplicity/backends/swiftbackend.py	2014-04-17 22:03:10 +0000
+++ duplicity/backends/swiftbackend.py	2014-04-28 02:49:55 +0000
@@ -19,14 +19,11 @@
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
 import os
-import time
 
 import duplicity.backend
-from duplicity import globals
 from duplicity import log
-from duplicity.errors import * #@UnusedWildImport
-from duplicity.util import exception_traceback
-from duplicity.backend import retry
+from duplicity.errors import BackendException
+
 
 class SwiftBackend(duplicity.backend.Backend):
     """
@@ -82,121 +79,30 @@
                            % (e.__class__.__name__, str(e)),
                            log.ErrorCode.connection_failed)
 
-    def put(self, source_path, remote_filename = None):
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
-
-        for n in range(1, globals.num_retries+1):
-            log.Info("Uploading '%s/%s' " % (self.container, remote_filename))
-            try:
-                self.conn.put_object(self.container,
-                                     remote_filename, 
-                                     file(source_path.name))
-                return
-            except self.resp_exc as error:
-                log.Warn("Upload of '%s' failed (attempt %d): Swift server returned: %s %s"
-                         % (remote_filename, n, error.http_status, error.message))
-            except Exception as e:
-                log.Warn("Upload of '%s' failed (attempt %s): %s: %s"
-                        % (remote_filename, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up uploading '%s' after %s attempts"
-                 % (remote_filename, globals.num_retries))
-        raise BackendException("Error uploading '%s'" % remote_filename)
-
-    def get(self, remote_filename, local_path):
-        for n in range(1, globals.num_retries+1):
-            log.Info("Downloading '%s/%s'" % (self.container, remote_filename))
-            try:
-                headers, body = self.conn.get_object(self.container,
-                                                     remote_filename)
-                f = open(local_path.name, 'w')
-                for chunk in body:
-                    f.write(chunk)
-                local_path.setdata()
-                return
-            except self.resp_exc as resperr:
-                log.Warn("Download of '%s' failed (attempt %s): Swift server returned: %s %s"
-                         % (remote_filename, n, resperr.http_status, resperr.message))
-            except Exception as e:
-                log.Warn("Download of '%s' failed (attempt %s): %s: %s"
-                         % (remote_filename, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up downloading '%s' after %s attempts"
-                 % (remote_filename, globals.num_retries))
-        raise BackendException("Error downloading '%s/%s'"
-                               % (self.container, remote_filename))
+    def _error_code(self, operation, e):
+        if isinstance(e, self.resp_exc):
+            if e.http_status == 404:
+                return log.ErrorCode.backend_not_found
+
+    def _put(self, source_path, remote_filename):
+        self.conn.put_object(self.container, remote_filename,
+                             file(source_path.name))
+
+    def _get(self, remote_filename, local_path):
+        headers, body = self.conn.get_object(self.container, remote_filename)
+        with open(local_path.name, 'wb') as f:
+            for chunk in body:
+                f.write(chunk)
 
     def _list(self):
-        for n in range(1, globals.num_retries+1):
-            log.Info("Listing '%s'" % (self.container))
-            try:
-                # Cloud Files will return a max of 10,000 objects.  We have
-                # to make multiple requests to get them all.
-                headers, objs = self.conn.get_container(self.container)
-                return [ o['name'] for o in objs ]
-            except self.resp_exc as resperr:
-                log.Warn("Listing of '%s' failed (attempt %s): Swift server returned: %s %s"
-                         % (self.container, n, resperr.http_status, resperr.message))
-            except Exception as e:
-                log.Warn("Listing of '%s' failed (attempt %s): %s: %s"
-                         % (self.container, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up listing of '%s' after %s attempts"
-                 % (self.container, globals.num_retries))
-        raise BackendException("Error listing '%s'"
-                               % (self.container))
-
-    def delete_one(self, remote_filename):
-        for n in range(1, globals.num_retries+1):
-            log.Info("Deleting '%s/%s'" % (self.container, remote_filename))
-            try:
-                self.conn.delete_object(self.container, remote_filename)
-                return
-            except self.resp_exc as resperr:
-                if n > 1 and resperr.http_status == 404:
-                    # We failed on a timeout, but delete succeeded on the server
-                    log.Warn("Delete of '%s' missing after retry - must have succeded earlier" % remote_filename )
-                    return
-                log.Warn("Delete of '%s' failed (attempt %s): Swift server returned: %s %s"
-                         % (remote_filename, n, resperr.http_status, resperr.message))
-            except Exception as e:
-                log.Warn("Delete of '%s' failed (attempt %s): %s: %s"
-                         % (remote_filename, n, e.__class__.__name__, str(e)))
-                log.Debug("Backtrace of previous error: %s"
-                          % exception_traceback())
-            time.sleep(30)
-        log.Warn("Giving up deleting '%s' after %s attempts"
-                 % (remote_filename, globals.num_retries))
-        raise BackendException("Error deleting '%s/%s'"
-                               % (self.container, remote_filename))
-
-    def delete(self, filename_list):
-        for file in filename_list:
-            self.delete_one(file)
-            log.Debug("Deleted '%s/%s'" % (self.container, file))
-
-    @retry
-    def _query_file_info(self, filename, raise_errors=False):
-        try:
-            sobject = self.conn.head_object(self.container, filename)
-            return {'size': int(sobject['content-length'])}
-        except self.resp_exc:
-            return {'size': -1}
-        except Exception as e:
-            log.Warn("Error querying '%s/%s': %s"
-                     "" % (self.container,
-                           filename,
-                           str(e)))
-            if raise_errors:
-                raise e
-            else:
-                return {'size': None}
+        headers, objs = self.conn.get_container(self.container)
+        return [ o['name'] for o in objs ]
+
+    def _delete(self, filename):
+        self.conn.delete_object(self.container, filename)
+
+    def _query(self, filename):
+        sobject = self.conn.head_object(self.container, filename)
+        return {'size': int(sobject['content-length'])}
 
 duplicity.backend.register_backend("swift", SwiftBackend)

=== modified file 'duplicity/backends/tahoebackend.py'
--- duplicity/backends/tahoebackend.py	2013-12-27 06:39:00 +0000
+++ duplicity/backends/tahoebackend.py	2014-04-28 02:49:55 +0000
@@ -20,9 +20,8 @@
 
 import duplicity.backend
 from duplicity import log
-from duplicity.errors import * #@UnusedWildImport
+from duplicity.errors import BackendException
 
-from commands import getstatusoutput
 
 class TAHOEBackend(duplicity.backend.Backend):
     """
@@ -36,10 +35,8 @@
 
         self.alias = url[0]
 
-        if len(url) > 2:
+        if len(url) > 1:
             self.directory = "/".join(url[1:])
-        elif len(url) == 2:
-            self.directory = url[1]
         else:
             self.directory = ""
 
@@ -59,28 +56,20 @@
 
     def run(self, *args):
         cmd = " ".join(args)
-        log.Debug("tahoe execute: %s" % cmd)
-        (status, output) = getstatusoutput(cmd)
-
-        if status != 0:
-            raise BackendException("Error running %s" % cmd)
-        else:
-            return output
-
-    def put(self, source_path, remote_filename=None):
+        _, output, _ = self.subprocess_popen(cmd)
+        return output
+
+    def _put(self, source_path, remote_filename):
         self.run("tahoe", "cp", source_path.name, self.get_remote_path(remote_filename))
 
-    def get(self, remote_filename, local_path):
+    def _get(self, remote_filename, local_path):
         self.run("tahoe", "cp", self.get_remote_path(remote_filename), local_path.name)
-        local_path.setdata()
 
     def _list(self):
-        log.Debug("tahoe: List")
-        return self.run("tahoe", "ls", self.get_remote_path()).split('\n')
+        output = self.run("tahoe", "ls", self.get_remote_path())
+        return output.split('\n') if output else []
 
-    def delete(self, filename_list):
-        log.Debug("tahoe: delete(%s)" % filename_list)
-        for filename in filename_list:
-            self.run("tahoe", "rm", self.get_remote_path(filename))
+    def _delete(self, filename):
+        self.run("tahoe", "rm", self.get_remote_path(filename))
 
 duplicity.backend.register_backend("tahoe", TAHOEBackend)

=== modified file 'duplicity/backends/webdavbackend.py'
--- duplicity/backends/webdavbackend.py	2014-04-25 15:03:00 +0000
+++ duplicity/backends/webdavbackend.py	2014-04-28 02:49:55 +0000
@@ -32,8 +32,7 @@
 import duplicity.backend
 from duplicity import globals
 from duplicity import log
-from duplicity.errors import * #@UnusedWildImport
-from duplicity.backend import retry_fatal
+from duplicity.errors import BackendException, FatalBackendException
 
 class CustomMethodRequest(urllib2.Request):
     """
@@ -54,7 +53,7 @@
                 global socket, ssl
                 import socket, ssl
             except ImportError:
-                raise FatalBackendError("Missing socket or ssl libraries.")
+                raise FatalBackendException("Missing socket or ssl libraries.")
 
             httplib.HTTPSConnection.__init__(self, *args, **kwargs)
 
@@ -71,21 +70,21 @@
                         break
             # still no cacert file, inform user
             if not self.cacert_file:
-                raise FatalBackendError("""For certificate verification a cacert database file is needed in one of these locations: %s
+                raise FatalBackendException("""For certificate verification a cacert database file is needed in one of these locations: %s
 Hints:
   Consult the man page, chapter 'SSL Certificate Verification'.
   Consider using the options --ssl-cacert-file, --ssl-no-check-certificate .""" % ", ".join(cacert_candidates) )
             # check if file is accessible (libssl errors are not very detailed)
             if not os.access(self.cacert_file, os.R_OK):
-                raise FatalBackendError("Cacert database file '%s' is not readable." % cacert_file)
+                raise FatalBackendException("Cacert database file '%s' is not readable." % cacert_file)
 
         def connect(self):
             # create new socket
             sock = socket.create_connection((self.host, self.port),
                                             self.timeout)
-            if self._tunnel_host:
+            if self.tunnel_host:
                 self.sock = sock
-                self._tunnel()
+                self.tunnel()
 
             # wrap the socket in ssl using verification
             self.sock = ssl.wrap_socket(sock,
@@ -126,7 +125,7 @@
 
         self.username = parsed_url.username
         self.password = self.get_password()
-        self.directory = self._sanitize_path(parsed_url.path)
+        self.directory = self.sanitize_path(parsed_url.path)
 
         log.Info("Using WebDAV protocol %s" % (globals.webdav_proto,))
         log.Info("Using WebDAV host %s port %s" % (parsed_url.hostname, parsed_url.port))
@@ -134,30 +133,33 @@
 
         self.conn = None
 
-    def _sanitize_path(self,path):
+    def sanitize_path(self,path):
         if path:
             foldpath = re.compile('/+')
             return foldpath.sub('/', path + '/' )
         else:
             return '/'
 
-    def _getText(self,nodelist):
+    def getText(self,nodelist):
         rc = ""
         for node in nodelist:
             if node.nodeType == node.TEXT_NODE:
                 rc = rc + node.data
         return rc
 
-    def _connect(self, forced=False):
+    def _retry_cleanup(self):
+        self.connect(forced=True)
+
+    def connect(self, forced=False):
         """
         Connect or re-connect to the server, updates self.conn
         # reconnect on errors as a precaution, there are errors e.g.
         # "[Errno 32] Broken pipe" or SSl errors that render the connection unusable
         """
-        if self.retry_count<=1 and self.conn \
+        if not forced and self.conn \
             and self.conn.host == self.parsed_url.hostname: return
 
-        log.Info("WebDAV create connection on '%s' (retry %s) " % (self.parsed_url.hostname,self.retry_count) )
+        log.Info("WebDAV create connection on '%s'" % (self.parsed_url.hostname))
         if self.conn: self.conn.close()
         # http schemes needed for redirect urls from servers
         if self.parsed_url.scheme in ['webdav','http']:
@@ -168,9 +170,9 @@
             else:
                 self.conn = VerifiedHTTPSConnection(self.parsed_url.hostname, self.parsed_url.port)
         else:
-            raise FatalBackendError("WebDAV Unknown URI scheme: %s" % (self.parsed_url.scheme))
+            raise FatalBackendException("WebDAV Unknown URI scheme: %s" % (self.parsed_url.scheme))
 
-    def close(self):
+    def _close(self):
         self.conn.close()
 
     def request(self, method, path, data=None, redirected=0):
@@ -178,7 +180,7 @@
         Wraps the connection.request method to retry once if authentication is
         required
         """
-        self._connect()
+        self.connect()
 
         quoted_path = urllib.quote(path,"/:~")
 
@@ -197,12 +199,12 @@
             if redirect_url:
                 log.Notice("WebDAV redirect to: %s " % urllib.unquote(redirect_url) )
                 if redirected > 10:
-                    raise FatalBackendError("WebDAV redirected 10 times. Giving up.")
+                    raise FatalBackendException("WebDAV redirected 10 times. Giving up.")
                 self.parsed_url = duplicity.backend.ParsedUrl(redirect_url)
-                self.directory = self._sanitize_path(self.parsed_url.path)
+                self.directory = self.sanitize_path(self.parsed_url.path)
                 return self.request(method,self.directory,data,redirected+1)
             else:
-                raise FatalBackendError("WebDAV missing location header in redirect response.")
+                raise FatalBackendException("WebDAV missing location header in redirect response.")
         elif response.status == 401:
             response.close()
             self.headers['Authorization'] = self.get_authorization(response, quoted_path)
@@ -261,10 +263,7 @@
         auth_string = self.digest_auth_handler.get_authorization(dummy_req, self.digest_challenge)
         return 'Digest %s' % auth_string
 
-    @retry_fatal
     def _list(self):
-        """List files in directory"""
-        log.Info("Listing directory %s on WebDAV server" % (self.directory,))
         response = None
         try:
             self.headers['Depth'] = "1"
@@ -289,7 +288,7 @@
             dom = xml.dom.minidom.parseString(document)
             result = []
             for href in dom.getElementsByTagName('d:href') + dom.getElementsByTagName('D:href'):
-                filename = self.__taste_href(href)
+                filename = self.taste_href(href)
                 if filename:
                     result.append(filename)
             return result
@@ -308,7 +307,7 @@
         for i in range(1,len(dirs)):
             d="/".join(dirs[0:i+1])+"/"
 
-            self.close() # or we get previous request's data or exception
+            self._close() # or we get previous request's data or exception
             self.headers['Depth'] = "1"
             response = self.request("PROPFIND", d)
             del self.headers['Depth']
@@ -317,21 +316,21 @@
 
             if response.status == 404:
                 log.Info("Creating missing directory %s" % d)
-                self.close() # or we get previous request's data or exception
+                self._close() # or we get previous request's data or exception
 
                 res = self.request("MKCOL", d)
                 if res.status != 201:
                     raise BackendException("WebDAV MKCOL %s failed: %s %s" % (d,res.status,res.reason))
-                self.close()
+                self._close()
 
-    def __taste_href(self, href):
+    def taste_href(self, href):
         """
         Internal helper to taste the given href node and, if
         it is a duplicity file, collect it as a result file.
 
         @return: A matching filename, or None if the href did not match.
         """
-        raw_filename = self._getText(href.childNodes).strip()
+        raw_filename = self.getText(href.childNodes).strip()
         parsed_url = urlparse.urlparse(urllib.unquote(raw_filename))
         filename = parsed_url.path
         log.Debug("webdav path decoding and translation: "
@@ -361,11 +360,8 @@
         else:
             return None
 
-    @retry_fatal
-    def get(self, remote_filename, local_path):
-        """Get remote filename, saving it to local_path"""
+    def _get(self, remote_filename, local_path):
         url = self.directory + remote_filename
-        log.Info("Retrieving %s from WebDAV server" % (url ,))
         response = None
         try:
             target_file = local_path.open("wb")
@@ -376,7 +372,6 @@
                 #import hashlib
                 #log.Info("WebDAV GOT %s bytes with md5=%s" % (len(data),hashlib.md5(data).hexdigest()) )
                 assert not target_file.close()
-                local_path.setdata()
                 response.close()
             else:
                 status = response.status
@@ -388,13 +383,8 @@
         finally:
             if response: response.close()
 
-    @retry_fatal
-    def put(self, source_path, remote_filename = None):
-        """Transfer source_path to remote_filename"""
-        if not remote_filename:
-            remote_filename = source_path.get_filename()
+    def _put(self, source_path, remote_filename):
         url = self.directory + remote_filename
-        log.Info("Saving %s on WebDAV server" % (url ,))
         response = None
         try:
             source_file = source_path.open("rb")
@@ -412,27 +402,23 @@
         finally:
             if response: response.close()
 
-    @retry_fatal
-    def delete(self, filename_list):
-        """Delete files in filename_list"""
-        for filename in filename_list:
-            url = self.directory + filename
-            log.Info("Deleting %s from WebDAV server" % (url ,))
-            response = None
-            try:
-                response = self.request("DELETE", url)
-                if response.status in [200, 204]:
-                    response.read()
-                    response.close()
-                else:
-                    status = response.status
-                    reason = response.reason
-                    response.close()
-                    raise BackendException("Bad status code %s reason %s." % (status,reason))
-            except Exception as e:
-                raise e
-            finally:
-                if response: response.close()
+    def _delete(self, filename):
+        url = self.directory + filename
+        response = None
+        try:
+            response = self.request("DELETE", url)
+            if response.status in [200, 204]:
+                response.read()
+                response.close()
+            else:
+                status = response.status
+                reason = response.reason
+                response.close()
+                raise BackendException("Bad status code %s reason %s." % (status,reason))
+        except Exception as e:
+            raise e
+        finally:
+            if response: response.close()
 
 duplicity.backend.register_backend("webdav", WebDAVBackend)
 duplicity.backend.register_backend("webdavs", WebDAVBackend)

=== modified file 'duplicity/commandline.py'
--- duplicity/commandline.py	2014-04-25 23:20:12 +0000
+++ duplicity/commandline.py	2014-04-28 02:49:55 +0000
@@ -210,13 +210,6 @@
     global select_opts, select_files, full_backup
     global list_current, collection_status, cleanup, remove_time, verify
 
-    def use_gio(*args):
-        try:
-            import duplicity.backends.giobackend
-            backend.force_backend(duplicity.backends.giobackend.GIOBackend)
-        except ImportError:
-            log.FatalError(_("Unable to load gio backend: %s") % str(sys.exc_info()[1]), log.ErrorCode.gio_not_available)
-
     def set_log_fd(fd):
         if fd < 1:
             raise optparse.OptionValueError("log-fd must be greater than zero.")
@@ -365,7 +358,9 @@
     # the time specified
     parser.add_option("--full-if-older-than", type = "time", dest = "full_force_time", metavar = _("time"))
 
-    parser.add_option("--gio", action = "callback", callback = use_gio)
+    parser.add_option("--gio",action = "callback", dest = "use_gio",
+                      callback = lambda o, s, v, p: (setattr(p.values, o.dest, True),
+                                                     old_fn_deprecation(s)))
 
     parser.add_option("--gpg-options", action = "extend", metavar = _("options"))
 
@@ -521,8 +516,8 @@
     # sftp command to use (ssh pexpect backend)
     parser.add_option("--sftp-command", metavar = _("command"))
 
-    # sftp command to use (ssh pexpect backend)
-    parser.add_option("--cf-command", metavar = _("command"))
+    # allow the user to switch cloudfiles backend
+    parser.add_option("--cf-backend", metavar = _("pyrax|cloudfiles"))
 
     # If set, use short (< 30 char) filenames for all the remote files.
     parser.add_option("--short-filenames", action = "callback",

=== modified file 'duplicity/errors.py'
--- duplicity/errors.py	2013-01-10 19:04:39 +0000
+++ duplicity/errors.py	2014-04-28 02:49:55 +0000
@@ -23,6 +23,8 @@
 Error/exception classes that do not fit naturally anywhere else.
 """
 
+from duplicity import log
+
 class DuplicityError(Exception):
     pass
 
@@ -68,9 +70,11 @@
     """
     Raised to indicate a backend specific problem.
     """
-    pass
+    def __init__(self, msg, code=log.ErrorCode.backend_error):
+        super(BackendException, self).__init__(msg)
+        self.code = code
 
-class FatalBackendError(DuplicityError):
+class FatalBackendException(BackendException):
     """
     Raised to indicate a backend failed fatally.
     """

=== modified file 'duplicity/globals.py'
--- duplicity/globals.py	2014-04-17 21:49:37 +0000
+++ duplicity/globals.py	2014-04-28 02:49:55 +0000
@@ -284,3 +284,6 @@
 
 # Level of Redundancy in % for Par2 files
 par2_redundancy = 10
+
+# Whether to enable gio backend
+use_gio = False

=== modified file 'po/duplicity.pot'
--- po/duplicity.pot	2014-04-25 15:03:00 +0000
+++ po/duplicity.pot	2014-04-28 02:49:55 +0000
@@ -8,7 +8,7 @@
 msgstr ""
 "Project-Id-Version: PACKAGE VERSION\n"
 "Report-Msgid-Bugs-To: Kenneth Loafman <[e-mail]>\n"
-"POT-Creation-Date: 2014-04-21 11:04-0500\n"
+"POT-Creation-Date: 2014-04-27 22:39-0400\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language-Team: LANGUAGE <[e-mail]>\n"
@@ -69,243 +69,243 @@
 "Continuing restart on file %s."
 msgstr ""
 
-#: ../bin/duplicity:299
+#: ../bin/duplicity:297
 #, python-format
 msgid "File %s was corrupted during upload."
 msgstr ""
 
-#: ../bin/duplicity:333
+#: ../bin/duplicity:331
 msgid ""
 "Restarting backup, but current encryption settings do not match original "
 "settings"
 msgstr ""
 
-#: ../bin/duplicity:356
+#: ../bin/duplicity:354
 #, python-format
 msgid "Restarting after volume %s, file %s, block %s"
 msgstr ""
 
-#: ../bin/duplicity:423
+#: ../bin/duplicity:421
 #, python-format
 msgid "Processed volume %d"
 msgstr ""
 
-#: ../bin/duplicity:572
+#: ../bin/duplicity:570
 msgid ""
 "Fatal Error: Unable to start incremental backup.  Old signatures not found "
 "and incremental specified"
 msgstr ""
 
-#: ../bin/duplicity:576
+#: ../bin/duplicity:574
 msgid "No signatures found, switching to full backup."
 msgstr ""
 
-#: ../bin/duplicity:590
+#: ../bin/duplicity:588
 msgid "Backup Statistics"
 msgstr ""
 
-#: ../bin/duplicity:695
+#: ../bin/duplicity:693
 #, python-format
 msgid "%s not found in archive, no files restored."
 msgstr ""
 
-#: ../bin/duplicity:699
+#: ../bin/duplicity:697
 msgid "No files found in archive - nothing restored."
 msgstr ""
 
-#: ../bin/duplicity:732
+#: ../bin/duplicity:730
 #, python-format
 msgid "Processed volume %d of %d"
 msgstr ""
 
+#: ../bin/duplicity:764
+#, python-format
+msgid "Invalid data - %s hash mismatch for file:"
+msgstr ""
+
 #: ../bin/duplicity:766
 #, python-format
-msgid "Invalid data - %s hash mismatch for file:"
-msgstr ""
-
-#: ../bin/duplicity:768
-#, python-format
 msgid "Calculated hash: %s"
 msgstr ""
 
-#: ../bin/duplicity:769
+#: ../bin/duplicity:767
 #, python-format
 msgid "Manifest hash: %s"
 msgstr ""
 
-#: ../bin/duplicity:807
+#: ../bin/duplicity:805
 #, python-format
 msgid "Volume was signed by key %s, not %s"
 msgstr ""
 
-#: ../bin/duplicity:837
+#: ../bin/duplicity:835
 #, python-format
 msgid "Verify complete: %s, %s."
 msgstr ""
 
-#: ../bin/duplicity:838
+#: ../bin/duplicity:836
 #, python-format
 msgid "%d file compared"
 msgid_plural "%d files compared"
 msgstr[0] ""
 msgstr[1] ""
 
-#: ../bin/duplicity:840
+#: ../bin/duplicity:838
 #, python-format
 msgid "%d difference found"
 msgid_plural "%d differences found"
 msgstr[0] ""
 msgstr[1] ""
 
-#: ../bin/duplicity:859
+#: ../bin/duplicity:857
 msgid "No extraneous files found, nothing deleted in cleanup."
 msgstr ""
 
-#: ../bin/duplicity:864
+#: ../bin/duplicity:862
 msgid "Deleting this file from backend:"
 msgid_plural "Deleting these files from backend:"
 msgstr[0] ""
 msgstr[1] ""
 
-#: ../bin/duplicity:876
+#: ../bin/duplicity:874
 msgid "Found the following file to delete:"
 msgid_plural "Found the following files to delete:"
 msgstr[0] ""
 msgstr[1] ""
 
-#: ../bin/duplicity:880
+#: ../bin/duplicity:878
 msgid "Run duplicity again with the --force option to actually delete."
 msgstr ""
 
+#: ../bin/duplicity:921
+msgid "There are backup set(s) at time(s):"
+msgstr ""
+
 #: ../bin/duplicity:923
-msgid "There are backup set(s) at time(s):"
-msgstr ""
-
-#: ../bin/duplicity:925
 msgid "Which can't be deleted because newer sets depend on them."
 msgstr ""
 
-#: ../bin/duplicity:929
+#: ../bin/duplicity:927
 msgid ""
 "Current active backup chain is older than specified time.  However, it will "
 "not be deleted.  To remove all your backups, manually purge the repository."
 msgstr ""
 
-#: ../bin/duplicity:935
+#: ../bin/duplicity:933
 msgid "No old backup sets found, nothing deleted."
 msgstr ""
 
-#: ../bin/duplicity:938
+#: ../bin/duplicity:936
 msgid "Deleting backup chain at time:"
 msgid_plural "Deleting backup chains at times:"
 msgstr[0] ""
 msgstr[1] ""
 
+#: ../bin/duplicity:947
+#, python-format
+msgid "Deleting incremental signature chain %s"
+msgstr ""
+
 #: ../bin/duplicity:949
 #, python-format
-msgid "Deleting incremental signature chain %s"
-msgstr ""
-
-#: ../bin/duplicity:951
-#, python-format
 msgid "Deleting incremental backup chain %s"
 msgstr ""
 
+#: ../bin/duplicity:952
+#, python-format
+msgid "Deleting complete signature chain %s"
+msgstr ""
+
 #: ../bin/duplicity:954
 #, python-format
-msgid "Deleting complete signature chain %s"
-msgstr ""
-
-#: ../bin/duplicity:956
-#, python-format
 msgid "Deleting complete backup chain %s"
 msgstr ""
 
-#: ../bin/duplicity:962
+#: ../bin/duplicity:960
 msgid "Found old backup chain at the following time:"
 msgid_plural "Found old backup chains at the following times:"
 msgstr[0] ""
 msgstr[1] ""
 
-#: ../bin/duplicity:966
+#: ../bin/duplicity:964
 msgid "Rerun command with --force option to actually delete."
 msgstr ""
 
-#: ../bin/duplicity:1043
+#: ../bin/duplicity:1041
 #, python-format
 msgid "Deleting local %s (not authoritative at backend)."
 msgstr ""
 
-#: ../bin/duplicity:1047
+#: ../bin/duplicity:1045
 #, python-format
 msgid "Unable to delete %s: %s"
 msgstr ""
 
-#: ../bin/duplicity:1075 ../duplicity/dup_temp.py:263
+#: ../bin/duplicity:1073 ../duplicity/dup_temp.py:263
 #, python-format
 msgid "Failed to read %s: %s"
 msgstr ""
 
-#: ../bin/duplicity:1089
+#: ../bin/duplicity:1087
 #, python-format
 msgid "Copying %s to local cache."
 msgstr ""
 
-#: ../bin/duplicity:1137
+#: ../bin/duplicity:1135
 msgid "Local and Remote metadata are synchronized, no sync needed."
 msgstr ""
 
-#: ../bin/duplicity:1142
+#: ../bin/duplicity:1140
 msgid "Synchronizing remote metadata to local cache..."
 msgstr ""
 
-#: ../bin/duplicity:1157
+#: ../bin/duplicity:1155
 msgid "Sync would copy the following from remote to local:"
 msgstr ""
 
-#: ../bin/duplicity:1160
+#: ../bin/duplicity:1158
 msgid "Sync would remove the following spurious local files:"
 msgstr ""
 
-#: ../bin/duplicity:1203
+#: ../bin/duplicity:1201
 msgid "Unable to get free space on temp."
 msgstr ""
 
-#: ../bin/duplicity:1211
+#: ../bin/duplicity:1209
 #, python-format
 msgid "Temp space has %d available, backup needs approx %d."
 msgstr ""
 
-#: ../bin/duplicity:1214
+#: ../bin/duplicity:1212
 #, python-format
 msgid "Temp has %d available, backup will use approx %d."
 msgstr ""
 
-#: ../bin/duplicity:1222
+#: ../bin/duplicity:1220
 msgid "Unable to get max open files."
 msgstr ""
 
-#: ../bin/duplicity:1226
+#: ../bin/duplicity:1224
 #, python-format
 msgid ""
 "Max open files of %s is too low, should be >= 1024.\n"
 "Use 'ulimit -n 1024' or higher to correct.\n"
 msgstr ""
 
-#: ../bin/duplicity:1275
+#: ../bin/duplicity:1273
 msgid ""
 "RESTART: The first volume failed to upload before termination.\n"
 "         Restart is impossible...starting backup from beginning."
 msgstr ""
 
-#: ../bin/duplicity:1281
+#: ../bin/duplicity:1279
 #, python-format
 msgid ""
 "RESTART: Volumes %d to %d failed to upload before termination.\n"
 "         Restarting backup at volume %d."
 msgstr ""
 
-#: ../bin/duplicity:1288
+#: ../bin/duplicity:1286
 #, python-format
 msgid ""
 "RESTART: Impossible backup state: manifest has %d vols, remote has %d vols.\n"
@@ -314,7 +314,7 @@
 "         backup then restart the backup from the beginning."
 msgstr ""
 
-#: ../bin/duplicity:1310
+#: ../bin/duplicity:1308
 msgid ""
 "\n"
 "PYTHONOPTIMIZE in the environment causes duplicity to fail to\n"
@@ -324,54 +324,54 @@
 "See https://bugs.launchpad.net/duplicity/+bug/931175\n"
 msgstr ""
 
-#: ../bin/duplicity:1401
+#: ../bin/duplicity:1399
 #, python-format
 msgid "Last %s backup left a partial set, restarting."
 msgstr ""
 
-#: ../bin/duplicity:1405
+#: ../bin/duplicity:1403
 #, python-format
 msgid "Cleaning up previous partial %s backup set, restarting."
 msgstr ""
 
+#: ../bin/duplicity:1414
+msgid "Last full backup date:"
+msgstr ""
+
 #: ../bin/duplicity:1416
-msgid "Last full backup date:"
+msgid "Last full backup date: none"
 msgstr ""
 
 #: ../bin/duplicity:1418
-msgid "Last full backup date: none"
-msgstr ""
-
-#: ../bin/duplicity:1420
 msgid "Last full backup is too old, forcing full backup"
 msgstr ""
 
-#: ../bin/duplicity:1463
+#: ../bin/duplicity:1461
 msgid ""
 "When using symmetric encryption, the signing passphrase must equal the "
 "encryption passphrase."
 msgstr ""
 
-#: ../bin/duplicity:1516
+#: ../bin/duplicity:1514
 msgid "INT intercepted...exiting."
 msgstr ""
 
-#: ../bin/duplicity:1524
+#: ../bin/duplicity:1522
 #, python-format
 msgid "GPG error detail: %s"
 msgstr ""
 
-#: ../bin/duplicity:1534
+#: ../bin/duplicity:1532
 #, python-format
 msgid "User error detail: %s"
 msgstr ""
 
-#: ../bin/duplicity:1544
+#: ../bin/duplicity:1542
 #, python-format
 msgid "Backend error detail: %s"
 msgstr ""
 
-#: ../bin/rdiffdir:56 ../duplicity/commandline.py:238
+#: ../bin/rdiffdir:56 ../duplicity/commandline.py:233
 #, python-format
 msgid "Error opening file %s"
 msgstr ""
@@ -381,33 +381,33 @@
 msgid "File %s already exists, will not overwrite."
 msgstr ""
 
-#: ../duplicity/selection.py:119
+#: ../duplicity/selection.py:121
 #, python-format
 msgid "Skipping socket %s"
 msgstr ""
 
-#: ../duplicity/selection.py:123
+#: ../duplicity/selection.py:125
 #, python-format
 msgid "Error initializing file %s"
 msgstr ""
 
-#: ../duplicity/selection.py:127 ../duplicity/selection.py:148
+#: ../duplicity/selection.py:129 ../duplicity/selection.py:150
 #, python-format
 msgid "Error accessing possibly locked file %s"
 msgstr ""
 
-#: ../duplicity/selection.py:163
+#: ../duplicity/selection.py:165
 #, python-format
 msgid "Warning: base %s doesn't exist, continuing"
 msgstr ""
 
-#: ../duplicity/selection.py:166 ../duplicity/selection.py:184
-#: ../duplicity/selection.py:187
+#: ../duplicity/selection.py:168 ../duplicity/selection.py:186
+#: ../duplicity/selection.py:189
 #, python-format
 msgid "Selecting %s"
 msgstr ""
 
-#: ../duplicity/selection.py:268
+#: ../duplicity/selection.py:270
 #, python-format
 msgid ""
 "Fatal Error: The file specification\n"
@@ -418,14 +418,14 @@
 "pattern (such as '**') which matches the base directory."
 msgstr ""
 
-#: ../duplicity/selection.py:276
+#: ../duplicity/selection.py:278
 #, python-format
 msgid ""
 "Fatal Error while processing expression\n"
 "%s"
 msgstr ""
 
-#: ../duplicity/selection.py:286
+#: ../duplicity/selection.py:288
 #, python-format
 msgid ""
 "Last selection expression:\n"
@@ -435,49 +435,49 @@
 "probably isn't what you meant."
 msgstr ""
 
-#: ../duplicity/selection.py:311
+#: ../duplicity/selection.py:313
 #, python-format
 msgid "Reading filelist %s"
 msgstr ""
 
-#: ../duplicity/selection.py:314
+#: ../duplicity/selection.py:316
 #, python-format
 msgid "Sorting filelist %s"
 msgstr ""
 
-#: ../duplicity/selection.py:341
+#: ../duplicity/selection.py:343
 #, python-format
 msgid ""
 "Warning: file specification '%s' in filelist %s\n"
 "doesn't start with correct prefix %s.  Ignoring."
 msgstr ""
 
-#: ../duplicity/selection.py:345
+#: ../duplicity/selection.py:347
 msgid "Future prefix errors will not be logged."
 msgstr ""
 
-#: ../duplicity/selection.py:361
+#: ../duplicity/selection.py:363
 #, python-format
 msgid "Error closing filelist %s"
 msgstr ""
 
-#: ../duplicity/selection.py:428
+#: ../duplicity/selection.py:430
 #, python-format
 msgid "Reading globbing filelist %s"
 msgstr ""
 
-#: ../duplicity/selection.py:461
+#: ../duplicity/selection.py:463
 #, python-format
 msgid "Error compiling regular expression %s"
 msgstr ""
 
-#: ../duplicity/selection.py:477
+#: ../duplicity/selection.py:479
 msgid ""
 "Warning: exclude-device-files is not the first selector.\n"
 "This may not be what you intended"