Duplicity ignores some FatalErrors

There are seemingly two bugs here: one is the error message itself, the other is that DD continues in the background after receiving what it thought was a fatal error.

The first one is, I believe, fixed with my 'retry' patch that made it into Duplicity. I had it ignore 'not found' errors when trying to delete files, which is what happened here. The cache indicated that it had left a .part file, but it wasn't there when it tried to delete it, so the backend errored out. But it shouldn't. So now that that's in Duplicity, I'm not worried about that message.

However, the second part where DD was ignoring the fatal error and secretly continuing is bad. I'm looking at that.

Do you still have that full log? I'd be very interested in seeing the whole thing (or at least, everything up to the error).

Figured out what's going on here.

1) Duplicity calls self.last_backup.delete() during a restart
2) Which executes the following:

try:
    self.backend.delete(rfn)
except:
    log.Debug("BackupSet.delete: missing %s" % rfn)
    pass

3) If the backend.delete function encounters a FatalError, sys.exit will be called, which should close duplicity.
4) But the way that except clause is written, it won't. Blanket "except:" clauses will catch even SystemExit exceptions. See http://stackoverflow.com/questions/730764/try-catch-in-python

The fix is to use "except Exception:" instead. I have a branch that does that for all except: lines. I assume that is correct, that we never intentionally want to block SystemExit or KeyboardInterrupt?

OK, it's merged. Thanks for the catch.

2011/6/17 Launchpad Bug Tracker <email address hidden>

> ** Branch linked: lp:~mterry/duplicity/797758
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/797758
>
> Title:
> Duplicity ignores some FatalErrors
>
> Status in Duplicity - Bandwidth Efficient Encrypted Backup:
> Triaged
>
> Bug description:
> When running a backup after the previous backup attempt failed (to the
> same target directory), an error similar to the following will be
> reported: "Backup location ‘/smb:/du-nas1/share/backup_miquel
> /duplicity-full.20110526T182540Z.manifest.part’ does not exist."
>
> The Deja Dup frontend treats this is a fatal error and reports to the
> user that the backup has failed, however Duplicity continues to
> operate in the background until Deja Dup is closed. DD should not
> treat this as a fatal error and should continue to report progress
> information as usual while Duplicity runs the backup.
>
> Another, possibly separate (Duplicity?) wrinkle to this bug is the
> fact that this error will occur even if the contents of the target
> directory have been removed after the failed backup but before the new
> attempt. Only changing the target directory will stop this error from
> occurring.
>
> Error is on line 60 of the attached log excerpt
>
> As reported in https://bugs.launchpad.net/duplicity/+bug/545486 and
> https://bugs.launchpad.net/deja-dup/+bug/779022
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/797758/+subscriptions
>

Ken, how would duplicity get in the position where it would try to delete a file on the backend that doesn't exist? With this change, such errors now stop duplicity (rightly). But this bug was noticed because duplicity did try to delete a file that didn't exist. Seems like it should know what's on the backend via list().

Sorry for the slow reply. A couple things:

> The first one is, I believe, fixed with my 'retry' patch that made it
> into Duplicity.
If this is the same one you had me try out, this was still occurring
even with that patch applied.

> Do you still have that full log? I'd be very interested in seeing the
> whole thing (or at least, everything up to the error).
Do you still need this? I can try to dig it up if so.

> With this change, such errors now stop duplicity (rightly).
Should this really be a fatal error? Granted, trying to delete a
non-existent file is a strange situation, but does it have any ill
effects? Aren't we arriving at the same destination (the file is
removed)? From a user perspective, I'd rather not see several hours of
backup work bomb out if there's any way to recover and continue.

Best regards,
Michael

On 18/06/11 15:09, Kenneth Loafman wrote:
> ** Changed in: duplicity
> Status: Fix Committed => Fix Released
>