Duplicity restore fails with UTF-8 chars in --file-to-restore

Same for me.

I use duplicity through deja-dup and in nautilus, I tried a "Restore Missing Files".
No problem with the backup, but impossible to restore the file. I got the same traceback as previous message.

I run Ubuntu 14.04 64bits, duplicity 0.6.23-1ubuntu4.1 and deja-dup 30.0.0ubutu4

Status changed to 'Confirmed' because the bug affects multiple users.

The problem exists also for filenames without non-ASCII characters but with pathname with non-ASCII characters.
For example, it is impossible to restore all my files under /home/user/Pùblico/....

This bug is really annoying since it indeed prevents restoring several files for me..

Thank you.

Do we have to wait until the offical update or is it possible to update right now ? And if yes, from where ?

You *could* update from the daily-build PPA [1], but that's not generally recommended for day-to-day use. Otherwise wait for an official release in the stable PPA [2].

I'm doubtful this fix will be backported to 14.04 because it's actually an odd case. You were experiencing the crash when duplicity was trying to say that the file you asked for didn't exist in the backup. When I tried restoring utf8 filenames that did exist in the backup, it worked fine. So you may have something else going on, or are passing the wrong arguments.

[1] https://launchpad.net/~duplicity-team/+archive/ubuntu/daily
[2] https://launchpad.net/~duplicity-team/+archive/ubuntu/ppa

This also affects me !

The Backup with DejaDup is executed without any error.
Then I wanted to restore a file

1. The Special characters are not recognized correctly so "dell´anima.doc" is shown as "dell
2. If i execute the restore nevertheless i get a FOLDER ! "dell�anima.doc (codifica non valida)" instead of a doc file.

Where is my data ?????

So the restore function with special characters for me as normal user is completly broken.
People are not aware that they can not redo their backup. -> Risk of complete data loss
This is why i consider this as a major bug. This should never happen with a offical release of a backup software !

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1494, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1488, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1337, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1422, in do_backup
    restore(col_stats)
  File "/usr/bin/duplicity", line 700, in restore
    % (globals.restore_dir,),
UnicodeDecodeError: 'ascii' codec can't decode byte 0xb4 in position 40: ordinal not in range(128)

Description: Ubuntu 14.04.1 LTS
Release: 14.04

deja-dup:
  Installato: 30.0-0ubuntu4
  Candidato: 30.0-0ubuntu4

LANG=it_IT.UTF-8
LANGUAGE=it:en
LC_CTYPE="it_IT.UTF-8"
LC_NUMERIC=it_IT.UTF-8
LC_TIME=it_IT.UTF-8
LC_COLLATE="it_IT.UTF-8"
LC_MONETARY=it_IT.UTF-8
LC_MESSAGES="it_IT.UTF-8"
LC_PAPER=it_IT.UTF-8
LC_NAME=it_IT.UTF-8
LC_ADDRESS=it_IT.UTF-8
LC_TELEPHONE=it_IT.UTF-8
LC_MEASUREMENT=it_IT.UTF-8
LC_IDENTIFICATION=it_IT.UTF-8
LC_ALL=

Speicality of the installation -> The Language was changed after installataion.

See https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1405128
I did not find this bug before .. so sorry for the duplication

A comment on
"I'm doubtful this fix will be backported to 14.04 because it's actually an odd case."

For me there was nothing odd. I did straight backup and then tried a restore.
I use LTS versions for havin some stability and long term support.

But what is this worth if the essential feature "restore" of the standard backup software which is shipped does not do what is expected ? If you need more information about this case please contact me.

As of today and revision bzr1048 installed from the duplicity daily-ppa, this problem persists on two unicorn, 64bit machines with fresh installs and language set to German respectively. Here is what I did:

I have an example directory with the path /home/paul/Musik/Clueso/Gute Musik/ that I wanted to backup via deja dup and that contains some files with and some files without special characters. What's worth noticing is that deja-dup seemingly shows how duplicity handles files with special characters different from such files without special characters.

This is the log from the deja-dup window:
[Einlesen is German and means "Reading" or "Indexing"]
[Sichern is German and means "Backing Up"]

Einlesen: /home/paul/Musik/Clueso/Gute Musik/.mediaartlocal/album-4e2e697e4e8f7556783434043ce052fe-7bad226fa0d8ca5da1405e4036652062.jpeg.gz
Einlesen: /home/paul/Musik/Clueso/Gute Musik/.mediaartlocal/album-70dd1a377d1e83914185be0f10e7cb47-7bad226fa0d8ca5da1405e4036652062.jpeg.gz
Einlesen: /home/paul/Musik/Clueso/Gute Musik/.mediaartlocal/album-84036bb3393f96f3e67553cf7361aacc-7bad226fa0d8ca5da1405e4036652062.jpeg.gz
Einlesen: /home/paul/Musik/Clueso/Gute Musik/.mediaartlocal/album-97f7883864ed3e64087ef4bb33844b17-7bad226fa0d8ca5da1405e4036652062.jpeg.gz
Einlesen: /home/paul/Musik/Clueso/Gute Musik/.mediaartlocal/album-b5af0a75d8062a26edc60a3841d2e1a7-7bad226fa0d8ca5da1405e4036652062.jpeg.gz
Einlesen: /home/paul/Musik/Clueso/Gute Musik/.mediaartlocal/album-b6c6d51a1e55f412332925a9f67d2b6e-7bad226fa0d8ca5da1405e4036652062.jpeg.gz
Einlesen: file:///home/paul/Musik/Clueso/Gute%20Musik/01%20Clueso%20mit%20J%FCrgen%20Kerth%20Band%20-%20Nacht%20unterwegs.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/02 Clueso - Das Level.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/03 Clueso - Uh Girl.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/04 Clueso - Gute Musik.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/05 Clueso mit Delhia - Dort wo du wohnst.mp3
Einlesen: file:///home/paul/Musik/Clueso/Gute%20Musik/06%20Clueso%20-%20Es%20wird%20hei%DF.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/07 Clueso mit Steer-M - Love the People.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/08 Clueso - Kalter Kaffee.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/09 Clueso - Bescheid.mp3
Einlesen: file:///home/paul/Musik/Clueso/Gute%20Musik/10%20Clueso%20-%20Vier%20kleine%20W%E4nde.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/11 Clueso - City-Pizza-Express.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/12 Clueso - Pizzaschachteln.mp3
Einlesen: file:///home/paul/Musik/Clueso/Gute%20Musik/13%20Clueso%20-%20Wie%20hei%DFt'n%20du_.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/14 Clueso - Wart mal.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/15 Clueso - Vergessen ist so leicht.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/16 Clueso mit Tim Neuhaus - Fanpost.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/17 Clueso mit Blumentopf - Egal wo.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/18 Clueso - Ich geh heim.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/19 Clueso - Herz Boom Boom.mp3
Einlesen: /home/paul/Musik/Clueso/Gute Musik/20 Clueso - Vorspiel.mp3
E...

Actually, I just figured that the restoration only doesn't work when using the deja-dup backend, with the duplicity command line options, it does indeed work as expected. I can restore the full directory without missing or renamed files or just restore the specific single file without a problem. It seems to be a problem with the deja-dup backend then. Sorry for bothering.

I thought being hit by this bug while having the same 'ascii' codec error message traing to restore a succesfully backed up file with accented characters in it's name.
However it proved to be a locale related problem I discovered while chasing an apparently unrelated issue of having English menu items instead of French ones in another package (found in http://doc.ubuntu-fr.org/cinelerra).
For me the following actions solved the duplicity file naming issue:
1. Add the following line in /var/lib/locales/supported.d/local :
   fr_CH.UTF-8 UTF-8
2. Reconfigure the locales
   sudo dpkg-reconfigure locales

Now th subject file is successfully restored.

OK, I uploaded the fix to trusty-proposed for an SRU. Am subscribing ubuntu-sru to review it.

Status changed to 'Confirmed' because the bug affects multiple users.

Hello Adrien, or anyone else affected,

Accepted duplicity into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/duplicity/0.6.23-1ubuntu4.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Did someone tried?

Hello!

I managed to test this now (with duplicity 0.6.23-1ubuntu4.2) and was able to restore files via "wiederherstellen"-button from deja-dup (German for restore), even if they or their folders contain ä, ö or è.
Unfortunately I got no result when trying to restore only one missing file (via context menu or duplicity --file-to-restore ./da file:///home/test/Sicherung .)! The terminal printed this message instead: "Datei im Archiv nicht gefunden, keine Dateien wiederhergestellt." which means something like "Unable to find file in archive, did not restored any files".

Best regards

Vej

I need to add something to my last comment:

The message I cited above indicates, that the stacktrace shown by Adrein Robin – when he opened this bug – is avoided by the fix.

This does not explain why this piece of code gets executed.

So if you see this bug only as the problem that occurred, after the line starting with "log.FatalError(" has been entered. It is fixed.

In this case I will remove all the duplicates, because they want their files to bee _restored_.

A fixed error-message is not very helpful if you lost your files to this bug ;).

Thank you for your attention

Vej

According to https://wiki.ubuntu.com/Bugs/Importance this bug must be marked as critcal!

What happens to this bug??

This bug is fixed in the latest versions, the 0.7 series at duplicity
<https://launchpad.net/duplicity/+download>. Please upgrade. If you are
on 0.6 series, please do not report new bugs about this, they won't be
fixed.

To upgrade to the newest,

   1. Remove the old version installed from the repository
   2. tar xf <tarball>
   3. cd <dir created>
   4. sudo python2 setup.py install

It is important to do step 1. Different distros use different paths and
the code can get confused if two versions are present.

On Sat, Jun 11, 2016 at 4:15 PM, Vej <email address hidden> wrote:

> What happens to this bug??
>
> --
> You received this bug notification because you are subscribed to
> duplicity in Ubuntu.
> https://bugs.launchpad.net/bugs/1356548
>
> Title:
> Duplicity restore fails with UTF-8 chars in --file-to-restore
>
> Status in Duplicity:
> Fix Released
> Status in duplicity package in Ubuntu:
> Fix Released
> Status in duplicity source package in Trusty:
> Fix Committed
>
> Bug description:
> Under Ubuntu 14.04 64bits, duplicity 0.6.23-1ubuntu4.1
>
> To reproduce the issue:
>
> 1) Create a backup including non-ASCII characters pathname, e.g.,
> "data/Thèse" for instance
>
> 2) Try a selective restore of the backup : duplicity --file-to-restore
> data/Thèse file:///home/user/Sauvegardes test/
>
> 3) Crash :
>
> Traceback (most recent call last):
> File "/usr/bin/duplicity", line 1494, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1488, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1337, in main
> do_backup(action)
> File "/usr/bin/duplicity", line 1422, in do_backup
> restore(col_stats)
> File "/usr/bin/duplicity", line 700, in restore
> % (globals.restore_dir,),
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 21:
> ordinal not in range(128)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1356548/+subscriptions
>

A new version of duplicity (0.6.23-1ubuntu4.2) is available for Trusty in trusty-proposed.

That should fix this bug, but it needs to be tested.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed.

The version of duplicity in the proposed pocket of Trusty that was purported to fix this bug report has been removed because the bugs that were to be fixed by the upload were not verified in a timely (105 days) fashion.

The version of duplicity in the proposed pocket of Trusty that was purported to fix this bug report has been removed because the bugs that were to be fixed by the upload were not verified in a timely (105 days) fashion.