Compressed/Uncompressed incremental backup fails on compressed full backup in Xtrabackup 2.0.0

To add, xtrabackup_checkpoints.qp is stored in full backup directory and not xtrabackup_checkpoints which is what the incremental backup script is looking for.

ok, i tried an uncompress, and it worked fine.

i agree, that this is a documentation bug.

# litle help for unpacking files
for bf in `find . -iname "*\.qp"`; do qpress -d $bf $(dirname $bf) && rm $bf; done

Thanks, Michael

This is absolutely not just a documentation bug. This new process requires you to extract the xtrabackup_checkpoints.qp file BEFORE you can run an incremental backup based on the existing full backup. The checkpoints file should clearly NOT be compressed if the innobackupex script doesn't expect it to be compressed.

** the rest of this is not really relevant to this bug, but was addressed by michael above is another issue altogether **

Furthermore, expecting users to loop through and decompress all of the files in a single threaded fashion defeats the purpose of having innobackupex parallelized. If you can write the files to disk in a parallelized way, why not make it so you can restore from disk in a parallel way? It's almost as if the restore process was forgotten about in this case.

Eli,

I think Michael meant to comment on bug #1003783, not this one.

As to this bug, I do not consider it as a documentation issue. I'm going to fix it in 2.0.1 by omitting xtrabackup_checkpoints from compression, so that a full backup could be used for incremental backups without decompression. I agree, it doesn't make much sense to compress it.

As to the other comments, there are 2 parts to this problem:

1) qpress can actually do multi-threaded decompression (the -T switch), and it uses 2 threads by default.
2) the dependency on an external utility to decompress backups makes the restore process confusing. To add insult to injury, qpress is unfortunately not packaged by all popular Linux distributions. So in addition to 1 more step with decompressing backups before restore, one may need to download and compile qpress before using compressed backups. I'm currently considering multiple options to make it less painful, but that will probably be implemented after 2.0.1.

I am using the 2.0.1-446.squeeze version of percona-xtrabackup and am still experiencing this issue. Is there a downloadable package that has this fix in it?

Simon,

Does 2.0.1 still create a compressed xtrabackup_checkpoints file for you? Or creating an incremental backup with a full backup created with a previous version does not work?

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-231