incremental snapshots store more files than needed

Bug #768080 reported by taka k.
26
This bug affects 4 people
Affects Status Importance Assigned to Milestone
sbackup
Status tracked in Trunk
0.11
Fix Released
Medium
Unassigned
Trunk
Fix Released
Medium
Unassigned

Bug Description

i am using version 0.11.4ppa1~karmic1 and in my sbackup.conf i have a line like this:

/media/fat32disk/ = 1

so that whenever i plug in my portable drive, which is formatted fat32, it gets backed up with the rest of the system.

i've noticed now that, after two backup sessions, sbackup seems to be taking another full backup of the /media/fat32disk.

for example, i haven't had fat32disk plugged in for a while, so there already existed a full backup of my system as well as several incremental backups. then i plug in fat32disk and sbackup makes another incremental, this time including /media/fat32disk since it's mounted and also backing up everything in /media/fat32disk since it hasn't been plugged in since the last full backup. so far so good.

however, next time the scheduled backup is run, the incremental backup again includes everything from /media/fat32disk even though practically none of the files have changed.

i'm not sure if this is due to the volume being formatted fat32, therefore causing some sort of problem reading file creation/modification times, or what. it was my first guess.

Revision history for this message
Jean-Peer Lorenz (peer.loz) wrote :

Thank you for using sbackup and for taking the time reporting this bug.

In order to track down the cause of the problem, please add a log file (debug mode).

Thanks for your help.

Changed in sbackup:
status: New → Incomplete
Revision history for this message
taka k. (scar) wrote :

i've since moved to bacula but am interestingly having the same issues with the incremental backups for the disk. so, unfortunately i cannot provide you with the requested information but it does appear to be an issue that is outside of sbackup.

summary: - improper incremental on fat32
+ incremental snapshots store more files than needed
Revision history for this message
Thibault Godouet (tglaunchpad) wrote :
Download full text (3.3 KiB)

Hi,

I've also experienced this problem. In my case I did a backup last WE, and did another one today: sbackup decided to do an incremental backup, which should be very small (actually empty) as pretty much no file changed in the last 90 days:

$ find /home/musique/ -ctime -90
/home/musique/
/home/musique/.config
/home/musique/.config/autostart
/home/musique/.config/autostart/indicator-weather.desktop

sbackup does think there is almost nothing to backup:

2011-10-22 10:11:07,092 - INFO: Last full backup is fresh (42 days old )-> make an increment
2011-10-22 10:11:07,093 - INFO: Instantané '2011-10-22_10.11.07.092778.workstation.inc' en cours de création.
2011-10-22 10:11:07,093 - INFO: Réglage de la Base à '2011-10-16_10.54.21.443230.workstation.inc'.
2011-10-22 10:11:07,093 - INFO: Réglage des fichiers de sauvegarde.
2011-10-22 10:11:07,233 - INFO: Réglage des exclusions de fichiers.
2011-10-22 10:11:07,233 - INFO: Réglage du format de compression à `0`
2011-10-22 10:11:07,234 - INFO: L'option 'Suivre les liens symboliques' est désactivée.
2011-10-22 10:11:07,234 - INFO: Inspection du système de fichiers et collecte des informations sur les fichiers
2011-10-22 10:11:08,956 - INFO: Résumé de la sauvegarde
2011-10-22 10:11:08,956 - INFO: Nombre de répertoires : 1026.
2011-10-22 10:11:08,956 - INFO: Nombre total de fichiers : 2837.
2011-10-22 10:11:08,956 - INFO: Nombre de liens symboliques : 0.
2011-10-22 10:11:08,956 - INFO: Nombre de fichiers inclus dans l'instantané : 1.
2011-10-22 10:11:08,957 - INFO: Nombre de nouveaux fichiers (également inclus) : 1.
2011-10-22 10:11:08,957 - INFO: Nombre de fichiers ignorés dans l'instantané incrémental : 2836.
2011-10-22 10:11:08,957 - INFO: Nombre de fichiers exclus de force : 0.
2011-10-22 10:11:08,957 - INFO: Nombre de fichiers exclus selon la configuration : 0.
2011-10-22 10:11:08,957 - INFO: L'espace libre maximum requis est '0 Mo 513 Ko 800'.

However the tar file is 9.6GB so far, and still growing. I've had this before and sbackup basically ended up doing a full backup.

I tried running the tar command sbackup uses -- I simply added a "-f /tmp/test.tar" to make things simpler and "-v" to see which files it backs up:

# /bin/tar -cS --directory=/ --ignore-failed-read --blocking-factor 20 --totals --files-from=/tmp/sbackup/includes.list --exclude-from=/tmp/sbackup/excludes.list --listed-incremental=/tmp/sbackup/files.snar -f /tmp/test.tar -v

Now if I add "--no-check-device", it pretty much doesn't backup anything apart from the folders (the file structure, not the file themselves), and I end up with a backup of 1.1MB:

# /bin/tar -cS --directory=/ --ignore-failed-read --blocking-factor 20 --totals --files-from=/tmp/sbackup/includes.list --exclude-from=/tmp/sbackup/excludes.list --listed-incremental=/tmp/sbackup/files.snar -f /tmp/test.tar -v --no-check-device
/home/musique/.config/
/home/musique/.config/autostart/
[...]
/home/musique/tmp/tmp/
Nombre total d'octets écrits: 1116160 (1,1MiB, 7,5MiB/s)

So I can only assume that the device number changes (see http://www.gnu.org/software/automake/manual/tar/Incremental-Dumps.html for more details) on my setup which has LVM on top of a software MD RA...

Read more...

Revision history for this message
Rogach (platon7pronko) wrote :

Same problem here (my main disk was changing device name after restart) - and adding "--no-check-device" to tar.py solved the problem.

Here is the diff (sbackup 0.11.3):

*** tar.py 2011-12-12 07:55:08.149261001 +0400
--- tar.py.orig 2011-12-12 07:53:11.209261001 +0400
***************
*** 249,256 ****
      options = ["-cS", "--directory=%s" % _FOP.pathsep,
                 "--ignore-failed-read",
                 "--blocking-factor", str(constants.TAR_BLOCKING_FACTOR),
! "--totals",
! "--no-check-device"
                ]

      # includes and excludes
--- 249,255 ----
      options = ["-cS", "--directory=%s" % _FOP.pathsep,
                 "--ignore-failed-read",
                 "--blocking-factor", str(constants.TAR_BLOCKING_FACTOR),
! "--totals"
                ]

      # includes and excludes

Revision history for this message
Felix Griewald (tiiunder) wrote :

I have/had the same problem here. I tried to write my backups into an EncFS-folder and the increment backups were to big.

Here is a little excerpt out of my debug log file. (I'm using Ubuntu 11.10 and sBackup 11.4)

I'm not sure if this helps, because the end of the log is maybe missing.

PS: I also tried the solution with "--no-check-device" and it worked the last two times.

Revision history for this message
Felix Griewald (tiiunder) wrote :
Revision history for this message
Jean-Peer Lorenz (peer.loz) wrote :

Thanks for your help. I've applied the fix to 0.11.4 and committed it to lp:sbackup/0.11. I'll prepare a bugfix release asap. Any help would be appreciated.

Changed in sbackup:
status: Incomplete → Fix Committed
Changed in sbackup:
importance: Undecided → Medium
milestone: none → 0.11.5
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.