OverflowError: signed integer is greater than maximum

Full log file

This one's fairly simple.

1) You need to upgrade to 0.6.23 ASAP,
2) Don't try to backup /dev or /proc

On Tue, Jun 10, 2014 at 3:10 PM, Andreas <email address hidden> wrote:

> Full log file
>
> ** Attachment removed: "Full log file"
>
> https://bugs.launchpad.net/duplicity/+bug/1327550/+attachment/4127009/+files/Duply-log-complete.log
>
> ** Attachment added: "Full log file"
>
> https://bugs.launchpad.net/duplicity/+bug/1327550/+attachment/4129210/+files/Duply-log-complete.log
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1327550
>
> Title:
> OverflowError: signed integer is greater than maximum
>
> Status in Duplicity - Bandwidth Efficient Encrypted Backup:
> New
>
> Bug description:
> Duplicity version: v1.5.2.3
> Python version: Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
> OS Distro and version: Debian GNU/Linux 6.0.9 (squeeze)
> Type of target filesystem: Linux (FTP)
> Log output: attached
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1327550/+subscriptions
>

I've updated to 0.6.24. /dev and /proc are excluded.

Error remains. Sorry.

Please post your /root/.duply/full/exclude file. Looks like /dev is not
really being excluded.

On Sat, Jun 14, 2014 at 4:38 AM, Andreas <email address hidden> wrote:

> I've updated to 0.6.24. /dev and /proc are excluded.
>
> Error remains. Sorry.
>
> ** Attachment added: "Complete log with 0.6.24 installed"
>
> https://bugs.launchpad.net/duplicity/+bug/1327550/+attachment/4131360/+files/Duply-log-complete-20140614.txt
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1327550
>
> Title:
> OverflowError: signed integer is greater than maximum
>
> Status in Duplicity - Bandwidth Efficient Encrypted Backup:
> New
>
> Bug description:
> Duplicity version: v1.5.2.3
> Python version: Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
> OS Distro and version: Debian GNU/Linux 6.0.9 (squeeze)
> Type of target filesystem: Linux (FTP)
> Log output: attached
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1327550/+subscriptions
>

/root/.duply/full/exclude:

/dev/*
/proc/*
/sys/*
/tmp/*
/var/tmp/*
/var/run/*

Is this running on a VM at a hosting service? I'd like to see an 'ls -l
/dev' listing, please.

On Sat, Jun 14, 2014 at 7:05 AM, Andreas <email address hidden> wrote:

> /root/.duply/full/exclude:
>
> /dev/*
> /proc/*
> /sys/*
> /tmp/*
> /var/tmp/*
> /var/run/*
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1327550
>
> Title:
> OverflowError: signed integer is greater than maximum
>
> Status in Duplicity - Bandwidth Efficient Encrypted Backup:
> New
>
> Bug description:
> Duplicity version: v1.5.2.3
> Python version: Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
> OS Distro and version: Debian GNU/Linux 6.0.9 (squeeze)
> Type of target filesystem: Linux (FTP)
> Log output: attached
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1327550/+subscriptions
>

It's running as a virtual root server (Parallels Virtuozzo)

That's how /dev looks:

% sudo ls -l /dev
total 4
crw-rw---- 1 root video 10, 175 May 30 2011 agpgart
crw-rw---- 1 root audio 14, 4 May 30 2011 audio
crw-rw---- 1 root audio 14, 20 May 30 2011 audio1
crw-rw---- 1 root audio 14, 36 May 30 2011 audio2
crw-rw---- 1 root audio 14, 52 May 30 2011 audio3
crw-rw---- 1 root audio 14, 7 May 30 2011 audioctl
crw------- 1 root root 5, 1 Jun 14 16:07 console
lrwxrwxrwx 1 root root 11 May 30 2011 core -> /proc/kcore
crw-rw---- 1 root audio 14, 3 May 30 2011 dsp
crw-rw---- 1 root audio 14, 19 May 30 2011 dsp1
crw-rw---- 1 root audio 14, 35 May 30 2011 dsp2
crw-rw---- 1 root audio 14, 51 May 30 2011 dsp3
lrwxrwxrwx 1 root root 13 May 30 2011 fd -> /proc/self/fd
crw-rw-rw- 1 root root 1, 7 May 30 2011 full
crw----r-x 1 root root 10, 229 Jun 14 16:06 fuse
prw------- 1 root root 0 Jun 14 16:06 initctl
crw-r----- 1 root kmem 1, 2 May 30 2011 kmem
srw-rw-rw- 1 root root 0 Jun 14 16:06 log
brw-rw---- 1 root disk 7, 0 May 30 2011 loop0
brw-rw---- 1 root disk 7, 1 May 30 2011 loop1
brw-rw---- 1 root disk 7, 2 May 30 2011 loop2
brw-rw---- 1 root disk 7, 3 May 30 2011 loop3
brw-rw---- 1 root disk 7, 4 May 30 2011 loop4
brw-rw---- 1 root disk 7, 5 May 30 2011 loop5
brw-rw---- 1 root disk 7, 6 May 30 2011 loop6
brw-rw---- 1 root disk 7, 7 May 30 2011 loop7
crw-r----- 1 root kmem 1, 1 May 30 2011 mem
crw-rw---- 1 root audio 35, 0 May 30 2011 midi0
crw-rw---- 1 root audio 14, 2 May 30 2011 midi00
crw-rw---- 1 root audio 14, 18 May 30 2011 midi01
crw-rw---- 1 root audio 14, 34 May 30 2011 midi02
crw-rw---- 1 root audio 14, 50 May 30 2011 midi03
crw-rw---- 1 root audio 35, 1 May 30 2011 midi1
crw-rw---- 1 root audio 35, 2 May 30 2011 midi2
crw-rw---- 1 root audio 35, 3 May 30 2011 midi3
crw-rw---- 1 root audio 14, 0 May 30 2011 mixer
crw-rw---- 1 root audio 14, 16 May 30 2011 mixer1
crw-rw---- 1 root audio 14, 32 May 30 2011 mixer2
crw-rw---- 1 root audio 14, 48 May 30 2011 mixer3
crw-rw---- 1 root audio 31, 0 May 30 2011 mpu401data
crw-rw---- 1 root audio 31, 1 May 30 2011 mpu401stat
drwxr-xr-x 2 root root 4096 Jun 17 2011 net
crw-rw-rw- 1 root root 1, 3 May 30 2011 null
brw------- 1 root root 182, 177649 Apr 10 01:20 ploop11103p1
brw------- 1 root root 182, 189681 Apr 9 23:46 ploop11855p1
brw------- 1 root root 182, 216577 Apr 9 23:54 ploop13536p1
b-----x--- 1 root root 182, 231121 Dec 2 2013 ploop14445p1
brw------- 1 root root 182, 239457 Apr 10 00:56 ploop14966p1
brw------- 1 root root 182, 255169 Apr 10 00:06 ploop15948p1
brw------- 1 root root 182, 359377 Apr 10 01:02 ploop22461p1
brw------- 1 root root 182, 366033 Apr 10 00:42 ploop22877p1
brw------- 1 root root 182, 434577 Apr 10 00:24 ploop27161p1
brw------- 1 root ...

I see the problem. Major and minor block numbers are supposed to be one
byte each. Someone has crafted a special system image using OpenVZ where
the major and minor block numbers are much larger (see ploop devices
above). I'm not sure what the final fix may be, but a quick fix for your
case would be to remove the lines 95-97 in path.py.

        if self.type in ("chr", "blk"):
            self.devnums = (os.major(self.stat.st_rdev),
                            os.minor(self.stat.st_rdev))

Or just comment them if you feel so inclined ('#' at the front of the line).

On Sat, Jun 14, 2014 at 9:50 AM, Andreas <email address hidden> wrote:

> It's running as a virtual root server (Parallels Virtuozzo)
>
> That's how /dev looks:
>
> % sudo ls -l /dev
> total 4
> crw-rw---- 1 root video 10, 175 May 30 2011 agpgart
> crw-rw---- 1 root audio 14, 4 May 30 2011 audio
> crw-rw---- 1 root audio 14, 20 May 30 2011 audio1
> crw-rw---- 1 root audio 14, 36 May 30 2011 audio2
> crw-rw---- 1 root audio 14, 52 May 30 2011 audio3
> crw-rw---- 1 root audio 14, 7 May 30 2011 audioctl
> crw------- 1 root root 5, 1 Jun 14 16:07 console
> lrwxrwxrwx 1 root root 11 May 30 2011 core -> /proc/kcore
> crw-rw---- 1 root audio 14, 3 May 30 2011 dsp
> crw-rw---- 1 root audio 14, 19 May 30 2011 dsp1
> crw-rw---- 1 root audio 14, 35 May 30 2011 dsp2
> crw-rw---- 1 root audio 14, 51 May 30 2011 dsp3
> lrwxrwxrwx 1 root root 13 May 30 2011 fd -> /proc/self/fd
> crw-rw-rw- 1 root root 1, 7 May 30 2011 full
> crw----r-x 1 root root 10, 229 Jun 14 16:06 fuse
> prw------- 1 root root 0 Jun 14 16:06 initctl
> crw-r----- 1 root kmem 1, 2 May 30 2011 kmem
> srw-rw-rw- 1 root root 0 Jun 14 16:06 log
> brw-rw---- 1 root disk 7, 0 May 30 2011 loop0
> brw-rw---- 1 root disk 7, 1 May 30 2011 loop1
> brw-rw---- 1 root disk 7, 2 May 30 2011 loop2
> brw-rw---- 1 root disk 7, 3 May 30 2011 loop3
> brw-rw---- 1 root disk 7, 4 May 30 2011 loop4
> brw-rw---- 1 root disk 7, 5 May 30 2011 loop5
> brw-rw---- 1 root disk 7, 6 May 30 2011 loop6
> brw-rw---- 1 root disk 7, 7 May 30 2011 loop7
> crw-r----- 1 root kmem 1, 1 May 30 2011 mem
> crw-rw---- 1 root audio 35, 0 May 30 2011 midi0
> crw-rw---- 1 root audio 14, 2 May 30 2011 midi00
> crw-rw---- 1 root audio 14, 18 May 30 2011 midi01
> crw-rw---- 1 root audio 14, 34 May 30 2011 midi02
> crw-rw---- 1 root audio 14, 50 May 30 2011 midi03
> crw-rw---- 1 root audio 35, 1 May 30 2011 midi1
> crw-rw---- 1 root audio 35, 2 May 30 2011 midi2
> crw-rw---- 1 root audio 35, 3 May 30 2011 midi3
> crw-rw---- 1 root audio 14, 0 May 30 2011 mixer
> crw-rw---- 1 root audio 14, 16 May 30 2011 mixer1
> crw-rw---- 1 root audio 14, 32 May 30 2011 mixer2
> crw-rw---- 1 root audio 14, 48 May 30 2011 mixer3
> crw-rw---- 1 root audio 31, 0 May 30 2011 mpu401data
> crw-rw---- 1 root au...

I've removed lines 95-97 in path.py and receive now:

Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 1502, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 1496, in with_tempdir
    fn()
  File "/usr/local/bin/duplicity", line 1345, in main
    do_backup(action)
  File "/usr/local/bin/duplicity", line 1466, in do_backup
    full_backup(col_stats)
  File "/usr/local/bin/duplicity", line 538, in full_backup
    globals.backend)
  File "/usr/local/bin/duplicity", line 399, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/usr/local/lib/python2.6/dist-packages/duplicity/gpg.py", line 331, in GPGWriteFile
    data = block_iter.next().data
  File "/usr/local/lib/python2.6/dist-packages/duplicity/diffdir.py", line 518, in next
    result = self.process(self.input_iter.next())
  File "/usr/local/lib/python2.6/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter
    (new_path, sig_path, sigTarFile))
  File "/usr/local/lib/python2.6/dist-packages/duplicity/robust.py", line 37, in check_common_error
    return function(*args)
  File "/usr/local/lib/python2.6/dist-packages/duplicity/diffdir.py", line 114, in get_delta_path
    ti = new_path.get_tarinfo()
  File "/usr/local/lib/python2.6/dist-packages/duplicity/path.py", line 276, in get_tarinfo
    ti.devmajor, ti.devminor = self.devnums
AttributeError: Path instance has no attribute 'devnums

Sorry about that. Replace the lines with these:

        try:
            if self.type in ("chr", "blk"):
                self.devnums = (os.major(self.stat.st_rdev),
                                os.minor(self.stat.st_rdev))
        except:
            log.Warn(_("Warning: %s invalid devnums (0x%X), treating as (0,
0).")
                     % (util.ufn(self.get_relative_path()),
self.stat.st_rdev))
            self.devnums = (0, 0)

On Sat, Jun 14, 2014 at 11:10 AM, Andreas <email address hidden>
wrote:

> I've removed lines 95-97 in path.py and receive now:
>
> Traceback (most recent call last):
> File "/usr/local/bin/duplicity", line 1502, in <module>
> with_tempdir(main)
> File "/usr/local/bin/duplicity", line 1496, in with_tempdir
> fn()
> File "/usr/local/bin/duplicity", line 1345, in main
> do_backup(action)
> File "/usr/local/bin/duplicity", line 1466, in do_backup
> full_backup(col_stats)
> File "/usr/local/bin/duplicity", line 538, in full_backup
> globals.backend)
> File "/usr/local/bin/duplicity", line 399, in write_multivol
> globals.gpg_profile, globals.volsize)
> File "/usr/local/lib/python2.6/dist-packages/duplicity/gpg.py", line
> 331, in GPGWriteFile
> data = block_iter.next().data
> File "/usr/local/lib/python2.6/dist-packages/duplicity/diffdir.py", line
> 518, in next
> result = self.process(self.input_iter.next())
> File "/usr/local/lib/python2.6/dist-packages/duplicity/diffdir.py", line
> 213, in get_delta_iter
> (new_path, sig_path, sigTarFile))
> File "/usr/local/lib/python2.6/dist-packages/duplicity/robust.py", line
> 37, in check_common_error
> return function(*args)
> File "/usr/local/lib/python2.6/dist-packages/duplicity/diffdir.py", line
> 114, in get_delta_path
> ti = new_path.get_tarinfo()
> File "/usr/local/lib/python2.6/dist-packages/duplicity/path.py", line
> 276, in get_tarinfo
> ti.devmajor, ti.devminor = self.devnums
> AttributeError: Path instance has no attribute 'devnums
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1327550
>
> Title:
> OverflowError: signed integer is greater than maximum
>
> Status in Duplicity - Bandwidth Efficient Encrypted Backup:
> New
>
> Bug description:
> Duplicity version: v1.5.2.3
> Python version: Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
> OS Distro and version: Debian GNU/Linux 6.0.9 (squeeze)
> Type of target filesystem: Linux (FTP)
> Log output: attached
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1327550/+subscriptions
>

That works! Thank you very much for your prompt help!

The reason of the problem:
Warning: dev/ploop33520p1 invalid devnums (0x82F0B601), treating as (0,0).
Warning: dev/ploop39497p1 invalid devnums (0x9A40B691), treating as (0,0).
Warning: dev/ploop39987p1 invalid devnums (0x9C30B631), treating as (0,0).
Warning: dev/ploop44394p1 invalid devnums (0xAD60B6A1), treating as (0,0).
Warning: dev/ploop44476p1 invalid devnums (0xADB0B6C1), treating as (0,0).
Warning: dev/ploop49876p1 invalid devnums (0xC2D0B641), treating as (0,0).
Warning: dev/ploop60386p1 invalid devnums (0xEBE0B621), treating as (0,0).
Warning: lib/udev/devices/ploop33520p1 invalid devnums (0x82F0B601), treating as (0,0).
Warning: lib/udev/devices/ploop39497p1 invalid devnums (0x9A40B691), treating as (0,0).
Warning: lib/udev/devices/ploop39987p1 invalid devnums (0x9C30B631), treating as (0,0).
Warning: lib/udev/devices/ploop44394p1 invalid devnums (0xAD60B6A1), treating as (0,0).
Warning: lib/udev/devices/ploop44476p1 invalid devnums (0xADB0B6C1), treating as (0,0).
Warning: lib/udev/devices/ploop49876p1 invalid devnums (0xC2D0B641), treating as (0,0).
Warning: lib/udev/devices/ploop60386p1 invalid devnums (0xEBE0B621), treating as (0,0).

Andreas, if you are still have this setup, could you please do me a favour? I am wanting to clean up this code, but do not want to break this again, and I have been unable to replicate this error on my machine.

If you can help, could you please:
1) Find the relevant block in duplicity/path.py:

        if self.type in ("chr", "blk"):
            try:
                self.devnums = (os.major(self.stat.st_rdev),
                                os.minor(self.stat.st_rdev))
            except:
                log.Warn(_("Warning: %s invalid devnums (0x%X), treating as (0, 0).")
                         % (util.ufn(self.get_relative_path()), self.stat.st_rdev))
                self.devnums = (0, 0)

2) Replace it with the old version that used to be broken and see if you get the OverflowError:

        if self.type in ("chr", "blk"):
            self.devnums = (os.major(self.stat.st_rdev),
                            os.minor(self.stat.st_rdev))

3) If you do still get the error, try this slightly modified version and see if it fixes the error:

        if self.type in ("chr", "blk"):
            try:
                self.devnums = (os.major(self.stat.st_rdev),
                                os.minor(self.stat.st_rdev))
            except OverflowError:
                log.Warn(_("Warning: %s invalid devnums (0x%X), treating as (0, 0).")
                         % (util.ufn(self.get_relative_path()), self.stat.st_rdev))
                self.devnums = (0, 0)