please backport rsync 3 from Intrepid to Hardy

Builds using PBuilder on Hardy with no source changes. I've uploaded it to my PPA for testing: https://launchpad.net/~chrisccoulson/+archive

Doesn't install though. It has a dependency on lsb-base > 3.2-14

Marking incomplete due to changes to the source package required for the backport.

The backport for Etch seems to work fine with lsb-base 3.1-23.2etch1. The requirement for Lenny is lsb-base (>= 3.0). It should be investigated as to whether the changed dependency in Ubuntu is necessary. If not, it should be changed in the source package for Intrepid. If so, then the backport should have its dependency information changed making it a "trivial/safe source change".

I'll also note, for those who are interested in using rsync 3 now without having to build from source, that the rsync 3.0.3 backport for Etch seems to work fine under Ubuntu 8.04. I'll be doing extensive testing tonight and if I encounter problems I will post here with that information.

I have re-packaged rsync 3 for Hardy in PPA after changing the dependency on lsb-base to make it installable. I have also removed the use of status_of_proc() from the init script, as it seems from the changelog that this is what the dependency was for. The package installs and seems to work ok, but I haven't done any extensive testing on it yet.

Setting to Confirmed now (as it builds and installs) to await feedback from testing.

I've been using this package quite a bit now to perform my usual backups and synchronization of files between my desktop and my pendrive, with no regressions noted so far.

Debdiff added for source change backport.

Changes are:
1)Relaxed dependency on lsb-base in debian/control.
2)Removed use of status_of_proc() in debian/init.d (looking through the changelog, it appears that this is the reason for having the lsb-base version 3.2-14 dependency).

Setting the bug status back to new and awaiting review as per the process for source change backports (sorry for all the bug traffic!)

No apologies needed; I'm happy to see this moving forward. I've switched almost all of my servers to your version. I haven't noticed any regressions. In fact, all I've noticed is that the 2 major problems I've experienced with rsync 2.6.9 have been disappeared:

1. Synchronizing very large files - For large files, like 10GB virtual disk images the older version would sometimes not properly synchronize an image. This could be seen by using rsync on the file and then comparing md5 or sha1 checksums. It would work only is the -c option was set, which is much slower. Rsync 3 seems to work without -c so it must use a more careful algorithm.

2. Synchronizing large numbers of files - I'd say rsync 3 has knocked an hour or so off of certain backup jobs that require backing up more than a million files. Previously to prevent rsync from using too much memory I had to break the job into multiple smaller pieces. Now rsync 3 works fine by itself with memory usage and running time greatly reduced.

So thus far I'd highly support this becoming available for Hardy. I think any user that upgrades will never want to go back.

Marking this as Triaged since it build, installs, and runs and no obvious regressions.

Debdiff looks good, only I'd suggest a comment for why the status stanza was commented out (i.e. LSB too old in Hardy) in the init script for confused sysadmins.

No problem John - I've attached an updated debdiff with extra comment.

Thanks

I've also attached is the diff.gz

Great job on the debdiff. Many people would have just arbitrarily changed the version requirement and not hunted down why it was there.

Next question: How many of the rdepends does this break:

$ apt-cache rdepends rsync
rsync
Reverse Depends:
  ubuntu-standard
  vzctl
  vserver-debiantools
  tramp
  tailor
  systemimager-server
  systemimager-common
  systemimager-client
  slack
  sisu
  simple-cdd
  rsyncrypto
  rssh
  rsnapshot
  replicator
  python-poker2d
  python-poker-network
  python-poker-engine
  python-codespeak-lib
  mrb
  libfile-rsync-perl
  keyanalyze
  jailtool
  hdup
  gtkrsync
  grsync
  gadmin-rsync
  flexbackup
  flashybrid
  falcon
  education-common
  dpsyco-skel
  dirvish
  debmirror
  debaux
  dak
  apt-proxy
  ubuntu-standard
  subversion-tools
  lsb-core
  livecd-rootfs
  git-core
  dput
  dpsyco-lib
  bzrtools
  backuppc

Scott - Good question. I'll try and work through that list of packages as time permits.

agent 8131 - Would you mind helping me out testing a few of these packages too?

We can tick off dput already - I used that quite a bit on Hardy with this backported version of rsync, and it seems to work with no regressions.

The same goes for grsync - I used that quite extensively with this version of rsync.

There isn't anything to test with ubuntu-standard and ubuntu-minimal.

livecd-rootfs is only used for building this live CD image isn't it? Wouldn't that mean it didn't need to be tested with this version of rsync?

> livecd-rootfs is only used for building this live CD image isn't it?
> Wouldn't that mean it didn't need to be tested with this version of
> rsync?

Yes. No need to test that.

I'm glad work is being done on this. It is really useful that 3.0.* be backported to Hardy. The protocol changed in 3.0.0. If a rsync >= 3.0.0 tries to talk to a rsync <= 3.0.0 at the other end the communication can fail. I've had failures like:

2008/10/31 14:00:58 [19814] --acls requires protocol 30 or higher (negotiated 29).
2008/10/31 14:00:58 [19814] rsync error: protocol incompatibility (code 2) at compat.c(195) [sender=3.0.3]

The sender runs Intrepid, the receiver runs Hardy.

Having 3.0.* backported to Hardy will simplify management for environments running Intrepid and Hardy side by side.

To ease testing, I put rsync 3 for hardy in our PPA. Its available there for anyone who wishes to grab it

That being said, I'm concerned about backporting rsync since it means that if you have backports enabled, it might break rsync against non-backport enabled hardy boxes. If we could have both packages installed side by side (or at least, have it so that it a seperate source package (i.e. rsync3), it might be a safer alternative.

rsync 3 gracefully degrades and properly supports older rsync (as both a client and server).

I know because we have a mix of rsync server in production, at my work, transferring ~10Gb between sites.

As the machines are re-installed with Hardy, we are also installing our own backport; it would be advantageous for the Ubuntu community to have all the features that rsync offers (identified earlier) easily available to them.

Ack from ubuntu-backporters then. Any motu can upload.

I was actually thinking about this yesterday, wondering why it hadn't moved forward. I've continued using version 3 without any problems whatsoever, and that includes in mixed 2.6 and 3.0 environments. I will point out a couple of things. The work that Chris Coulson did was for version 3.0.3. Currently 3.0.4 is available in Jaunty and 3.0.5 was released Dec 28th and is in Debian Sid. Looking over the bugfixes for 3.0.5 I'd say it might be worth trying to get 3.0.5 into Jaunty and then backporting that into hardy (and possibly intrepid if people are interested).

http://samba.anu.edu.au/ftp/rsync/src/rsync-3.0.5-NEWS

For people looking to try the various versions:

3.0.3 can still be found here and can be installed with no other changes:
https://launchpad.net/~chrisccoulson/+archive

3.0.4 can be found here but also requires installing backported lsb-base (available from the same archive):
https://launchpad.net/~pgquiles/+archive

It also looks like 3.0.5 work is beginning in that latter archive:
https://launchpad.net/~pgquiles/+archive/+index?field.name_filter=rsync&field.status_filter=any

Uploaded.

Reading other comments, I perceive that there is confusion about the compatibility between 2.6 and 3.0.x.

It is simply not true that *in all cases* 2.6 can talk with 3.0.x.

Whether or not a 2.6 can talk to a 3.0.x depends on the the *actual arguments* passed to the client. In my earlier report you can see that the 2.6 and 3.0.3 cannot talk together because the parameter "--acls" was used. If "--acls" had not been used, they would probably have talked together fine. 3.0.x degrades its protocol only as much as it possibly can while at the same time still provide the features new to the 3.0.x series.

The only way to have full compatibility *in all cases* is to have the same version (or maybe different version of the same series is enough) on both the client and the server.

This bug was fixed in the package rsync - 3.0.4-3ubuntu1~hardy1

---------------
rsync (3.0.4-3ubuntu1~hardy1) hardy-backports; urgency=low

  [ Chris Coulson ]
  * Source backport for Hardy Backports (LP: #257211)
    - debian/control: Relaxed dependency on lsb-base from (>=3.2-14) to
      (>=3.2-4)
    - debian/init.d: Removed usage of status_of_proc(), as lsb-base in
      Hardy is too old.

 -- Scott Kitterman <email address hidden> Tue, 27 Jan 2009 21:35:04 -0500

Accepted.