Merge ~mirespace/ubuntu/+source/rsync:bionic-hang-when-overabundance-messages-lp1528921 into ubuntu/+source/rsync:ubuntu/bionic-devel

[Miriam España Acebal]

This is same fix that the proposed for Focal here https://code.launchpad.net/~mirespace/ubuntu/+source/rsync/+git/rsync/+merge/415244

----------------
Hi Team,

PPA with the package (rsync_3.1.2-2.1ubuntu1.3) is here:

https://launchpad.net/~mirespace/+archive/ubuntu/sru-rsync-lp1528921-overabundance-messages

sudo add-apt-repository ppa:mirespace/sru-rsync-lp1528921-overabundance-messages
sudo apt-get update

What the user suffering from this bug experiences is that the big amount of informative messages
related to the copy process with the three spawned processes(sender, receiver and generator) exhaust the I/O buffers and the sync gets stuck, either because there are too many files to synchronise and/or because too many detail messages (levels of verbose mode) have been requested in the output. These messages generated by rsync go in the protocol stream.

The solution applied here (incrementing the size of I/O buffer) comes from upstream (https://github.com/WayneD/rsync/commit/01b9bbb0f94e4ddd4b1e3ad678b220ca09bc0ad0), starting in 3.2.0. version.

No DEP-8 tests are present. There is a testsuite in the package, but is not used in the building either.

Lintian also passed (there is a lot of warnings about the absence of the series files):

W: rsync source: binary-nmu-debian-revision-in-source 3.1.2-2.1ubuntu1.3
W: rsync source: patch-file-present-but-not-mentioned-in-series 0001-Enforce-trailing-0-when-receiving-xattr-name-values.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series 0002-Check-fname-in-recv_files-sooner.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series 0003-Sanitize-xname-in-read_ndx_and_attrs.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series 0004-Check-daemon-filter-against-fnamecmp-in-recv_files.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series CVE-2016-9840.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series CVE-2016-9841.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series CVE-2016-9842_1.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series CVE-2016-9842_2.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series CVE-2016-9843.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series CVE-2018-5764.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series allow-missing-parent-dir-delete-missing-args.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series avoid-deadlock-huge-amounts-verbose-messages.patch
W: rsync source: patch-file-present-but-not-mentioned-in-series backup-dir-implies-backup.diff
W: rsync source: patch-file-present-but-not-mentioned-in-series copy-devices.diff
W: rsync source: patch-file-present-but-not-mentioned-in-series logdir.diff
W: rsync source: patch-file-present-but-not-mentioned-in-series manpage-compress-level.diff
W: rsync source: patch-file-present-but-not-mentioned-in-series noatime.diff
W: rsync source: patch-file-present-but-not-mentioned-in-series speedup-xattrs.diff
W: rsync source: patch-file-present-but-not-mentioned-in-series ssh-6-option.diff
W: rsync source: patch-file-present-but-not-mentioned-in-series templines.diff
W: rsync source: patch-file-present-but-not-mentioned-in-series time-limit.diff
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_BUILD_ARCH (line 29)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_BUILD_GNU_CPU (line 30)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_BUILD_GNU_SYSTEM (line 32)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_BUILD_GNU_SYSTEM (line 34)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_BUILD_GNU_TYPE (line 36)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_HOST_ARCH (line 38)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_HOST_GNU_CPU (line 39)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_HOST_GNU_SYSTEM (line 40)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_HOST_GNU_TYPE (line 41)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_BUILD_GNU_TYPE (line 43)
W: rsync source: debian-rules-sets-dpkg-architecture-variable DEB_HOST_GNU_TYPE (line 44)
W: rsync source: debian-watch-could-verify-download debian/upstream/signing-key.pgp
N: 3 tags overridden (1 error, 2 info)

I: Lintian run was successful.

SRU template was filed in the bug with the steps for reproducing, bad and good cases.

Thanks in advance for your time in reviewing this.

[Andreas Hasenack (ahasenack)]

Same patch as the other MP. Fun, I don't remember seeing a package that applied patches manually via d/rules like this one, with no debian/series ;)

+1

review: Approve

[Miriam España Acebal (mirespace)]

> Same patch as the other MP. Fun, I don't remember seeing a package that
> applied patches manually via d/rules like this one, with no debian/series ;)
>
> +1

Me neither, and I was totally crazy looking for the series file... until Lucas pointed me to the rules file... maybe I should have explicitly mentioned it in the MP's description... sorry for the fun!

[Andreas Hasenack (ahasenack)]

Uploaded
$ dput ubuntu ../rsync_3.1.2-2.1ubuntu1.3_source.changes
D: Setting host argument.
Checking signature on .changes
gpg: ../rsync_3.1.2-2.1ubuntu1.3_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../rsync_3.1.2-2.1ubuntu1.3.dsc: Valid signature from AC983EB5BF6BCBA9
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading rsync_3.1.2-2.1ubuntu1.3.dsc: done.
  Uploading rsync_3.1.2-2.1ubuntu1.3.debian.tar.xz: done.
  Uploading rsync_3.1.2-2.1ubuntu1.3_source.buildinfo: done.
  Uploading rsync_3.1.2-2.1ubuntu1.3_source.changes: done.
Successfully uploaded packages.

diff --git a/debian/changelog b/debian/changelog
index 7bec431..04c64a9 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,14 @@
+rsync (3.1.2-2.1ubuntu1.3) bionic; urgency=medium
+
+  * d/p/avoid-deadlock-huge-amounts-verbose-messages.patch:
+    Allow the receiver to increase their iobuf.msg xbuf if it fills
+    up. This ensures that the receiver will never block trying to
+    output a message, and thus it will always drain the data from
+    the sender and keep the whole thing from clogging up. Thanks to
+    Wayne Davison <[e-mail]>. (LP: #1528921)
+
+ -- Miriam España Acebal <[e-mail]>  Tue, 08 Feb 2022 13:26:20 +0100
+
 rsync (3.1.2-2.1ubuntu1.2) bionic; urgency=medium
 
   * d/p/allow-missing-parent-dir-delete-missing-args.patch:
diff --git a/debian/patches/avoid-deadlock-huge-amounts-verbose-messages.patch b/debian/patches/avoid-deadlock-huge-amounts-verbose-messages.patch
new file mode 100644
index 0000000..f8324c4
--- /dev/null
+++ b/debian/patches/avoid-deadlock-huge-amounts-verbose-messages.patch
@@ -0,0 +1,44 @@
+Description: Avoid a deadlock due to huge amounts of verbose messages.
+ Allow the receiver to increase their iobuf.msg xbuf if it fills up.
+ This ensures that the receiver will never block trying to output a
+ message, and thus it will always drain the data from the sender and
+ keep the whole thing from clogging up.
+Author: Wayne Davison <[e-mail]>
+Origin: upstream, https://github.com/WayneD/rsync
+Bug: https://bugzilla.samba.org/show_bug.cgi?id=11166
+Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1528921
+Forwarded: not-needed
+Applied-Upstream: https://github.com/WayneD/rsync/commit/01b9bbb0f94e4ddd4b1e3ad678b220ca09bc0ad0
+Last-Update: 2022-02-07
+---
+--- a/io.c
++++ b/io.c
+@@ -963,8 +963,17 @@
+ 	} else
+ #endif
+ 		needed = len + 4 + 3;
+-	if (iobuf.msg.len + needed > iobuf.msg.size)
+-		perform_io(needed, PIO_NEED_MSGROOM);
++	if (iobuf.msg.len + needed > iobuf.msg.size) {
++		if (!am_receiver)
++			perform_io(needed, PIO_NEED_MSGROOM);
++		else { /* We allow the receiver to increase their iobuf.msg size to avoid a deadlock. */
++			size_t old_size = iobuf.msg.size;
++			restore_iobuf_size(&iobuf.msg);
++			realloc_xbuf(&iobuf.msg, iobuf.msg.size * 2);
++			if (iobuf.msg.pos + iobuf.msg.len > old_size)
++				memcpy(iobuf.msg.buf + old_size, iobuf.msg.buf, iobuf.msg.pos + iobuf.msg.len - old_size);
++		}
++	}
+ 
+ 	pos = iobuf.msg.pos + iobuf.msg.len; /* Must be set after any flushing. */
+ 	if (pos >= iobuf.msg.size)
+@@ -1185,7 +1194,7 @@ int read_line(int fd, char *buf, size_t
+ 
+ #ifdef ICONV_OPTION
+ 	if (flags & RL_CONVERT && iconv_buf.size < bufsiz)
+-		realloc_xbuf(&iconv_buf, bufsiz + 1024);
++		realloc_xbuf(&iconv_buf, ROUND_UP_1024(bufsiz) + 1024);
+ #endif
+ 
+   start: