Merge ~lucaskanashiro/ubuntu/+source/ruby2.7:fix-hash-iteration into ubuntu/+source/ruby2.7:ubuntu/focal-devel

[Lucas Kanashiro]

Fix LP: #2018215. It is a straight backport of a upstream patch:

PPA with proposed package:

https://launchpad.net/~lucaskanashiro/+archive/ubuntu/testing/+packages

autopkgtest summary:

autopkgtest [06:20:58]: @@@@@@@@@@@@@@@@@@@@ summary
run-all PASS
bundled-gems PASS
builtin-extensions PASS
rubyconfig PASS

[Bryce Harrington (bryce)]

The patch does look straightforward, and on visual inspection looks good to me.

I didn't spot the package in the PPA though, did it get uploaded? There's an older version, 2.7.0-5ubuntu1.9~ppa2, but the changelog includes another fix, and I'm unsure if the autopkgtest results for that version are valid?

  - ruby2.7/2.7.0-5ubuntu1.9~ppa2
    + ❌ ruby2.7 on focal for ppc64el @ 03.05.23 14:46:37 Log️ 🗒️
      • run-all FAIL 🟥
      • bundled-gems PASS 🟩
      • builtin-extensions PASS 🟩
      • rubyconfig PASS 🟩

Unfortunately, downgrading to this version did not allow validating the fix:

root@ruby27-hash-iteration:~# apt-cache policy ruby
ruby:
  Installed: 1:2.7+1
  Candidate: 1:2.7+1
  Version table:
 *** 1:2.7+1 500
        500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
root@ruby27-hash-iteration:~# apt-cache policy ruby2.7
ruby2.7:
  Installed: 2.7.0-5ubuntu1.9
  Candidate: 2.7.0-5ubuntu1.9
  Version table:
 *** 2.7.0-5ubuntu1.9 500
        500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
        100 /var/lib/dpkg/status
     2.7.0-5ubuntu1.9~ppa2 500
        500 http://ppa.launchpad.net/lucaskanashiro/testing/ubuntu focal/main amd64 Packages
     2.7.0-5ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
root@ruby27-hash-iteration:~# apt-get install ruby=2.7.0-5ubuntu1.9~ppa2
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Version '2.7.0-5ubuntu1.9~ppa2' for 'ruby' was not found
root@ruby27-hash-iteration:~# apt-get install ruby2.7=2.7.0-5ubuntu1.9~ppa2
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following packages will be DOWNGRADED:
  ruby2.7
0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 0 not upgraded.
Need to get 108 kB of archives.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://ppa.launchpad.net/lucaskanashiro/testing/ubuntu focal/main amd64 ruby2.7 amd64 2.7.0-5ubuntu1.9~ppa2 [108 kB]
Fetched 108 kB in 1s (142 kB/s)
dpkg: warning: downgrading ruby2.7 from 2.7.0-5ubuntu1.9 to 2.7.0-5ubuntu1.9~ppa2
(Reading database ... 33926 files and directories currently installed.)
Preparing to unpack .../ruby2.7_2.7.0-5ubuntu1.9~ppa2_amd64.deb ...
Unpacking ruby2.7 (2.7.0-5ubuntu1.9~ppa2) over (2.7.0-5ubuntu1.9) ...
Setting up ruby2.7 (2.7.0-5ubuntu1.9~ppa2) ...
Processing triggers for man-db (2.9.1-1) ...
root@ruby27-hash-iteration:~# ruby bug.rb
Traceback (most recent call last):
 1: from bug.rb:2:in `<main>'
bug.rb:2:in `each': ret: 2, hash modified during iteration (RuntimeError)

I think it'd be worthwhile to validate this in a PPA since the effect of the RHASH_AR_TABLE_REF() macro is not immediately obvious.

The SRU text looks good, I've run through the test case to verify it reproduces th...

review: Needs Information

[Bryce Harrington (bryce)]

$ ppa create ruby2.7-review-lp2018215
PPA 'ruby2.7-review-lp2018215' created for the following architectures:

  i386, amd64, armhf, ppc64el, s390x, arm64, powerpc

The PPA can be viewed at:

  https://launchpad.net/~bryce/+archive/ubuntu/ruby2.7-review-lp2018215

$ dput ppa:bryce/ruby2.7-review-lp2018215 ../ruby2.7_2.7.0-5ubuntu1.10_source.changes

$ ppa wait ppa:bryce/ruby2.7-review-lp2018215
* Still waiting on these builds:
  - ruby2.7 (2.7.0-5ubuntu1.10) amd64: Currently building
  - ruby2.7 (2.7.0-5ubuntu1.10) arm64: Currently building
  - ruby2.7 (2.7.0-5ubuntu1.10) armhf: Currently building
  - ruby2.7 (2.7.0-5ubuntu1.10) i386: Currently building
  - ruby2.7 (2.7.0-5ubuntu1.10) ppc64el: Currently building
  - ruby2.7 (2.7.0-5ubuntu1.10) s390x: Currently building

[Bryce Harrington (bryce)]

It took a few rebuilds but the packages are all successfully built in the PPA, and after installing the PPA to the test lxc container and upgrading, the test case passes:

root@ruby27-hash-iteration:~# cat bug.rb
h = {a:1, b:2, c:3, d:4, e:5, f:6, g:7, h:8}
h.each{|k,v| GC.start; h.delete(k)}
root@ruby27-hash-iteration:~# ruby bug.rb
root@ruby27-hash-iteration:~# echo $?
0

It'd be nice to have bug.rb print something to positively indicate success, but the exit code is good enough.

review: Approve

[git-ubuntu bot (git-ubuntu-bot)]

Approvers: lucaskanashiro, bryce
Uploaders: lucaskanashiro, bryce
MP auto-approved

review: Approve

[Lucas Kanashiro (lucaskanashiro)]

I did rebase my branch to fetch the latest security upload fixing a regression.

[Lucas Kanashiro (lucaskanashiro)]

Package uploaded:

Uploading ruby2.7_2.7.0-5ubuntu1.11.dsc
Uploading ruby2.7_2.7.0-5ubuntu1.11.debian.tar.xz
Uploading ruby2.7_2.7.0-5ubuntu1.11_source.changes

diff --git a/debian/changelog b/debian/changelog
index 076a677..2559c59 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+ruby2.7 (2.7.0-5ubuntu1.11) focal; urgency=medium
+
+  * d/p/0026-reload-AR-table-body-for-transient-heap.patch: Fix hash iteration
+    (LP: #2018215).
+
+ -- Lucas Kanashiro <[e-mail]>  Wed, 03 May 2023 04:51:06 -0300
+
 ruby2.7 (2.7.0-5ubuntu1.10) focal-security; urgency=medium
 
   * SECURITY REGRESSION: URI.parse returning empty when it should return nil
diff --git a/debian/patches/0026-reload-AR-table-body-for-transient-heap.patch b/debian/patches/0026-reload-AR-table-body-for-transient-heap.patch
new file mode 100644
index 0000000..5a1a971
--- /dev/null
+++ b/debian/patches/0026-reload-AR-table-body-for-transient-heap.patch
@@ -0,0 +1,28 @@
+From: Koichi Sasada <[e-mail]>
+Date: Mon, 13 Jan 2020 03:36:47 +0900
+Subject: reload AR table body for transient heap.
+
+ar_talbe (Hash representation for <=8 size) can use transient heap
+and the memory area can move. So we need to restore `pair' ptr after
+`func` call (which can run any programs) because of moving.
+[Bug #16503]
+
+Bug: https://bugs.ruby-lang.org/issues/16503
+Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/ruby2.7/+bug/2018215
+Origin: upstream, https://github.com/ruby/ruby/commit/8e8841f6bf58031a1fe5b0dbacb5a1fb442102df
+---
+ hash.c | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/hash.c b/hash.c
+index 82383e9..1d97cca 100644
+--- a/hash.c
++++ b/hash.c
+@@ -961,6 +961,7 @@ ar_foreach_check(VALUE hash, st_foreach_check_callback_func *func, st_data_t arg
+ 
+             switch (retval) {
+               case ST_CHECK: {
++                  pair = RHASH_AR_TABLE_REF(hash, i);
+                   if (pair->key == never) break;
+                   ret = ar_find_entry_hint(hash, hint, key);
+                   if (ret == RHASH_AR_TABLE_MAX_BOUND) {
diff --git a/debian/patches/series b/debian/patches/series
index c8434da..01fa9d3 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -26,3 +26,4 @@ CVE-2021-33621-3.patch
 fix_tzdata-2022.patch
 CVE-2023-28756-1.patch
 CVE-2023-28756-2.patch
+0026-reload-AR-table-body-for-transient-heap.patch