Merge ~vpa1977/ubuntu/+source/openjdk-20:03_security_release_debug into ubuntu/+source/openjdk-20:ubuntu/devel

[Vladimir Petko]

Please do not merge, this merge proposal is a part in the series [1][2][3][4]:

Summary of changes:
 - backport openjdk-17 debug symbols/exports fix

PPA: ppa:vpa1977/openjdk-20-april[5]

Testing:
 - it is possible to debug openjdk with and without symbols
 - NMT shows allocation locations

[1] https://code.launchpad.net/~vpa1977/ubuntu/+source/openjdk-20/+git/openjdk-20/+merge/442093
[2] https://code.launchpad.net/~vpa1977/ubuntu/+source/openjdk-20/+git/openjdk-20/+merge/442097
[3] https://code.launchpad.net/~vpa1977/ubuntu/+source/openjdk-20/+git/openjdk-20/+merge/442098
[4] https://code.launchpad.net/~vpa1977/ubuntu/+source/openjdk-20/+git/openjdk-20/+merge/442099
[5] https://launchpad.net/~vpa1977/+archive/ubuntu/openjdk-20-april

[Vladimir Petko (vpa1977)]

Debug without symbols:
--------------------cut-----------------------
gdb java
GNU gdb (Ubuntu 13.1-2ubuntu2) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from java...

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.ubuntu.com>
Enable debuginfod for this session? (y or [n])
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
(No debugging symbols found in java)
(gdb) break JNI_CreateJavaVM
Function "JNI_CreateJavaVM" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (JNI_CreateJavaVM) pending.
(gdb) run -cp . infinite
Starting program: /usr/bin/java -cp . infinite
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7bff6c0 (LWP 7644)]
[Switching to Thread 0x7ffff7bff6c0 (LWP 7644)]

Thread 2 "java" hit Breakpoint 1, 0x00007ffff6f8cdd0 in JNI_CreateJavaVM ()
   from /usr/lib/jvm/java-20-openjdk-amd64/lib/server/libjvm.so
(gdb) where
#0 0x00007ffff6f8cdd0 in JNI_CreateJavaVM ()
   from /usr/lib/jvm/java-20-openjdk-amd64/lib/server/libjvm.so
#1 0x00007ffff7fb2d93 in JavaMain ()
   from /usr/lib/jvm/java-20-openjdk-amd64/bin/../lib/libjli.so
#2 0x00007ffff7fb709d in ThreadJavaMain ()
   from /usr/lib/jvm/java-20-openjdk-amd64/bin/../lib/libjli.so
#3 0x00007ffff7c8f18a in start_thread (arg=<optimized out>)
    at ./nptl/pthread_create.c:444
#4 0x00007ffff7d1dbd0 in clone3 ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) info local
No symbol table info available.
(gdb)
------------------cut-------------------------------------

[Vladimir Petko (vpa1977)]

NMT traces:
---------------------------cut-----------------------
bsp@bsp-Standard-PC-Q35-ICH9-2009:~$ java -XX:NativeMemoryTracking=detail -cp . infinite&
[1] 7772
bsp@bsp-Standard-PC-Q35-ICH9-2009:~$ jcmd 7772 VM.native_memory detail
7772:

Native Memory Tracking:

(Omitting categories weighting less than 1KB)

Total: reserved=2457146KB, committed=127906KB
       malloc: 6690KB #7945
       mmap: reserved=2450456KB, committed=121216KB

- Java Heap (reserved=999424KB, committed=65536KB)
                            (mmap: reserved=999424KB, committed=65536KB)

- Class (reserved=1048635KB, committed=187KB)
                            (classes #518)
                            ( instance classes #431, array classes #87)
                            (malloc=59KB #980)
                            (mmap: reserved=1048576KB, committed=128KB)
                            ( Metadata: )
                            ( reserved=65536KB, committed=128KB)
                            ( used=64KB)
                            ( waste=64KB =49.82%)
                            ( Class space:)
                            ( reserved=1048576KB, committed=128KB)
                            ( used=4KB)
                            ( waste=124KB =97.11%)

- Thread (reserved=18481KB, committed=817KB)
                            (thread #18)
                            (stack: reserved=18432KB, committed=768KB)
                            (malloc=29KB #112)
                            (arena=19KB #34)

- Code (reserved=247729KB, committed=7589KB)
                            (malloc=41KB #820)
                            (mmap: reserved=247688KB, committed=7548KB)

- GC (reserved=58868KB, committed=38804KB)
                            (malloc=4496KB #351)
                            (mmap: reserved=54372KB, committed=34308KB)

- GCCardSet (reserved=15KB, committed=15KB)
                            (malloc=15KB #195)

- Compiler (reserved=167KB, committed=167KB)
                            (malloc=2KB #20)
                            (arena=165KB #5)

- Internal (reserved=197KB, committed=197KB)
                            (malloc=161KB #1410)
                            (mmap: reserved=36KB, committed=36KB)

- Symbol (reserved=1130KB, committed=1130KB)
                            (malloc=770KB #46)
                            (arena=360KB #1)

- Native Memory Tracking (reserved=250KB, committed=250KB)
                            (malloc=126KB #1787)
                            (tracking overhead=124KB)

- Shared class space (reserved=16384KB, committed=12756KB)
                            (mmap: reserved=16384KB, committed=12756KB)

- Arena Chunk (reserved=174KB, committed=174KB)
                            (malloc=174KB #54)

- Module (reserved=39KB, committed=39KB)
                            (malloc=39KB #1212)

- Safepoint (reserved=8KB, committed=8KB)
                            (mmap: reser...

[Vladimir Petko (vpa1977)]

Debugging with symbols installed:
-----------cut------------
 gdb java
GNU gdb (Ubuntu 13.1-2ubuntu2) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from java...
Reading symbols from /usr/lib/debug/.build-id/53/aa656ed842955b3fccb534e44e7e2a1d7f0fef.debug...
(gdb) break JNI_CreateJavaVM
Function "JNI_CreateJavaVM" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (JNI_CreateJavaVM) pending.
(gdb) run -cp . infinite
Starting program: /usr/bin/java -cp . infinite

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.ubuntu.com>
Enable debuginfod for this session? (y or [n])
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7bff6c0 (LWP 8249)]
[Switching to Thread 0x7ffff7bff6c0 (LWP 8249)]

Thread 2 "java" hit Breakpoint 1, JNI_CreateJavaVM (vm=0x7ffff7bfee60, penv=0x7ffff7bfee68,
    args=0x7ffff7bfee70) at src/hotspot/share/prims/jni.cpp:3704
3704 src/hotspot/share/prims/jni.cpp: No such file or directory.
(gdb) where
#0 JNI_CreateJavaVM (vm=0x7ffff7bfee60, penv=0x7ffff7bfee68, args=0x7ffff7bfee70)
    at src/hotspot/share/prims/jni.cpp:3704
#1 0x00007ffff7fb2d93 in InitializeJVM (ifn=<synthetic pointer>, penv=0x7ffff7bfee68,
    pvm=0x7ffff7bfee60) at src/java.base/share/native/libjli/java.c:1459
#2 JavaMain (_args=<optimized out>) at src/java.base/share/native/libjli/java.c:413
#3 0x00007ffff7fb709d in ThreadJavaMain (args=<optimized out>)
    at src/java.base/unix/native/libjli/java_md.c:650
#4 0x00007ffff7c8f18a in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
#5 0x00007ffff7d1dbd0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) info local
result = -1
(gdb)

-----------------cut-----

symbols table present, exports present

diff --git a/debian/JB-jre.overrides.in b/debian/JB-jre.overrides.in
index c7cb389..679d44c 100644
--- a/debian/JB-jre.overrides.in
+++ b/debian/JB-jre.overrides.in
@@ -1,2 +1,6 @@
 # these are in the -jre-headless package, -jre depends on it
 @basename@-jre binary: desktop-command-not-in-package
+
+# LP: #2012326 - in order to print allocation locations,
+# the libraries need to retain symbols
+@basename@-jre binary: unstripped-binary-or-object
diff --git a/debian/changelog b/debian/changelog
index 1424525..b132b7b 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -7,6 +7,8 @@ openjdk-20 (20.0.1+9~us1-0ubuntu1) UNRELEASED; urgency=medium
       TODO: waiting for announcement
   * d/p/*: refresh patches.
   * d/watch: update version format for the security release.
+  * d/rules: pack external debug symbols with build-id, do not pack duplicate
+    symbols, do not strip JVM shared libraries (LP: #2012326, LP: #2016739).
 
  -- Vladimir Petko <[e-mail]>  Thu, 20 Apr 2023 15:22:10 +1200
 
diff --git a/debian/rules b/debian/rules
index fa7c9f1..2febbe5 100755
--- a/debian/rules
+++ b/debian/rules
@@ -1533,15 +1533,45 @@ ifneq (,$(wildcard $(builddir)/$(sdkimg)/lib/*.diz))
 	) > debian/$(p_dbg).install
 endif
 
-# *.debuginfo should be located in /usr/lib/debug/ move files
+# the logic here -> move external debuginfo files into usr/lib/debug
+# check binaries and if the debuginfo is found, check if the binary has build id
+# debug information with build ids is moved under usr/lib/debug/.build-id
+# since we rewrite the build id with_debugedit=yes, patch the build id inside
+# the debug symbols file
+# otherwise (no buildid) just lib debuginfo file where it is
 	mkdir -p $(d)/usr/lib/debug
 	(cd $(d) && find . -path './$(basedir)/*' -name '*.debuginfo' | cpio -pdm usr/lib/debug/)
 	find $(d)/$(basedir)/ -name '*.debuginfo' -exec rm {} \;
-	( \
-	  echo 'usr/lib/debug/$(basedir)/bin/*.debuginfo'; \
-	  echo 'usr/lib/debug/$(basedir)/lib/*.debuginfo'; \
-	  echo 'usr/lib/debug/$(basedir)/lib/*/*.debuginfo'; \
-	) > debian/$(p_dbg).install
+	echo > debian/$(p_dbg).install
+	for binary in `(cd $(d) && find . -path './$(basedir)/*' -print)`; do \
+	  dir=`dirname $$binary`; \
+	  base=`basename $$binary .so`; \
+	  debuginfo=usr/lib/debug/$$dir/$$base.debuginfo; \
+	  [ -f $(d)/$$debuginfo ] || continue; \
+	  $(OBJCOPY) --compress-debug-sections $(d)/$$debuginfo; \
+	  b_id=$$(LC_ALL=C readelf -n $(d)/$$binary | sed -n 's/ *Build ID: *\([0-9a-f][0-9a-f]*\)/\1/p'); \
+	  if [ -z "$$b_id" ]; then \
+	    echo $$debuginfo >> debian/$(p_dbg).install; \
+	  else \
+	    debugdir=usr/lib/debug/.build-id/$${b_id:0:2}; \
+	    f=$${b_id:2}.debug; \
+	    mkdir -p $(d)/$$debugdir; \
+	    mv $(d)/$$debuginfo $(d)/$$debugdir/$$f; \
+	    chmod 644 $(d)/$$debugdir/$$f; \
+	    echo $$debugdir/$$f >> debian/$(p_dbg).install; \
+	    notes=$$(mktemp); \
+	    $(OBJCOPY) --dump-section=.note.gnu.build-id=$$notes $(d)/$$binary; \
+	    $(OBJCOPY) --update-section=.note.gnu.build-id=$$notes $(d)/$$debugdir/$$f; \
+	    rm $$notes; \
+	  fi; \
+	  if [[ "$(DEB_BUILD_OPTIONS)" != *nostrip* ]]; then  \
+	    $(OBJCOPY) --remove-section=.comment --remove-section=.note $(d)/$$binary; \
+	    if [[ "$$binary" != *.so ]]; then \
+	      $(STRIP) --strip-unneeded $(d)/$$binary; \
+	    fi; \
+	  fi; \
+	done;
+	find $(d)/usr/lib/debug/ -type d -empty -delete
 
 #	: # FIXME: the targets are not installed ...
 #	rm -rfv $(d)/$(basedir)/lib/jfr
@@ -1839,33 +1869,12 @@ endif
 	set -e; if command -v dh_strip_nondeterminism >/dev/null; then \
 	  dh_strip_nondeterminism -a $(nodemo) $(nojrez); \
 	fi
-	dh_strip -v -a $(nodemo) $(nojrez) \
-		-Xlibjvm.so -X.debuginfo --dbg-package=$(p_dbg)
+#	The strip operation is performed manually, because the build packages
+#	external debug symbols (so that native libraries are included in the modules)
+#	This step would override the actual debug symbols with the empty sections
+#	dh_strip -v -a $(nodemo) $(nojrez) \
+#		-Xlibjvm.so -X.debuginfo --dbg-package=$(p_dbg)
 
-ifeq (,$(findstring nostrip, $(DEB_BUILD_OPTIONS)))
-	set -ex; \
-	for i in {$(d_jrehl),$(d_jre),$(d_jrez)}/$(basedir)/lib/*/libjvm.so; do \
-	  [ -f $$i ] || continue; \
-	  b_id=$$(LC_ALL=C readelf -n $$i | sed -n 's/ *Build ID: *\([0-9a-f][0-9a-f]*\)/\1/p'); \
-	  if [ -z "$$b_id" ]; then \
-	    id=$$(echo $$i | sed -r 's,debian/[^/]+,$(d_dbg)/usr/lib/debug,'); \
-	    echo strip $$i; \
-	    mkdir -p $$(dirname $$id); \
-	    $(OBJCOPY) --only-keep-debug $$i $$id; \
-	    chmod 644 $$id; \
-	    $(STRIP) --remove-section=.comment --remove-section=.note \
-	      --strip-debug $$i; \
-	    $(OBJCOPY) --add-gnu-debuglink $$id $$i; \
-	  else \
-	    d=usr/lib/debug/.build-id/$${b_id:0:2}; \
-	    f=$${b_id:2}.debug; \
-	    mkdir -p $(d_dbg)/$$d; \
-	    $(OBJCOPY) --only-keep-debug --compress-debug-sections $$i $(d_dbg)/$$d/$$f; \
-	    chmod 644 $(d_dbg)/$$d/$$f; \
-	    $(STRIP) --remove-section=.comment --remove-section=.note $$i; \
-	  fi; \
-	done
-endif
 	dh_compress -a $(nodemo) $(nojrez) -Xexamples -Xdemo -Xpackage-list
 	dh_fixperms -a $(nodemo) $(nojrez)
 	dh_makeshlibs -p$(p_jrehl) -p$(p_jre)