Merge ~vpa1977/ubuntu/+source/openjdk-lts:02_security_release_debug into ubuntu/+source/openjdk-lts:ubuntu/devel

[Vladimir Petko]

Please do not merge, this merge proposal is a part in the series [1][2][3]:

Summary of changes:
 - backport debug symbols fix and symbols fix from openjdk-17

PPA: ppa:vpa1977/openjdk-11-april[4]

Testing:
 - debugging java works
     - without symbols
     - with symbols
 - NMT traces contain allocation locations

$ echo "public class infinite { public static void main(String[] args) { while (true); } }" > infinite.java
# javac infinite.java

debugging without symbols installed:
---------------cut--------------------------------
 gdb java
GNU gdb (Ubuntu 13.1-2ubuntu2) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from java...

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.ubuntu.com>
Enable debuginfod for this session? (y or [n])
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
(No debugging symbols found in java)
(gdb) break JNI_CreateJavaVM
Function "JNI_CreateJavaVM" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (JNI_CreateJavaVM) pending.
(gdb) run -cp . infinite
Starting program: /usr/bin/java -cp . infinite
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7bff6c0 (LWP 4568)]
[Switching to Thread 0x7ffff7bff6c0 (LWP 4568)]

Thread 2 "java" hit Breakpoint 1, 0x00007ffff70f3ad0 in JNI_CreateJavaVM ()
   from /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
(gdb) where
#0 0x00007ffff70f3ad0 in JNI_CreateJavaVM ()
   from /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
#1 0x00007ffff7fb2d11 in JavaMain ()
   from /usr/lib/jvm/java-11-openjdk-amd64/bin/../lib/jli/libjli.so
#2 0x00007ffff7fb798d in ThreadJavaMain ()
   from /usr/lib/jvm/java-11-openjdk-amd64/bin/../lib/jli/libjli.so
#3 0x00007ffff7c8f18a in start_thread (arg=<optimized out>)
    at ./nptl/pthread_create.c:444
#4 0x00007ffff7d1dbd0 in clone3 ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) info local
No symbol table info available.
(gdb)

---------------cut--------------------------------
No symbol table found, exports are present

debugging with symbols:
-------------------cut------------------------
$ gdb java
GNU gdb (Ubuntu 13.1-2ubuntu2) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from java...
Reading symbols from /usr/lib/debug/.build-id/70/0c9fdbc99cc23d7fc0aeb4fdc58c5748384fbf.debug...
(gdb) break JNI_CreateJavaVM
Function "JNI_CreateJavaVM" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (JNI_CreateJavaVM) pending.
(gdb) run -cp . infinite
Starting program: /usr/bin/java -cp . infinite

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.ubuntu.com>
Enable debuginfod for this session? (y or [n])
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7bff6c0 (LWP 4897)]
[Switching to Thread 0x7ffff7bff6c0 (LWP 4897)]

Thread 2 "java" hit Breakpoint 1, JNI_CreateJavaVM (vm=0x7ffff7bfee60, penv=0x7ffff7bfee68,
    args=0x7ffff7bfee70)
    at /usr/src/openjdk-lts-11.0.19+7~us1-0ubuntu1~ppa2/src/hotspot/share/prims/jni.cpp:4095
4095 /usr/src/openjdk-lts-11.0.19+7~us1-0ubuntu1~ppa2/src/hotspot/share/prims/jni.cpp: No such file or directory.
(gdb)
(gdb) where
#0 JNI_CreateJavaVM (vm=0x7ffff7bfee60, penv=0x7ffff7bfee68, args=0x7ffff7bfee70)
    at /usr/src/openjdk-lts-11.0.19+7~us1-0ubuntu1~ppa2/src/hotspot/share/prims/jni.cpp:4095
#1 0x00007ffff7fb2d11 in InitializeJVM (ifn=<synthetic pointer>, penv=0x7ffff7bfee68,
    pvm=0x7ffff7bfee60)
    at /usr/src/openjdk-lts-11.0.19+7~us1-0ubuntu1~ppa2/src/java.base/share/native/libjli/java.c:1527
#2 JavaMain (_args=<optimized out>)
    at /usr/src/openjdk-lts-11.0.19+7~us1-0ubuntu1~ppa2/src/java.base/share/native/libjli/java.c:414
#3 0x00007ffff7fb798d in ThreadJavaMain (args=<optimized out>)
    at /usr/src/openjdk-lts-11.0.19+7~us1-0ubuntu1~ppa2/src/java.base/unix/native/libjli/java_md_solinux.c:743
#4 0x00007ffff7c8f18a in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
#5 0x00007ffff7d1dbd0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) info local
result = -1
(gdb)

---------------------------------------------------
Symbols table is present, exports are present.

NMT tracing:
---------------cut---------------------------
java -XX:NativeMemoryTracking=detail -cp . infinite&
[1] 4673
bsp@bsp-Standard-PC-Q35-ICH9-2009:~$ jcmd 4673 VM.native_memory detail
4673:

Native Memory Tracking:

...

Virtual memory map:

[0x00000000c3000000 - 0x0000000100000000] reserved 999424KB for Java Heap from
    [0x00007f5a04ea419f] ReservedHeapSpace::try_reserve_heap(unsigned long, unsigned long, bool, char*)+0xbf
    [0x00007f5a04ea4eac] ReservedHeapSpace::initialize_compressed_heap(unsigned long, unsigned long, bool)+0x75c
    [0x00007f5a04ea514e] ReservedHeapSpace::ReservedHeapSpace(unsigned long, unsigned long, bool, char const*)+0x13e
    [0x00007f5a04e66e72] Universe::reserve_heap(unsigned long, unsigned long)+0x52

 [0x00000000c3000000 - 0x00000000c6e00000] committed 63488KB from
            [0x00007f5a04798e05] G1PageBasedVirtualSpace::commit_preferred_pages(unsigned long, unsigned long)+0x65
            [0x00007f5a047991e8] G1PageBasedVirtualSpace::commit(unsigned long, unsigned long)+0x1a8
            [0x00007f5a047a4587] G1RegionsLargerThanCommitSizeMapper::commit_regions(unsigned int, unsigned long, WorkGang*)+0x47
            [0x00007f5a048299a8] HeapRegionManager::commit_regions(unsigned int, unsigned long, WorkGang*)+0x58
.....
---------------------cut-------------------

allocation locations are present.

[1] https://code.launchpad.net/~vpa1977/ubuntu/+source/openjdk-lts/+git/openjdk-lts/+merge/442078
[2] https://code.launchpad.net/~vpa1977/ubuntu/+source/openjdk-lts/+git/openjdk-lts/+merge/442081
[3] https://code.launchpad.net/~vpa1977/ubuntu/+source/openjdk-lts/+git/openjdk-lts/+merge/442082
[4] https://launchpad.net/~vpa1977/+archive/ubuntu/openjdk-11-april

[Graham Inggs (ginggs)]

review: Approve

diff --git a/debian/JB-jre.overrides.in b/debian/JB-jre.overrides.in
index c7cb389..fb8107e 100644
--- a/debian/JB-jre.overrides.in
+++ b/debian/JB-jre.overrides.in
@@ -1,2 +1,6 @@
 # these are in the -jre-headless package, -jre depends on it
 @basename@-jre binary: desktop-command-not-in-package
+
+# LP: #2012326 - in order to print allocation locations,
+# the libraries need to retain symbols
+@basename@-jre binary: unstripped-binary-or-object
\ No newline at end of file
diff --git a/debian/changelog b/debian/changelog
index 60120cf..8a532d8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -6,6 +6,8 @@ openjdk-lts (11.0.19+7~us1-0ubuntu1) UNRELEASED; urgency=medium
     - Release notes:
       TODO: waiting for announcement
     - d/p/*: refresh patches.
+  * d/rules: pack external debug symbols with build-id, do not pack duplicate
+    symbols, do not strip JVM shared libraries (LP: #2012326, LP: #2016739).
 
  -- Vladimir Petko <[e-mail]>  Thu, 20 Apr 2023 14:25:28 +1200
 
diff --git a/debian/rules b/debian/rules
index 5ba542f..9892b38 100755
--- a/debian/rules
+++ b/debian/rules
@@ -1513,15 +1513,46 @@ ifneq (,$(wildcard $(builddir)/$(sdkimg)/lib/*.diz))
 	) > debian/$(p_dbg).install
 endif
 
-# *.debuginfo should be located in /usr/lib/debug/ move files
+# the logic here -> move external debuginfo files into usr/lib/debug
+# check binaries and if the debuginfo is found, check if the binary has build id
+# debug information with build ids is moved under usr/lib/debug/.build-id
+# since we rewrite the build id with_debugedit=yes, patch the build id inside
+# the debug symbols file
+# otherwise (no buildid) just lib debuginfo file where it is
 	mkdir -p $(d)/usr/lib/debug
 	(cd $(d) && find . -path './$(basedir)/*' -name '*.debuginfo' | cpio -pdm usr/lib/debug/)
 	find $(d)/$(basedir)/ -name '*.debuginfo' -exec rm {} \;
-	( \
-	  echo 'usr/lib/debug/$(basedir)/bin/*.debuginfo'; \
-	  echo 'usr/lib/debug/$(basedir)/lib/*.debuginfo'; \
-	  echo 'usr/lib/debug/$(basedir)/lib/*/*.debuginfo'; \
-	) > debian/$(p_dbg).install
+	echo > debian/$(p_dbg).install
+	for binary in `(cd $(d) && find . -path './$(basedir)/*' -print)`; do \
+	  dir=`dirname $$binary`; \
+	  base=`basename $$binary .so`; \
+	  debuginfo=usr/lib/debug/$$dir/$$base.debuginfo; \
+	  [ -f $(d)/$$debuginfo ] || continue; \
+	  $(OBJCOPY) --compress-debug-sections $(d)/$$debuginfo; \
+	  b_id=$$(LC_ALL=C readelf -n $(d)/$$binary | sed -n 's/ *Build ID: *\([0-9a-f][0-9a-f]*\)/\1/p'); \
+	  if [ -z "$$b_id" ]; then \
+	    echo $$debuginfo >> debian/$(p_dbg).install; \
+	  else \
+	    debugdir=usr/lib/debug/.build-id/$${b_id:0:2}; \
+	    f=$${b_id:2}.debug; \
+	    mkdir -p $(d)/$$debugdir; \
+	    mv $(d)/$$debuginfo $(d)/$$debugdir/$$f; \
+	    chmod 644 $(d)/$$debugdir/$$f; \
+	    echo $$debugdir/$$f >> debian/$(p_dbg).install; \
+	    notes=$$(mktemp); \
+	    $(OBJCOPY) --dump-section=.note.gnu.build-id=$$notes $(d)/$$binary; \
+	    $(OBJCOPY) --update-section=.note.gnu.build-id=$$notes $(d)/$$debugdir/$$f; \
+	    rm $$notes; \
+	  fi; \
+	  if [[ "$(DEB_BUILD_OPTIONS)" != *nostrip* ]]; then  \
+	    $(OBJCOPY) --remove-section=.comment --remove-section=.note $(d)/$$binary; \
+	    if [[ "$$binary" != *.so ]]; then \
+	      $(STRIP) --strip-unneeded $(d)/$$binary; \
+	    fi; \
+	  fi; \
+	done;
+	find $(d)/usr/lib/debug/ -type d -empty -delete
+
 
 	dh_install --sourcedir=debian/tmp -XLICENSE
 ifeq (,$(findstring old, $(old_debhelper)))
@@ -1817,33 +1848,12 @@ endif
 	set -e; if which dh_strip_nondeterminism >/dev/null; then \
 	  dh_strip_nondeterminism -a $(nodemo) $(nojrez); \
 	fi
-	dh_strip -a $(nodemo) $(nojrez) \
-		-Xlibjvm.so -X.debuginfo --dbg-package=$(p_dbg)
+#	The strip operation is performed manually, because the build packages
+#	external debug symbols (so that native libraries are included in the modules)
+#	This step would override the actual debug symbols with the empty sections
+#	dh_strip -v -a $(nodemo) $(nojrez) \
+#		-Xlibjvm.so -X.debuginfo --dbg-package=$(p_dbg)
 
-ifeq (,$(findstring nostrip, $(DEB_BUILD_OPTIONS)))
-	set -ex; \
-	for i in {$(d_jrehl),$(d_jre),$(d_jrez)}/$(basedir)/lib/*/libjvm.so; do \
-	  [ -f $$i ] || continue; \
-	  b_id=$$(LC_ALL=C readelf -n $$i | sed -n 's/ *Build ID: *\([0-9a-f][0-9a-f]*\)/\1/p'); \
-	  if [ -z "$$b_id" ]; then \
-	    id=$$(echo $$i | sed -r 's,debian/[^/]+,$(d_dbg)/usr/lib/debug,'); \
-	    echo strip $$i; \
-	    mkdir -p $$(dirname $$id); \
-	    $(OBJCOPY) --only-keep-debug $$i $$id; \
-	    chmod 644 $$id; \
-	    $(STRIP) --remove-section=.comment --remove-section=.note \
-	      --strip-debug $$i; \
-	    $(OBJCOPY) --add-gnu-debuglink $$id $$i; \
-	  else \
-	    d=usr/lib/debug/.build-id/$${b_id:0:2}; \
-	    f=$${b_id:2}.debug; \
-	    mkdir -p $(d_dbg)/$$d; \
-	    $(OBJCOPY) --only-keep-debug --compress-debug-sections $$i $(d_dbg)/$$d/$$f; \
-	    chmod 644 $(d_dbg)/$$d/$$f; \
-	    $(STRIP) --remove-section=.comment --remove-section=.note $$i; \
-	  fi; \
-	done
-endif
 	dh_compress -a $(nodemo) $(nojrez) -Xexamples -Xdemo -Xpackage-list
 	dh_fixperms -a $(nodemo) $(nojrez)
 	dh_makeshlibs -p$(p_jrehl) -p$(p_jre)