[Bionic][ARM64]Failure debugging linux kernel

Bug #1830796 reported by Manoj Iyer
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
gdb (Ubuntu)
Fix Released
High
Manoj Iyer
Bionic
Fix Released
High
Unassigned

Bug Description

[Impact]
GDB fails to debug ARM64 vmlinux debug image with proc/kcore information. For example it is unable to print values of variables like 'jiffies_64'.

[Test]
# gdb /usr/lib/debug/boot/vmlinux-4.18.0-20-generic /proc/kcore
[New process 1]
Core was generated by `BOOT_IMAGE=/boot/vmlinuz-4.18.0-20-generic root=UUID=edb5e5a7-8272-4e13-aa25-37'.
#0 0x0000000000000000 in ?? ()
(gdb) p jiffies_64
Cannot access memory at address 0xffff000009616980
(gdb)

[Fix]
This issue was fixed upstream (git://sourceware.org/git/binutils-gdb.git) by the following patch:
8727de56b0 Fix tagged pointer support

[Regression Potential]
The risk of regression after applying this patch could be to architectures other than ARM64 due to changes to gdb/util.c. Please see comment #2 where I have tested the PPA package on a ppc64el system and found it does not introduce any regressions.

Manoj Iyer (manjo)
Changed in gdb (Ubuntu):
status: New → In Progress
Revision history for this message
Manoj Iyer (manjo) wrote :

Test package is available in PPA:
ppa:manjo/gdb-lp1830796

ubuntu@hotdog:~$ apt policy gdb
gdb:
  Installed: 8.1-0ubuntu4
  Candidate: 8.1-0ubuntu4
  Version table:
 *** 8.1-0ubuntu4 500
        500 http://ppa.launchpad.net/manjo/gdb-lp1830796/ubuntu bionic/main arm64 Packages
        100 /var/lib/dpkg/status
     8.1-0ubuntu3 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic/main arm64 Packages
ubuntu@hotdog:~$

root@hotdog:~# gdb /usr/lib/debug/boot/vmlinux-4.18.0-20-generic /proc/kcore
GNU gdb (Ubuntu 8.1-0ubuntu4) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib/debug/boot/vmlinux-4.18.0-20-generic...done.

warning: core file may not match specified executable file.
[New process 1]
Core was generated by `BOOT_IMAGE=/boot/vmlinuz-4.18.0-20-generic root=UUID=edb5e5a7-8272-4e13-aa25-37'.
#0 0x0000000000000000 in ?? ()
(gdb) p jiffies_64
$1 = 4406710034
(gdb)

Revision history for this message
Manoj Iyer (manjo) wrote :

== Testing on PPC64EL ==

ubuntu@tiselius:~$ apt policy gdb
gdb:
  Installed: 8.1-0ubuntu4
  Candidate: 8.1-0ubuntu4
  Version table:
 *** 8.1-0ubuntu4 500
        500 http://ppa.launchpad.net/manjo/gdb-lp1830796/ubuntu bionic/main ppc64el Packages
        100 /var/lib/dpkg/status
     8.1-0ubuntu3 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic/main ppc64el Packages
ubuntu@tiselius:~$

root@hotdog:~# gdb /usr/lib/debug/boot/vmlinux-4.18.0-20-generic /proc/kcore
GNU gdb (Ubuntu 8.1-0ubuntu4) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib/debug/boot/vmlinux-4.18.0-20-generic...done.

warning: core file may not match specified executable file.
[New process 1]
Core was generated by `BOOT_IMAGE=/boot/vmlinuz-4.18.0-20-generic root=UUID=edb5e5a7-8272-4e13-aa25-37'.
#0 0x0000000000000000 in ?? ()
(gdb) p jiffies_64
$1 = 4406710034
(gdb)

Revision history for this message
Manoj Iyer (manjo) wrote :

The attached debdiff to gdb fixes the issue. The patch was cleanly applied from the upstream GDB git repo, and test results are as posted to this bug report. No regression were found.

Manoj Iyer (manjo)
description: updated
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "Fix tagged pointer support." seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Manoj Iyer (manjo)
description: updated
Mathew Hodson (mhodson)
Changed in gdb (Ubuntu Bionic):
status: New → In Progress
importance: Undecided → High
Changed in gdb (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Andrew Cloke (andrew-cloke) wrote :

Next step is for this patch to go into the bionic gdb SRU queue.

Revision history for this message
Brian Murray (brian-murray) wrote :

I've uploaded this to the SRU queue for Ubuntu 18.04 so it can be reviewed by the SRU team.

Revision history for this message
Brian Murray (brian-murray) wrote :

Actually the version number used in the debdiff (8.1-0ubuntu4) had already been uploaded to the archive and so it was rejected. This is part of why https://wiki.ubuntu.com/SecurityTeam/UpdatePreparation#Update_the_packaging recommends appending a .1 to the package version number, which is what I did.

Revision history for this message
Manoj Iyer (manjo) wrote :

== Regression test on PPC64EL ==

ubuntu@tiselius:~$ uname -a
Linux tiselius 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:14:44 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux
ubuntu@tiselius:~$

ubuntu@tiselius:~$ apt policy gdb
gdb:
  Installed: 8.1-0ubuntu4
  Candidate: 8.1-0ubuntu4
  Version table:
 *** 8.1-0ubuntu4 500
        500 http://ppa.launchpad.net/manjo/gdb-lp1830796/ubuntu bionic/main ppc64el Packages
        100 /var/lib/dpkg/status
     8.1-0ubuntu3 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic/main ppc64el Packages
ubuntu@tiselius:~$

root@tiselius:/home/ubuntu# gdb /usr/lib/debug/boot/vmlinux-4.15.0-20-generic /proc/kcore
GNU gdb (Ubuntu 8.1-0ubuntu4) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "powerpc64le-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib/debug/boot/vmlinux-4.15.0-20-generic...done.

warning: core file may not match specified executable file.
[New process 1]
Core was generated by `root=UUID=295f571b-b731-4ebb-b752-60aadc80fc1b ro console=hvc0'.
#0 0x0000000000000000 in ?? ()
(gdb) p jiffies_64
$1 = 4294952364
(gdb)

Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Manoj, or anyone else affected,

Accepted gdb into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/gdb/8.1-0ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in gdb (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (gdb/8.1-0ubuntu3.1)

All autopkgtests for the newly accepted gdb (8.1-0ubuntu3.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

apport/2.20.9-0ubuntu7.7 (amd64)
python3.6/3.6.8-1~18.04.1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#gdb

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Manoj Iyer (manjo) wrote :

== Proposed testing results ==

ubuntu@recht:~$ apt policy gdb
gdb:
  Installed: 8.1-0ubuntu3.1
  Candidate: 8.1-0ubuntu3.1
  Version table:
 *** 8.1-0ubuntu3.1 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/main arm64 Packages
        100 /var/lib/dpkg/status
     8.1-0ubuntu3 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic/main arm64 Packages
ubuntu@recht:~$

ubuntu@recht:~$ sudo gdb /usr/lib/debug/boot/vmlinux-4.15.0-62-generic /proc/kcore
GNU gdb (Ubuntu 8.1-0ubuntu3.1) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib/debug/boot/vmlinux-4.15.0-62-generic...done.

warning: core file may not match specified executable file.
[New process 1]
Core was generated by `BOOT_IMAGE=/boot/vmlinuz-4.15.0-62-generic root=UUID=4e57d549-7669-4f93-82d9-8c'.
#0 0x0000000000000000 in ?? ()
(gdb) p jiffies_64
$1 = 4295689599
(gdb)

tags: added: verification-done-bionic
removed: verification-needed
tags: added: verification-done
removed: verification-needed-bionic
Revision history for this message
Brian Murray (brian-murray) wrote :

I retried python3.6 and it passed, however apport failed again but that is due to a timeout when connecting to Lanchpad so we've bad-tested that failure.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package gdb - 8.1-0ubuntu3.1

---------------
gdb (8.1-0ubuntu3.1) bionic; urgency=medium

  * Fix tagged pointer support. LP: #1830796

 -- Manoj Iyer <email address hidden> Fri, 28 Jun 2019 08:26:38 -0700

Changed in gdb (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for gdb has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.