"Failed to parse bus message: Invalid argument" with Linux 5.8

Bug #1905245 reported by Gary van der Merwe
38
This bug affects 6 people
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Fix Released
Undecided
Unassigned
Bionic
Fix Released
High
Dan Streetman
Focal
Fix Released
High
Dan Streetman

Bug Description

[impact]

newer kernels introduced a new capability, and existing systemd doesn't have the name mapping for the new cap (since the mapping table is generated at systemd compile time), so it fails when trying to map the capability to a user-facing name, which causes failure when running commands like 'systemctl show'

[test case]

install a focal system, and install the 5.8 (or newer) kernel, e.g. from linux-generic-hwe-20.04-edge, and reboot into the new kernel.

Find any service that does not specify its CapabilityBoundingSet; e.g. 'apparmor', and run systemctl show on it:

ubuntu@lp1905245-f:~$ systemctl show -p CapabilityBoundingSet apparmor
Failed to parse bus message: Invalid argument

the command should correctly show the value, e.g.:
$ systemctl show -p CapabilityBoundingSet apparmor
CapabilityBoundingSet=cap_chown cap_dac_override ...etc...

[regression potential]

a regression would likely occur while systemd is parsing or printing or otherwise handling kernel capabilities. A regression could happen when running systemd commands, such as systemctl, or when pid1 is managing services.

[scope]

this is needed only in focal and bionic.

This is fixed upstream by PR 16424:
https://github.com/systemd/systemd/pull/16424
which was first included in v246, so this is already fixed in groovy and later.

This was introduced upstream in systemd by commit 52610b020c077ee769c6923249f7e6c4e99d2980 which was first included in v235, so this bug does not exist in Xenial.

This bug will reproduce on any system running under the 5.8 kernel, with the new capability, if the systemd binary was compiled with kernel headers that do not include the new capability. This means this is reproducable on bare-metal/vm instances running 5.8, as well as containers on hosts running 5.8. Therefore, while bionic may not ever receive a new kernel with added capability, it still needs to be patched to avoid the bug on a bionic container running on a host with the 5.8 kernel.

[other info]

there is a testcase-only related bug 1905044

[original description]

When I run `systemctl show myservice.service`, I get the following error message:

   Failed to parse bus message: Invalid argument

systemd version: 245.4-4ubuntu3.3
linux version: 5.8.0-29-generic #31~20.04.1-Ubuntu (From linux-generic-hwe-20.04-edge)

This is a bug that has been fixed in Debian. See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964926

Please can we port the fix to the ubuntu 20.04 version.

Revision history for this message
lotuspsychje (lotuspsychje) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please execute the following command only once, as it will automatically gather debugging information, in a terminal:
apport-collect 1905245

When reporting bugs in the future please use apport by using 'ubuntu-bug' and the name of the package affected. You can learn more about this functionality at https://wiki.ubuntu.com/ReportingBugs.

Revision history for this message
Dan Streetman (ddstreet) wrote :
tags: added: rls-bb-incoming rls-ff-incoming
Dan Streetman (ddstreet)
description: updated
Changed in systemd (Ubuntu):
status: New → Fix Released
Dan Streetman (ddstreet)
description: updated
Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Focal):
assignee: nobody → Dan Streetman (ddstreet)
importance: Undecided → Medium
status: New → In Progress
tags: added: fr-963
tags: removed: rls-bb-incoming rls-ff-incoming
Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Bionic):
assignee: nobody → Dan Streetman (ddstreet)
importance: Undecided → Medium
status: New → In Progress
Dan Streetman (ddstreet)
description: updated
description: updated
Revision history for this message
Stéphane Graber (stgraber) wrote :

Now that we have the 5.8 kernel in HWE for focal, all my servers are hitting this, both on metal and in containers. This causes things like "systemctl show" to show an error as well as partial data.

It really feels like this should have been fixed before the rollout of the 5.8 kernel through HWE...

Revision history for this message
Dan Streetman (ddstreet) wrote :

Sorry for the delay; the patched systemd is in the upload queue:
https://launchpad.net/ubuntu/focal/+queue?queue_state=1&queue_text=

Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Focal):
importance: Medium → High
Changed in systemd (Ubuntu Bionic):
importance: Medium → High
Dan Streetman (ddstreet)
description: updated
Revision history for this message
Chris Halse Rogers (raof) wrote : Please test proposed package

Hello Gary, or anyone else affected,

Accepted systemd into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/245.4-4ubuntu3.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Chris Halse Rogers (raof) wrote :

Hello Gary, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.44 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/245.4-4ubuntu3.4)

All autopkgtests for the newly accepted systemd (245.4-4ubuntu3.4) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

linux-hwe-5.8/5.8.0-37.42~20.04.1 (arm64)
netplan.io/0.100-0ubuntu4~20.04.3 (amd64)
apt/2.0.2ubuntu0.2 (armhf)
munin/2.0.56-1ubuntu1 (ppc64el)
gvfs/1.44.1-1ubuntu1 (amd64, arm64)
prometheus-alertmanager/0.15.3+ds-3ubuntu1 (armhf)
lxc/1:4.0.2-0ubuntu1 (amd64)
indicator-session/17.3.20+19.10.20190921-0ubuntu1 (ppc64el)
pyudev/0.21.0-3ubuntu1 (s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/237-3ubuntu10.44)

All autopkgtests for the newly accepted systemd (237-3ubuntu10.44) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

linux-azure-4.15/4.15.0-1104.116 (amd64)
munin/2.0.37-1ubuntu0.1 (arm64)
gvfs/1.36.1-0ubuntu1.3.3 (amd64)
libvirt/4.0.0-1ubuntu8.17 (i386)
apt/1.6.12ubuntu0.2 (armhf)
netplan.io/0.99-0ubuntu3~18.04.3 (amd64)
linux-hwe-5.0/5.0.0-63.69 (armhf)
openssh/1:7.6p1-4ubuntu0.3 (armhf, s390x, ppc64el, arm64, i386, amd64)
corosync/2.4.3-0ubuntu1.1 (armhf)
nut/2.7.4-5.1ubuntu2 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Dan Streetman (ddstreet) wrote :

focal host reproduction:

root@lp1905245-f:~# uname -a
Linux lp1905245-f 5.8.0-36-generic #40~20.04.1-Ubuntu SMP Wed Jan 6 10:15:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
root@lp1905245-f:~# dpkg -l systemd|grep systemd
ii systemd 245.4-4ubuntu3.3 amd64 system and service manager
root@lp1905245-f:~# systemctl show -p CapabilityBoundingSet apparmor
Failed to parse bus message: Invalid argument
root@lp1905245-f:~# echo $?
1

focal container repro:

root@lp1905245-f:~# lxc shell focal
root@focal:~# dpkg -l systemd|grep systemd
ii systemd 245.4-4ubuntu3.3 amd64 system and service manager
root@focal:~# systemctl show -p CapabilityBoundingSet apparmor
Failed to parse bus message: Invalid argument
root@focal:~# echo $?
1

bionic container repro:

root@lp1905245-f:~# lxc shell bionic
root@bionic:~# dpkg -l systemd|grep systemd
ii systemd 237-3ubuntu10.43 amd64 system and service manager
root@bionic:~# systemctl show -p CapabilityBoundingSet apparmor
Failed to parse bus message: Invalid argument
root@bionic:~# echo $?
1

focal host verification:

root@lp1905245-f:~# dpkg -l systemd|grep systemd
ii systemd 245.4-4ubuntu3.4 amd64 system and service manager
root@lp1905245-f:~# systemctl show -p CapabilityBoundingSet apparmor
CapabilityBoundingSet=cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_>
root@lp1905245-f:~# echo $?
0

focal container verification:

root@focal:~# dpkg -l systemd|grep systemd
ii systemd 245.4-4ubuntu3.4 amd64 system and service manager
root@focal:~# systemctl show -p CapabilityBoundingSet apparmor
CapabilityBoundingSet=cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_>
root@focal:~# echo $?
0

bionic container verification:

root@bionic:~# dpkg -l systemd|grep systemd
ii systemd 237-3ubuntu10.44 amd64 system and service manager
root@bionic:~# systemctl show -p CapabilityBoundingSet apparmor
CapabilityBoundingSet=cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_r
root@bionic:~# echo $?
0

tags: added: verification-done verification-done-bionic verification-done-focal
removed: verification-needed verification-needed-bionic verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 245.4-4ubuntu3.4

---------------
systemd (245.4-4ubuntu3.4) focal; urgency=medium

  * d/p/lp1905245/0001-basic-cap-list-parse-print-numerical-capabilities.patch,
    d/p/lp1905245/0002-basic-capability-util-let-cap_last_cap-return-unsign.patch,
    d/p/lp1905245/0003-basic-cap-list-reduce-scope-of-variables.patch:
    - print number of unknown capabilities instead of failing
      (LP: #1905245)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=5cd98102e16a6e4acc1444b10db3308d87930933
  * d/p/lp1890448-hwdb-Add-EliteBook-to-use-micmute-hotkey.patch:
    Add EliteBook to use micmute hotkey (LP: #1890448)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=238c8c1a7b9d75f69bdeafb1d55f1faf00acb063
  * d/extra/dhclient-enter-resolved-hook:
    suppress output of cmp command in dhclient hook (LP: #1878955)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=83df4fc182f8ffe87256f5d7c4b49cee5192529a
  * d/p/lp1905044-test-use-cap_last_cap-for-max-supported-cap-number-n.patch:
    test: use cap_last_cap() instead of capability_list_length()
    (LP: #1905044)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ff21f41e624d9e603f3be463846ce981a433842a
  * d/p/lp1903300/0001-network-VXLan-fix-adding-Group-address.patch,
    d/p/lp1903300/0002-network-VXLan-Add-support-for-remote-address.patch,
    d/p/lp1903300/0003-networkctl-Add-support-to-display-VXLan-remote-addre.patch:
    set vxlan multicast group when specified (LP: #1903300)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=9deff4b7c5495dbe738561ca47daf3756df9fcde
  * d/p/lp1907306/0001-sd-dhcp-client-don-t-log-timeouts-if-already-expired.patch,
    d/p/lp1907306/0002-sd-dhcp-client-track-dhcp4-t1-t2-expire-times.patch,
    d/p/lp1907306/0003-sd-dhcp-client-add-RFC2131-retransmission-details.patch,
    d/p/lp1907306/0004-sd-dhcp-client-simplify-dhcp4-t1-t2-parsing.patch,
    d/p/lp1907306/0005-sd-dhcp-client-correct-dhcpv4-renew-rebind-retransmi.patch,
    d/p/lp1907306/0006-sd-dhcp-client-correct-retransmission-timeout-to-mat.patch,
    d/p/lp1907306/0007-test-network-increase-wait_online-timeout-to-handle-.patch,
    d/p/lp1907306/0008-sd-dhcp-client-fix-renew-rebind-timeout-calculation-.patch:
    Send correct number of dhcpv4 renew and rebind requests
    (LP: #1907306)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=a73c51d0df284dcc38e6924d40eed810554bab2e
  * d/p/lp1902960-udev-re-assign-ID_NET_DRIVER-ID_NET_LINK_FILE-ID_NET.patch:
    Run net_setup_link on 'change' uevents (LP: #1902960)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ec7ba2358aa68d8d6276ed56ef91caafc287cecf
  * d/t/root-unittests:
    Remove any corrupt journal files (LP: #1881947)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=5481fececdb3cb35ca7118598cad537681b5ff14

 -- Dan Streetman <email address hidden> Wed, 06 Jan 2021 15:47:39 -0500

Changed in systemd (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 237-3ubuntu10.44

---------------
systemd (237-3ubuntu10.44) bionic; urgency=medium

  * d/extra/dhclient-enter-resolved-hook:
    suppress output of cmp command in dhclient hook (LP: #1878955)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=c5a2db69aafc7a3ab4e71bae44fd7ad9dd955c97
  * d/p/lp1905044/0001-capability-add-a-way-to-get-a-uint64_t-with-all-caps.patch,
    d/p/lp1905044/0002-test-use-cap_last_cap-for-max-supported-cap-number-n.patch:
    test: use cap_last_cap() instead of capability_list_length()
    (LP: #1905044)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=34ebc6e28e63881d40c91c5839597acc2fdab546
  * d/p/lp1905245/0001-basic-cap-list-parse-print-numerical-capabilities.patch:
    print number of unknown capabilities instead of failing
    (LP: #1905245)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=5ab225b7f731c6cf6b4655cb27c3a842150c4c1a
  * d/p/lp1907306/0001-sd-dhcp-client-don-t-log-timeouts-if-already-expired.patch,
    d/p/lp1907306/0002-sd-dhcp-client-track-dhcp4-t1-t2-expire-times.patch,
    d/p/lp1907306/0003-sd-dhcp-client-add-RFC2131-retransmission-details.patch,
    d/p/lp1907306/0004-sd-dhcp-client-simplify-dhcp4-t1-t2-parsing.patch,
    d/p/lp1907306/0005-sd-dhcp-client-correct-dhcpv4-renew-rebind-retransmi.patch,
    d/p/lp1907306/0008-sd-dhcp-client-fix-renew-rebind-timeout-calculation-.patch:
    Send correct number of dhcpv4 renew and rebind requests
    (LP: #1907306)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=036230cac8232bf4f970e565c355ee1a82fc2ee6
  * d/t/root-unittests:
    Remove any corrupt journal files (LP: #1881947)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=b284b93e40b6cb834bb40dd3db94850853ab5bb8

 -- Dan Streetman <email address hidden> Wed, 06 Jan 2021 16:04:25 -0500

Changed in systemd (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.