udev: boot script does not trigger subsystem coldplug

Bug #1713536 reported by bugproxy
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
Medium
Canonical Foundations Team
systemd (Ubuntu)
Fix Released
Undecided
Skipper Bug Screeners
Xenial
Fix Released
Undecided
Unassigned
Zesty
Fix Released
Undecided
Unassigned
Artful
Fix Released
Undecided
Skipper Bug Screeners

Bug Description

[Impact]
subsystems udev rules are not processed on boot, thus resulting in missing devices on boot / before rootfs is mounted.

[Solution]
trigger udev subsystems and devices, in the initramfs, in that order.

[Testcase]
Boot s390x system with chzdev configured devices, and cio_ignore=all kernel command line parameter. The chzdev configured devices should still be discovered on boot.

[Original Bug report]

The udev initramfs-tools boot script does not trigger subsystem "add" uevents. As a result, udev rules that listen to subsystem "add" events are never activated. This problem exists on at least Ubuntu 16.04 and 17.10.

On s390, this results in a boot failure if the kernel is configured to start with an active device black list (kernel parameter cio_ignore=all,!condev). An example for an affected udev rule looks like this:

    ACTION=="add", SUBSYSTEM=="subsystem", KERNEL=="ccw", RUN{program}+="/bin/sh -c 'echo free 0009,ec30,ec32,f5f0-f5f2 > /proc/cio_ignore'"

A proposed fix would be:

Modify /usr/share/initramfs-tools/scripts/init-top/udev:

Replace line
        udevadm trigger --action=add
with
        udevadm trigger --type=subsystems --action=add
        udevadm trigger --type=devices --action=add

This would also be consistent with the steps that the systemd udev coldplug unit file performs (see /lib/systemd/system/systemd-udev-trigger.service).

[Regression Potential]
More udev rules will be now triggered, earlier, during initramfs stage of boot rather than post-pivot-root. However, this is inline with current rootfs behaviour and thus should not regress behaviour - simply some rules will get triggered earlier.

bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-158070 severity-medium targetmilestone-inin1604
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → initramfs-tools-ubuntu-touch (Ubuntu)
Steve Langasek (vorlon)
affects: initramfs-tools-ubuntu-touch (Ubuntu) → systemd (Ubuntu)
Changed in ubuntu-power-systems:
importance: Undecided → Medium
assignee: nobody → Canonical Foundations Team (canonical-foundations)
affects: ubuntu-power-systems → ubuntu-z-systems
Changed in systemd (Ubuntu):
status: New → Confirmed
Changed in systemd (Ubuntu Artful):
status: Confirmed → Fix Committed
Changed in systemd (Ubuntu Zesty):
status: New → In Progress
description: updated
description: updated
Changed in systemd (Ubuntu Xenial):
status: New → In Progress
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted systemd into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/232-21ubuntu6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Zesty):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-zesty
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: New → In Progress
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2017-09-11 08:12 EDT-------
Fix successfully verified on Ubuntu 17.04 with udev package version 232-21ubuntu6.

tags: added: verification-done verification-done-zesty
removed: verification-needed verification-needed-zesty
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 234-2ubuntu10

---------------
systemd (234-2ubuntu10) artful; urgency=medium

  * Do not fail debootstrap if /etc/resolv.conf is immutable. (LP: #1713212)
  * Revert "Create /etc/resolv.conf on resolved start, if it is an empty file."
    As it is ineffective, and correct creation of /etc/resolv.conf has been fixed.
    This reverts commit ccba42504f216f6ffbc54eb2c9af347355f8d86b.
  * initramfs-tools: trigger udevadm add actions with subsystems first.
    This updates the initramfs-tools init-top udev script to trigger udevadm
    actions with type specified. This mimicks the
    systemd-udev-trigger.service. Without type specified only devices are
    triggered, but triggering subsystems may also be required and should happen
    before triggering the devices. This is the case for example on s390x with zdev
    generated udev rules. (LP: #1713536)

 -- Dimitri John Ledkov <email address hidden> Wed, 30 Aug 2017 11:22:41 +0100

Changed in systemd (Ubuntu Artful):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted systemd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/229-4ubuntu20 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
removed: verification-done
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Used 229-4ubuntu19, added set +x at the top of the init-top/udev script and booted with break=bottom to clearly observe initramfs boot messages and check that there is only one (devices) trigger.

Upgraded to 229-4ubuntu20, added set +x at the top of the init-top/udev script booted again and clearly boserved that subsystems and devices are triggered.

tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello bugproxy, or anyone else affected,

Accepted systemd into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/232-21ubuntu7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed verification-needed-zesty
removed: verification-done verification-done-zesty
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

With systemd 232-21ubuntu7 installed, changing default boot param to break=bottom, and added set -x to the udev init-top script and rebooting.

Observed that in the console boot log that both subsystems and devices are triggered.

tags: added: verification-done verification-done-zesty
removed: verification-needed verification-needed-zesty
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 229-4ubuntu20

---------------
systemd (229-4ubuntu20) xenial; urgency=medium

  * resolved: recognize DNS names with more than one trailing dot as invalid
    (LP: #1600000)
  * Ignore failures to set Nice priority on services in containers.
    (LP: #1709536)
  * networkd: accept `:' in ifnames in systemd/networkd. (LP: #1714933)
  * initramfs-tools: trigger udevadm add actions with subsystems first.
    (LP: #1713536)
  * networkd: Add support to set STP value on a bridge. (LP: #1665088)
  * networkd: add support for AgeingTImeSec, Priority and DefaultPVID settings.
    (LP: #1715131)
    - Drop cherrypick of uint16 config parser, superseeded by above commit.
  * networkd: add support to set ActiveSlave and PrimarySlave. (LP: #1709135)
    - networkd: add support to configure ARP, depedency of Primary/ActiveSlave.

 -- Dimitri John Ledkov <email address hidden> Tue, 05 Sep 2017 14:01:51 +0100

Changed in systemd (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 232-21ubuntu7

---------------
systemd (232-21ubuntu7) zesty; urgency=medium

  * networkd: accept `:' in ifnames in systemd/networkd. (LP: #1714933)
  * networkd: add support for ActiveSlave and PrimarySlave netdev options.
    (LP: #1709135)
  * Cherrypick upstream fix for a race between .mount and .automount units,
    which currently may result in automounts hanging. (LP: #1709649)
  * systemd.postinst: Fix-up version number check in the previous sru.
    The version check in the postinst was too tight, thus the SRU fix failed
    validation. (LP: #1710410)

systemd (232-21ubuntu6) zesty; urgency=medium

  * link: Fix offload features initialization.
    This fixes a regression introduced in v232 which caused TCP
    segmentation offloads being disabled by default, resulting in
    significant performance issues under certain conditions. (Closes: #864073)
    (LP: #1703393)
  * loginctl: Fix loginctl ignoring user given session IDs at command-line
    (LP: #1682154)
  * Disable fallback DNS servers.
    This causes resolved to call-home to google, attempt to access network when
    none is available, and spams logs. (LP: #1449001)
  * initramfs-tools: trigger udevadm add actions with subsystems first.
    This updates the initramfs-tools init-top udev script to trigger udevadm
    actions with type specified. This mimicks the
    systemd-udev-trigger.service. Without type specified only devices are
    triggered, but triggering subsystems may also be required and should happen
    before triggering the devices. This is the case for example on s390x with zdev
    generated udev rules. (LP: #1713536)
  * Enable systemd-resolved by default. (LP: #1710410)
  * core: fix systemd failing to serialize tasks correctly on daemon-reload.
    (LP: #1702823)

 -- Dimitri John Ledkov <email address hidden> Wed, 04 Oct 2017 14:22:02 +0100

Changed in systemd (Ubuntu Zesty):
status: Fix Committed → Fix Released
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2017-11-17 04:55 EDT-------
IBM buzgilla status-> closed, available within all distros in the field

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.