systemd-resolved constantly restarts on Bionic upgraded from Xenial

Bug #1805183 reported by Neil Wilson
90
This bug affects 13 people
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Fix Released
Low
Unassigned
Bionic
Fix Released
Medium
Louis Bouchard
Cosmic
Won't Fix
Undecided
Unassigned
Disco
Fix Released
Low
Unassigned
Eoan
Fix Released
Undecided
Unassigned
Focal
Fix Released
Low
Unassigned

Bug Description

[Impact]
Log noise due to needless restart of resolved on lease expiry, maybe loss of cached state?
Application that require Name Resolution may fail while the service is being unnecessarily restarted

[Test case]
(1) Append make_resolv_conf to the end of the file, so it gets executed
(2) Execute the file with bash -x and different settings and ensure there are no restarts if the settings are the same, and that there are if settings change; for example:

sudo new_domain_name_servers=8.8.4.4 interface="wlp61s0" reason=REBIND bash -x debian/extra/dhclient-enter-resolved-hook
sudo new_domain_name_servers=8.8.4.4 interface="wlp61s0" reason=REBIND bash -x debian/extra/dhclient-enter-resolved-hook
=> no restart
sudo new_domain_name_servers=8.8.8.8 interface="wlp61s0" reason=REBIND bash -x debian/extra/dhclient-enter-resolved-hook
=> should restart
sudo new_domain_name_servers=8.8.8.8 interface="wlp61s0" reason=REBIND bash -x debian/extra/dhclient-enter-resolved-hook
=> no restart
sudo new_domain_name_servers=8.8.4.4 interface="wlp61s0" reason=REBIND bash -x debian/extra/dhclient-enter-resolved-hook
=> should restart

[Regression potential]
The change only restarts resolved when the settings change. If there's a bug in the logic, resolved might not be restarted when it should be. Also, since there will be less restarts of resolved, it will run longer, so if there are memory leaks they will become more apparent.

[other info]

this fix was included in the initial release of systemd for eoan, but the fix required the additional change in bug 1849608. Both the original patch plus that change (to avoid using bash-specific &>) are included in the b/d patch for this bug.

[Original bug report]
If a cloud server is upgraded from Xenial to Bionic, the dhclient system remains in place and any DHCP lease refreshes cause a needless restart of the system-resolved daemon

Nov 26 16:59:41 srv-qvjhx dhclient[825]: DHCPREQUEST of 10.226.209.106 on ens3 to 10.226.209.105 port 67 (xid=0x2bd41d7d)
Nov 26 16:59:41 srv-qvjhx dhclient[825]: DHCPACK of 10.226.209.106 from 10.226.209.105
Nov 26 16:59:41 srv-qvjhx systemd[1]: Stopping Network Name Resolution...
Nov 26 16:59:41 srv-qvjhx systemd[1]: Stopped Network Name Resolution.
Nov 26 16:59:41 srv-qvjhx systemd[1]: Starting Network Name Resolution...
Nov 26 16:59:41 srv-qvjhx systemd-resolved[1609]: Positive Trust Anchors:
Nov 26 16:59:41 srv-qvjhx systemd-resolved[1609]: . IN DS 19036 8 2 49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
Nov 26 16:59:41 srv-qvjhx systemd-resolved[1609]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Nov 26 16:59:41 srv-qvjhx systemd-resolved[1609]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 1
Nov 26 16:59:41 srv-qvjhx systemd-resolved[1609]: Using system hostname 'srv-qvjhx'.
Nov 26 16:59:41 srv-qvjhx systemd[1]: Started Network Name Resolution.
Nov 26 16:59:41 srv-qvjhx systemd[1]: Starting resolvconf-pull-resolved.service...
Nov 26 16:59:41 srv-qvjhx dhclient[825]: bound to 10.226.209.106 -- renewal in 1466 seconds.
Nov 26 16:59:41 srv-qvjhx systemd[1]: Started resolvconf-pull-resolved.service.

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: ubuntu-release-upgrader-core 1:16.04.25
ProcVersionSignature: Ubuntu 4.4.0-139.165-generic 4.4.160
Uname: Linux 4.4.0-139-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.18
Architecture: amd64
CrashDB: ubuntu
Date: Mon Nov 26 16:17:52 2018
PackageArchitecture: all
SourcePackage: ubuntu-release-upgrader
UpgradeStatus: No upgrade log present (probably fresh install)

Related branches

Revision history for this message
Neil Wilson (neil-aldur) wrote :
summary: - systems-resolved constantly restarts on Bionic upgraded from Xenial
+ systemd-resolved constantly restarts on Bionic upgraded from Xenial
tags: added: id-5c0011e969ed904c67dda9ee
Revision history for this message
Julian Andres Klode (juliank) wrote :

First of all, dhclient remaining in place is not a bug. Systems are not switching to netplan and networkd on upgrade.

The restart caused by /etc/dhcp/dhclient-enter-hooks.d/resolved to tell systemd-resolved about new servers and search domains, and hence expected to happen (resolved does not support a reload action, hence restart). I'm not sure what the alternative would look like, maybe a bunch of busctl calls feeding that information to the running daemon via dbus?

Revision history for this message
Julian Andres Klode (juliank) wrote :

The dbus API is only available in 239, which means we can't use it in bionic.

We could maybe make the hook just restart resolved if the new files it wrote are different from the old files.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Ah no, 229, not 239 it seems.

no longer affects: ubuntu-release-upgrader (Ubuntu)
Changed in systemd (Ubuntu):
status: New → Triaged
importance: Undecided → Low
Revision history for this message
Neil Wilson (neil-aldur) wrote :

I think just a delta change process would be fine. It's restarting when there is no change in lease details, and just clogging up the logs.

btw I am not suggesting leaving dhclient there is a bug - hence the title of the bug.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Here's a patch while the git branch is pushing.

Changed in systemd (Ubuntu):
status: Triaged → In Progress
Revision history for this message
Julian Andres Klode (juliank) wrote :

If we want to backport this to stable releases, we need a test case. What I came up with, as I was not able to simulate dhcp lease refreshes, is:

(1) Append make_resolv_conf to the end of the file, so it gets executed
(2) Execute the file with bash -x and different settings and ensure there are no restarts if the settings are the same, and that there are if settings change.

description: updated
description: updated
tags: added: patch
Revision history for this message
dat (dat-1982) wrote :

@juliank
I *believe* this is impacting our AWS ubuntu EC2 machines and causing wild DNS errors impacting our users, I tried to apply the patch in this thread but it doesn't work, it complains like this

    /etc/dhcp/dhclient-enter-hooks.d/resolved: Syntax error: "(" unexpected

about this line

    if ! cmp --quiet $oldstate <(md5sum $statedir/isc-dhcp-v4-$interface.conf $statedir/isc-dhcp-v6-$interface.conf 2>&1); then

Also for me this line

+ md5sum $statedir/isc-dhcp-v4-$interface.conf $statedir/isc-dhcp-v6-$interface.conf &> $oldstate

outputs to stdout

   $ sudo dhclient
   RTNETLINK answers: File exists
   d41d8cd98f00b204e9800998ecf8427e /run/systemd/resolved.conf.d/isc-dhcp-v4-ens5.conf
   md5sum: /run/systemd/resolved.conf.d/isc-dhcp-v6-ens5.conf: No such file or directory

and the resulting temp file is empty.

Dunno why your patch misbehaves like this but I really have no time to investigate further.

I've attached a patch that seems to be working on our servers in case others are experiencing the same issue and need a quick fix.

----
Adding few server details in case you need 'em:

$ uname -a
Linux ip-172-31-14-255 4.15.0-42-generic #45-Ubuntu SMP Thu Nov 15 19:32:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic
$ dhclient --version
isc-dhclient-4.3.5
$ systemd --version
systemd 237
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid
$ bash --version
GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-gnu)

Revision history for this message
dat (dat-1982) wrote :

been running for 1 hour, seems to be working as expected

finally got one minute to create a patch against systemd repo instead of a live server

Changed in systemd (Ubuntu Bionic):
status: New → Confirmed
Changed in systemd (Ubuntu Cosmic):
status: New → Confirmed
Changed in systemd (Ubuntu Disco):
status: In Progress → Confirmed
Revision history for this message
Rafael David Tinoco (rafaeldtinoco) wrote :

xnox, it looks like you're missing a "$" in md5sum command from:

https://launchpadlibrarian.net/426017250/systemd_240-6ubuntu7_240-6ubuntu8.diff.gz

for the -proposed systemd version:

/sbin/dhclient-script: 57: /etc/dhcp/dhclient-enter-hooks.d/resolved: Syntax error: "(" unexpected

Adding it solved my boot problem (systemd stucks in networking services).

cheers o/

Revision history for this message
Dimitri John Ledkov (xnox) wrote : Re: [Bug 1805183] Re: systemd-resolved constantly restarts on Bionic upgraded from Xenial

Lol

On Thu, 30 May 2019, 20:20 Rafael David Tinoco, <
<email address hidden>> wrote:

> xnox, it looks like you're missing a "$" in md5sum command from:
>
>
> https://launchpadlibrarian.net/426017250/systemd_240-6ubuntu7_240-6ubuntu8.diff.gz
>
> for the -proposed systemd version:
>
> /sbin/dhclient-script: 57: /etc/dhcp/dhclient-enter-hooks.d/resolved:
> Syntax error: "(" unexpected
>
> Adding it solved my boot problem (systemd stucks in networking
> services).
>
> cheers o/
>
> --
> You received this bug notification because you are subscribed to systemd
> in Ubuntu.
> Matching subscriptions: systemd
> https://bugs.launchpad.net/bugs/1805183
>
> Title:
> systemd-resolved constantly restarts on Bionic upgraded from Xenial
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1805183/+subscriptions
>

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (3.5 KiB)

This bug was fixed in the package systemd - 240-6ubuntu9

---------------
systemd (240-6ubuntu9) eoan; urgency=medium

  * Fix typpo in storage test.
    File: debian/tests/storage
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=f28aa5fe4ab175b99b6ea702559c59ca473b4ca8

  * Fix bashism
    File: debian/extra/dhclient-enter-resolved-hook
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=0725c1169ddde4f41cacba7af3e546704e2206be

systemd (240-6ubuntu8) eoan; urgency=medium

  * Only restart resolved on changes in dhclient enter hook.
    This prevents spurious restarts of resolved on rebounds when
    the addresses did not change. (LP: #1805183)
    Author: Julian Andres Klode
    File: debian/extra/dhclient-enter-resolved-hook
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=258893bae8cbb12670e4807636fe8f7e9fb5407a

  * Wait for cryptsetup unit to start, before stopping.
    Patch from cascardo. Plus small refactor for readability. (LP: #1814373)
    File: debian/tests/storage
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=b65aa350be7e61c65927fbc0921a750fcfaa51cd

  * Wait for systemctl is-system-running state.
    File: debian/tests/boot-smoke
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=776998f1f55c445b6e385cab69a4219c42d00838

systemd (240-6ubuntu7) eoan; urgency=medium

  * Revert "Add check to switch VTs only between K_XLATE or K_UNICODE"
    This reverts commit 60407728a1a453104e3975ecfdf25a254dd7cc44.
    Files:
    - debian/patches/Add-check-to-switch-VTs-only-between-K_XLATE-or-K_UNICODE.patch
    - debian/patches/Move-verify_vc_kbmode-to-terminal-util.c-as-vt_verify_kbm.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=18029ab5ff436bfb3b401f24cd1e3a4cf2a1579c

  * Cherrypick missing systemd-stable patches to unbreak wireguard peer endpoints.
    Signed-off-by: Dimitri John Ledkov <email address hidden> (LP: #1825378)
    Author: Dan Streetman
    Files:
    - debian/patches/network-wireguard-fixes-sending-wireguard-peer-setti.patch
    - debian/patches/network-wireguard-use-sd_netlink_message_append_sock.patch
    - debian/patches/sd-netlink-introduce-sd_netlink_message_append_socka.patch
    - debian/patches/test-network-add-more-checks-in-NetworkdNetDevTests..patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=4046f515e40c4dc80d18d2303466737f1f451f11

  * Remove expected failure from passing test.
    Signed-off-by: Dimitri John Ledkov <email address hidden> (LP: #1829450)
    Author: Dan Streetman
    File: debian/tests/systemd-fsckd
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=c43b12037d08555dc1d26593307726d7c7992df0

  * Fix false negative checking for running jobs after boot.
    Signed-off-by: Dimitri John Ledkov <email address hidden> (LP: #1825997)
    Author: Dan Streetman
    File: debian/tests/boot-smoke
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=aeb01631efbaf3fe851dee15d496e0b66b5c347f

  * Cherrypick ask-password: prevent buffer ...

Read more...

Changed in systemd (Ubuntu):
status: Confirmed → Fix Released
Louis Bouchard (louis)
Changed in systemd (Ubuntu Bionic):
importance: Undecided → Medium
assignee: nobody → Louis Bouchard (louis)
status: Confirmed → In Progress
Revision history for this message
Louis Bouchard (louis) wrote :

For Info, I'm repearing an SRU upload for Bionic hopefully available by End Of Day

description: updated
Revision history for this message
tf8 (tifeit) wrote :

This still looks like an issue in Ubuntu 18.04 LTS. At least on systems that was upgraded from 14.04 release. I will disable systemd-resolved and revert to good old resolv.conf but what is the actual recommended workaround while the fix is not released?

Revision history for this message
dat (dat-1982) wrote :

@tf8 the patch in https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1805183/comments/8 against a live server has been working fine for us on AWS for almost a year now.

Revision history for this message
Steve Langasek (vorlon) wrote :

+ md5sum $statedir/isc-dhcp-v4-$interface.conf $statedir/isc-dhcp-v6-$interface.conf &> $oldstate

"&>" is a bashism. /sbin/dhclient-script uses /bin/sh as its interpreter. This does not work as intended.

$ dash
$ echo foo &> output
foo
$ cat output
[1] + Done echo foo
$ ^D
$

Revision history for this message
Steve Langasek (vorlon) wrote : Proposed package upload rejected

An upload of systemd to bionic-proposed has been rejected from the upload queue for the following reason: "wrong use of bashisms in shell script, does not work as intended".

Changed in systemd (Ubuntu Cosmic):
status: Confirmed → Won't Fix
Revision history for this message
Steve Langasek (vorlon) wrote :

This was also pointed out specifically in comment #8.

I'm reopening the trunk task on this bug as well since the bug is still there in eoan.

Changed in systemd (Ubuntu):
status: Fix Released → Triaged
Revision history for this message
dat (dat-1982) wrote :

Steve in comment #9 I left a patch against the repo that iirc was working fine on my machines. Maybe it can be helpful. Thanks

Revision history for this message
Louis Bouchard (louis) wrote :

Ok, I'll have a look at it; I only backported the Eoan fix to Bionic

Revision history for this message
Ivan Brawley (brawley) wrote :

I've found the patch applied to Eoan was the broken version where it used the bash'ism instead of the much more basic (and more portable) sh'ism of redirecting both STDOUT and STDERR.

I have confirmed this behaviour in Eoan (see bug #1850009) after doing an upgrade to this release.

I did a quick fix patch that is basically the same as #9 (I redirect only STDOUT however where as #9 redirects both STDOUT and STDERR).

Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Eoan):
status: New → Fix Released
Changed in systemd (Ubuntu Focal):
status: Triaged → Fix Released
description: updated
Revision history for this message
Steve Langasek (vorlon) wrote :

This has been wrongly marked as fixed in eoan. It's not; the fix for this is still in eoan-proposed in systemd 242-7ubuntu3.2. I don't know why the separate bug LP: #1849608 was used for that SRU vs this one. Maybe the argument is that in eoan, systemd-resolved is *never* restarted, instead of being *always* restarted? But I think they should be treated as the same bug.

Changed in systemd (Ubuntu Eoan):
status: Fix Released → Triaged
status: Triaged → Fix Committed
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Neil, or anyone else affected,

Accepted systemd into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/240-6ubuntu5.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Disco):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-disco
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/240-6ubuntu5.8)

All autopkgtests for the newly accepted systemd (240-6ubuntu5.8) for disco have finished running.
The following regressions have been reported in tests triggered by the package:

prometheus-bind-exporter/unknown (armhf)
php7.2/7.2.24-0ubuntu0.19.04.1 (armhf)
gvfs/1.40.1-1ubuntu0.1 (ppc64el)
pdns-recursor/unknown (armhf)
webhook/unknown (armhf)
munin/2.0.47-1ubuntu3 (armhf, arm64)
systemd/240-6ubuntu5.8 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/disco/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Steven Shiau (stevenshiau) wrote :

I have tested Ubuntu Focal with systemd 243-3ubuntu1:
https://launchpad.net/ubuntu/+source/systemd/243-3ubuntu1
and the issue I have reported in Bug 1849608, which is marked as the duplicate of this bug, is gone now. Hence I confirmed the fix works here.

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Neil, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.32 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Dan Streetman (ddstreet) wrote :

with dnsmasq server setup to provide dhcp with 1m lease timeout, just run dhclient on test system:

ubuntu@lp1805183-d:~$ dpkg -l systemd|grep ii
ii systemd 240-6ubuntu5.7 amd64 system and service manager
ubuntu@lp1805183-d:~$ journalctl -b -u systemd-resolved | grep Started
Nov 14 16:02:12 lp1805183-d systemd[1]: Started Network Name Resolution.
ubuntu@lp1805183-d:~$ sudo dhclient ens8
ubuntu@lp1805183-d:~$ journalctl -b -u systemd-resolved | grep Started
Nov 14 16:02:12 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:04:32 lp1805183-d systemd[1]: Started Network Name Resolution.
ubuntu@lp1805183-d:~$ sleep 900 ; journalctl -b -u systemd-resolved | grep Started
Nov 14 16:02:12 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:04:32 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:05:22 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:06:11 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:06:57 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:07:52 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:08:41 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:09:25 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:10:17 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:11:03 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:11:51 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:12:33 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:13:24 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:14:16 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:15:11 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:16:03 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:16:46 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:17:34 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:18:23 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:19:16 lp1805183-d systemd[1]: Started Network Name Resolution.

ubuntu@lp1805183-d:~$ dpkg -l systemd|grep ii
ii systemd 240-6ubuntu5.8 amd64 system and service manager
ubuntu@lp1805183-d:~$ journalctl -b -u systemd-resolved | grep Started
Nov 14 16:21:20 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:29:41 lp1805183-d systemd[1]: Started Network Name Resolution.
ubuntu@lp1805183-d:~$ sudo dhclient ens8
cmp: EOF on /tmp/tmp.qY6fTBCaSb which is empty
ubuntu@lp1805183-d:~$ sleep 900 ; journalctl -b -u systemd-resolved | grep Started
Nov 14 16:21:20 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:29:41 lp1805183-d systemd[1]: Started Network Name Resolution.
Nov 14 16:30:51 lp1805183-d systemd[1]: Started Network Name Resolution.

tags: added: verification-done-disco
removed: verification-needed-disco
Revision history for this message
Neil Wilson (neil-aldur) wrote :

LGTM on bionic

ubuntu@srv-ywz63:~$ dpkg -l systemd | grep ii
ii systemd 237-3ubuntu10.32 i386 system and service manager
ubuntu@srv-ywz63:~$ journalctl -b -u systemd-resolved | grep Started
Nov 14 17:05:43 srv-ywz63 systemd[1]: Started Network Name Resolution.
Nov 14 17:05:44 srv-ywz63 systemd[1]: Started Network Name Resolution.
ubuntu@srv-ywz63:~$ sudo dhclient eth0
RTNETLINK answers: File exists
ubuntu@srv-ywz63:~$ journalctl -b -u systemd-resolved | grep Started
Nov 14 17:05:43 srv-ywz63 systemd[1]: Started Network Name Resolution.
Nov 14 17:05:44 srv-ywz63 systemd[1]: Started Network Name Resolution.
ubuntu@srv-ywz63:~$

tags: added: verification-done-bionic
removed: verification-needed-bionic
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/237-3ubuntu10.32)

All autopkgtests for the newly accepted systemd (237-3ubuntu10.32) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

gvfs/1.36.1-0ubuntu1.3.3 (ppc64el)
linux/unknown (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Dan Streetman (ddstreet) wrote :

> ubuntu@srv-ywz63:~$ sudo dhclient eth0
> RTNETLINK answers: File exists

I think you're trying to start dhclient on an interface that's already setup.

root@lp1805183-b:~# dpkg -l systemd|grep ii
ii systemd 237-3ubuntu10.31 amd64 system and service manager
root@lp1805183-b:~# journalctl -b -u systemd-resolved | grep Started
Nov 15 12:38:54 lp1805183-b systemd[1]: Started Network Name Resolution.
root@lp1805183-b:~# dhclient ens8
root@lp1805183-b:~# journalctl -b -u systemd-resolved | grep Started
Nov 15 12:38:54 lp1805183-b systemd[1]: Started Network Name Resolution.
Nov 15 12:45:42 lp1805183-b systemd[1]: Started Network Name Resolution.
root@lp1805183-b:~# sleep 300 ; journalctl -b -u systemd-resolved | grep Started
Nov 15 12:38:54 lp1805183-b systemd[1]: Started Network Name Resolution.
Nov 15 12:45:42 lp1805183-b systemd[1]: Started Network Name Resolution.
Nov 15 12:46:29 lp1805183-b systemd[1]: Started Network Name Resolution.
Nov 15 12:47:22 lp1805183-b systemd[1]: Started Network Name Resolution.
Nov 15 12:48:10 lp1805183-b systemd[1]: Started Network Name Resolution.
Nov 15 12:49:00 lp1805183-b systemd[1]: Started Network Name Resolution.
Nov 15 12:49:51 lp1805183-b systemd[1]: Started Network Name Resolution.

root@lp1805183-b:~# dpkg -l systemd|grep ii
ii systemd 237-3ubuntu10.32 amd64 system and service manager
root@lp1805183-b:~# journalctl -b -u systemd-resolved | grep Started
Nov 15 13:19:35 lp1805183-b systemd[1]: Started Network Name Resolution.
root@lp1805183-b:~# dhclient ens8
cmp: EOF on /tmp/tmp.64FqHLxW32 which is empty
root@lp1805183-b:~# sleep 300 ; journalctl -b -u systemd-resolved | grep Started
Nov 15 13:19:35 lp1805183-b systemd[1]: Started Network Name Resolution.
Nov 15 13:20:25 lp1805183-b systemd[1]: Started Network Name Resolution.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Neil Wilson (neil-aldur) wrote :

"I think you're trying to start dhclient on an interface that's already setup."

It just triggers a lease refresh on the existing interface, which has the same issue.

Nov 15 14:17:10 srv-ywz63 systemd-logind[981]: New session 27 of user ubuntu.
Nov 15 14:17:10 srv-ywz63 systemd[1]: Started Session 27 of user ubuntu.
Nov 15 14:17:30 srv-ywz63 sudo[12484]: ubuntu : TTY=pts/2 ; PWD=/home/ubuntu ; USER=root ; COMMAND=/sbin/dhclient eth0
Nov 15 14:17:30 srv-ywz63 sudo[12484]: pam_unix(sudo:session): session opened for user root by ubuntu(uid=0)
Nov 15 14:17:30 srv-ywz63 dhclient[12485]: DHCPREQUEST of 10.241.196.178 on eth0 to 255.255.255.255 port 67 (xid=0x511efd40)
Nov 15 14:17:30 srv-ywz63 dhclient[12485]: DHCPACK of 10.241.196.178 from 10.241.196.177
Nov 15 14:17:30 srv-ywz63 dhclient[12485]: bound to 10.241.196.178 -- renewal in 1421 seconds.
Nov 15 14:17:30 srv-ywz63 sudo[12484]: pam_unix(sudo:session): session closed for user root

Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Neil, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.33 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed verification-needed-bionic
removed: verification-done verification-done-bionic
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/237-3ubuntu10.33)

All autopkgtests for the newly accepted systemd (237-3ubuntu10.33) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

gvfs/1.36.1-0ubuntu1.3.3 (ppc64el, amd64)
dovecot/1:2.2.33.2-1ubuntu4.5 (armhf)
umockdev/0.11.1-1 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 240-6ubuntu5.8

---------------
systemd (240-6ubuntu5.8) disco; urgency=medium

  [ Victor Tapia ]
  * d/p/resolved_disable-connection-downgrade-when-DNSSEC-yes.patch
    Fix regression introduced by
    resolved-Mitigate-DVE-2018-0001-by-retrying-NXDOMAIN-with.patch when
    DNSSEC=yes (LP: #1796501)

  [ Dan Streetman ]
  * d/p/lp1840640-shared-seccomp-add-sync_file_range2.patch:
    allow sync_file_range2 in nspawn container (LP: #1840640)
  * d/p/lp1847527-journal-remote-do-not-request-Content-Length-if-Tran.patch:
    do not request Content-Length if Transfer-Encoding is chunked
    (LP: #1847527)
  * d/t/storage: fix flaky test
    (LP: #1847815)
  * d/p/lp1843381-dell_passthrough_skip_rename_retry.patch,
    debian/extra/rules/73-usb-net-by-mac.rules:
    fix rename delay for systems using "Dell MAC passthrough"
    (LP: #1843381)
  * d/p/lp1849733/0001-resolved-if-we-can-t-append-EDNS-OPT-RR-then-indicat.patch,
    d/p/lp1849733/0002-resolved-don-t-let-EDNS0-OPT-dgram-size-affect-TCP.patch:
    ignore EDNS0 payload limit when responding over TCP (LP: #1849733)
  * d/p/lp1849658-resolved-set-stream-type-during-DnsStream-creation.patch:
    - Fix bug in refcounting TCP stream types (LP: #1849658)
  * d/extra/dhclient-enter-resolved-hook:
    - only restart resolved if dhclient conf changed (LP: #1805183)

  [ Balint Reczey ]
  * d/p/test-execute-Filter-dev-.lxc-in-exec-dynamicuser-statedir.patch:
    fix test breakage due to running in nested lxd container
    (LP: #1845337)

 -- Dan Streetman <email address hidden> Fri, 04 Oct 2019 09:06:58 -0400

Changed in systemd (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Dan Streetman (ddstreet) wrote :

root@lp1805183-b:~# dpkg -l systemd|grep ii
ii systemd 237-3ubuntu10.33 amd64 system and service manager
root@lp1805183-b:~# journalctl -b -u systemd-resolved | grep Started
Nov 26 03:04:30 lp1805183-b systemd[1]: Started Network Name Resolution.
root@lp1805183-b:~# dhclient ens8
cmp: EOF on /tmp/tmp.yJwQabpfMe which is empty
root@lp1805183-b:~# sleep 300 ; journalctl -b -u systemd-resolved | grep Started
Nov 26 03:04:30 lp1805183-b systemd[1]: Started Network Name Resolution.
Nov 26 03:06:06 lp1805183-b systemd[1]: Started Network Name Resolution.

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 237-3ubuntu10.33

---------------
systemd (237-3ubuntu10.33) bionic; urgency=medium

  * d/p/lp1852754/0001-network-do-not-re-set-MTU-when-current-and-requested.patch,
    d/p/lp1852754/0002-network-call-link_acquire_conf-and-link_enter_join_n.patch,
    d/p/lp1852754/0003-network-prohibit-to-set-MTUBytes-and-UseMTU-simultan.patch:
    - Complete link setup after setting mtu (LP: #1852754)

systemd (237-3ubuntu10.32) bionic; urgency=medium

  [ Victor Tapia ]
  * d/p/resolved_disable-connection-downgrade-when-DNSSEC-yes.patch
    Fix regression introduced by
    resolved-Mitigate-DVE-2018-0001-by-retrying-NXDOMAIN-with.patch when
    DNSSEC=yes (LP: #1796501)

  [ Dan Streetman ]
  * d/p/fix-typo-lp1668771-resolved-switch-cache-option-to-a-tri-state-option-s.patch:
    - Fix typo in previous patch
  * d/p/lp1840640-shared-seccomp-add-sync_file_range2.patch:
    - allow sync_file_range2 in nspawn container
      (LP: #1840640)
  * d/p/lp1783994-dissect-Don-t-count-RPMB-and-boot-partitions-8609.patch:
    - avoid systemd-gpt-auto-generator failure if mmc dev present
      (LP: #1783994)
  * d/p/lp1832672-resolved-rework-parsing-of-etc-hosts.patch:
    - do not fail entire file on error when parsing /etc/hosts
    - parse # char anywhere in line as start of comment
      (LP: #1832672)
  * d/p/lp1843381-dell_passthrough_skip_rename_retry.patch,
    debian/extra/rules/73-usb-net-by-mac.rules:
    - fix rename delay for systems using "Dell MAC passthrough"
      (LP: #1843381)
  * d/p/lp1849733/0001-resolved-longlived-TCP-connections.patch,
    d/p/lp1849733/0002-resolved-line-split-dns_stream_new-function-signatur.patch,
    d/p/lp1849733/0003-resolved-add-some-assert-s.patch,
    d/p/lp1849733/0004-stream-track-type-of-DnsStream-object.patch,
    d/p/lp1849733/0005-llmnr-add-comment-why-we-install-no-complete-handler.patch,
    d/p/lp1849733/0006-resolved-restart-stream-timeout-whenever-we-managed-.patch,
    d/p/lp1849733/0007-resolved-only-call-complete-with-zero-argument-in-LL.patch,
    d/p/lp1849733/0008-resolved-add-comment-to-dns_stream_complete-about-it.patch,
    d/p/lp1849733/0009-resolved-keep-stub-stream-connections-up-for-as-long.patch,
    d/p/lp1849733/0010-resolved-if-we-can-t-append-EDNS-OPT-RR-then-indicat.patch,
    d/p/lp1849733/0011-resolved-don-t-let-EDNS0-OPT-dgram-size-affect-TCP.patch,
    d/p/lp1849733/0012-resolved-add-new-accessor-dns_stream_take_read_packe.patch,
    d/p/lp1849733/0013-resolve-do-not-complete-stream-transaction-when-it-i.patch:
    - add TCP pipelining to handle getaddrinfo() fallback to TCP
    - ignore EDNS0 payload limit when responding over TCP (LP: #1849733)
  * d/p/lp1849658-resolved-set-stream-type-during-DnsStream-creation.patch:
    - Fix bug in refcounting TCP stream types (LP: #1849658)
  * d/p/lp1850704/0001-networkd-Unify-set-MTU.patch,
    d/p/lp1850704/0002-network-drop-redundant-lines.patch:
    - Fix setting mtu if interface already up (LP: #1850704)
  * d/extra/dhclient-enter-resolved-hook:
    - only restart resolved if dhclient conf changed (LP: #1805183)

 -- Dan Streetman <email address hidden> Fri, 15 Nov 2019 10:01:16 -0500

Changed in systemd (Ubuntu Bionic):
status: Fix Committed → Fix Released
Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Eoan):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.