extra domains not removed from resolv.conf when VPN disconnects

Could you paste the output from
$ systemd-resolved --status
when the system is in a bad state? That is, system mentions the domains in /etc/resolv.conf when it should not have.

 $ systemd-resolve webvpn.purestorage.com
webvpn.purestorage.com: resolve call failed: No appropriate name servers or networks for name found
 $ systemd-resolve --status
Global
          DNS Domain: home.digitalvampire.org
                      purestorage.com
                      dev.purestorage.com
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 3 (wlp4s0)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 2001:470:1f05:221::1
                      10.1.0.1
          DNS Domain: home.digitalvampire.org

Link 2 (enp0s31f6)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

I wonder if the issue has anything to do with the fact that the VPN creates a new network link that disappears when the VPN goes down - note that the purestorage.com domains are listed for tun0 when the VPN is up. When I turn off the VPN, tun0 disappears but the purestorage.com domains stay in the Global part of the status output:

 $ systemd-resolve --status
Global
          DNS Domain: home.digitalvampire.org
                      purestorage.com
                      dev.purestorage.com
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 17 (tun0)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 10.231.255.252
                      10.230.255.252
          DNS Domain: purestorage.com\032dev.purestorage.com
                      purestorage.com
                      dev.purestorage.com

Link 3 (wlp4s0)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 2001:470:1f05:221::1
                      10.1.0.1
          DNS Domain: home.digitalvampire.org

Link 2 (enp0s31f6)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

I believe I have a fix for this. There is a feedback loop which was not closed, with resolved re-reading /etc/resolv.conf and thus DNS domains leaking from per-link scope, into the global scope, and staying there, even when the link is dead / gone.

Great, let me know if you want me to try anything out.

I'm experiencing this same issue.

A workaround that works for me is to remove my vpn domain from /etc/resolv.conf. The last line of this file looks as follows after a successful VPN connection:

search Home myvpndomain

where the last term is the domain of my vpn (vpn.bla.com). The workaround is to remove it and leave:
search Home
then restart the network-manager service and the VPN reconnects.

Otherwise I have to restart the computer to be able to connect to the VPN after a successful connection. I mean: after a computer restart, VPN always connects, but if I disconnect I cannot reconnect to the VPN unless I restart the computer or do the workaround.

I have a big confusion on how /etc/resolv.conf is managed nowadays in Ubuntu. I have at least three different system processes generating a resolv.conf.

systemd-resolved:
/run/systemd/resolve/resolv.conf

network-manager:
/run/NetworkManager/resolv.conf

and resolvconf:
/run/resolvconf/resolv.conf

My actual /etc/resolv.conf is a symbolic link to this last one. At some point I tried to have only network-manager manage /etc/resolv.conf. Everything works fine, but when I connect to VPN it can't resolve any DNS outside of the private network.

The resolv.conf generated by systemd-resolved has the same problem as the one generated by resolvconf, when the VPN reconnects the VPN domain is not removed from the "search" line.

This bug was fixed in the package systemd - 234-2ubuntu12

---------------
systemd (234-2ubuntu12) artful; urgency=medium

  [ Dimitri John Ledkov ]
  * debian/rules: do not strip test-copy.
    This insures test-copy is large enough for test-copy tests to pass.
    (LP: #1721203)

  [ Michael Biebl ]
  * Drop systemd-timesyncd.service.d/disable-with-time-daemon.conf.
    All major NTP implementations ship a native service file nowadays with a
    Conflicts=systemd-timesyncd.service so this drop-in is no longer
    necessary. (Closes: #873185) (LP: #1721204)

 -- Dimitri John Ledkov <email address hidden> Wed, 04 Oct 2017 13:28:34 +0100

I still have this problem. When I disconnect from a VPN, the various resolv files retain a search line and DNS lookup fails for the domain associated with the VPN (in this case, my university). Luckily, this is fairly easy to work around (I no longer reboot to try to fix my network). Did I miss something? Or has the fix been committed, but not yet propagated out into a new systemd package in the repos? The post above seems to indicate that the version 234-2ubuntu12 and above would have the fix. I have 234-2ubuntu12.2 which I presume is newer.

here's output from dpkg-query -s systemd:

fanslow@sano:~$ dpkg-query -s systemd
Package: systemd
Status: install ok installed
Priority: important
Section: admin
Installed-Size: 11704
Maintainer: Ubuntu Developers <email address hidden>
Architecture: amd64
Multi-Arch: foreign
Version: 234-2ubuntu12.2
Replaces: udev (<< 228-5)
Depends: libacl1 (>= 2.2.51-8), libapparmor1 (>= 2.9.0-3+exp2), libaudit1 (>= 1:2.2.1), libblkid1 (>= 2.19.1), libc6 (>= 2.25), libcap2 (>= 1:2.10), libcryptsetup4 (>= 2:1.4.3), libgcrypt20 (>= 1.7.0), libgpg-error0 (>= 1.14), libidn11 (>= 1.13), libip4tc0 (>= 1.6.0+snapshot20161117), libkmod2 (>= 5~), liblz4-1 (>= 0.0~r130), liblzma5 (>= 5.1.1alpha+20120614), libmount1 (>= 2.26.2), libpam0g (>= 0.99.7.1), libseccomp2 (>= 2.3.1), libselinux1 (>= 2.1.9), libsystemd0 (= 234-2ubuntu12.2), util-linux (>= 2.27.1), mount (>= 2.26), adduser, procps
Pre-Depends: libc6 (>= 2.8)
Recommends: libpam-systemd, dbus
Suggests: systemd-container, policykit-1
Breaks: apparmor (<< 2.9.2-1), ifupdown (<< 0.8.5~), laptop-mode-tools (<< 1.68~), systemd-shim (<< 10-3~), udev (<< 228-5)
Conffiles:
 /etc/dhcp/dhclient-enter-hooks.d/resolved 17323c120a8bb9f8453c24b43d900203
 /etc/dhcp/dhclient-exit-hooks.d/timesyncd a891f21f45b0648b7082d999bf424591
 /etc/pam.d/systemd-user 3d97692a0125712fcfbd7ddf756f7696
 /etc/systemd/journald.conf b89a22c219efce8b3700feafc61e0c18
 /etc/systemd/logind.conf aeeac805b4bfb7450183ce346999dd38
 /etc/systemd/resolved.conf cda783b1a7bc2e01f2268b9df3c71934
 /etc/systemd/system.conf adc308320471c75a60e07184cf2ebb05
 /etc/systemd/timesyncd.conf 4e1cf2b5f85dff6c1f186135b9f66a85
 /etc/systemd/user.conf 675370e2d80a4ad957202e68c1b4aaee
Description: system and service manager
 systemd is a system and service manager for Linux. It provides aggressive
 parallelization capabilities, uses socket and D-Bus activation for starting
 services, offers on-demand starting of daemons, keeps track of processes using
 Linux control groups, maintains mount and automount points and implements an
 elaborate transactional dependency-based service control logic.
 .
 systemd is compatible with SysV and LSB init scripts and can work as a
 drop-in replacement for sysvinit.
 .
 Installing the systemd package will not switch your init system unless you
 boot with init=/bin/systemd or install systemd-sysv in addition.
Homepage: https://www.freedesktop.org/wiki/Software/systemd
Original-Maintainer: Debian systemd Maintainers <email address hidden>

Same for me. I really don't understand how /etc/resolv.conf should be managed in Ubuntu 17.04. I have the setup described in comment #7. I tried to have only network-manager manage /etc/resolv.conf, but then I loose name resolution in containers.

I think we should open a new bug report, as this one marked with "Fix released" will never get attention. It may also something worth investigating upstream.

It seems this systemd bug is related and is supposed to be fixed. I still experience the problem however:
https://github.com/systemd/systemd/issues/7305

The fix is incorporated in 236, but the systemd developer suggests the fix could be backported to 234. What version of systemd will be in bionic?

I have had the same exact problem since I have upgraded to 17.10. In my case it does not even involve the VPN. When I come back from work, if I had suspended the laptop without restarting it, then I cannot connect to the computers from work.

If I restart resolvconf with:
$ sudo /etc/init.d/resolvconf restart
The problem persists.

If I edit the file /run/resolvconf/interface/systemd-resolved with
$ sudo emacs -nw /run/resolvconf/interface/systemd-resolved
... remove the work computer name in the line starting with search ...
$ sudo /etc/init.d/resolvconf restart
Then it works fine.

I hope this bug is well understood and the fix is on its way. It seems like a bit of a corner case, but it is indeed a serious bug.

Still have it in Ubuntu 19.10.

I still see this in 20.04