Ubuntu
systemd package

"Too many levels of symbolic links” error when using systemd to mount sshfs

Bug #1902891 reported by Sami Pietila
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
systemd
Fix Released
Unknown
systemd (Ubuntu)
Fix Released
Undecided
Unassigned
Bionic
Won't Fix
Undecided
Unassigned
Focal
Fix Released
Low
Dan Streetman
Groovy
Fix Released
Low
Dan Streetman

Bug Description

[impact]

sshfs mount configured in fstab fails with 'too many levels of symbolic links'

[test case]

configure a sshfs mount in fstab using a remote system where the local user performing the mount (usually, root) does not have the remote host ssh key in its known hosts db

then attempt to access any files under the mount, and check the journal

see original description for more detail

[regression potential]

any regression would likely cause systemd-mounted filesystem(s) to fail to be correctly mounted

[scope]

this is needed for g and earlier

this is fixed upstream with commit 2fa0bd7d57863dffda89190a70a83c51bd7d114a which is included in v247, so this is fixed in h and later

[original description]

I have asked question at askubuntu about "Too many levels of symbolic links” error when trying to use systemd to mount sshfs (https://askubuntu.com/questions/1286375/too-many-levels-of-symbolic-links-error-when-using-systemd)

I have not got any responses, so I am submitting a bug report. If this is not a bug, please advice how to mount sshfs share with systemd.

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: systemd 245.4-4ubuntu3.2
ProcVersionSignature: Ubuntu 5.4.0-52.57-generic 5.4.65
Uname: Linux 5.4.0-52-generic x86_64
NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
ApportVersion: 2.20.11-0ubuntu27.10
Architecture: amd64
CasperMD5CheckResult: skip
CurrentDesktop: ubuntu:GNOME
Date: Wed Nov 4 16:17:04 2020
InstallationDate: Installed on 2019-11-03 (367 days ago)
InstallationMedia: Ubuntu 19.10 "Eoan Ermine" - Release amd64 (20191017)
MachineType: LENOVO 81H1
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-52-generic root=UUID=7f92666a-65f8-485e-b998-046c2c596aa5 ro rootflags=subvol=@ quiet splash vt.handoff=7
SourcePackage: systemd
UpgradeStatus: Upgraded to focal on 2020-03-28 (220 days ago)
dmi.bios.date: 08/17/2018
dmi.bios.vendor: LENOVO
dmi.bios.version: 8PCN45WW
dmi.board.asset.tag: NO Asset Tag
dmi.board.name: LNVNB161216
dmi.board.vendor: LENOVO
dmi.board.version: SDK0J40700 WIN
dmi.chassis.asset.tag: NO Asset Tag
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Lenovo ideapad 530S-14ARR
dmi.modalias: dmi:bvnLENOVO:bvr8PCN45WW:bd08/17/2018:svnLENOVO:pn81H1:pvrLenovoideapad530S-14ARR:rvnLENOVO:rnLNVNB161216:rvrSDK0J40700WIN:cvnLENOVO:ct10:cvrLenovoideapad530S-14ARR:
dmi.product.family: ideapad 530S-14ARR
dmi.product.name: 81H1
dmi.product.sku: LENOVO_MT_81H1_BU_idea_FM_ideapad 530S-14ARR
dmi.product.version: Lenovo ideapad 530S-14ARR
dmi.sys.vendor: LENOVO

See original description
Revision history for this message
Sami Pietila (sampie) wrote :
Sami Pietila (sampie)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in systemd (Ubuntu):
status: New → Confirmed
Revision history for this message
Dan Streetman (ddstreet) wrote :

I was able to reproduce this.

Revision history for this message
Sami Pietila (sampie) wrote :

This was inspected at systemd github incident-pages. Automount (executed as root) requires the target machine to be in /root/.ssh/known_hosts file. Unfortunately, the failing mount does not seem to give any helpful log or error messages.

Revision history for this message
Dan Streetman (ddstreet) wrote :

@sampie you have a reference to the upstream issue?

Revision history for this message
Sami Pietila (sampie) wrote :

This is the upstream issue:

https://github.com/systemd/systemd/issues/17545

Revision history for this message
Dan Streetman (ddstreet) wrote :

@sampie thanks, I marked that as the upstream bug for this.

For the upstream PR that fixes this:
https://github.com/systemd/systemd/pull/17631

If I'm reading that correctly, that won't actually fix this (the mount will still fail), and it won't add any log messages indicating the actual underlying problem (missing known_hosts ssh key), it will just avoid the repeated mount attempts, right?

I haven't tested the fix myself, just trying to understand if it's "enough" to backport, or if more is needed, either to backport and/or fix upstream. It seems like "fully" fixing this (i.e. adding the remote host key to known_hosts) isn't really something that should be done automatically, since the whole point of asking the user to confirm the host key is because the system can't know if the host key is correct, or a MITM attack. Probably the "best" that could be done, besides existing upstream patch, is to log a message indicating that sshfs failed and possibly include some sshfs detail so users understand what the actual problem is?

Revision history for this message
Sami Pietila (sampie) wrote :

Yes, it would be good to have an informative error message indicating ssh has failed and that the host key needs to be added to /root/.ssh/known_hosts.

Bug Watch Updater (bug-watch-updater)
Changed in systemd:
status: Unknown → Fix Released
Revision history for this message
Dan Streetman (ddstreet) wrote :

marking this as wontfix for bionic, as the fix doesn't actually make anything work, it only prevents the confusing error and fails the mount instead. Without someone specifically asking for this to be fixed in bionic the chance of regression definitely outweighs the benefit.

I'll queue this up for focal (and groovy), but it would also be good to have upstream changed to somehow notify the end user (system admin) of what the actual problem was.

description: updated
Changed in systemd (Ubuntu Groovy):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Focal):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Groovy):
importance: Undecided → Low
Changed in systemd (Ubuntu Focal):
importance: Undecided → Low
Changed in systemd (Ubuntu Groovy):
status: New → In Progress
Changed in systemd (Ubuntu Focal):
status: New → In Progress
Changed in systemd (Ubuntu Bionic):
status: New → Won't Fix
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Sami, or anyone else affected,

Accepted systemd into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/246.6-1ubuntu1.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Groovy):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-groovy
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Sami, or anyone else affected,

Accepted systemd into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/245.4-4ubuntu3.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Dan Streetman (ddstreet) wrote :

ubuntu@lp1902891-g:~$ grep node /etc/fstab
ddstreet@192.168.122.1:/home/ddstreet /mnt/node fuse.sshfs noauto,x-systemd.automount,_netdev,users,idmap=user,IdentityFile=/home/ddstreet/.ssh/id_rsa,allow_other,reconnect 0 0
ubuntu@lp1902891-g:~$ dpkg -l systemd|grep systemd
ii systemd 246.6-1ubuntu1.3 amd64 system and service manager
ubuntu@lp1902891-g:~$ ls -l /mnt/node/
ls: cannot access '/mnt/node/': Too many levels of symbolic links
ubuntu@lp1902891-g:~$ journalctl -b -u mnt-node.mount
-- Logs begin at Fri 2021-06-04 15:01:24 UTC, end at Fri 2021-06-04 17:11:20 UTC. --
Jun 04 17:11:09 lp1902891-g systemd[1]: Mounting /mnt/node...
Jun 04 17:11:09 lp1902891-g mount[968]: read: Connection reset by peer
Jun 04 17:11:09 lp1902891-g systemd[1]: mnt-node.mount: Mount process exited, code=exited, status=1/FAILURE
Jun 04 17:11:09 lp1902891-g systemd[1]: Mounted /mnt/node.

ubuntu@lp1902891-g:~$ grep node /etc/fstab
ddstreet@192.168.122.1:/home/ddstreet /mnt/node fuse.sshfs noauto,x-systemd.automount,_netdev,users,idmap=user,IdentityFile=/home/ddstreet/.ssh/id_rsa,allow_other,reconnect 0 0
ubuntu@lp1902891-g:~$ dpkg -l systemd|grep systemd
ii systemd 246.6-1ubuntu1.4 amd64 system and service manager
ubuntu@lp1902891-g:~$ ls -l /mnt/node/
ls: cannot access '/mnt/node/': No such device
ubuntu@lp1902891-g:~$ journalctl -b -u mnt-node.mount
-- Logs begin at Fri 2021-06-04 15:01:24 UTC, end at Fri 2021-06-04 17:13:16 UTC. --
Jun 04 17:13:04 lp1902891-g systemd[1]: Mounting /mnt/node...
Jun 04 17:13:04 lp1902891-g mount[970]: read: Connection reset by peer
Jun 04 17:13:04 lp1902891-g systemd[1]: mnt-node.mount: Mount process exited, code=exited, status=1/FAILURE
Jun 04 17:13:04 lp1902891-g systemd[1]: mnt-node.mount: Failed with result 'exit-code'.
Jun 04 17:13:04 lp1902891-g systemd[1]: Failed to mount /mnt/node.
Jun 04 17:13:16 lp1902891-g systemd[1]: Mounting /mnt/node...
Jun 04 17:13:16 lp1902891-g mount[983]: read: Connection reset by peer
Jun 04 17:13:16 lp1902891-g systemd[1]: mnt-node.mount: Mount process exited, code=exited, status=1/FAILURE
Jun 04 17:13:16 lp1902891-g systemd[1]: mnt-node.mount: Failed with result 'exit-code'.
Jun 04 17:13:16 lp1902891-g systemd[1]: Failed to mount /mnt/node.

tags: added: verification-done-groovy
removed: verification-needed-groovy
Revision history for this message
Dan Streetman (ddstreet) wrote :

ubuntu@lp1902891-f:~$ grep node /etc/fstab
ddstreet@192.168.122.1:/home/ddstreet /mnt/node fuse.sshfs noauto,x-systemd.automount,_netdev,users,idmap=user,IdentityFile=/home/ddstreet/.ssh/id_rsa,allow_other,reconnect 0 0
ubuntu@lp1902891-f:~$ dpkg -l systemd|grep systemd
ii systemd 245.4-4ubuntu3.6 amd64 system and service manager
ubuntu@lp1902891-f:~$ ls -l /mnt/node
ls: cannot open directory '/mnt/node': Too many levels of symbolic links
ubuntu@lp1902891-f:~$ journalctl -b -u mnt-node.mount
-- Logs begin at Fri 2021-06-04 15:01:27 UTC, end at Fri 2021-06-04 17:24:29 UTC. --
Jun 04 17:24:07 lp1902891-f systemd[1]: Mounting /mnt/node...
Jun 04 17:24:07 lp1902891-f mount[1302]: read: Connection reset by peer
Jun 04 17:24:07 lp1902891-f systemd[1]: mnt-node.mount: Mount process exited, code=exited, status=1/FAILURE
Jun 04 17:24:07 lp1902891-f systemd[1]: Mounted /mnt/node.

ubuntu@lp1902891-f:~$ grep node /etc/fstab
ddstreet@192.168.122.1:/home/ddstreet /mnt/node fuse.sshfs noauto,x-systemd.automount,_netdev,users,idmap=user,IdentityFile=/home/ddstreet/.ssh/id_rsa,allow_other,reconnect 0 0
ubuntu@lp1902891-f:~$ dpkg -l systemd|grep systemd
ii systemd 245.4-4ubuntu3.7 amd64 system and service manager
ubuntu@lp1902891-f:~$ ls -l /mnt/node
ls: cannot open directory '/mnt/node': No such device
ubuntu@lp1902891-f:~$ journalctl -b -u mnt-node.mount
-- Logs begin at Fri 2021-06-04 15:01:27 UTC, end at Fri 2021-06-04 17:30:52 UTC. --
Jun 04 17:30:52 lp1902891-f systemd[1]: Mounting /mnt/node...
Jun 04 17:30:52 lp1902891-f mount[1023]: read: Connection reset by peer
Jun 04 17:30:52 lp1902891-f systemd[1]: mnt-node.mount: Mount process exited, code=exited, status=1/FAILURE
Jun 04 17:30:52 lp1902891-f systemd[1]: mnt-node.mount: Failed with result 'exit-code'.
Jun 04 17:30:52 lp1902891-f systemd[1]: Failed to mount /mnt/node.

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/246.6-1ubuntu1.4)

All autopkgtests for the newly accepted systemd (246.6-1ubuntu1.4) for groovy have finished running.
The following regressions have been reported in tests triggered by the package:

casync/2+20190213-1 (s390x)
udisks2/2.9.1-2ubuntu1 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/groovy/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 246.6-1ubuntu1.4

---------------
systemd (246.6-1ubuntu1.4) groovy; urgency=medium

  [ Andy Chi ]
  * debian/patches/lp1926547-hwdb-60-keyboard-Update-Dell-Privacy-Local-Mic-Mute-.patch
    - Apply upstream patch to correct key and device mapping.
      (LP: #1926547)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=a4c14d1c8370445e315ffa59b0a7ce593a79fbf4

  [ Dan Streetman ]
  * d/p/lp1921696/0001-rfkill-improve-error-logging.patch,
    d/p/lp1921696/0002-rfkill-use-short-writes-and-accept-long-reads.patch:
    Handle rfkill api change in kernel 5.10 (LP: #1921696)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=b683c842c74b5f193555fd79bc76e574a025f5b1
  * d/p/lp1929560-network-move-set-MAC-and-set-nomaster-operations-out.patch:
    Move link mac and master config out of link_up() (LP: #1929560)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=a90963d52a70c0ed1b429b1025b95f8c0fa6e7aa
  * d/p/lp1902891-core-mount-mount-command-may-fail-after-adding-the-c.patch:
    Handle failed mount command (LP: #1902891)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=d83f3bce38e04395c6406c3c45efbc9977ae1138
  * d/p/debian/UBUNTU-resolved-Mitigate-DVE-2018-0001-by-retrying-NXDOMAIN-with.patch,
    d/p/lp1880258-log-nxdomain-as-debug.patch,
    d/p/lp1785383-resolved-address-DVE-2018-0001.patch:
    - Use upstream patch for DVE-2018-0001 handling (LP: #1785383)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=eb311ad89c5fd570bf3af362d8a2af1b357be3dd

  [ Łukasz 'sil2100' Zemczak ]
  * d/p/lp1664844/0001-network-add-ActivationPolicy-configuration-parameter.patch,
    d/p/lp1664844/0002-test-add-ActivationPolicy-unit-tests.patch,
    d/p/lp1664844/0003-save-link-activation-policy-to-state-file-and-displa.patch:
    - add support for configuring the activation policy for an interface
      (LP: #1664844)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=0b0a9202964b24fb8d9fb4b28bdb22c6aadd25b7

 -- Dan Streetman <email address hidden> Thu, 27 May 2021 11:13:07 -0400

Changed in systemd (Ubuntu Groovy):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 245.4-4ubuntu3.7

---------------
systemd (245.4-4ubuntu3.7) focal; urgency=medium

  [ Andy Chi ]
  * debian/patches/lp1926547-hwdb-60-keyboard-Update-Dell-Privacy-Local-Mic-Mute-.patch
    - Apply upstream patch to correct key and device mapping.
      (LP: #1926547)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=62c3ce6d6b2cab762b24aa610d6d135a67bdd76a

  [ Dan Streetman ]
  * d/p/lp1921696/0001-rfkill-improve-error-logging.patch,
    d/p/lp1921696/0002-rfkill-use-short-writes-and-accept-long-reads.patch:
    Handle rfkill api change in kernel 5.10 (LP: #1921696)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ff0c23ba4fbcfa7f68e98adb6d62798ce54ca1da
  * d/p/lp1929122-network-check-that-received-ifindex-is-valid.patch:
    Check if ifindex is valid (LP: #1929122)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=6378191818bc7d169b657e6f7a2b50cfddb4275e
  * d/p/lp1929560-network-move-set-MAC-and-set-nomaster-operations-out.patch:
    Move link mac and master config out of link_up() (LP: #1929560)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=28cff7ee02a9ebd4ab93026af9fceaa2283725b3
  * d/p/lp1902891-core-mount-mount-command-may-fail-after-adding-the-c.patch:
    Handle failed mount command (LP: #1902891)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=b425189a483d7455db870b0ec5b2443c0eea7d76
  * d/p/resolved-Mitigate-DVE-2018-0001-by-retrying-NXDOMAIN-with.patch,
    d/p/lp1880258-log-nxdomain-as-debug.patch,
    d/p/lp1785383-resolved-address-DVE-2018-0001.patch:
    - Use upstream patch for DVE-2018-0001 handling (LP: #1785383)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ec45ebfee362ad3e429642f7519e8b88f16dc221

  [ Łukasz 'sil2100' Zemczak ]
  * d/p/lp1664844/0001-network-add-ActivationPolicy-configuration-parameter.patch,
    d/p/lp1664844/0002-test-add-ActivationPolicy-unit-tests.patch,
    d/p/lp1664844/0003-save-link-activation-policy-to-state-file-and-displa.patch:
    - add support for configuring the activation policy for an interface
      (LP: #1664844)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=94f7b72d8128c743f35b308101a87d2c53a4074c

 -- Dan Streetman <email address hidden> Thu, 27 May 2021 11:16:17 -0400

Changed in systemd (Ubuntu Focal):
status: Fix Committed → Fix Released
Dan Streetman (ddstreet)
Changed in systemd (Ubuntu):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.