nfs shares not mounted at startup

Bug #45842 reported by Marc Schiffbauer
66
This bug affects 3 people
Affects Status Importance Assigned to Milestone
sysvinit (Ubuntu)
Fix Released
Medium
Ubuntu Server
Declined for Feisty by Henrik Nilsen Omma
Declined for Gutsy by Henrik Nilsen Omma

Bug Description

One update from the last few days must have caused, that nfs shares do not get mounted at system startup anymore.

I have some nfs shares in my /etc/fstab
which used to work all the time until three days ago or so.

But a "mount -a" as root after startup does the job. So I guess there has been changed something in the initscript order or something that causes this.

These are the nfs entriees from my /etc/fstab:
# lisa nfs
lisa:/ /home/lisa nfs rw,addr=192.168.1.1 0 0
lisa:/home /home/lisa/home nfs rw,addr=192.168.1.1 0 0
lisa:/home2 /home/lisa/home2 nfs rw,addr=192.168.1.1 0 0
lisa:/var /home/lisa/var nfs rw,addr=192.168.1.1 0 0
lisa:/usr /home/lisa/usr nfs rw,addr=192.168.1.1 0 0

Cheers
-Marc

Related branches

Revision history for this message
Laurent Bigonville (bigon) wrote :

I think this bug is a duplicate of bug #44836.

Could you have a look at /proc/mount to see if your nfs share are actually not mounted

Revision history for this message
Marc Schiffbauer (mschiff) wrote :

OK. And no, there is really nothing mounted after system startup. And yes, these not-mounted NFS shares are NOT going to be mounted onto the root-fs.

Maybe its really a duplicate of #44836.

BTW: The tmpfs seem to be double- and tripple-mounted:

mschiff@homer:~$ cat /proc/mounts
rootfs / rootfs rw 0 0
none /sys sysfs rw 0 0
none /proc proc rw,nodiratime 0 0
udev /dev tmpfs rw 0 0
/dev/hda8 / xfs rw 0 0
/dev/hda8 /dev/.static/dev xfs rw 0 0
tmpfs /var/run tmpfs rw 0 0
tmpfs /var/lock tmpfs rw 0 0
usbfs /proc/bus/usb usbfs rw 0 0
tmpfs /lib/modules/2.6.15-23-k7/volatile tmpfs rw 0 0
devpts /dev/pts devpts rw 0 0
tmpfs /dev/shm tmpfs rw 0 0
tmpfs /var/run tmpfs rw 0 0
tmpfs /var/lock tmpfs rw 0 0
tmpfs /var/run tmpfs rw 0 0
tmpfs /var/lock tmpfs rw 0 0
/dev/hda7 /boot ext3 rw,data=ordered 0 0
/dev/hda11 /home xfs rw 0 0
/dev/hda9 /usr xfs rw 0 0
/dev/hda10 /var xfs rw 0 0
/dev/hda10 /tmp xfs rw 0 0
mschiff@homer:~$

-Marc

Revision history for this message
david (david-kabelverhau) wrote :

I have similar issues:
My NFS shares are not mounted on startup. This is totally deterministic (always the case), so I don't think that this is the same problem as #44836 or #46516. They are really not mounted (it's not the mtab issue). I can also relieably mount them using mount -a as root.

I have investigated a bit further in the problem: nfs shares are mounted by the /etc/init.d/networking script, which starts ifup, which runs /etc/network/if-up.d/mountnfs. Now in my case, when /etc/init.d/networking runs, eth0 is already configured (probably by some plug'n'play script and DHCP), so ifup does not start any interfaces, thus does not execute the if-up.d scripts, thus nfs is not mounted. I did no modifications to the init scripts so far. I found that out by inserting "ifconfig" at the beginning of the /etc/init.d/networking script, which showed me that my interfaces were already up.

Also, if I do /etc/init.d/networking stop; /etc/init.d/networking start manually after booting up, then all my nfs shares get mounted, which shows that the script itself is okay, but it makes wrong assumptions about the state of the system before it runs.

My system is Dapper Flight 7 AMD64 with the latest updates, so it should be the same as the release candidate, last update was made today (26. may 2006).

Revision history for this message
Marc Schiffbauer (mschiff) wrote :

Hi,

this bug is still there in edgy. Would be nice if this was fixed in edgy release because it is really annoying and IMO must not happen in such a good Distro as ubuntu ;)

Do you need further Information?

-Marc

Revision history for this message
Matthew Caron (matt-mattcaron) wrote :

I had the same problem intermittently in dapper, and now it appears to always be the case in Edgy. Neither server nor client logfiles report anything useful.

Revision history for this message
drew einhorn (drew-einhorn) wrote :

Don't know if I'm seeing this exact bug or a similar one.

I'm just getting a small lan set up and so far I'm only
seeing this problem on my fastest box a 3.4 GHz P4
with 1GB ram.

sudo mount -a

does solve the problem till the next reboot.

Revision history for this message
Marc Schiffbauer (mschiff) wrote :

This bug is still present in feisty and its been there for two releases now...

Ubuntu is still not able to mount NFS-Shares at startup.

I would consider this a major bug because this really could damage ubuntu's reputation IMO... :-(

-Marc

Revision history for this message
Matthew Caron (matt-mattcaron) wrote :

Oh, it's worse than that - NFS has been broken in Edgy for many people since release (worked fine in Beta 2, if I remember correctly). I would expect that to be worse to Ubuntu's reputation than other things. I know I'm not too happy about it. See Bug #62308 for more information on this.

Revision history for this message
Wido den Hollander (wido) wrote :

Hi, i also experience this bug on around 35 Ubuntu Dapper servers.

This is a really serious bug since this causes a lot of troubles on our mailclusters.

Revision history for this message
Marc Schiffbauer (mschiff) wrote :

My workaround until this nasty bug will be fixed is this:

mschiff@homer:~$ cat /etc/cron.d/mount_all_workaround
@reboot root mount -a
mschiff@homer:~$

This lets cron do a "mount -a" one time after the machine has booted.
Works for me but for servers I would create another initscript that will do it at the right time.

-Marc

Revision history for this message
Wido den Hollander (wido) wrote :

But i think your too late right, since cron has 89 as priority to start after a boot.

Revision history for this message
Matthew Caron (matt-mattcaron) wrote :

I believe that the problem is related to the mount command in /etc/init.d/mountall.sh. Specifically, the command:

mount -a -v -t noproc,nfs,nfs4,smbfs,cifs,ncp,ncpfs,coda,ocfs2,gfs

doesn't actually report trying to mount the nfs4 filesystems. An alternative construction:

mount -a -v -t nfs,nfs4,smbfs,cifs,ncp,ncpfs,coda,ocfs2,gfs,noproc

only tries to mount nfs filesystems - it does not try to mount local filesystems.

Changing it to:

mount -a -v

or

mount -a

for the less verbose version seems to fix the issue.

Now, this is further complicated by the fact that it only seems to run when you are going into single user mode. However, this will add it to runlevel 2 (the default runlevel)

sudo update-rc.d -f mountall.sh remove
sudo update-rc.d mountall.sh start 35 2 S .

Revision history for this message
gizero (gizero) wrote :

I investigated a little bit more around what david said on 2006-05-26 and found that /etc/network/if-up.d/mountnfs is not executed at boot because eth0 is already configured by the udev scripts. I checked this by removing my /etc/udev/rules.d/85-ifupdown.rules (causing the interface to be brought up before /etc/init.d/networking execution). This works for me and seems to have no side effects, but I have no idea if/how this could affect systems with hotpluggable network interfaces. Maybe it is a matter of fixing the udev rule or, if udev is really supposed to be run before networking scripts, to allow these to be aware of a possibly already configured iface.

If I have some more time I'll check how this is arranged in other distros

Revision history for this message
Marc Schiffbauer (mschiff) wrote :

This nasty bug is still present in gutsy... ubuntu is still not able to mount NFS shares automatically

Revision history for this message
Hassan El Jacifi (waver) wrote :

I confirm that this bug is still present in gutsy server.

Revision history for this message
Wido den Hollander (wido) wrote :

I also confirm this.

We had a power outage a few weeks ago..... We have 72 Ubuntu Dapper servers who rely on NFS for their operations. After those 72 servers rebooted, i had to log on manually to type "mount -a" on 72 servers!

This caused a lot of downtime which could have been prevented if Ubuntu mounts NFS correctly on boot.

I say this bug has a pretty high priority and should/could be fixed very soon!

Hassan El Jacifi (waver)
Changed in sysvinit:
status: New → Confirmed
Revision history for this message
Paul Filo (paul-filo) wrote :

I had the same problem and a lot of troubles with network interfaces names with Gutsy. Between boot from initrd and real boot, eth devices changed their names and, for me, that's why nfs doesn't mount.
My workaround :
edit /etc/udev/rules.d/70-persistent-net.rules, delete or comment all rules and write rules with driver (not with MAC) and with names found on first boot, for example :

# dmesg | grep eth
[ 20.491562] eth0: SiS 900 PCI Fast Ethernet at 0xdc00, IRQ 12, 00:0a:e6:41:1d:b6.
[ 21.252607] eth1: RTL8169sb/8110sb at 0xe0814f00, 00:1b:11:59:b5:91, XID 10000000 IRQ 3

/etc/udev/rules.d/70-persistent-net.rules :
SUBSYSTEM=="net", DRIVERS=="sis900", NAME="eth0"
SUBSYSTEM=="net", DRIVERS=="r8169", NAME="eth1"

Now my NFS are mounted at startup.
Hope this help you.

Revision history for this message
chenel (jeremy-wolcott) wrote :

One more for the same problem under Gutsy. I didn't have Paul Filo's problem mentioned on 2007-10-29, but gizero's workaround from 2007-08-22 did fix it. Nevertheless I don't think we should have to muck around with udev just to get NFS to mount on startup...

Revision history for this message
Stefan Grothkopp (lp-sg) wrote :

Gutsy desktop suffers the same problem.
My workaround is to link to the nfsmount script in rcS.d:

cd /etc/rcS.d
ln -s ../network/if-up.d/mountnfs S41mountnfs

Revision history for this message
Andreas Sandberg (andysan) wrote :

This bug is causing me quite a lot of headache as well, this is a problem with all recent Ubuntu based installations I use. Both at work, at the computer club, and it will be on my home computer (as soon as|if) I upgrade to Gutsy. This bug really needs attention!

Revision history for this message
chris2000 (christopher-fanning) wrote :

Hi,

Stefan's solution worked for me on Gutsy
cd /etc/rcS.d
ln -s ../network/if-up.d/mountnfs S41mountnfs
Thanks.

This is the first ubuntu installation I've done and this bug has taken me by suprise.
you know, mounting nfs from fstab is an old classic in the linux world.
Then again, I am doing this with the desktop edition. Perhaps I should be using the server edition?
By the way, what's the difference between the two?

Changed in sysvinit:
assignee: nobody → ubuntu-server
Revision history for this message
Martin Emrich (emme) wrote :

Around two weeks ago, this problem appeared on my desktop (gutsy i386) out of the blue. Only one of my three NFS mounts were mounted.

At first, I typed "mount -a -t nfs" manually after boot, then I added this line to /etc/rc.local. But this had no effect. Then I added "sleep 10" before the mount command in my rc.local, now it works. So maybe it has something to do with the way upstart starts services, network etc. in parallel?

Ciao

Martin

Revision history for this message
Hermann Lacheiner (fidohl) wrote :

I experienced the same problem, not mounting nfs shares at boot time on hardy (diskless client).
I solved the problem adding "auto eth0" to /etc/network/interfaces. Actually eth0 is already configured by the kernel when booting from the net, but the interface is not brought up "officially" when the keyword auto is missing in /etc/network/interfaces, according to /etc/udev/rules.d/85-ifupdown.rules.

Revision history for this message
David Portwood (dzportwood) wrote :

This seems to be fixed in Feisty and Gusty, as both have the "nfs4" bit in '/etc/init.d/mountall.sh', both work fine for me.
Thanks,
David P.

Revision history for this message
David Portwood (dzportwood) wrote :

I should also mention, I could not reproduce the bug on Gutsy, Feisty or Hardy server install (32bit). Anyone else still experiencing this, are you on a server install?
Thanks,
David P.

Revision history for this message
Marc Schiffbauer (mschiff) wrote :

Hi David,

no I experienced this on Desktop installs. But isnt this the same repo anyway?
So I think I does not matter whether you have server, desktop (ubuntu/kunbuntu) ...

-marc

Revision history for this message
David Portwood (dzportwood) wrote :

No, the server installs do not use network-manager, the desktop installs do, this could be the root of the problem.

Revision history for this message
Laurent Bigonville (bigon) wrote :

I think this was happening before n-m was installed by default on desktop machine.

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

The attached debdiff solves this problem, as I've reproduced it on both Gutsy and Hardy. I've tested this fix successfully on both Gutsy and Hardy.

Additionally, this debdiff solves the related Bug #44836, where NFS mounts show up in /proc/mounts, but not in mtab.

Note that there are a number of "NFS shares not mounted at bootup" bugs that should, perhaps, be re-tested with this fix.

Revision history for this message
Kees Cook (kees) wrote :

Excellent work. I think this is very close to a solution. I think some more testing and some additional tweaking is needed. My notes after looking over all the scripts' interactions:

- if-up.d/mountnfs should not run unless the required rc scripts are finished (i.e. finished with rcS.d)
    - note the existing hack to start portmap/nfs-common
    - other infrastructure may not be ready (mtab)
- init.d/waitnfs.sh could then run after all the interfaces came up and already aborted their (too early) attempt to mount, so it will need to initiate a mount-all-netfs too.
- races from all directions (udev, manual ifup, waitnfs) is (and will be) a problem since the current locking isn't good enough (mkdir "lock" isn't a sane lock for tight races)
    - perhaps use /lib/udev/watershed to run a new nfs-mounting-helper script from both if-up.d/mountnfs and init.d/waitnfs.sh ?

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

I'm attaching an updated patch against sysvinit.

I believe this patch to fix:
Bug 44836
Bug 45842
Bug 46145
Bug 46516

All of these are NFS mounting bugs. Two of them have to do with mounts succeeding, but not ending up in mtab. The other two do not actually mount NFS shares during boot up, but will mount at some point shortly after.

This patch does a couple of things. It fixes up the documentation inline in waitnfs.sh, which was a bit misleading about its effect (it only actually waits on /usr and /var nfs mounts). It also fixes the mtab race condition that kept mtab from being updated sometimes. Finally, it establishes a common script that mounts all networked shares (nfs, samba, etc). This script is called using watershed both in the ifup scripts, as well as in waitnfs.sh. This should cover any other bases, where the mount may not have been issued (or not issued late enough), while handling the various race conditions via watershed.

:-Dustin

Revision history for this message
Wido den Hollander (wido) wrote :

Great! Any idea if it will be fixed before hardy?

Revision history for this message
Jorge Allyson Azevedo (allyson-land) wrote :

It would be great to have an official bugfix update for Gusty. Is that possible any time soon?

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Updated patch, added udev as a dependency package, due to newly introduced dependency on watershed.

This should be in Hardy shortly. Gutsy would require an SRU, if anyone is interested in driving that process...

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sysvinit - 2.86.ds1-14.1ubuntu37

---------------
sysvinit (2.86.ds1-14.1ubuntu37) hardy; urgency=low

  * debian/initscripts/etc/init.d/mtab.sh:
    Explicitly add mtab entries for networked file systems; race condition
    exists where mtab can get overwritten *after* networked file systems are
    mounted (LP: #46145, #44836).
  * debian/initscripts/etc/init.d/waitnfs.sh, debian/control:
    Update inline documentation, as waitnfs ONLY waits on /usr and /var mounted
    filesystems.
    Before waiting for net file systems, try mounting them (with watershed)
    just in case the mount has not been attempted elsewhere.
    Added udev as dependency due to use of watershed.
  * debian/initscripts/etc/network/if-up.d/mountnfs:
    Remove the bulk of this script to a common library script
    Call the common library script with watershed for locking purposes
    (LP: #45842, #46516).
  * debian/initscripts/lib/init/mountall-net-fs, debian/rules:
    Common library script for mounting all network file systems (nfs, cifs,
    samba, etc).
    Script can be called in various meaningful locations; should be wrapped
    with watershed to correctly handle race conditions.

 -- Dustin Kirkland <email address hidden> Tue, 26 Feb 2008 16:10:56 -0600

Changed in sysvinit:
status: Confirmed → Fix Released
Revision history for this message
toer (tore-ericsson) wrote :

Just a question since I have never done any [start-script] patching before: Should I run the command

       patch -p0 < sysvinit_2.86.ds1-14.1ubuntu37.debdiff

(or -p or -p1?) in a root konsole, and all will be fixed?

This concerns Kubuntu 7.10 Gutsy laptops suffering from not having nfs wlan-disks automounted at startup (the network nicely autoconnects through Wicd, super!).

Thanks on beforehand / Tore

Revision history for this message
T. Ribbrock (emgaron) wrote :

I suprised that his fix only seems to affect "sysvinit" - I just ran into this problem on Gutsy once again (very annoying) and noticed that sysvinit is not installed on that box. This is an out-of-box 7.10 install with updates. "sysvinit" is in the list of packages that can be installed, however, if I try to install it, several other packages will be uninstalled:

- apparmor
- apparmor-utils
- startup-tasks
- system-services
- ubuntu-minimal
- upstart
- upstart-compat-sysv
- upstart-logd

If Hardy works the same, I wonder whether the fix in sysvinit is sufficient if it isn't used. What am I missing here?

Revision history for this message
Matthew Caron (matt-mattcaron) wrote :

I think upstart-compat-sysv provides the same sysv functionality is sysvinit, which is why a bunch of stuff is going to be removed.

Perhaps a similar patch needs to be written for upstart?

Revision history for this message
Scott James Remnant (Canonical) (canonical-scott) wrote :

sysvinit is a source package that contains, as well as sysvinit itself, the sysv-rc script runner and initscripts that we still do use.

Revision history for this message
toer (tore-ericsson) wrote :

Is this bug possibly related to <a href="https://bugs.launchpad.net/ubuntu/+bug/213444">Bug #213444</a>, recently reported in Hardy Heron, where auto mount now works, but where the 'mount -a' command is extremely slow and issues unexpected messages?

Revision history for this message
Dustin Kirkland  (kirkland) wrote : Re: [Bug 45842] Re: nfs shares not mounted at startup

Hi-

I took a look at Bug #213444 and it does not appear to be related, in
my opinion.

Bug #45842 (this bug) was caused by a race condition, which
specifically only occured on boot.

The slowness of mount.nfs you speak of in Bug #213444 usually points
to an rpc/portmap or networking problem, in my experience.

:-Dustin

Revision history for this message
Alessandro Morgantini (gpz500) wrote :

Hi all!
I've just noticed that, if in the /etc/fstab file the server is inserted in the form 'servername.local' (mDNS name), the NFS volume isn't mounted at boot time. If, after the boot, I do "sudo mount -a" is ok: obviously the avahi-daemon is not ready yet at boot time.

But the weird thing is that if I configure my network interface eth0 in Roaming mode, so is NetworkManager that manages it, everything is OK at the boot time also, as expected!

I'm in the situation that I can't use NetworkManager, because I must set up some special routing rules and, simultaneously, I would like to use the .local name, because in my LAN there isn't any DNS server and, also, I can't use fixed IP addresses for the PCs.

Someone has any idea?

Revision history for this message
Ashwin (cybernytrix) wrote :

I am running hardy-server x64 and this really sux.
Someone please explain why this is so:

sudo mount -a -t noproc,nfs,nfs4,smbfs,cifs,ncp,ncpfs,coda,ocfs2,gfs # this does not work

sudo mount -a -t nfs,nfs4,smbfs,cifs,ncp,ncpfs,coda,ocfs2,gfs,noproc # This WORKS!!!

What kinda hacky thing is this?

Can you patch /etc/init.d/mountall to have this new order?

Thanks!

Revision history for this message
Mateusz Marzantowicz (mmarzantowicz) wrote :

Hello,

as you can guess I have the problem with mounting NFS at boot time.

My /etc/lsb-release is:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=9.04
DISTRIB_CODENAME=jaunty
DISTRIB_DESCRIPTION="Ubuntu 9.04"

I've installed this system as VMware virtual machine (F4 during install CD boot, etc.).

This bug is really annoying. I've added mount -a to my /etc/rc.local which temporarily solved the problem but it's ugly hack!

Is there any solution for this bug?

Revision history for this message
Thierry Carrez (ttx) wrote :

@Mateusz: your bug is different, as this precise one was fixed.
Yours looks a lot like bug 275451.

Revision history for this message
Tim Keitt (tkeitt) wrote :

This appears not to be fixed in karmic. I have sysvinit-utils version 2.87dsf and my NSF mounts are not happening on reboot.

Revision history for this message
Kris OMealy (komealy) wrote :

I confirm that this is still happening in karmic, I have the same issue, and a lot of the fixes above either don't work, or the file that they are changing doesn't exist to begin with. I'm running a fresh install using kubuntu on an amd64 platform.

Revision history for this message
Kris OMealy (komealy) wrote :

I did find that the rc.local fix that was mentioned works, I didn't even have to include the sleep command, just the mount -a -t nfs.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.