NFSv4 mounts often missing after boot

You may change the duplicate status of a bug by clicking
"Mark as duplicate" in the left-hand menu.

There appears to be enough information in bug 46145 to
reproduce the race.

I thought I better post this here as the bug I posted too (twice, oops sorry) was marked the duplicate.

I have the same problem here after doing a dist-upgrade to Dapper from Breezy. fstab hasn't changed and was working fine in Breezy last night.

I have only rebooted 3 times and all times NFS mounts weren't mounted but could be (I use Kwidisk).

I will do some more investigating with thise info at hand.
I have previously been digging around in my rc folders to see if NFS mounts are attemtped to be mounted BEFORE the network is brought up. So far no luck as I am no expert on RC scripts.

I have a similar problem with a fully patched clean install of Dapper. I have 5 nfs v3 partions and 1 smbfs share in /etc/fstab that always mount after a reboot but /etc/mtab usually only shows one of the remote file systems. Sometimes it is one of the nfs shares and sometimes it is the smbfs share that shows.

cat /etc/fstab
192.168.x.x:/vol/postfixc /etc/postfixc nfs defaults 0 0
192.168.x.x:/vol/mail /var/mail nfs defaults 0 0
192.168.x.x:/vol/mail/mailman /var/lib/mailman nfs defaults 0 0
192.168.x.x:/vol/www /var/www nfs defaults 0 0
192.168.x.x:/vol/logs /var/log/www nfs defaults 0 0
//192.168.x.x/ahfc\044 /mnt/ahfc/rates smbfs defaults credentials=/secret,uid=x,gid=x 0 0

cat /proc/fstab
/dev/sda1 / ext3 rw,errors=remount-ro 0 0
proc /proc proc rw 0 0
varrun /var/run tmpfs rw 0 0
udev /dev tmpfs rw 0 0
devpts /dev/pts devpts rw,gid=5,mode=620 0 0
/var/run/saslauthd /var/spool/postfix/var/run/saslauthd bind rw,bind 0 0
//192.168.x.x/ahfc$ /mnt/ahfc/rates smbfs rw 0 0

cat /proc/mounts
rootfs / rootfs rw 0 0
none /proc proc rw,nodiratime 0 0
udev /dev tmpfs rw 0 0
/dev/sda1 / ext3 rw,data=ordered 0 0
/dev/sda1 /dev/.static/dev ext3 rw,data=ordered 0 0
tmpfs /var/run tmpfs rw 0 0
tmpfs /var/lock tmpfs rw 0 0
tmpfs /dev/shm/var.run tmpfs rw 0 0
tmpfs /dev/shm/var.lock tmpfs rw 0 0
devpts /dev/pts devpts rw 0 0
192.168.x.x:/vol/postfixc /etc/postfixc nfs rw,v3,rsize=32768,wsize=32768,hard,lock,proto=tcp,addr=192.168.162.100 0 0
192.168.x.x:/vol/mail /var/mail nfs rw,v3,rsize=32768,wsize=32768,hard,lock,proto=tcp,addr=192.168.162.100 0 0
192.168.x.x:/vol/mail/mailman /var/lib/mailman nfs rw,v3,rsize=32768,wsize=32768,hard,lock,proto=tcp,addr=192.168.162.100 0 0
192.168.x.x:/vol/www /var/www nfs rw,v3,rsize=32768,wsize=32768,hard,lock,proto=tcp,addr=192.168.162.100 0 0
192.168.x.x:/vol/logs /var/log/www nfs rw,v3,rsize=32768,wsize=32768,hard,lock,proto=tcp,addr=192.168.162.100 0 0
tmpfs /var/run tmpfs rw 0 0
tmpfs /var/spool/postfix/var/run/saslauthd tmpfs rw 0 0
//192.168.x.x/ahfc$ /mnt/ahfc/rates smbfs rw,nodiratime,nosuid,nodev,uid=x,gid=x,file_mode=0755,dir_mode=0755 0 0

Sorry, regarding my earlier post. cat /proc/fstab should have read cat /etc/mtab.

I've tried to debug this further, and I'm pretty sure that if-up.d/mountnfs isn't run at all when 'ifup -a' is run in rcS.d/S40networking:

Feb 2 16:06:17 rcS: * Starting basic networking...
Feb 2 16:06:18 rcS: + PATH=/sbin:/bin
Feb 2 16:06:18 rcS: + . /lib/init/vars.sh
Feb 2 16:06:18 rcS: + [ -f /etc/default/rcS ]
Feb 2 16:06:18 rcS: + . /etc/default/rcS
Feb 2 16:06:18 rcS: + TMPTIME=0
Feb 2 16:06:18 rcS: + SULOGIN=no
Feb 2 16:06:18 rcS: + DELAYLOGIN=no
Feb 2 16:06:18 rcS: + UTC=yes
Feb 2 16:06:18 rcS: + VERBOSE=no
Feb 2 16:06:18 rcS: + FSCKFIX=no
Feb 2 16:06:18 rcS: + [ ]
Feb 2 16:06:18 rcS: + . /lib/lsb/init-functions
Feb 2 16:06:18 rcS: + FANCYTTY=
Feb 2 16:06:18 rcS: + [ -e /etc/lsb-base-logging.sh ]
Feb 2 16:06:18 rcS: + . /etc/lsb-base-logging.sh
Feb 2 16:06:18 rcS: + . /lib/init/mount-functions.sh
Feb 2 16:06:18 rcS: + [ lo != lo ]
Feb 2 16:06:18 rcS: + exit 0
Feb 2 16:06:18 rcS: ...done.
<snip>
Feb 2 16:06:25 rcS: * Configuring network interfaces... ESC[80G ^MESC[74G[ OK ]
Feb 2 16:06:25 rcS: * Starting portmap daemon... ESC[80G * Already running. ESC[80G ^MESC[74G[ OK ]

notice that I've put 'set -x' in the mountnfs -script, which is shown only when loopback is ifup'd.

This is with feisty. NFSv3 mounts somehow get mounted, but NFSv4+krb5i not.

it seems that none of the scripts in if-up.d are run.. for instance avahi-daemon is started later when dbus is started.

setting severity as high, since a lot of functionality expects if-up.d -scripts to be run.

I suffered the same problem and was able to nail it down to udev! (see also bug 77325)

Indeed, the mountnfs script in if-up.d is never run, because interfaces are already up when rcS reaches S40networking.
The reason for this is, that udev get's notice of the hardware and calls itself (via /etc/udev/rules/85-ifupdown.rules) an "ifup --allow auto $env{INTERFACE}", which enables the interface.

This is obviously too early in the boot process (S10), to successfully initialize NFS(4). Since it is called in the background (start-stop-daemon), it may eventually be delayed so that it may succeed.

My fix and suggestion is to change the "auto-CLASS" (see man ifup) for the udev invocation.
When changing "--allow auto" to "--allow udev" only those interfaces marked with e.g. "allow-udev eth0" are brought up by udev.
This gives the admin control where interfaces are handled (udev or S40networking).

Not related to ifupdown

I started running into this or something similar to it after changing motherboards.

Old motherboard was an nforce4 with a "Vitesse VSC8201" ethernet adapter. With this my nfs entries in fstab were mounted normally at boot.

New motherboard is an "msi p6n sli" with the ethernet adapter "Realtek RTL8211BL".
with this new motherboard my nfs4 share was not being mounted at boot and I would have to do it manually after logging in.

the workaround I am currently using is adding the line:
pre-up sleep 5
at the bottom of my "eth0" section in /etc/network/interfaces

I've spent many hours getting around this problem in Dapper. None of the workarounds in the bug comments have worked for me. Finally I arrived across a solution that worked reliably:

Add the mountnfs command manually to the init script for the first program that requires NFS for me. In this case it's backuppc.

sudo sed -i '/"Starting $NAME..."/s@"Starting $NAME..."@&\n\t/etc/network/if-up.d/mountnfs@' /etc/init.d/backuppc

Next I made sure nfs-common started earlier than S20backuppc so that idmapd is started for successful mounting of the nfs4 directory.

sudo mv /etc/rc2.d/S21nfs-common /etc/rc2.d/S19nfs-common

It's a shame Ubuntu has such major problems with nfs4 that they aren't even mounted on boot when they're in the fstab file. The system that was setup depends on NFS being available, even for start up services, so a bug like this is unacceptable.

I just wanted to add that I have this exact same problem with NFSv3 mounts on Feisty. *None* of the mounts listed in /etc/fstab get mounted during boot - I had to add additional mount commands to /etc/rc.local to get them mounted.

Example line from /etc/fstab:
192.168.1.1:/export/src /usr/local/src nfs nouser,wsize=8192,rsize=8192,atime,auto,rw,nodev,noexec,suid 0 0

This share will not get mounted during boot. It will, however, get mounted by the following command in rc.local:
mount /usr/local/src
which means that the fstab entry *as such* works.

The client is Kubuntu 7.04 'Feisty' AMD64 (nfs-common is installed and all updates are applied) and the server runs OpenBSD 3.8 on Sparc64. I've been using the same server set-up (OpenBSD on Sparc) for years now with varying Linux clients (Red Hat, Fedora, Mandrake, SuSE, Aurora) and never had any issues, so I tend to assume that the server is ok... :-)

Bug still present in Gutsy. Two nfs shares in /etc/fstab, they are never mounted after a reboot. If I run the /etc/network/if-up.d/mountnfs script manually, the fs'es get mounted, so it looks like the same bug: on a fast machine during boot the script never gets called.

I can also confirm this behavior in Gutsy.

Removing "noproc" from the mount command in /etc/init.d/mountall.sh cured this issue for me (Gutsy). The mountall.sh problem was reported in another bug.

Try "perl -p -i -e 's/noproc\,//' /etc/init.d/mountal.sh"

Removing "noproc" from the mount command in /etc/init.d/mountall.sh cured this issue for me (Gutsy). The mountall.sh problem was reported in another bug.

Try "perl -p -i -e 's/noproc\,//' /etc/init.d/mountall.sh"

Changing mountall.sh in this manner is the wrong thing to do. mountall.sh is responsible for mounting *local* (i.e. non-networked) filesystems, and by removing the 'no' prefix from -t you change it so it only mounts networked filesystems instead.

Not to mention that mountall.sh is run at a point where you're not guaranteed to have networking up and running -- it depends on how fast your udev is while it's running in the background.

I am also experiencing this problem on gutsy

john@dad-desktop:~$ grep : /proc/mounts
192.168.1.100:/home /home nfs rw,vers=3,rsize=8192,wsize=8192,hard,intr,proto=tcp,timeo=14,retrans=2,sec=sys,addr=192.168.1.100 0 0

This bug was fixed in the package sysvinit - 2.86.ds1-14.1ubuntu37

---------------
sysvinit (2.86.ds1-14.1ubuntu37) hardy; urgency=low

  * debian/initscripts/etc/init.d/mtab.sh:
    Explicitly add mtab entries for networked file systems; race condition
    exists where mtab can get overwritten *after* networked file systems are
    mounted (LP: #46145, #44836).
  * debian/initscripts/etc/init.d/waitnfs.sh, debian/control:
    Update inline documentation, as waitnfs ONLY waits on /usr and /var mounted
    filesystems.
    Before waiting for net file systems, try mounting them (with watershed)
    just in case the mount has not been attempted elsewhere.
    Added udev as dependency due to use of watershed.
  * debian/initscripts/etc/network/if-up.d/mountnfs:
    Remove the bulk of this script to a common library script
    Call the common library script with watershed for locking purposes
    (LP: #45842, #46516).
  * debian/initscripts/lib/init/mountall-net-fs, debian/rules:
    Common library script for mounting all network file systems (nfs, cifs,
    samba, etc).
    Script can be called in various meaningful locations; should be wrapped
    with watershed to correctly handle race conditions.

 -- Dustin Kirkland <email address hidden> Tue, 26 Feb 2008 16:10:56 -0600

I see this happen on Natty. In one of five boot processes, an nfs4 entry in /etc/fstab seems to be ignored.

Should this be re-opened or should I file a separate bug?

i am seeing this bug in

Linux box04 2.6.38-10-server #46~lucid1-Ubuntu SMP Wed Jul 6 20:19:32 UTC 2011 x86_64 GNU/Linux

still seeing this bug on backport of natty kernel to lucid

Linux 2.6.38-12-generic #51~lucid1-Ubuntu SMP Thu Sep 29 19:55:22 UTC 2011 i686 GNU/Linux
Linux 2.6.38-12-server #51~lucid1-Ubuntu SMP Thu Sep 29 20:09:53 UTC 2011 x86_64 GNU/Linux