portmap does not stop during shutdown, causing possible root fs corruption

Bug #711425 reported by Clint Byrum
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
portmap (Ubuntu)
Fix Released
High
Clint Byrum
Lucid
Fix Released
High
Clint Byrum
Maverick
Won't Fix
High
Unassigned
sysvinit (Ubuntu)
Fix Released
High
Clint Byrum
Lucid
Fix Released
High
Clint Byrum
Maverick
Won't Fix
High
Unassigned

Bug Description

= SRU JUSTIFICATION =

= IMPACT =

This bug can cause data corruption in rare cases, and at the very least, forces fsck on reboot unnecessarily.

= TEST CASE: =

1. apt-get install portmap
2. apt-get install libc6 --reinstall
3. reboot
4. affected systems will show orphaned inodes in 'dmesg' on the next boot. fixed systems should not.

Binary package hint: portmap

This is related, but not the same, as bug #672177

portmap should be stopped as soon as it is no longer needed, which is right after NFS filesystems are unmounted. The unmounting is done by unmountnfs.sh from sysvinit. It properly defines an LSB header to say that portmap should be stopped after it. The trouble is that nothing interprets this header anymore as upstart does not support this method of shutdown.

When libc6 is upgraded, this means portmap keeps /lib/libc.so.xxxxx open, and so / cannot be remounted as readonly, causing filesystem corruption (this was reported in the comments of bug #672177 by "ingo".

So portmap should define an event to stop on and sysvinit should emit that event when it happens.

Revision history for this message
ingo (ingo-steiner) wrote :

On my machine portmap is started/running anyhow (together with rpc.statd), also if no nfs share has been mounted ever. In Lucid I have just installed 'nfs-common' (not the kernel server!). It is only used when I temporarly mount some nfs3-exports from guests in VBox, or nfs4-exports from my nas (running Lenny-armel).

You need to install nfs-common also if you only want just the client - and that brings portmap and rpc.statd.
Amaizingly; stopping portmap also stops rpc.statd?

Portmap is not included in nfs-common, it's a separate package. But it is refereced by nfs-common as "required"!

ingo (ingo-steiner)
Changed in portmap (Ubuntu):
status: New → Confirmed
Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Setting this to Triaged. The fix is clear: we need to stop portmap after umountnfs runs.

Changed in portmap (Ubuntu):
importance: Undecided → High
status: Confirmed → Triaged
assignee: nobody → Clint Byrum (clint-fewbar)
Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Adding sysvinit as it currently provides no facility to stop things right after umountnfs, which portmap's old init.d script clearly defined as the point at which it should be stopped.

Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Unfortunately sysvinit's package import is still broken.

Attached is the debdiff to add emitting 'unmounted-remote-filesystems' to umountnfs.sh

Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Ok the merge proposal submitted fixes the portmap upstart job, and the debdiff will provide the event. Neither malfunctions without the other in any way, so its not critical that they are fixed at the same time, but it won't be possible to test the fix without both changes.

tags: added: upstart
Changed in sysvinit (Ubuntu):
status: New → In Progress
assignee: nobody → Clint Byrum (clint-fewbar)
Changed in portmap (Ubuntu):
status: Triaged → In Progress
Changed in sysvinit (Ubuntu):
importance: Undecided → High
Changed in portmap (Ubuntu Lucid):
importance: Undecided → High
status: New → Triaged
Changed in portmap (Ubuntu Maverick):
importance: Undecided → High
status: New → Triaged
Changed in sysvinit (Ubuntu Lucid):
importance: Undecided → High
status: New → Triaged
Changed in sysvinit (Ubuntu Maverick):
importance: Undecided → High
status: New → Triaged
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package portmap - 6.0.0-2ubuntu4

---------------
portmap (6.0.0-2ubuntu4) natty; urgency=low

  [ Steve Langasek ]
  * rename debian/portmap.portmap.upstart to debian/portmap.upstart; this
    will let debhelper DTRT with both the upstart job and the init script
    under the new policy.

  [ Clint Byrum ]
  * debian/portmap.upstart: stop on unmounted-remote-filesystems, a new
    event emitted by umountnfs.sh. (LP: #711425)
 -- Dave Walker (Daviey) <email address hidden> Wed, 02 Mar 2011 13:51:55 +0000

Changed in portmap (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Benjamin Drung (bdrung) wrote :

The debdiff was sponsored - I am unsubscribing ubuntu-sponsors.

Revision history for this message
Benjamin Drung (bdrung) wrote :

subscribing ubuntu-sponsors again - portmap was sponsored, but not sysvinit.

Revision history for this message
Dave Walker (davewalker) wrote :

@Clint, your debdiff for sysvinit FTBFS:

gcc -ansi -W -Wall -O2 -fomit-frame-pointer -D_GNU_SOURCE -c -o bootlogd.o bootlogd.c
bootlogd.c: In function 'consolename':
bootlogd.c:264:15: error: 'ENOIOCTLCMD' undeclared (first use in this function)
bootlogd.c:264:15: note: each undeclared identifier is reported only once for each function it appears in
bootlogd.c: In function 'findtty':
bootlogd.c:116:8: warning: ignoring return value of 'chdir', declared with attribute warn_unused_result
bootlogd.c:131:10: warning: ignoring return value of 'chdir', declared with attribute warn_unused_result
bootlogd.c:142:10: warning: ignoring return value of 'chdir', declared with attribute warn_unused_result
bootlogd.c:147:10: warning: ignoring return value of 'chdir', declared with attribute warn_unused_result
bootlogd.c:154:7: warning: ignoring return value of 'chdir', declared with attribute warn_unused_result
make[1]: *** [bootlogd.o] Error 1
make[1]: Leaving directory `/tmp/buildd/sysvinit-2.87dsf/src'
make: *** [build-stamp] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

Revision history for this message
Dave Walker (davewalker) wrote :

Updated debdiff, fixes FTBFS for Natty. Wanting ack from foundations team before uploading.

Thanks.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sysvinit - 2.87dsf-4ubuntu21

---------------
sysvinit (2.87dsf-4ubuntu21) natty; urgency=low

  * debian/initscripts/etc/init.d/umountnfs.sh: emit a new event,
    unmounted-remote-filesystems, to allow stopping portmap
    and others. (LP: #711425)
 -- Clint Byrum <email address hidden> Fri, 04 Mar 2011 12:18:15 +0000

Changed in sysvinit (Ubuntu):
status: In Progress → Fix Released
Changed in portmap (Ubuntu Lucid):
assignee: nobody → Clint Byrum (clint-fewbar)
Changed in sysvinit (Ubuntu Lucid):
assignee: nobody → Clint Byrum (clint-fewbar)
Changed in portmap (Ubuntu Maverick):
assignee: nobody → Clint Byrum (clint-fewbar)
Changed in sysvinit (Ubuntu Maverick):
assignee: nobody → Clint Byrum (clint-fewbar)
Revision history for this message
ingo (ingo-steiner) wrote :

This is just to avoid expiry of this bug, affecting all that use nfs.

Revision history for this message
Clint Byrum (clint-fewbar) wrote : Re: [Bug 711425] Re: portmap does not stop during shutdown, causing possible root fs corruption

Excerpts from ingo's message of Fri May 20 17:41:49 UTC 2011:
> This is just to avoid expiry of this bug, affecting all that use nfs.
>

Ingo, only "Incomplete" tasks expire. None of the tasks associated with
this bug are Incomplete.

I did just notice that the milestone was not set for these. We definitely
want to fix this before 10.04.3 is cut and released in ~10 weeks. Setting
milestone accordingly.

Changed in portmap (Ubuntu Lucid):
milestone: none → ubuntu-10.04.3
Changed in sysvinit (Ubuntu Lucid):
milestone: none → ubuntu-10.04.3
Colin Watson (cjwatson)
Changed in portmap (Ubuntu Lucid):
milestone: ubuntu-10.04.3 → ubuntu-10.04.4
Changed in sysvinit (Ubuntu Lucid):
milestone: ubuntu-10.04.3 → ubuntu-10.04.4
Revision history for this message
ingo (ingo-steiner) wrote :

With other words:

*REST in PEACE*

on this huge graveyard named Launchpad

Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Excerpts from ingo's message of Tue Jul 19 14:04:10 UTC 2011:
> With other words:
>
> *REST in PEACE*
>
> on this huge graveyard named Launchpad
>

Ingo, thanks for staying with us through the long spooky night.

There are some ghosts that need vanquishing before we can move on
this one.

Namely bug #665185 needs to be verified before we add this bug on top
of it.

Revision history for this message
Martin Pitt (pitti) wrote :

As bug 665185 is now fixed in lucid, I guess this is no longer blocked?

description: updated
Revision history for this message
Martin Pitt (pitti) wrote : Please test proposed package

Hello Clint, or anyone else affected,

Accepted sysvinit into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in sysvinit (Ubuntu Lucid):
status: Triaged → Fix Committed
tags: added: verification-needed
Changed in portmap (Ubuntu Lucid):
status: Triaged → Fix Committed
Revision history for this message
Martin Pitt (pitti) wrote :

Hello Clint, or anyone else affected,

Accepted portmap into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Revision history for this message
Martin Pitt (pitti) wrote :

Maverick survived most of its live without this fix, and we hardly get any feedback for it any more, so wontfixing.

Changed in portmap (Ubuntu Maverick):
assignee: Clint Byrum (clint-fewbar) → nobody
status: Triaged → Won't Fix
Changed in sysvinit (Ubuntu Maverick):
status: Triaged → Won't Fix
assignee: Clint Byrum (clint-fewbar) → nobody
Revision history for this message
ingo (ingo-steiner) wrote :

Hello Clint,

great to hear that this bug has resurrected for beeing cured!

I did test the 'portmap' package from lucid-proposed. It also pulls in package 'initscripts' and all is fine now (before I regularly obtained 4 orphaned inodes).

Test was done in a VM running Lucid-amd64 with 2 CPU's (I do not have it on real hardware anymore).

So from my point of view it's fixed finally.

Martin Pitt (pitti)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package portmap - 6.0.0-1ubuntu2.2

---------------
portmap (6.0.0-1ubuntu2.2) lucid-proposed; urgency=low

  * debian/portmap.upstart: stop on unmounted-remote-filesystems, a new
    event emitted by umountnfs.sh. (LP: #711425)
  * d/control: Use versioned dep on initscripts to ensure portmap is
    stopped at the appropriate moment.
 -- Clint Byrum <email address hidden> Fri, 20 Jan 2012 01:37:05 -0800

Changed in portmap (Ubuntu Lucid):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sysvinit - 2.87dsf-4ubuntu17.5

---------------
sysvinit (2.87dsf-4ubuntu17.5) lucid-proposed; urgency=low

  * debian/initscripts/etc/init.d/umountnfs.sh: emit a new event,
    unmounted-remote-filesystems, to allow stopping portmap
    and others. (LP: #711425)
 -- Clint Byrum <email address hidden> Fri, 20 Jan 2012 01:15:42 -0800

Changed in sysvinit (Ubuntu Lucid):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.