Intermittent mount failure after growroot

Running the mount by hand from the busybox prompt works.

The 'wait-for-root' command won't guard for the events that the BLKRRPART ioctl generates.

BLKRRPART generates the following:

ubuntu@srv-hgww8:~$ sudo udevadm --debug monitor --udev
run_command: calling: monitor
monitor will print the received events for:
UDEV - the event which udev sends out after rule processing

UDEV [1303300813.480716] remove /devices/pci0000:00/0000:00:06.0/virtio3/block/vdb/vdb1 (block)
UDEV [1303300813.538753] change /devices/pci0000:00/0000:00:06.0/virtio3/block/vdb (block)
UDEV [1303300813.570692] add /devices/pci0000:00/0000:00:06.0/virtio3/block/vdb/vdb1 (block)

The 'wait-for-root' command is written presuming that there is no device entry in the database and you're just waiting for it to arrive. If it runs before the kernel has queued or processed the 'remove'event, then it will succeed in any event.

The mount will then fail if it runs after the 'remove' event but before the 'add' event completes.

Neil,
  Thank you for reporting this issue (and fixing it).
  I've pulled your fix into the "upstream" location at https://code.launchpad.net/~cloud-init-dev/ubuntu/natty/cloud-initramfs-tools/natty . I just pushed that right now.

  I've also pushed a build to my ppa at https://launchpad.net/~smoser/+archive/ppa .

  Could you test that and report back?
  It also has a change to rescuevol that should not affect growroot at all.

@Neil,
  Could you please let me know if the suggested patches / debs fix the problem?

Yep,

Testing now. Been on hols since before Easter

On 3 May 2011 14:19, Scott Moser <email address hidden> wrote:
> @Neil,
>  Could you please let me know if the suggested patches / debs fix the problem?
--
Neil Wilson

I've tested the rebuilt package from your ppa as best I can with launchpad's PPA builder in virtual stasis :)

The fixes seem to do the job - thanks for incorporating them.

It looks like this has been merged into Scott's upstream branch and will be uploaded to Oneiric. I'm not sure this patch can be incorporated into an SRU as-is, but it looks like Scott has tweaked it a bit to use wait-for-root *and* udevadm settle, which seems safer.

Once this has landed in Oneiric, the bit that fixes this bug only should be extracted and prepared as an SRU to natty.

This bug was fixed in the package cloud-initramfs-tools - 0.2ubuntu1

---------------
cloud-initramfs-tools (0.2ubuntu1) oneiric; urgency=low

  * invoke blkid to search for the rescue volume rather than relying on
    /dev/disk/by-label/LABEL . The by-label entries would sometimes
    not exist while the device would.
  * improve failed boot message
  * update debian/README.source to point at upstream location

  [Neil Wilson]
  * fix race condition that resulted in intermittent mount failures
    after 'growroot'. (LP: #765843)
 -- Scott Moser <email address hidden> Wed, 08 Jun 2011 09:46:57 -0400