os-prober fails to find OSes on uncleanly-unmounted filesystems

I am starting to make some headway on my os-prober / grub2 issue.

Today, there was a new kernel for Natty. When it installed, it gave a grub2 syntax error. I got around the error by running install-grub, which ran without error, but again failed to detect Maverick on the other partition. When I rebooted to Natty, it was using the new kernel.

But, as Maverick is my main-use installation, I have to be able to boot to it. So, once again, I booted to the Maverick Live DVD and installed Maverick's grub2. That detected both Maverick and Natty. Maverick booted fine, but when I booted Natty, it was back on the previous kernel version.

So, on a hunch, I manually ran os-prober in Natty, then ran grub-install, again. That did the trick. I can now selectively boot Natty or Maverick, and when I boot to Natty, it runs the current latest kernel.

This is all to say, things are still quite a mess, but I have learned how to force them to work.

These are the relevant errors:

  EXT4-fs (sda2): INFO: recovery required on readonly filesystem
  EXT4-fs (sda2): write access unavailable, cannot proceed

Unfortunately, there is no way to mount an ext3 filesystem that requires recovery without replaying the journal and thus causing writes to that partition. os-prober used to replay the journal, but that caused serious data-loss bugs in some cases so we stopped doing that.

The workaround is simply to perform recovery on the filesystem in question using the 'fsck' tool. I regret the need for this, but unfortunately the alternative is worse.

As for why your Maverick system always needs recovery, I can't say. Perhaps it isn't shutting down cleanly?

error: out of memory.
error: syntax error.
error: Incorrect command.
error: syntax error.
error: line no: 285
Syntax errors are detected in generated GRUB config file.
Ensure that there are no errors in /etc/default/grub
and /etc/grub.d/* files or please file a bug report with
/boot/grub/grub.cfg.new file attached.
done

I don't know which file... os-prober in Natty doesn't find Maverick

Renate, here is what you need to do to work around the problem. You will need to know wher hardware device Maverick is installed on, e.g., /dev/sda2 - yours may be the same, or it may be different.

Then, when booted into Natty, run command "sudo fsck /dev/sda2" <== Maks sure to use the correct device for your system

This will fix your Maverick filesystem, then you can run "sudo update-grub" in Natty and it will detect and configure Maverick, too.

Tim

Yes that worked. Thanks

Same issue, same relevant error messages, same solution (fsck) on my system today that was updated to 2.6.37-12-generic #26 (I only boot into Natty ever occasionally).

Re: comment #3 (Colin): I'm confused by "Unfortunately, there is no way to mount an ext3 filesystem that requires recovery"; my Maverick fs is ext4 as is Tim's. Further, the Maverick partition boots just fine when mounted with an alternate grub2 boot disk & shows no requirement for "recovery". It is only when Natty does a update that the problem occurs. Yesterday I ran the first updates since the19 (I'll be happy to attach a history.log if that will help) & that included grub 1.99~rc1-1ubuntu2.

ext3 or ext4 - they have the same relevant property.

requires recovery - this doesn't necessarily mean requires *manual* recovery; the kernel can often do automatic recovery without you knowing it. It just means that it wasn't unmounted cleanly.

This may well be happening as a consequence of bug 672177.

On 01/25/2011 03:13 AM, Colin Watson wrote:
> ext3 or ext4 - they have the same relevant property.
>
> requires recovery - this doesn't necessarily mean requires *manual*
> recovery; the kernel can often do automatic recovery without you knowing
> it. It just means that it wasn't unmounted cleanly.
>
> This may well be happening as a consequence of bug 672177.
>

Interesting... thanks for the reference to 672177. I tried the 'test':
$ sudo apt-get install --reinstall libc6 && sudo shutdown -r now

->
$ dmesg | grep orphan

and sure enough:

EXT4-fs (sda5 (my Maverick partition)): 14 orphan inodes deleted

I'll join 672177 as well.

Still occuring. I've been following 672177 and applied all the 'fixes' there. However the most recent update to 2.6.38.5-generic on natty reinstalls grub2 and again finds no maverick partition. fsck of the maverick partition (from natty) is still necessary & have to reinstall grub2 from the maverick partition to get my maverick grub menu back.

With todays upgrade I had to fsck my Maverick-Installation again:
sudo fsck /dev/sdaX
fsck from util-linux-ng 2.17.2
e2fsck 1.41.14 (22-Dec-2010)
/dev/sdaX: stelle das Journal wieder her
Bereinige verwaist Inode 27918417 (uid=0, gid=0, mode=0100644, size=2719432)
Afterwards I was able to recover Maverick within GRUB2 via sudo update-grub.

As did I. My grub is installed from Maverick & slightly customised, so after I am able to get back into Maverick I need to reinstall from Maverick as Natty has overtaken grub during the update.

It's to the point that I actually created a launcher script on Natty:

$ sudo fsck /dev/sda5 && sudo update-grub

And when I get my Maverick partion back I have one on there as well:

$ sudo grub-install && sudo update-grub

Please chance the status from 'Won't Fix' as this is a critical issue that *needs* to be *fixed* & hopefully *before* Natty goes to release.

You don't need to fsck - mount/unmount will do.

Unfortunately there's no simple solution to this. The prior situation caused filesystem corruption in the context of hibernation.

@Colin re comment #14: I'm not sure that I understand "filesystem corruption in the context of hibernation". The system hasn't been hibernated since installing Natty on the /dev/sda7 partition. Today's Natty upgrade did the following:

Setting up grub-common (1.99~rc1-8ubuntu1) ...
Installing new version of config file /etc/grub.d/10_linux ...
Setting up grub-pc (1.99~rc1-8ubuntu1) ...
Installation finished. No error reported.
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.38-7-generic
Found initrd image: /boot/initrd.img-2.6.38-7-generic
Found linux image: /boot/vmlinuz-2.6.38-6-generic
Found initrd image: /boot/initrd.img-2.6.38-6-generic
Found memtest86+ image: /boot/memtest86+.bin
Found Windows 7 (loader) on /dev/sda1
Found Windows Recovery Environment (loader) on /dev/sda2
Found Windows Recovery Environment (loader) on /dev/sda3
done

Maverick is installed on /dev/sda5 (and it's grub is the default install):

 sudo fdisk -l

Disk /dev/sda: 320.1 GB, 320072933376 bytes
5 heads, 63 sectors/track, 1984579 cylinders
Units = cylinders of 315 * 512 = 161280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x135c058f

   Device Boot Start End Blocks Id System
/dev/sda1 * 7 1301 203776 7 HPFS/NTFS
/dev/sda2 1301 651462 102400360+ 7 HPFS/NTFS
/dev/sda3 1905278 1984571 12488704 7 HPFS/NTFS
/dev/sda4 651463 1905277 197475862+ 5 Extended
/dev/sda5 651463 1738212 171163093+ 83 Linux
/dev/sda6 1868262 1905277 5829988+ 82 Linux swap / Solaris
/dev/sda7 1738213 1868258 20482048 83 Linux sudo fdisk -l

Disk /dev/sda: 320.1 GB, 320072933376 bytes
5 heads, 63 sectors/track, 1984579 cylinders
Units = cylinders of 315 * 512 = 161280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x135c058f

   Device Boot Start End Blocks Id System
/dev/sda1 * 7 1301 203776 7 HPFS/NTFS
/dev/sda2 1301 651462 102400360+ 7 HPFS/NTFS
/dev/sda3 1905278 1984571 12488704 7 HPFS/NTFS
/dev/sda4 651463 1905277 197475862+ 5 Extended
/dev/sda5 651463 1738212 171163093+ 83 Linux
/dev/sda6 1868262 1905277 5829988+ 82 Linux swap / Solaris
/dev/sda7 1738213 1868258 20482048 83 Linux

Natty overwrites the existing grub install (maverick's) and afterwards fails to recognize that maverick (on /dev/sda5) exists.

While mount/unmount works:

$ sudo mount /dev/sda5 /mnt
[sudo] password for natty:
natty@mavericklaptop:~$ mount
/dev/sda7 on / type ext4 (rw,errors=remount-ro,commit=0)
proc on /proc type proc (rw,noexec,nosuid,nodev)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /dev type devtmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,...

On Tue, Apr 05, 2011 at 02:30:39AM -0000, NoOp wrote:
> @Colin re comment #14: I'm not sure that I understand "filesystem
> corruption in the context of hibernation". The system hasn't been
> hibernated since installing Natty on the /dev/sda7 partition.

I wasn't saying that your bug was due to hibernation. I was saying that
if I reverted the os-prober changes that caused this bug, then that
would have the effect of resurrecting an old bug that caused filesystem
corruption in the context of hibernation.

> BTW:
> $ sudo fsck /dev/sda5 && sudo update-grub
> seems simpler than:
> $ sudo mount /dev/sda5 /mnt && sudo umount /dev/sda5 && sudo update-grub

It may be less typing, but it may also take considerably longer to run,
depending on what needs to be checked.

> Either way something has to give, as continuing to have to go through
> contortions every time that natty installs/overwrites/changes grub is
> growing very tiresome. Next step will be to install grub to it's own
> partition so that natty can't mess with it at all.

I realise that this bug is a problem. All I'm saying is that simply
reverting to the previous state isn't an option.

I'm working on fixing this with the aid of grub-mount (a utility available in an upstream GRUB branch). With any luck I should get that done by 11.04.

That'd be really awesome, thank you!

This bug was fixed in the package os-prober - 1.44ubuntu1

---------------
os-prober (1.44ubuntu1) natty; urgency=low

  * Cherry-pick from trunk:
    - Use grub-mount if it exists. This lets us do true read-only mounts,
      and works better on journalling filesystems that were mounted
      uncleanly (LP: #683355).
    - Attempt to load the fuse module, to improve the chances of grub-mount
      working.
  * Depend on grub-mount-udeb.
 -- Colin Watson <email address hidden> Sun, 10 Apr 2011 00:36:21 +0100

Thanks Colin. Works.
Still overwrites my maverick grub2 but I can live with that. Thanks again.