Karmic: Re-generation of grub.cfg takes long time.

os-prober 1.31 will appear soon (not uploaded to Ubuntu yet - I'm waiting for it to land in Debian unstable first). When it does, could you please try with that and let me know if it's noticeably faster? It ought to be.

Regarding the duplicates, you can probably remove /etc/grub.d/30_otheros, which was mistakenly created by the installer (and won't be created in fresh installations as of Alpha 5).

Hello and thanks for your answer.

I will wait till the os-prober 1.31 package is available and will then do a new test.

This is the current "os-prober" package:

$ sudo apt-cache show os-prober
Package: os-prober
Priority: optional
Section: utils
Installed-Size: 168
Maintainer: Ubuntu Installer Team <email address hidden>
Original-Maintainer: Debian Install System Team <email address hidden>
Architecture: amd64
Version: 1.30ubuntu1
Filename: pool/main/o/os-prober/os-prober_1.30ubuntu1_amd64.deb
----

I also removed the "/etc/grub.d/30_otheros" script.

$ ls -l /etc/grub.d/
-rwxr-xr-x 1 root root 3175 2009-09-04 21:42 00_header
-rwxr-xr-x 1 root root 1154 2009-08-10 14:18 05_debian_theme
-rwxr-xr-x 1 root root 3557 2009-09-04 21:42 10_linux
-rwxr-xr-x 1 root root 476 2009-09-04 23:26 20_memtest86+
-rwxr-xr-x 1 root root 5245 2009-09-04 21:42 30_os-prober
-rwxr-xr-x 1 root root 84 2009-07-16 12:18 40_custom
-rw-r--r-- 1 root root 483 2009-07-16 12:18 README
----

Talk to you later.

Re-hello,

Ok, new updates just arrived with os-prober v1.31 package.

a) First, upgrading an existing system. Here is the message log after running
$ sudo aptitude update && sudo aptitude upgrade

...
(hd0) /dev/sda
(hd1) /dev/sdb
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.31-9-generic
Found initrd image: /boot/initrd.img-2.6.31-9-generic
Found memtest86+ image: /boot/memtest86+.bin
Found Ubuntu 9.04 (9.04) on /dev/sda1
Found Ubuntu 8.10 (8.10) on /dev/sda3
Found Ubuntu 8.04 (8.04) on /dev/sda5
Found Linux Mint 7 Gloria - x64 Edition (7) on /dev/sda9
Found Ubuntu 8.04.1 (8.04) on /dev/sdb1
Found Ubuntu 9.04 (9.04) on /dev/sdb2
Found Linux Mint 7 Gloria - Main Edition (7) on /dev/sdb3
done
--

Now the GRUB2 update took only a few seconds (maybe 5 - 6 seconds or so).

But some menu entries are missing.
The newly-generated /boot/grub/grub.cfg file contains only entries from this actual Ubuntu installation + rescue mode and memtest86. It does not include the various Linuxes from other partitions. Most notably the "Other operating systems:" section of grub.cfg is completly empty. See the attached grub.cfg.
-----

The version of os-prober is now:

$ sudo apt-cache show os-prober
Package: os-prober
Priority: optional
Section: utils
Installed-Size: 180
Maintainer: Ubuntu Developers <email address hidden>
Original-Maintainer: Debian Install System Team <email address hidden>
Architecture: amd64
Version: 1.31
Depends: libc6 (>= 2.4)
Filename: pool/main/o/os-prober/os-prober_1.31_amd64.deb
Size: 20784
MD5sum: 77a765fd961d3c972ef59b05d2572a61
...
--------

b) I also did a totally new 64bit Ubuntu Alpha5 installation.
The grub.cfg after first boot is exactly 100% same as in the upgrade step a).

And files in /etc/grub.d/ are:
$ ls -l /etc/grub.d/
-rwxr-xr-x 1 root root 2944 2009-09-01 23:53 00_header
-rwxr-xr-x 1 root root 1154 2009-09-01 23:41 05_debian_theme
-rwxr-xr-x 1 root root 3434 2009-09-01 23:53 10_linux
-rwxr-xr-x 1 root root 472 2009-06-24 18:04 20_memtest86+
-rwxr-xr-x 1 root root 5123 2009-09-01 23:53 30_os-prober
-rwxr-xr-x 1 root root 84 2009-09-01 23:53 40_custom
-rw-r--r-- 1 root root 483 2009-09-01 23:53 README
--------

The missing menu entries (the other partitions/Linuces) are not a big problem because I always maintain my own grub.cfg manually.

But maybe the "30_otheros" script is still required? Or maybe 30_os-prober script needs som fine-tuning?

See the attached grub.cfg.

Do not hesitate to ask me to do additional tests. I'll very happy to help. That's what I do.
--------

The missing entries for other Linux systems are due to bug 426061.
os-prober 1.32 is on its way to correct this.

Re-hi,

The updates of today brought os-prober v1.32 which seems to work right.
It generated a new grub.cfg by scanning all the partitions once and in order, with correct result.

...
Running postinst hook script /usr/sbin/update-grub.
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.31-10-generic
Found initrd image: /boot/initrd.img-2.6.31-10-generic
Found linux image: /boot/vmlinuz-2.6.31-9-generic
Found initrd image: /boot/initrd.img-2.6.31-9-generic
Found memtest86+ image: /boot/memtest86+.bin
Found Ubuntu 9.04 (9.04) on /dev/sda1
Found Ubuntu 8.10 (8.10) on /dev/sda3
Found Ubuntu karmic (development branch) (9.10) on /dev/sda5
Found Linux Mint 7 Gloria - x64 Edition (7) on /dev/sda9
Found Ubuntu 8.04.1 (8.04) on /dev/sdb1
Found Ubuntu 9.04 (9.04) on /dev/sdb2
Found Linux Mint 7 Gloria - Main Edition (7) on /dev/sdb3
done
----

$ apt-cache show os-prober
Package: os-prober
Priority: optional
Section: utils
Installed-Size: 180
Maintainer: Ubuntu Developers <email address hidden>
Original-Maintainer: Debian Install System Team <email address hidden>
Architecture: amd64
Version: 1.32
Depends: libc6 (>= 2.4)
Filename: pool/main/o/os-prober/os-prober_1.32_amd64.deb
Size: 20870
MD5sum: 0b861b394f23756f2f174f6453632416
----

On Wed, Sep 09, 2009 at 12:49:37PM -0000, moma wrote:
> The updates of today brought os-prober v1.32 which seems to work right.
> It generated a new grub.cfg by scanning all the partitions once and in order, with correct result.

OK, great. Was it still acceptably quick? If so, I think this bug can be
closed.

Hallo,

Yes, it is acceptably quick and normal.
This machine has 11 partitions and totally 25 kernels, so it takes ca. 65 seconds to re-generate the entire grub.cfg. That is not bad at all. Very normal indeed.

Excellent - thanks for following up. Of course we may see if we can make it faster still (I have at least one idea that might help).

The problem still exists for me:

During "Generating grub.cfg..." the disk is under heavy load, braking out the whole system. The update takes approx. 20min.

System is:
$ uname -a
Linux mythos-e6500 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:05:01 UTC 2009 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 9.10
Release: 9.10
Codename: karmic

$ sudo fdisk -l

Platte /dev/sda: 160.0 GByte, 160041885696 Byte
255 Köpfe, 63 Sektoren/Spuren, 19457 Zylinder
Einheiten = Zylinder von 16065 × 512 = 8225280 Bytes
Disk identifier: 0xc8000000

   Gerät boot. Anfang Ende Blöcke Id System
/dev/sda1 1 17 136521 de Dell Utility
/dev/sda2 18 279 2097152 7 HPFS/NTFS
Partition 2 endet nicht an einer Zylindergrenze.
/dev/sda3 * 279 5379 40970703+ 7 HPFS/NTFS
/dev/sda4 5380 19457 113081535 5 Erweiterte
/dev/sda5 5380 8619 26025268+ 83 Linux
/dev/sda6 8620 11231 20980858+ 83 Linux
/dev/sda7 11232 18880 61440561 83 Linux
/dev/sda8 18881 19457 4634721 82 Linux Swap / Solaris

$ mount
/dev/sda6 on / type ext4 (rw,relatime,errors=remount-ro)
proc on /proc type proc (rw)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
none on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
/dev/sda7 on /home type xfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
gvfs-fuse-daemon on /home/mythos/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=mythos)

Still affects me

I think my problem could have been related to an 32-bit xfs partition on an 64-bit OS. I did a 'xfs_repair' and now the search time is much shorter.

I'm seeing it also. The commands that are taking so long are:

root 17586 0.0 0.0 4072 672 pts/7 S+ 11:27 0:00 | \_ /bin/sh -e /usr/sbin/grub-mkconfig -o /boot/grub/grub.cfg 2.6.31-17-generic /boot/vmlinuz-2.6.31-17-generic
root 17636 76.4 0.1 17988 4672 pts/7 D+ 11:28 0:11 | \_ /usr/sbin/grub-probe -t fs /usr/share/grub/unicode.pf2

... running from within the update-grub from a postrm script, I believe. What is 'unicode.pf2' and why does grub-probe need to scan it?

This is untested, but I bet it solves the problem.

I've tested that patch, and it works just fine. Please apply it to the package before you release Lucid Lynx or you'll be hearing about it a lot more.

Sorry, I think this patch is incorrect as it stands. The point of using grub-probe is to ensure that GRUB will be able to read it at boot time. 'test -r' would not achieve this - it would only check for file permissions, which are irrelevant here (we've already done 'test -e').

grub-probe should not take a long time, and if it is doing so then *that* is what should be fixed. That particular call is not in an inner loop or anything - indeed I think it's only called once per update-grub run, and there's no intrinsic reason it should be a bottleneck.

Oh, and I don't normally find "Please <do X> or you'll be hearing about it a lot more" particularly persuasive, by the way - I prefer solid technical arguments!

I'm currently seeking review for http://lists.gnu.org/archive/html/grub-devel/2010-03/msg00008.html; if I don't get serious objections upstream (and so far people seem to like it, although it's a bit risky for 1.98 which is coming out very soon), I'll apply it to Lucid.

This bug was fixed in the package grub2 - 1.98~20100128-1ubuntu4

---------------
grub2 (1.98~20100128-1ubuntu4) lucid; urgency=low

  [ Colin Watson ]
  * Update DM-RAID probing patch, removing some configure.ac cruft.
  * Add patch from Colin King to shave eight bytes off the
    pre-partition-table part of boot.img.
  * Don't display "GRUB loading" unless Shift is held down.
  * Use light-gray as the default background for highlighted menu entries;
    white is not safe because some systems interpret the top bit as blink
    (thanks, proski; LP: #527334).
  * Adjust versions of grub-doc and grub-legacy-doc conflicts to tolerate
    our backport of the grub-doc split (LP: #493968).
  * Optimise hostdisk device handling, substantially speeding up grub-probe
    filesystem reads (LP: #425650).
 -- Colin Watson <email address hidden> Wed, 03 Mar 2010 20:33:03 +0000

I have the same problem on 11.04 (Natty) now using 64-bit kernel 2.6.38-8-generic and grub-probe version 1.99~rc1-13ubuntu3
The reason is mystery to me as the update log come clean like this,

run-parts: executing /etc/kernel/postinst.d/initramfs-tools 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
run-parts: executing /etc/kernel/postinst.d/nvidia-common 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
run-parts: executing /etc/kernel/postinst.d/pm-utils 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 2.6.38-10-generic /boot/vmlinuz-2.6.38-10-generic
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.38-10-generic
Found initrd image: /boot/initrd.img-2.6.38-10-generic
Found linux image: /boot/vmlinuz-2.6.38-8-generic
Found initrd image: /boot/initrd.img-2.6.38-8-generic
Found memtest86+ image: /memtest86+.bin
done

It's just a long wait (10-12 minutes) after "Generating grub.cfg ..." message.
In fact, it takes so long that I had aborted my first installation on Natty thinking it had failed at this point.