update-grub needs to ignore linux-ec2 kernels

Bug #671097 reported by Scott Moser
18
This bug affects 1 person
Affects Status Importance Assigned to Milestone
grub2 (Ubuntu)
Fix Released
Low
Unassigned
Lucid
Fix Released
Medium
Unassigned

Bug Description

Binary package hint: grub2

I'm investigating backporting support for guest managed kernels into 10.04 guest UEC images.

Doing so would allow 10.04 guest images that were running on EC2 and 10.10 UEC hosts to run 'apt-get dist-upgrade && reboot' and boot into their new kernel. Currently, the 10.04 guests cannot manage their own kernel under either UEC or EC2.

The solution in place for 10.10 is to use grub-pc to load the kernel on UEC and legacy-grub-ec2 to load the kernel on EC2.

That same solution will almost work for 10.04. The one point of failure is that 10.04 images have 2 kernels in them (linux-ec2 and linux-virtual). Currently the linux-ec2 kernel is 2.6.32.309.10 and linux-virtual is 2.6.32.25.27. That would mean that when update-grub runs, it will choose the linux-ec2 kernel as the newest, and, on reboot, the guest would reboot into that -ec2 kernel on UEC.

To avoid that, we need to have update-grub explicitly ignore the -ec2 kernel.

I had a discussion on this solution with cjwatson at http://irclogs.ubuntu.com/2010/11/03/%23ubuntu-devel.html .

The key point of that discussion is that it will *never* be useful for the -ec2 kernel to be loaded by grub-pc. the only possible case that would occur is the future development of a grub2 based xen pv-grub.

One other bit of information, is that the upgrade path from lucid to maverick is preserved. The maverick based grub-pc do not exclude the linux-ec2, but do not need to. Maverick does not include a linux-ec2 kernel (well, it is in the archive, but not updated). Thus, when a do-release-upgrade from lucid->maverick would occur, there will be a -virtual kernel installed that will appear to maverick's grub-pc to be newest. It would be selected, and all future upgrades would have -virtual kernels that were newer than -ec2 kernel.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: grub-pc 1.98-1ubuntu7
ProcVersionSignature: Ubuntu 2.6.32-309.18-ec2 2.6.32.21+drm33.7
Uname: Linux 2.6.32-309-ec2 i686
Architecture: i386
Date: Thu Nov 4 18:34:45 2010
Ec2AMI: ami-480df921
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-east-1d
Ec2InstanceType: t1.micro
Ec2Kernel: aki-6603f70f
Ec2Ramdisk: unavailable
ProcEnviron:
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: grub2

===== SRU Information =====
* impact of the bug: In the maverick cycle, work was done to make instances launched from UEC images capable of managing their own kernel using grub-legacy-ec2 on EC2 and grub2-pc on UEC. Without those changes, lucid images are not able to manage their kernel. What that means is that an instance running in UEC cannot have a security upgrade applied to a kernel by a simple 'apt-get upgrade && reboot'.
  In the lucid images, we have 2 flavors insalled (linux-ec2 and linux-virtual). The linux-ec2 version is higher than the linux-virtual version. As such, update-grub from grub-pc would select the the ec2 kernel for booting on UEC, and the instance would fail to reboot.
* how the bug has been addressed: In maverick, the linux-ec2 kernel was removed entirely. There is no '-ec2' kernel to be ignored. In lucid, the fix is to explicitly ignore kernels named '*-ec2'.
* minimal patch: The changes are available in my branch at [1], the diff can be seen from tip of that branch to revision 69 (current lucid-updates) [2].
* instructions on how to reproduce the bug:
  * Run grub-install /dev/sda1 and update-grub in a uec image.
  * verify that the linux-ec2 kernel is the first kernel listed in /boot/grub/grub.cfg
* regression potential: The possibility for regression is very low. There is no current situation where grub-pc would ever be intended to load an Ubuntu 'linux-ec2' kernel.
--
[1] https://code.launchpad.net/~smoser/ubuntu/lucid/grub2/lucid-kernel-upgrades
[2] http://bazaar.launchpad.net/~smoser/ubuntu/lucid/grub2/lucid-kernel-upgrades/revision/71?remember=69&compare_revid=69
=====

Revision history for this message
Scott Moser (smoser) wrote :
description: updated
Revision history for this message
Scott Moser (smoser) wrote :

I'm marking this as 'fix-released', as it is not relevant for maverick or natty.

Changed in grub2 (Ubuntu):
importance: Undecided → Low
status: New → Fix Released
Scott Moser (smoser)
description: updated
Revision history for this message
Scott Moser (smoser) wrote :

There is one scenario where this could cause regression. If xen is being used as a hypervisor and pygrub being used to load kernels, and linux-ec2 being used as a general purpose xen guest kernel.

linux-ec2 is not intended to be a general purpose guest kernel, but it could be being used by some parties in that manner.

If deemed necessary, I could modify the above logic to allow a config file (or debconf seed) to specify a list of blacklisted strings. I could then write that config/seed in the building of the uec images, so they would blacklist *-ec2.

Revision history for this message
Scott Moser (smoser) wrote :

Colin,
  I'd appreciate your review of the changes, and also of comment 3. I've done some general tests (making sure linux-ec2 is ignored) and also that grub.cfg still functions with other kernels installed.

Revision history for this message
Scott Moser (smoser) wrote :
Revision history for this message
Colin Watson (cjwatson) wrote :

Looks good to me, thanks. Merging.

Re comment 3: this doesn't seem likely to be a widespread use case to me, not that I'd necessarily know about it if it were; we can resolve this if it comes up. Would this patch even affect use of pygrub?

Revision history for this message
Scott Moser (smoser) wrote :

Apparently pygrub supports reading grub2 config files (/boot/grub/grub.cfg).
This patch would affect that scenario if the user were using pygrub with grub2 support to load the linux-ec2 kernel in an Ubuntu guest. After installing the updated grub-pc, the linux-ec2 kernels would no longer be listed in /boot/grub/grub.cfg, and would then not be seen by pygrub.

Again, I think this is likely a very small use case if any. More than likely people are not using linux-ec2 kernel and pygrub in xen domU.

Revision history for this message
Scott Moser (smoser) wrote :

Just mentioning here, that cjwatson said this upload will wait for bug 581760 getting tested and moved to -updates.

Scott Moser (smoser)
Changed in grub2 (Ubuntu Lucid):
importance: Undecided → Medium
status: New → Triaged
Colin Watson (cjwatson)
Changed in grub2 (Ubuntu Lucid):
milestone: none → ubuntu-10.04.2
Revision history for this message
Martin Pitt (pitti) wrote : Please test proposed package

Accepted grub2 into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in grub2 (Ubuntu Lucid):
status: Triaged → Fix Committed
tags: added: verification-needed
Revision history for this message
Scott Moser (smoser) wrote :

I can verify this fix:

$ apt-cache policy grub-pc
grub-pc:
  Installed: 1.98-1ubuntu7
  Candidate: 1.98-1ubuntu9
  Version table:
     1.98-1ubuntu9 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid-proposed/main Packages
 *** 1.98-1ubuntu7 0
        500 http://us-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
        100 /var/lib/dpkg/status
     1.98-1ubuntu5 0
        500 http://us-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid/main Packages

$ sudo update-grub; echo $?
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.32-309-ec2
Found initrd image: /boot/initrd.img-2.6.32-309-ec2
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
Found linux image: /boot/vmlinuz-2.6.32-25-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-25-generic-pae
Found memtest86+ image: /boot/memtest86+.bin
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
done
0

## Note above that there are '-ec2' kernels listed. Also, note that the
## errors can be ignored. This was fixed in maverick under bug 623609
## (1.98+20100804-4ubuntu6), but there, the 'update-grub' would exit with
## non-zero, which would cause problems with kernel installation. Because
## kernel it exits zero, we dont care.

## Now, install the new version from proposed:
##

$ sudo apt-get install grub-pc
...
$ dpkg-query --show grub-pc
grub-pc 1.98-1ubuntu9
$ sudo update-grub
Generating grub.cfg ...
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-309-ec2
Found linux image: /boot/vmlinuz-2.6.32-25-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-25-generic-pae
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
Found memtest86+ image: /boot/memtest86+.bin
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
done

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package grub2 - 1.98-1ubuntu9

---------------
grub2 (1.98-1ubuntu9) lucid-proposed; urgency=low

  * in update-grub, explicitly ignore kernels ending in -ec2 (LP: #671097)
 -- Scott Moser <email address hidden> Mon, 22 Nov 2010 16:15:12 +0000

Changed in grub2 (Ubuntu Lucid):
status: Fix Committed → Fix Released
Revision history for this message
Khairul Aizat Kamarudzzaman (fenris) wrote :

ubuntu@ip-10-128-211-172:/etc/apt$ sudo update-grub
Generating grub.cfg ...
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
ubuntu@ip-10-128-211-172:/etc/apt$ sudo update-grub; echo $?
Generating grub.cfg ...
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
1
ubuntu@ip-10-128-211-172:/etc/apt$ apt-cache policy grub-pc
grub-pc:
  Installed: 1.98-1ubuntu9
  Candidate: 1.98-1ubuntu9
  Version table:
 *** 1.98-1ubuntu9 0
        500 http://ap-southeast-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
        100 /var/lib/dpkg/status
     1.98-1ubuntu5 0
        500 http://ap-southeast-1.ec2.archive.ubuntu.com/ubuntu/ lucid/main Packages
ubuntu@ip-10-128-211-172:/etc/apt$ sudo apt-get install grub-pc
Reading package lists... Done
Building dependency tree
Reading state information... Done
grub-pc is already the newest version.
grub-pc set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
ubuntu@ip-10-128-211-172:/etc/apt$ dpkg-query --show grub-pc
grub-pc 1.98-1ubuntu9

im using proposed but it seem it still having prob ..

Revision history for this message
Khairul Aizat Kamarudzzaman (fenris) wrote :

im still having this problem .. please help ...

Revision history for this message
Scott Moser (smoser) wrote :

@Khairul,
  Your issue is is actually bug 623609. It is worked around in the newer UEC image builds, and can be worked around in your existing instance the same way. See https://bugs.launchpad.net/ubuntu/lucid/+source/cloud-init/+bug/623609/comments/14 for more information.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.