Grub2 fails to install to non-standard device path

I wrote e-mail to Colin Watson who maintains "grub-installer". Below are my message to Colin and his replay. He suggest to me to open a bug ticket but for unknown for me reasons I failed to do this. However I send to him a syslog file to Colin. He may already did some work.
"
> I am writing this e-mail because you are an owner of Ubuntu
> grub-installer.
> I work in FusionIO. Our company develops PCI SSDs. We tries to use
> Ubuntu 12.04 x64 server on our device. We want to install Ubuntu on
> FusionIO devices and boot. We have a problem during installing Ubuntu.
> The installer recognizes our disk as /dev/fioa device, which is non
> standard name but it works fine on non-boot disk, installs almost
> everything but failed in the end of installation. The error is
> following
> "main-menu[402]: WARNING **: Configuring 'grub-installer' failed with
> error code 1
> "main-menu[402]: WARNING **: menu item 'grub-installer' failed.

If you're happy for this to be public (which is best - it means there's an audit trail for any change), could you file a bug and attach the full installer syslog to it, and then give me a reference to the bug number?
Otherwise, I'll at least need the syslog.

> The installer creates a EFI system partition, but there is nothing in this partition.
> I believe the problem is in our name "/de/fioa" which grub-installer doesn't like.
> I analyzed the code and tried to add "/dev/fioa" in grub-installer/grub-installer file but there is not success yet.

It's quite possible that GRUB itself needs some adjustment. I'd be happy to have a look given a full log.
"

Yes, I expect this is a lack of support in grub-probe for this device.

Is the source code for your driver available, in which case I can probably work out the answer to the following questions by myself? Failing that, I need to know a few things:

 * What are the characteristics of your device naming? Ideally I'd like to have something in the format of Documentation/devices.txt from the kernel, for example:

"""
  8 block SCSI disk devices (0-15)
                  0 = /dev/sda First SCSI disk whole disk
                 16 = /dev/sdb Second SCSI disk whole disk
                 32 = /dev/sdc Third SCSI disk whole disk
                    ...
                240 = /dev/sdp Sixteenth SCSI disk whole disk

                Partitions are handled in the same way as for IDE
                disks (see major number 3) except that the limit on
                partitions is 15.
"""

 * Can the firmware access your device directly by way of the usual UEFI block device methods, or is anything special needed?

It would also be helpful if you would be able to test our current development release. I don't expect it to work right now, so don't spend time testing just yet as you'll no doubt just run into much the same bug, but our standard operating procedure is to fix bugs first in the development release and then backport as necessary, so it would be good to ensure that you would be able to confirm fixes in our development release.

Incidentally, I expect that the device map issue you mention is all part of the same root cause, so no need for a separate bug on that.

Here is a link to the daily builds you can test with: http://cdimage.ubuntu.com/daily-live/current/

For reference, here are the Suacy milestones: https://wiki.ubuntu.com/SaucySalamander/ReleaseSchedule

Thank you Colin and David. Unfortunately our driver is not open source, but I can answer the first part of your question, Colin, and I believe Igor will be able to answer the UEFI questions you have.

We start the naming of our devices where the first device is /dev/fioa and each following device moves forward in the alphabet the same as the SCSI example above (/dev/fiob, /dev/fioc, etc). If we get more than 26 devices in a system, we would have /dev/fioaa and even go as far as /dev/fiozzz (though this isn't likely to ever happen). We also handle partitions in the same way as IDE disks, but I am not sure what, if any, partition limit we have. I can find this out if necessary.

I will also do a sample test of the development release just to make sure I don't have any problems when the time comes to test any changes you may have. Thank you for your help.

For what it's worth, I'd just like to say that your organization should reconsider opening the source of the driver. Personally I have long wanted to see these kind of devices on the market and would love to look over the source and play around with the hardware. You are also likely to get some help finding bugs and improving the driver that you otherwise could miss. More importantly, it would be much better for users to have the driver integrated into the kernel so that it just works out of the box instead of requiring them to go get proprietary drivers, which are often fraught with problems whenever a new kernel comes out.

I will add some information about UEFI implementation in our devices.
The device exposes blockIO interface. In UEFI terminology the device registers BlockIO Protocol.
However, the device doesn't belong to any standard UEFI storage buses like ATA, SCSI or USB. It register as a LUN device in the Media Part of Device Path.
The typical Device Path of our device would be following:

PciRoot (0X0)/Pci (0x1,0x0)/Pci (0x0,0x0) /Unit(0x0)/HD(1,GPT,<GUID>)

It doesn't do any special stuff in terms of UEFI implementaion.

I modified grub-installer/grub-installer file by adding /dev/fioa device. Now, we could pass the error which we reported.
The installation of Ubuntu is completed without any error now.
I attached "grub-installer file, which I modified.
However, we have two different problems now.
The first, the installation doesn't create a boot entry in UEFI Boot menu. The installer removed my old Ubuntu boot entry ( I had installed Ubuntu on SATA drive before) but doesn't created the new one.
I tried to boot installed system from UEFI Shell by running /EFI/ubuntu/grubx64.efi. Ubuntu started booting but gives following error

"
Gave up waiting for root device. Common problems:
- Boot args (cat /proc/cmdline)
 - check rootdelay= (did the system wait long enough?)
 - Check root= (did the system wait for right device?)
-Missing modules (cat /proc/modules; ls /dev)
ALERT /dev/disk/by-uuid/<GUID> does not exist. Dropping to a shell!
"

I used for installation my modified *.iso image , where I added our *.ko file into initrd.gz file. Maybe I missed something else.

I linked to the wrong daily, here is the server daily (prefer to do all initial test/dev here):

http://cdimages.ubuntu.com/ubuntu-server/daily/current/

There is also a precise-daily as well:

http://cdimages.ubuntu.com/ubuntu-server/precise/daily/current/

with attached grub-installer I am getting the following error http://paste.ubuntu.com/6252941/ Need to look into it why summy device has been used here first for grub-install and it fails. While on legacy BIOS system correct device have been used which is /dev/fioa1

grub-efi uses a dummy device sine it does not install to a device, but to a file in /boot/grub/efi, which is the EFI system partition.

grub-installer attached to email was old and exposing a old bug. Here is a new grub-installer with merge changes for fioa device installs successfully without any issue. But during the boot process EFI does not know the fusion io block device and I am not sure how to start the grub now as no file system has been mapped.

If anyone can point me how to find the fusion io device in the uEFI shell then I can try to load the grub here and see whether install worked fine or not? Attaching the new grub-installer file with the fix.

Thank you, Narinder. I will send you an e-mail with instructions for what you are asking, but basically you need to load our efi driver from the UEFI shell because the device you have probably doesn't have our optrom loaded and therefore the system isn't automatically loading the device driver for you. The basic command for that is 'load <drivername>.efi'. I will be sure to send you the correct driver for that along with instructions.

Attaching a full log from an installation. Right now, we're having to base things on 13.04 because we're having problems getting a working kernel module for the fusionio device on the 3.11-based kernels in 13.10 and higher.

Steps taken:

1) Copy driver packages to a USB stick with the OEMDRV driver disk format.
2) Insert the driver disk and installation media.
3) add "anna/choose_modules=driver-injection-disk-detect" to the boot command line so drivers will load and we have a device available in step #4
4) Auto-partition the resulting fioa device.
5) Also copy the updated grub-installer script in comment #13 to /usr/bin/ inside the installer environment so grub-install dummy doesn't crash at the end.
6) Reboot the server and drop to an efi shell. The fusion-io.efi drivers must be loaded in by hand at this point.
7) Once the device is present in the efi firmware, navigate to the EFI\ubuntu directory and attempt to execute grubx64.efi.

At that point we are presented with an error message that states "no such device xxxxx-xxxxx-xxxx", where xxx is the uuid of the disk. From there we are dropped to a grub rescue prompt. Running 'ls' in grub rescue gives is (hd0) (hd1) (hd2) with no partitons available.

It is possible to boot from the Fusion io device by doing the following:

1) Install on the Fusionio card and verify that it won't boot up using the steps above.
2) Re-install the OS, this time on a local SATA disk in the same machine as the FusionIO card.
3) Boot from the SATA disk. The grub menu should have two entries...one for the SATA drive and another for the fusion IO card device. The Fusion IO entry is typically listed second.
4) Select the Fusion IO entry. Initially you will get an error message complaining about "no such device xxxx-xxxxx-xxxxx", however, if you select the 'press any key to continue' option, the OS will boot up from the FusionIO device.

Attaching a grub.cfg from this type of installation. The relevant Fusionio entries begin in the /etc/grub.d/30_os-prober section of the file.

I can verify the same behavior with Intel desktop also with comment #16. But I saw the kernel panic with fusio-io driver when i tried to run init 0 on the operating system.

In the case where you are using the SATA disk to load grub then boot to the ioDrive:

Is it using the grub/ directory off the SATA disk or ioDrive to get grub up and going?

Is it pulling the kernel and initrd off of the SATA disk or the ioDrive?

Just wondering if it could be finding the grub files and kernel+initrd off the SATA drive and then after the kernel+initrd are loaded off the SATA disk the it looks at the root= and does the pivot root to the ioDrive (which the linux driver properly finds).

I tried to grock the grub.conf provided but am not familure with the intricacies of the "search" parameter which I suspect is what determines where it is pulling the grub/ files from...

I know for sure that it's using the grub directory off of the SATA drive to get things going. Not so sure about the kernel and initrd. I'll have to take a closer look at that and see if I can determine something.

looks like bootcurrent under grub-installer has few issues. Here is the extract of syslog from both the installs.

grub-installer on the fusion io device:

Oct 24 03:33:27 grub-installer: info: Installing grub on 'dummy'
Oct 24 03:33:27 grub-installer: info: grub-install does not support --no-floppy
Oct 24 03:33:27 grub-installer: info: Running chroot /target grub-install --force
Oct 24 03:33:28 grub-installer: invalid numeric value
Oct 24 03:33:28 grub-installer: Installation finished. No error reported.
Oct 24 03:33:28 grub-installer:
Oct 24 03:33:28 grub-installer: info: grub-install ran successfully

grub-installer on the data disk device:

Oct 24 15:51:04 grub-installer: info: Installing grub on 'dummy'
Oct 24 15:51:04 grub-installer: info: grub-install does not support --no-floppy
Oct 24 15:51:04 grub-installer: info: Running chroot /target grub-install --force
Oct 24 15:51:05 grub-installer: BootCurrent: 0007
Oct 24 15:51:05 grub-installer: Timeout: 2 seconds
Oct 24 15:51:05 grub-installer: BootOrder: 0006,0002,0004,0005,0001,0003,0000,000A,000B,0008,0009,0007
Oct 24 15:51:05 grub-installer: Boot0000* EFI USB Device
Oct 24 15:51:05 grub-installer: Boot0001* Boot0001
Oct 24 15:51:05 grub-installer: Boot0002* EFI USB Device 1
Oct 24 15:51:05 grub-installer: Boot0003* Boot0003
Oct 24 15:51:05 grub-installer: Boot0004* EFI Network
Oct 24 15:51:05 grub-installer: Boot0005* EFI Network 1
Oct 24 15:51:05 grub-installer: Boot0006* EFI Internal Shell
Oct 24 15:51:05 grub-installer: Boot0007* EFI USB Device 2
Oct 24 15:51:05 grub-installer: Boot0008* Primary Slave CDROM
Oct 24 15:51:05 grub-installer: Boot0009* Harddisk 0
Oct 24 15:51:05 grub-installer: Boot000A* EFI Hard Drive
Oct 24 15:51:05 grub-installer: Boot000B* EFI DVD/CDROM
Oct 24 15:51:05 grub-installer: BootCurrent: 0007
Oct 24 15:51:05 grub-installer: Timeout: 2 seconds
Oct 24 15:51:05 grub-installer: BootOrder: 000C,0006,0002,0004,0005,0001,0003,0000,000A,000B,0008,0009,0007
Oct 24 15:51:05 grub-installer: Boot0000* EFI USB Device
Oct 24 15:51:05 grub-installer: Boot0001* Boot0001
Oct 24 15:51:05 grub-installer: Boot0002* EFI USB Device 1
Oct 24 15:51:05 grub-installer: Boot0003* Boot0003
Oct 24 15:51:05 grub-installer: Boot0004* EFI Network
Oct 24 15:51:05 grub-installer: Boot0005* EFI Network 1
Oct 24 15:51:05 grub-installer: Boot0006* EFI Internal Shell
Oct 24 15:51:05 grub-installer: Boot0007* EFI USB Device 2
Oct 24 15:51:05 grub-installer: Boot0008* Primary Slave CDROM
Oct 24 15:51:05 grub-installer: Boot0009* Harddisk 0
Oct 24 15:51:05 grub-installer: Boot000A* EFI Hard Drive
Oct 24 15:51:05 grub-installer: Boot000B* EFI DVD/CDROM
Oct 24 15:51:05 grub-installer: Boot000C* ubuntu
Oct 24 15:51:05 grub-installer: Installation finished. No error reported.
Oct 24 15:51:05 grub-installer:
Oct 24 15:51:05 grub-installer: info: grub-install ran successfully

attaching syslog on sata disk

I'm pretty sure that the "Oct 24 03:33:28 grub-installer: invalid numeric value" error comes from grub-install running efibootmgr against the Fusion-IO device. The naming scheme, of course, for that adapter is /dev/fioa1...the grub-install script, however, runs the following at around line 892:

efidir_part="$(echo "$efidir_drive" | sed 's/^([^,]*,[^0-9]*//; s/[^0-9].*//')"
efibootmgr $efi_quiet -c -d "efidir_disk" -p "$efidir_part" -w\
    -L "$bootloader_id" -l "\\EFI\\$efi_distributor\\$efi_file"

It looks like the regex search is returning basically a null value for efidir_part, thus, the "-p" parameter in efibootmgr isn't valid. The regex seems to be tailored for traditional devices such as /dev/sda1, which does return a sane value.

Ultimately, though, this isn't the total cause of efibootmgr not working properly...if I go in and 'hard code' the -p value to 1 then re-run grub install against the fusion IO card, efibootmgr still doesn't seem to want to add the device to its list. Even just running efibootmgr by hand also produces the same result.

The efibootmgr problems mentioned above are most likely resolved already. In 13.04, there was a kernel bug where the kernel would sometimes fail to update EFI vars. I think that's partly what we were hitting here. Here's the bug, and the issue has since been fixed:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1173423

If I boot to the SATA drive, update the kernel to the latest 13.04 one, add in the updated iomemory-vsl driver for that kernel and then try to add an efi entry for the fioa device via efibootmgr, it at least succeeds:

efibootmgr -c -w -L FusionIO -d /dev/fioa1 -p 1 -l 'EFI\ubuntu\grubx64.efi'

...but, I'm not sure if what's added is entirely correct (see attached output from efibootmgr -v)

At this point, it is indeed now every important that we test things with the latest releases, given the issue I just hit here.

Brandon at FusionIO is working with his build team to see what can be done about getting a kernel module built that will work on 13.10 and higher releases. If we can get that, it'll hopefully help shake out some things that may have already been addressed.

Also, the grub-installer script that runs at install time identifies the fioa device's partition label as loop.

Oct 23 18:17:10 grub-installer: info: Identified partition label for /dev/fioa2: loop

When installing to the ATA device, the sda devices are seen as gpt:

Oct 24 15:50:51 grub-installer: info: Identified partition label for /dev/sda2: gpt

Tried again with the latest 14.04 build for Ubuntu server along with an updated fusionio driver. No surprise, but, we still aren't able to boot directly from the device. Attaching latest logs from an install attempt.

Looks like your problem is here:

kernel: [ 744.123969] efivars: set_variable() failed: status=-28

Which means your board is out of nvram and can't hold more boot configs.

@Joshua,

To answer your question, it definitely looks as if the OS is using the grub/ directory off the SATA disk get grub up and going in the case where you have a parallel install of Ubuntu on the HDD in addition to the FIO device. It is also pulling the kernel and initrd off of the SATA disk. Once the kernel is loaded, it looks like we hand things off to the FIO devices and the OS comes up on the card.

Although it doesn't change the overall outcome with regard to booting straight from the FusionIO card, it also looks like I can get past the efivars error by passing "efi_no_storage_paranoia" at the kernel command line at install time.

Igor: Thanks for your suggested grub-installer patches. There were a few problems with it, mainly misunderstandings about exactly what semantics were expected of various code paths, and the patch would have mishandled /dev/fiob* and later devices. I've uploaded a corrected (I think) version to https://launchpad.net/~cjwatson/+archive/grub/+packages for trusty.

Narinder: The use of "dummy" is correct. The problem you were encountering was because you dropped a grub-installer version from precise directly into a raring installer, and as a result you reverted the fix for bug 1054323 along the way. Needless to say I don't recommend this approach. :-)

Kent: The reason that regex is failing is that patching grub-installer isn't enough; it's also necessary to patch GRUB itself to handle probing these devices. grub2 2.00-19ubuntu4~ppa1 in https://launchpad.net/~cjwatson/+archive/grub/+packages should have the necessary patch.

There is now the slightly fiddly question of how to test all this. I'd very much like to test this on trusty before uploading backported patches to precise. However, this will only work if you've resolved the issues with newer kernels. If not, please let me know and I'll prepare versions for precise. If you can manage to use trusty, then the simplest way to test it is probably to do the following:

 * Boot the installer with the following in its preseed file:

  d-i apt-setup/local0/repository string http://ppa.launchpad.net/cjwatson/grub/ubuntu trusty main
  d-i apt-setup/local0/key string http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x6A547A8B977102C0

 * Run through the installer until after the "Loading additional components" stage.

 * Switch to tty2 (Alt-F2), run "nano /usr/bin/grub-installer", and manually apply the grub-installer part of the change in https://launchpadlibrarian.net/156124415/grub-installer_1.78ubuntu8_1.78ubuntu9~ppa1.diff.gz to that file; the change is short enough that you can do this in place with a bit of care. Save (Ctrl-o) and exit (Ctrl-x).

 * Switch back to tty1 (Alt-F1) and continue installation.

Thank you, Colin! I believe we can build the FusionIO modules for 14.04. We got some updated code that plays nicer with 3.10 and newer kernels.

It will be Monday morning before I can hop onto this, but, we will definitely get back to you as soon as we get something put together and tested.

Thanks, again.

I tried this earlier today, and the installer is failing at the grub installation when trying to install grub-efi-amd64-signed. I think this is because grub-efi-amd64-signed depends on grub-efi-amd64 2.00-19ubuntu3 and our packages from the ppa are at ubuntu4.

I do, however, see some improvement in behavior in that fioa2 is now seen properly as a gpt partition instead of a loop one.

grub-installer: info: Identified partition label for /dev/fioa2: gpt

Does an updated version of grub-efi-amd64-signed also need to be in the ppa? Log attached.

Ah, yeah, that bit's annoying. I'm just going to go ahead and upload this directly to trusty (with a matching grub2-signed) and we'll see how we get on there.

This bug was fixed in the package grub-installer - 1.78ubuntu9

---------------
grub-installer (1.78ubuntu9) trusty; urgency=low

  * Handle FusionIO devices, based on work by Igor Sharovar (LP: #1237519).
 -- Colin Watson <email address hidden> Tue, 12 Nov 2013 16:50:04 +0000

This bug was fixed in the package grub2 - 2.00-19ubuntu4

---------------
grub2 (2.00-19ubuntu4) trusty; urgency=low

  * Probe FusionIO devices (LP: #1237519).
 -- Colin Watson <email address hidden> Tue, 12 Nov 2013 16:46:19 +0000

Hi,

Do the above close notices mean that ioDrives are successfully booting with grub?

Or is it that the grub installer is now properly setting up the config and writing the installer to the ioDrive?

Or both?

Thanks!

Things still need to be tested to ensure these are working properly. We tried to test the 14.04 images today, but, they don't contain the updated grub packages, yet. Once they do, then we will know for sure if all the fixes are in or if more work needs to be done.

It didn't make it onto the images due to some kind of strange locking problem. I've poked a fresh build.

Tried today's build that looks to have the needed packages. Everything went fine until grub2-signed tried to install and that's where things failed. Attaching partman logs as well as syslog from today's attempt.

Looks like grub-probe doesn't quite know how to handle the fioa partitions:

Nov 19 17:33:01 in-target: Creating config file /etc/default/grub with new version
Nov 19 17:33:02 in-target: /usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/fioa1. Check your device.map.
Nov 19 17:33:02 in-target: /boot/efi doesn't look like an EFI partition.
Nov 19 17:33:02 in-target: Path `/boot/grub' is not readable by GRUB on boot. Installation is impossible. Aborting.
Nov 19 17:33:02 in-target: dpkg: error processing grub-efi-amd64 (--configure):
Nov 19 17:33:02 in-target: subprocess installed post-installation script returned error exit status 1
Nov 19 17:33:02 in-target: dpkg: dependency problems prevent configuration of grub-efi-amd64-signed:
Nov 19 17:33:02 in-target: grub-efi-amd64-signed depends on grub-efi-amd64 (= 2.00-20); however:
Nov 19 17:33:02 in-target: Package grub-efi-amd64 is not configured yet.

I investigated this with the aid of some kindly-provided remote access and gdb.

GRUB needs to be able to find the start sector of each partition within a disk in order to accurately match up OS-level devices with GRUB devices. On Linux, it uses the HDIO_GETGEO ioctl to do this; although most of the information returned by that ioctl is obsolete, it returns the start sector. Most Linux block device drivers implement this ioctl; unfortunately the FusionIO block device driver does not, and therefore GRUB has no way to safely identify /dev/fioa1.

Since this ioctl is typically rather easy to implement (you need to implement the "getgeo" member of block_device_operations), I think this should be fixed in the FusionIO driver. If you don't have meaningful cylinder/head/sector-type information to fill in, you can probably fake it up, perhaps in the same way that drivers/block/umem.c does. The start sector is filled in from generic code in block/ioctl.c.

There is only one other way I know of to get the start sector of a partition from userspace, and that's to read its "start" attribute from sysfs. However, that's considerably more cumbersome to do from C code, and I'd prefer to avoid that if at all possible. If you could add this fairly simple interface to the FusionIO driver, it would make things considerably easier. Alternatively, if there's some other interface you can suggest that's generic in Linux rather than being FusionIO-specific, I'd be happy to look into using that.

The HDIO_GETGEO ioctl has been depreciated for years and will be removed "some day", so user space tools have been moving to reading the sysfs attribute, so I'd prefer not to see more use of the obsolete ioctl. While it isn't quite as simple as the ioctl, it isn't that bad and you should be able to copy/paste the code that does this out of parted's libparted/arch/linux.c, or util-linux's lib/sysfs.c is a bit more complete but complex interface.

I've asked to have the ioctl implemented in an upcoming release to allow for greater backward compatibility even if you start using the sysfs (or some other) method.

I've been looking at the HDIO_GETGEO ioctl a bit, and it may be worth a discussion with those who have more information as to its usefulness. Particularly, the notes about the IOCTL say:

"Not particularly useful with modern disk drives, whose geometry
is a polite fiction anyway. Modern drives are addressed
purely by sector number nowadays (lba addressing), and the
drive geometry is an abstraction which is actually subject
to change. Currently (as of Nov 2004), the geometry values
are the "bios" values -- presumably the values the drive had
when Linux first booted.

In addition, the cylinders field of the hd_geometry is an
unsigned short, meaning that on most architectures, this
ioctl will not return a meaningful value on drives with more
than 65535 tracks.

The start field is unsigned long, meaning that it will not
contain a meaningful value for disks over 219 Gb in size."

It is this last paragraph about the start sector value that is most concerning to me since it is the start sector value that is needed for this ticket. From what I could gather, it appears that people lean toward Phillip Susi's suggestion of going the sysfs route.

I did come across the mention of another IOCTL called BLKPG_GET_PARTITION, but I couldn't find any good documentation on it to assess its usefulness for our purposes. Does anyone have any input on this ioctl or know where I can find good documentation?

Finally, it is worth mentioning that RHEL works with our devices and installs grub without any problems. From what I understand, they use Grub 0.97 (probably patched many times) and the grub error from this ticket does not happen with them. There is the potential that one of these patches uses the sysfs method that is needed here. If anyone has some experience in RHEL, they may be able to offer some input here, but this may be a worthy avenue to explore.

Phillip: Fair enough. I'm working on a GRUB patch to read the sysfs attribute now.

Brandon: The point about hd_geometry.start being unsigned long is one I hadn't noticed, and is indeed relevant!

I didn't actually see your mail until after I'd written a patch for this. As it happens, GRUB 0.97 is sufficiently different from GRUB 2 that it's generally easier to just do things from scratch. If I'm reading the Fedora patch set correctly, it looks like they're doing something very complex involving reading the geometry directly out of the partition table; I think using the sysfs attribute should work just as well and be a lot more maintainable. I've posted my patch upstream for review, although it hasn't yet appeared in the grub-devel web archives.

It looks as though BLKPG_GET_PARTITION was an ioctl proposed by Phillip that never actually got added to the mainline kernel.

Thank you for that information, Colin, I'm glad to hear you were able to get stuff working with sysfs. I had also been wondering how Fedora was working but haven't had time to really dig into it myself.

I've now tested this and confirmed that it works with FusionIO disks. I'll include it in the next trusty and precise uploads.

Hi Colin,

Thanks for getting this done. Please let us know when these packages will be available so we can run some tests on a couple of other machines too.

lborda

Incidentally, the sysfs part of this is upstream too now.

  http://git.savannah.gnu.org/gitweb/?p=grub.git;a=commitdiff;h=0480665b9de31aefc7f1d95448efdf8349997fe9
  http://git.savannah.gnu.org/gitweb/?p=grub.git;a=commitdiff;h=59b38922a1ea2b27995a686c3c2de8558a0f26dc

Leonardo: there'll be automatic messages when the packages are available.

Hello Brandon, or anyone else affected,

Accepted grub2 into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/grub2/1.99-21ubuntu3.14 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Brandon, or anyone else affected,

Accepted grub-installer into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/grub-installer/1.68ubuntu5.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I am working on getting this verified and will update the ticket as soon as I can.

I tested the fix using the daily build from http://cdimages.ubuntu.com/ubuntu-server/precise/daily/current that was "Last Modified" 09-Dec-2013 09:36. I installed using the 'precise-server-amd64.iso' image. Ubuntu successfully installed on our device: /dev/fioa. It appears this bug has been fixed. Thank you.

@Brandon,

Thanks for testing that on Precise. I also assume that you were able to successfully boot from that point (I think the efi module for the fio device has to be loaded by hand)? I just want to confirm so that way I won't be testing the same thing twice.

Once the grub2 fixes go out for 14.04 I'll also test it here on my server. I just tested 14.04 today not realizing that any fixes have gone out for grub2 on Trusty yet. I'll try again as soon as I see the status change.

Thanks, again, for testing precise.

Thanks for checking!

Thank you Kent. Unfortunately I was not able to successfully boot after
the grub stage. I am not sure if that is our bug or not, though.
Following is the processed I used, but first I'll mention that I was
unable to use the insmod method of installing our driver during Ubuntu
install in order to have our device recognized. It kept telling me I
had an incorrect module format. I made sure the kernel versions matched
up, but was unsuccessful using this method.

Instead, I used a driver injection disk to have our device recognized
and that worked: the os and grub both installed successfully. Then,
because I don't have an option rom loaded on my card, I loaded our
driver from the uefi shell and invoked grubx64.efi from the shell off of
our device. This brought the grub menu up and I chose to start Ubuntu
Server. I then get few print messages from our device and the OS does
not boot.

The reason I mentioned the insmod method is because I'm not sure if,
even though the driver injection disk files worked, there was still some
sort of conflict that I didn't see. Do you have any thoughts on that?
Is there a different way of testing the fixes you would like me to try?
I apologize I am by no means a Linux expert. Thank you for your time.

Brandon

On 12/10/2013 11:09 AM, Kent Baxley wrote:
> @Brandon,
>
> Thanks for testing that on Precise. I also assume that you were able to
> successfully boot from that point (I think the efi module for the fio
> device has to be loaded by hand)? I just want to confirm so that way I
> won't be testing the same thing twice.
>
> Once the grub2 fixes go out for 14.04 I'll also test it here on my
> server. I just tested 14.04 today not realizing that any fixes have
> gone out for grub2 on Trusty yet. I'll try again as soon as I see the
> status change.
>
> Thanks, again, for testing precise.
>

This e-mail (and any attachments) is confidential and may be privileged. Any unauthorized use, copying, disclosure or dissemination of this communication is prohibited. If you are not the intended recipient, please notify the sender immediately and delete all copies of the message and its attachments.

@Brandon,

No thoughts on that without seeing the actual output of what happened during boot. The driver disk is certainly the way to go for this. I think I have a theory on what that might be with regard to your boot failure. I need to test it first, though.

@Brandon,

I was able to successfully boot into 12.04. The trick, I think, is that you need two sets of driver disks in order to pull this off.

I tested this on a PowerEdge C8000 series server, which allowed me to boot trouble-free into an EFI shell to load the Fusion-io.efi module.

The reason is because there are two kernels involved on this daily image of 12.04. The installer kernel is running 3.8.0-29 while the final kernel that gets installed is 3.8.0-35. Since the kernel modules are built for a specific kernel, I'm guessing what happened to you was that you had 3.8.0-29 drivers but didn't have any for the 3.8.0-35 kernel, thus, when the OS tried to boot it fell down on you.

Here's what I did:

1) Because of this situation, I had to create kernel modules for both 3.8.0-29 and 3.8.0-35, create packages, and put the respective packages on my driver disk USB stick.

2) Once those were in place, I booted up with the kernel command line "anna/choose_modules=driver-injection-disk-detect"

3) I verified both sets of kernel modules were installed. The installer picked up the fioa device and I could install without trouble..grub gave me no problems.

4) Once that was done, I rebooted into an EFI shell, loaded the Fusion-IO module, then booted from grubx64.efi on the FIO device.

5) From the grub prompt I saw an "efidisk write error". This can be safely ignored and after a few seconds the Operating Sytem booted up just fine from the fioa disk partitions.

So, 12.04.4's daily builds are indeed good to go from the testing I was able to do.

I'll send along the packages I used along with the steps I used to boot this machine up successfully.

This bug was fixed in the package grub2 - 2.00-22

---------------
grub2 (2.00-22) unstable; urgency=low

  * Backport from upstream:
    - On Linux, read partition start offsets from sysfs if possible
      (LP: #1237519).
    - Fix sector number when writing to non-512B disks (LP: #1253443).
  * Regularise indentation of "recordfail" in /etc/grub.d/10_linux.

 -- Colin Watson <email address hidden> Thu, 12 Dec 2013 01:24:11 +0000

The verification of the Stable Release Update for grub-installer has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

This bug was fixed in the package grub2 - 1.99-21ubuntu3.14

---------------
grub2 (1.99-21ubuntu3.14) precise; urgency=low

  * Handle FAT filesystems on non-512B disks (LP: #1065281).
  * Probe FusionIO devices (LP: #1237519).
  * On Linux, read partition start offsets from sysfs if possible
    (LP: #1237519).

grub2 (1.99-21ubuntu3.13) precise; urgency=low

  * Revamp hidden timeout handling by adding a new timeout_style environment
    variable and a corresponding GRUB_TIMEOUT_STYLE configuration key for
    grub-mkconfig. This controls hidden-timeout handling more simply than
    the previous arrangements, and pressing any hotkeys associated with menu
    entries during the hidden timeout will now boot the corresponding menu
    entry immediately (LP: #1178618). As part of merging this, radically
    simplify /etc/grub.d/30_os-prober; if it finds other OSes it can now
    just set timeout_style=menu and make sure the timeout is non-zero.
  * Fix mismerge of GRUB_RECOVERY_TITLE option in 1.99-21ubuntu3.12.

grub2 (1.99-21ubuntu3.12) precise; urgency=low

  * debian/build-efi-images: Where possible, make use of the device path
    derived from the EFI Loaded Image Protocol to compute the prefix
    (LP: #1097570).
  * Add GRUB_RECOVERY_TITLE option, to allow the controversial "recovery
    mode" text to be customised (LP: #1240360).
 -- Colin Watson <email address hidden> Thu, 05 Dec 2013 16:53:48 +0000

This bug was fixed in the package grub-installer - 1.68ubuntu5.3

---------------
grub-installer (1.68ubuntu5.3) precise; urgency=low

  * Handle FusionIO devices, based on work by Igor Sharovar (LP: #1237519).
 -- Colin Watson <email address hidden> Thu, 05 Dec 2013 17:15:06 +0000