dmraid not activating on boot

edit d-a, log to /dev/.initramfs/foo, set -x

I appreciate it looks like a dup, but perhaps you could let me finish investigating before marking as the duplicate of a fix-released bug.

If it is the same, sure, we'll collapse them - and we'll need to open a bug task on jaunty, because if its a common issue jaunty is rather broken ;)

Robert Collins ha scritto:
> I appreciate it looks like a dup, but perhaps you could let me finish
> investigating before marking as the duplicate of a fix-released bug.
>

Good point, but perhaps you shouldn't mark as confirmed a bug that seems me
pretty incomplete :)

Cheers,
Giuseppe

Robert Collins ha scritto:
> edit d-a, log to /dev/.initramfs/foo, set -x
>

In case you need some debug logs, the tar.gz in attachment could help:

> I need some debug logs.
> Extract the attached .tar.gz, and you have a dmraid-debug directory
> Backup /etc/udev/rules.d/85_dmraid.rules and /usr/share/initramfs-tools/hooks/dmraid
>
> cd dmraid-debug
> cp -a sbin/* /sbin/
> cp -a 85_dmraid.rules /etc/udev/rules.d/
> cat dmraid.initramfs-hook/dmraid > /usr/share/initramfs-tools/hooks/dmraid
> update-initramfs -u -k `uname -r`
>
>
> Reboot and send me /dev/dmraid.log and /dev/dmraid-activate.log
>

Cheers,
Giuseppe.

Giuseppe Iuculano ha scritto:
> Robert Collins ha scritto:
>> edit d-a, log to /dev/.initramfs/foo, set -x
>>
>
> In case you need some debug logs, the tar.gz in attachment could help:

Oh yes, But I need also to attach that tar.gz :-)

Cheers,
Giuseppe.

On Fri, 2009-05-15 at 15:26 +0000, Giuseppe Iuculano wrote:
> Robert Collins ha scritto:
> > I appreciate it looks like a dup, but perhaps you could let me finish
> > investigating before marking as the duplicate of a fix-released bug.
> >
>
> Good point, but perhaps you shouldn't mark as confirmed a bug that seems me
> pretty incomplete :)

A bug can be confirmed without having enough information for a developer
to fix it; and as its assigned to me it does in fact have a dev working
on it. Launchpad uses 'incomplete' as a lifecycle tool to tell users to
gather more data - its not a problem to have a bug not precisely match
its lifecycle state.

That said, I have yet to grab the -6 -> -7 delta from karmic, I should
get that done today. Is my HPA fix in -7 ?

-Rob

On Fri, 2009-05-15 at 15:49 +0000, Giuseppe Iuculano wrote:
> Giuseppe Iuculano ha scritto:
> > Robert Collins ha scritto:
> >> edit d-a, log to /dev/.initramfs/foo, set -x
> >>
> >
> > In case you need some debug logs, the tar.gz in attachment could help:

Thanks!

-Rob

Robert Collins ha scritto:

> A bug can be confirmed without having enough information for a developer
> to fix it; and as its assigned to me it does in fact have a dev working

I'm not expert with ubuntu bugs status, but in your wiki I read[1]:

Confirmed:

    * Another reporter has experienced the same bug, this can come in the form
of a duplicate bug or a bug comment
    * Confirmed bugs require confirmation from someone other than the original
reporter
    * This helps ensure that the bug is applicable to Ubuntu in general, and not
a problem with the reporter's system, therefore...
    * Please don't confirm your own bugs!

> That said, I have yet to grab the -6 -> -7 delta from karmic, I should
> get that done today. Is my HPA fix in -7 ?

No, but Luke forwarded your patch in the upstream ml, so I guess he will commit
that in our git repository if it's fine.

[1]https://wiki.ubuntu.com/Bugs/Status

Cheers,
Giuseppe.

On Sat, 2009-05-16 at 08:18 +0000, Giuseppe Iuculano wrote:
> Robert Collins ha scritto:
>
> > A bug can be confirmed without having enough information for a developer
> > to fix it; and as its assigned to me it does in fact have a dev working
>
> I'm not expert with ubuntu bugs status, but in your wiki I read[1]:

Yes, but the broader point is as part of the triage system:
end user reports a bug
non-experts apply triage
developers see actual bugs, with some sort of sort to allow choosing the
right ones to work on

in this case, I'm a dev, and taking responsibility for it. So in the
broad set of goals it is consistent.

Of course, it could be set to unconfirmed just as well - in fact any bug
status will do, as no status will stop me poking at it ;).

Looking at your tarball, it looks like thats just something to unpack to
change the hooks used in the initramfs. Have you considered having that
built into the package via a build switch, or even a custom -debug
package users could install that would divert the original hooks?

-Rob

I tried -7 with my hpa patch applied, still fails to boot au naturale :). So it looks like not a dupe to me, or my HPA patch interferes with the fix.

I've modified dmraid-activate to allow debugging if invoked with DMRAID_DEBUG=1, using your debug code, patch for both attached. I'll attach log files in a second.

running
DMRAID_DEBUG=1 /sbin/dmraid-activate /dev/sda
fails

running
DMRAID_DEBUG=1 /sbin/dmraid-activate sda
works

Now, I'm not sure how udev will precisely invoke dmraid-activate, but if its using the former syntax, that would explain it. If its meant to use the latter syntax, I have to assume udev is failing to invoke dmraid-activate.

So, these two logs are from udev running dmraid-activate, as opposed to me running it by hand. This confirms that udev is running dmraid-activate.

(Sorry for all the spam :P). Anyway, as I read the log, udev is invoking dmraid-activate 4 times:
/sbin/dmraid-activate no
/sbin/dmraid-activate block
/sbin/dmraid-activate devices
/sbin/dmraid-activate found

(I base this by looking at grep -- --degraded boot/dmraid-activate.log, which is the first inspection of $1.

Robert Collins ha scritto:
> So, these two logs are from udev running dmraid-activate, as opposed to
> me running it by hand. This confirms that udev is running dmraid-
> activate.

> ERROR: either the required RAID set not found or more options required
> no raid sets and with names: "no block devices found"

Is this ^^^ happening also without your hpa patch, and with ata_ignore_hpa=0 ?

Robert Collins ha scritto:
> (Sorry for all the spam :P). Anyway, as I read the log, udev is invoking dmraid-activate 4 times:
> /sbin/dmraid-activate no
> /sbin/dmraid-activate block
> /sbin/dmraid-activate devices
> /sbin/dmraid-activate found
>
> (I base this by looking at grep -- --degraded boot/dmraid-activate.log,
> which is the first inspection of $1.
>

I don't understand, the dmraid rules is:

SUBSYSTEM=="block", ACTION=="add|change", ENV{ID_TYPE}=="disk",
ENV{ID_FS_USAGE}=="raid", KERNEL=="hd[a-z]|sd[a-z]", \
        RUN+="/sbin/dmraid-activate %k"

So udev invokes "/sbin/dmraid-activate %k" where %k is the node name.

Robert Collins ha scritto:

> Now, I'm not sure how udev will precisely invoke dmraid-activate, but if
> its using the former syntax, that would explain it. If its meant to use
> the latter syntax, I have to assume udev is failing to invoke dmraid-
> activate.

Udev invokes dmraid-activate node-name.

On Sat, 2009-05-16 at 11:11 +0000, Giuseppe Iuculano wrote:
> Robert Collins ha scritto:
> > So, these two logs are from udev running dmraid-activate, as opposed to
> > me running it by hand. This confirms that udev is running dmraid-
> > activate.
>
>
> > ERROR: either the required RAID set not found or more options required
> > no raid sets and with names: "no block devices found"
>
> Is this ^^^ happening also without your hpa patch, and with
> ata_ignore_hpa=0 ?

yes and no :). I have now built and installed the 7ubuntu1 package, and
booted with libata.ignore_hpa=0 ata_ignore_hpa=0 both added to the
command line.

I see four "no block devices found" echoed on screen, then the machine
boots, but only if I have cold booted it; otherwise the BIOS seems not
to provide HPA details.

Its late here now, and I'm travelling shortly. I suspect my debugging
output may be confusing dmraid-activate; if I get time before I travel
I'll try commenting that out and booting with and without my patched
dmraid-activate too. The isw debug code is only output when -d is
passed, but perhaps its being passed at an inappropriate point/needs to
be a lower debug level (if dmraid has that concept).

-Rob

I'm attaching a new debdiff that adds a slight tweak to dmraid-activate to output the command line params.

It shows, pretty clearly, that dmraid-activate is being called with
'no'
'block'
'devices'
'found'

I'm really not sure where to go from here :(

This is cleaned up; it removes noise due to dmraid-activate not handling dmraid errors properly when probing for raid names.

dmraid.log when configured to log on boot shows:
dmraid-activate run with no
dmraid -i -r -cr /dev/no
no block devices found
dmraid-activate run with block
dmraid -i -r -cr /dev/block
no block devices found
dmraid-activate run with devices
dmraid -i -r -cr /dev/devices
no block devices found
dmraid-activate run with found
dmraid -i -r -cr /dev/found
no block devices found

This bug was fixed in the package dmraid - 1.0.0.rc15-11build1

---------------
dmraid (1.0.0.rc15-11build1) karmic; urgency=low

  * Fakesync from Debian

dmraid (1.0.0.rc15-11) unstable; urgency=low

  * [ce4e2dc] 15_activate_multiple_raid_sets.patch: Continue onto all
    raid sets instead of returning after processing the first. (LP: #401713)
  * [58a1426] debian/initramfs/dmraid.initramfs-local-top/dmraid: Check
    the exit code before parsing output, thanks to Tormod Volden.
    (Closes: #542256) (LP: #415280, #376792)
  * [91d2b67] Updated to standards version 3.8.3 (No changes needed)

 -- Luke Yelavich <email address hidden> Thu, 27 Aug 2009 08:29:15 +1000