Broken PATH set by casper (Was: swap partitions not automounted by the LiveCD)

We already do this and has done it since hoary.

Uhm... Edgy beta LiveCD *doesn't* on my laptop.
Ergo reopening.

I think the script which is supposed to find my swap partition is: /usr/share/initramfs-tools/scripts/casper-bottom/13swap.
I can't understand what's wrong with it: I know it's run without any error (see casper.log, which I'm going to attach) and I've checked that my swap partition is of type SWAPSPACE2, but no swap line is put into my fstab.
If I manually do a "swapon /dev/hda5", it gets activated without any problem.

Some info which may be useful to debug:

root@ubuntu:~# fdisk -l /dev/hda
Disk /dev/hda: 20.0 GB, 20003880960 bytes
16 heads, 63 sectors/track, 38760 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
   Device Boot Start End Blocks Id System
/dev/hda1 * 1 16257 8193118+ b W95 FAT32
/dev/hda2 16257 16783 265072+ 1c Hidden W95 FAT32 (LBA)
/dev/hda3 16783 34585 8972302+ 5 Extended
/dev/hda5 16783 17819 522081 82 Linux swap / Solaris
/dev/hda6 17819 33931 8120857 83 Linux
/dev/hda7 33932 34585 329301 83 Linux

root@ubuntu:~# cat /etc/fstab
unionfs / unionfs rw 0 0
tmpfs /tmp tmpfs nosuid,nodev 0 0

root@ubuntu:~# dd if=/dev/hda5 bs=4086 skip=1 count=1 2>/dev/null | dd bs=10 count=1 2>/dev/null | hd
00000000 53 57 41 50 53 50 41 43 45 32 |SWAPSPACE2|
0000000a

root@ubuntu:~# cat /proc/cmdline
BOOT_IMAGE=/casper/vmlinuz file=/cdrom/preseed/ubuntu.seed boot=casper xforcevesa initrd=/casper/initrd.gz ramdisk_size=1048576 root=/dev/ram rw -- acpi=force locale=it_IT console-setup/layoutcode=it

The feature is implemented, but is not working now.
I tested it on another PC and casper fails to find the swap partition there too.
(Tollef, are you sure it does work on your machine? Have you tested it recently?)

However I think I found a solution!

I tried to debug using "break=mount" when booting my Edgy-RC LiveCD, then editing the casper-bottom/13swap script (using cat, grep and sed,... there's no text editor in busybox... argh!).

It seems that, for some *strange* reason, when dd is called in /usr/share/initramfs-tools/scripts/casper-bottom/13swap, it is resolved to the one on the (just) mounted root fs instead of the one in the initramfs.
So it fails to run, because, unlike the latter, it's not a static binary and it can't find the libraries it requires to be executed.
Notice that this error doesn't show up in casper.log because dd's stderr is redirected to /dev/null.

The incriminated line is this one:

magic=$(dd if="$device" bs=4086 skip=1 count=1 2>/dev/null | dd bs=10 count=1 2>/dev/null) || continue

I solved the issue changing "dd" with "/bin/dd" there (twice, of course).

Confirmation anyone?

I had a similar experience installing the Dapper Live CD on a PPC iMac. See:

https://launchpad.net/distros/ubuntu/+source/xorg/+bug/64934

In fact I had to turn on the swap manually before installation and even so the installer turned off the swap during execution of partitioning bringing the system to freeze due to too little memory. I had to to the following trick on the Live CD ramdisk to make it work for me:

cp /bin/true /sbin/swapoff

I have no idea why the installer turned off the already working swap, either a bug or poor design.

I was checking the live and the scripts and it's true, the fstab generation fails creatin the swap entry because "dd" is not compiled inside the busybox and /root/bin/dd is not in the PATH.

Here is the commands compiled for the busybox who is inside the initramfs:
----
BusyBox v1.1.3 (Debian 1:1.1.3-2ubuntu3) multi-call binary

Usage: busybox [function] [arguments]...
   or: [function] [arguments]...

        BusyBox is a multi-call binary that combines many common Unix
        utilities into a single executable. Most people will create a
        link to busybox for each function they wish to use and BusyBox
        will act like whatever it was invoked as!

Currently defined functions:
        [, [[, ash, basename, busybox, cat, chmod, chroot, chvt, clear,
        cmp, cp, cut, deallocvt, dumpkmap, echo, egrep, env, expr, false,
        fbset, fdflush, fgrep, grep, hostname, ifconfig, ip, kill, ln,
        loadfont, loadkmap, ls, mkdir, mkfifo, mknod, mkswap, mktemp,
        more, mount, mv, openvt, printf, ps, pwd, readlink, reset, rm,
        rmdir, sed, setkeycodes, sh, sleep, sort, stat, sync, tail, tee,
        test, touch, tr, true, tty, umount, uname, uniq, yes
----

We need to add "/root/bin" to the PATH or to put "/root/bin/dd" instead of just "dd".

BTW, there is not any bug on turning off the swap for the instalation. It's necessary because you can't change the partitions table if it is in use (turn on the swap is considered use).

dd in root/bin is linked against librt (for the timing statistic) but there's no librt.so in /lib...

You're right.

Maybe the best solution for this is to activate the "dd" in the busybox's config. Something like:
--- .config 2006-12-13 00:15:55.000000000 +0100
+++ debian/config-initramfs 2006-12-13 00:10:01.000000000 +0100
@@ -115,7 +115,7 @@
 CONFIG_CUT=y
 # CONFIG_DATE is not set
 # CONFIG_FEATURE_DATE_ISOFMT is not set
-CONFIG_DD=y
+# CONFIG_DD is not set
 # CONFIG_DF is not set
 # CONFIG_DIRNAME is not set
 # CONFIG_DOS2UNIX is not set

What do you think?

Juanje: /bin/dd *is* in the initramfs.
I don't know where it comes from. Probably, as you noticed in a provious post, not from busybox. However it's there and does work.

The problem is that it's *not used*. Instead /root/bin/dd is used, and that fails because it's a dynamically linked binary.
It simply is a PATH issue.
In fact, PATH=/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin:/usr/bin:/usr/sbin:/bin:/sbin when /scripts/casper-bottom/13swap is executed. (tested booting with "break=casper-bottom")

This is broken!
I'm investigating why this happens.

rubepisc: My fault. I didn't check first other binaries in the initramfs...

The initramfs' /bin/dd came from the package 'klibc-utils' through the mkinitramfs (line 168):
ln -s /usr/lib/klibc/bin/* ${DESTDIR}/bin

It's a statically linked binary:
juanje@mac:~$ file /usr/lib/klibc/bin/dd
/usr/lib/klibc/bin/dd: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked (uses shared libs), stripped

So, as you said the problem is not in the busybox, but in the PATH.

That PATH is exported in the 'casper' script (/usr/share/initramfs-tools/scripts/casper) at line 5:
export PATH=/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin:/usr/bin:/usr/sbin:/bin:/sbin

I guess there is a reason for that, but I don't know if it's a problem to reverse this path list...
It could fix the problem, don't you think?

I'll try tomorrow and I'll report it here. If this is not possible, I guess the solution could be to put the absolute PATH of the static dd (/bin/dd) instead of just the command (dd).

We'll see.

As I said yesterday I've tested the PATH changed in '/usr/share/initramfs-tools/scripts/casper' and works fine :-)

The patch could be like this:
--- /usr/share/initramfs-tools/scripts/casper 2006-10-25 13:45:45.000000000 +0200
+++ /usr/share/initramfs-tools/scripts/casper.new 2006-12-19 12:33:39.000000000 +0100
@@ -2,7 +2,7 @@

 # set -e

-export PATH=/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin:/usr/bin:/usr/sbin:/bin:/sbin
+export PATH=/usr/bin:/usr/sbin:/bin:/sbin:/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin

 mountpoint=/cdrom

This doesn't seem to break nothing (the live system is working normally), but the swap partition is now added to the '/etc/fstab'.

With this change (which is quite tiny and clean) is working for me. So I propose to make this little change to fix the bug.

What do you think guys?

> As I said yesterday I've tested the PATH changed in '/usr/share
> /initramfs-tools/scripts/casper' and works fine :-)
>
> The patch could be like this:
> --- /usr/share/initramfs-tools/scripts/casper 2006-10-25 13:45:45.000000000 +0200
> +++ /usr/share/initramfs-tools/scripts/casper.new 2006-12-19 12:33:39.000000000 +0100
> @@ -2,7 +2,7 @@
>
> # set -e
>
> -export PATH=/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin:/usr/bin:/usr/sbin:/bin:/sbin
> +export PATH=/usr/bin:/usr/sbin:/bin:/sbin:/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin
>
> mountpoint=/cdrom
>
>
> This doesn't seem to break nothing (the live system is working
> normally), but the swap partition is now added to the '/etc/fstab'.
>
> With this change (which is quite tiny and clean) is working for me. So I
> propose to make this little change to fix the bug.
>
> What do you think guys?
It will failed elsewhere later because:
- softwares in /usr/bin, /bin; /sbin don't implement all GNU options
expected by commands in /root/bin.
- some softwares not in /usr/bin, /bin will still fail to load.

The whole PATH=/root... is brittle and IMO a disaster in waiting. If a
casper script needs a command not in the initrd image then it must
chroot to /root.

You are right.
I don't know if this is the best way to do it, but exporting LD_LIBRARY_PATH=/root/usr/lib:/root/lib:/usr/lib:/lib (and leaving the PATH as is) solves the problems with dynamically linked binaries.
However it seems a little hackish to me.

The only, and IMHO better, alternative is to completely remove /root/* dirs from the PATH and only use tools from the busybox, so that the initramfs is somehow self-contained and its scripts are indipendent from the rootfs, as much as possible.

I'm currently checking all the scripts in /scripts/casper-bottom in order to find eventual commands which require full featured GNU tools or tools not in busybox.

> > As I said yesterday I've tested the PATH changed in '/usr/share
> > /initramfs-tools/scripts/casper' and works fine :-)
[...]
> > -export PATH=/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin:/usr/bin:/usr/sbin:/bin:/sbin
> > +export PATH=/usr/bin:/usr/sbin:/bin:/sbin:/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin
[...]
> > With this change (which is quite tiny and clean) is working for me. So I
> > propose to make this little change to fix the bug.
> >
> > What do you think guys?
>
> It will failed elsewhere later because:
> - softwares in /usr/bin, /bin; /sbin don't implement all GNU options
> expected by commands in /root/bin.

Well, I believe the problem is not really the path but how the scripts
in casper are implemented. If we don't use GNU options which are not
implemented in the static binaries inside the initramfs, we have
nothing to worry about.

> - some softwares not in /usr/bin, /bin will still fail to load.

Yes, but this is a problem of the person who writes the scripts. I
mean, if that person take care of not to use software that is not in
the initrd, or run those through a chroot, nothing bad should happen.

> The whole PATH=/root... is brittle and IMO a disaster in waiting. If a
> casper script needs a command not in the initrd image then it must
> chroot to /root.

I'm totally agree with you in this point. If casper need something
from the distro it should get through chroot.

So we all more or less agree :-)

I checked, one by one, *all* the scripts in /scripts (not only the ones in /scripts/casper-bottom, just to be sure).
Nothing uses any functionality not already present in the initramfs or options not supported by busybox.
What needs commands from /root/ already uses chroot.

To confirm this I also compared two casper.log files from two boots: the first one with the default PATH setting, the latter with all /root/* dirs removed from the PATH (in /scripts/casper). I didn't find any difference or new error.

So PATH= /usr/bin:/usr/sbin:/bin:/sbin is safe and sane.

Tollef, Colin: are you here? What is your opinion?

PS. In the process I discovered an error in casper-bottom/23networking (independent from the PATH issue). At line 30 there is a call to udevtrigger:

   udevtrigger -Bpci -Iclass="0x02*"

I don't know what it is supposed to do, but it doesn't work. As also man udevtrigger says, udevtrigger doesn't support any -B or -I options! So it fails.

Changed title and corrected summary.

(BTW, the udevtrigger issue is discussed in bug #77438)

robepisc wrote:
> Changed title and corrected summary.
>
> (BTW, the udevtrigger issue is discussed in bug #77438)
>
>
> ** Summary changed:
>
> - swap partitions not automounted by the LiveCD
> + Broken PATH set by casper (Was: swap partitions not automounted by the LiveCD)
>
> ** Description changed:
>
> - The LiveCD should detect and automount swap partitions.
> - It's important, in particular, for system like mine, which only has 256M of memory.
> - Of course it shouldn't consider swap partitions currently filled in by hibernation data.
> + The LiveCD can't detect and automount swap partitions.
>
> - Last time I used knoppix (2 years ago...) it mounted any swap partition already present on my HD.
> - Whenever it couldn't find any swap partition and if memory was not enough to start KDE, it even used to ask the user if it had to create a swap file in Windows' partition and run KDE or startup in text mode!
> + It turned out (read the comments) that /scripts/casper-bottom/13swap
> + fails to recognize swap partitions because of a broken PATH setting in
> + /scripts/casper (it's a fortune that the other casper scripts work at
> + all with that PATH...).
>

For people like me who are rolling their own custom LiveCDs before
Feisty is released, is there a patch to the script(s) that can be
applied to fix the problem?

thanks

Aniruddha Shankar

Aniruddha Shankar:
> For people like me who are rolling their own custom LiveCDs before
> Feisty is released, is there a patch to the script(s) that can be
> applied to fix the problem?

Aniruddha, you need to change the PATH set in the /scripts/casper script which is inside the initramfs image (/casper/initrd.gz) of the LiveCD.

For an HOWTO about extracting and rebuilding an initrd.gz file, see https://wiki.ubuntu.com/CustomizeLiveInitrd (section "Modify the initrd").

Inside the script /scripts/casper, edit the line which sets the PATH ("export PATH=/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin:/usr/bin:/usr/sbin:/bin:/sbin") to simply be:
  export PATH=/usr/bin:/usr/sbin:/bin:/sbin

Then rebuild the initrd.gz file and put it in the /casper/ directory of your customized LiveCD.

If you have some problem, feel free to mail me directly (you can find my email address at https://launchpad.net/~robepisc).

or just change in
scripts/casper-bottom/13swap
dd with /bin/dd

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

didier wrote:
> or just change in
> scripts/casper-bottom/13swap
> dd with /bin/dd
>
AFAIRecall, I did this change in the latest version of my custom LiveCD
but the swap was not recognised. I shall try Robepsic's directions
tomorrow/dayafter and let you know.

cheers,

Aniruddha Shankar
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.1 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFrnijhJkrd6A3rSsRApUoAJoDP5ZDGClAXJGBsVx1Vc4skEQt8QCZARMe
pxem+NoqBjtQVP1RFFgwmQ8=
=aGOC
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Successful! :)

This should be changed for feisty. Am detailing the steps I took, below, to help
anyone who needs to duplicate this.

Thanks,

Aniruddha Shankar

root@bhim:/cybermohalla/karim# mkdir initrd-tmp
root@bhim:/cybermohalla/karim# cd initrd-tmp/
gzip -dc /cybermohalla/karim/edgy-remaster/casper/initrd.gz | cpio -id
31444 blocks
root@bhim:/cybermohalla/karim/initrd-tmp# ls
bin conf etc init lib modules sbin scripts usr
root@bhim:/cybermohalla/karim/initrd-tmp# cd scripts/

root@bhim:/cybermohalla/karim/initrd-tmp/scripts# nano -w casper
root@bhim:/cybermohalla/karim/initrd-tmp/scripts# cat casper | head -

#!/bin/sh

# set -e

# Changed to try and fix bug 62868
# export
PATH=/root/usr/bin:/root/usr/sbin:/root/bin:/root/sbin:/usr/bin:/usr/sbin:/bin:/sbin
export PATH=/usr/bin:/usr/sbin:/bin:/sbin
root@bhim:/cybermohalla/karim/initrd-tmp/scripts# cd ..
root@bhim:/cybermohalla/karim/initrd-tmp# ls
bin conf etc init lib modules sbin scripts usr
# find . | cpio --quiet --dereference -o -H newc | gzip -9 > ~/new-initrd.gz
# mv new-initrd.gz /cybermohalla/karim/edgy-remaster/casper/initrd.gz

root@bhim:/cybermohalla/karim/edgy-remaster/casper# cd
/cybermohalla/karim/edgy-remaster/casper
root@bhim:/cybermohalla/karim/edgy-remaster/casper# ls -la
total 708616
dr-xr-xr-x 2 root root 4096 2007-01-31 17:30 .
dr-xr-xr-x 8 root root 4096 2006-11-22 15:22 ..
- -r--r--r-- 1 root root 26968 2006-12-27 17:36 filesystem.manifest
- -r--r--r-- 1 root root 26799 2006-12-27 17:36 filesystem.manifest-desktop
- -r-xr-xr-x 1 root root 717520896 2006-12-27 17:47 filesystem.squashfs
- -rw-r--r-- 1 root root 5670414 2007-01-31 17:27 initrd.gz
- -r--r--r-- 1 root root 1636681 2006-10-25 19:43 vmlinuz

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFwL7qhJkrd6A3rSsRAgNnAKDDQ1s6INrqLPKabaPCPekCWj3JswCeKpJi
CPacW2rgLMAGeLurCDqN9d8=
=I951
-----END PGP SIGNATURE-----

Hey, linux-swap partitions are still not automounted by the Ubuntu Feisty Herd5 (2007-03) LiveCD
I think importance of this bug should be increased.

I have tested this as well and to make the liveCD use existing swap partitions, the 13swap script should call /bin/dd not dd.
With 256M memory and no swap available ubiquity freezes during install. If there's swap to use, it finishes and feels a lot more responsive.