Merge lp:~mwhudson/livecd-rootfs/casper-be-gone into lp:livecd-rootfs

[Michael Hudson-Doyle]

Currently casper ends up in the base filesystem (and so on the installed system) which isn't what we want. So we prevent that, by preventing lb_chroot_live-packages from running at all. Then of course, we need to install casper in the installer layer, and then regenerate the initramfs and make sure the later steps of the build pick it up (which is pretty ugly).

I haven't tested a CD built with this branch but I have run the livefs build https://launchpad.net/~mwhudson/+livefs/ubuntu/bionic/live-server-test/+build/118096 and checked that the generated initramfs is all but identical to the one generated by the default server-live livefs (only timestamps and randomly generated data differ).

[Michael Hudson-Doyle (mwhudson)]

FWIW, this doesn't work, casper probably needs to change too somehow.

[Michael Hudson-Doyle (mwhudson)]

Actually it does seem to work, I must have screwed up my testing before.

[Steve Langasek (vorlon)]

Could you factor the bulk of this code out into a couple of functions, for easy consumption of other flavor builds in the future?

review: Needs Fixing

[Mathieu Trudel-Lapierre (cyphermox)]

I'm worried about not installing live-config as would normally be done in lb_chroot_live-packages (which is what you disabled by adding the touch command).

I have no idea if it's actually all that useful on our live images, but live-config is meant to do some late-boot "configuration" of the system, such as messing with sudoers, locale-gen, timezone, etc. for the live session. I just don't recall explicitly noticing that it was live-config rather than casper doing that work.

review: Needs Information

[Michael Hudson-Doyle (mwhudson)]

> Could you factor the bulk of this code out into a couple of functions, for
> easy consumption of other flavor builds in the future?

I've tried to do this.

[lp:~mwhudson/livecd-rootfs/casper-be-gone]

1603. By Michael Hudson-Doyle on 2018-02-22

respond to some review feedback

1604. By Michael Hudson-Doyle on 2018-02-22

make another possibly more generally useful function

[Michael Hudson-Doyle (mwhudson)]

> I'm worried about not installing live-config as would normally be done in
> lb_chroot_live-packages (which is what you disabled by adding the touch
> command).

Nah, that bit of chroot_live-packages didn't fire (I guess we don't set LB_INITSYSTEM?)

> I have no idea if it's actually all that useful on our live images, but live-
> config is meant to do some late-boot "configuration" of the system, such as
> messing with sudoers, locale-gen, timezone, etc. for the live session. I just
> don't recall explicitly noticing that it was live-config rather than casper
> doing that work.

You can see that live-config is not installed in the manifest of the current livefs builds.

[lp:~mwhudson/livecd-rootfs/casper-be-gone]

1605. By Michael Hudson-Doyle on 2018-02-22

move skip_lb_stage to config

1606. By Michael Hudson-Doyle on 2018-02-22

call recreate_initramfs with the argument it needs

[Steve Langasek (vorlon)]

review: Needs Fixing

[lp:~mwhudson/livecd-rootfs/casper-be-gone]

1607. By Michael Hudson-Doyle on 2018-02-22

remove some of the cargo culted scorpions

[Steve Langasek (vorlon)]

review: Approve

[lp:~mwhudson/livecd-rootfs/casper-be-gone]

1608. By Michael Hudson-Doyle on 2018-02-23

source config/binary in 032-installer-squashfs.binary

1609. By Michael Hudson-Doyle on 2018-02-23

clean up changelog

1610. By Michael Hudson-Doyle on 2018-02-23

it is config/common, not config/binary, grr

=== modified file 'debian/changelog'
--- debian/changelog	2018-02-20 21:40:04 +0000
+++ debian/changelog	2018-02-23 01:58:51 +0000
@@ -1,3 +1,9 @@
+livecd-rootfs (2.505) UNRELEASED; urgency=medium
+
+  * Move casper from filesystem.squashfs to installer.squashfs. 
+
+ -- Michael Hudson-Doyle <[e-mail]>  Fri, 23 Feb 2018 13:40:35 +1300
+
 livecd-rootfs (2.504) bionic; urgency=medium
 
   * Override JobRunningTimeoutSec to 0s on the .device unit that

=== modified file 'live-build/auto/config'
--- live-build/auto/config	2018-02-14 20:31:07 +0000
+++ live-build/auto/config	2018-02-23 01:58:51 +0000
@@ -116,6 +116,12 @@
 	esac
 fi
 
+skip_lb_stage() {
+	STAGE="$1"
+	mkdir -p .build
+	touch ".build/$STAGE"
+}
+
 case $IMAGEFORMAT in
 	ext2|ext3|ext4)
 		OPTS="${OPTS:+$OPTS }--initramfs none --chroot-filesystem $IMAGEFORMAT"
@@ -141,7 +147,9 @@
 		INITRAMFS_TYPE=none
 		case $PROJECT:${SUBPROJECT:-} in
 			ubuntu-server:live)
-				add_package live lupin-casper
+				# Stop lp installing casper into filesystem.squashfs
+				#  by skipping lb_chroot_live-packages.
+				skip_lb_stage chroot_live-packages
 				INITRAMFS_TYPE=auto
 				;;
 			*)

=== modified file 'live-build/functions'
--- live-build/functions	2017-11-23 19:27:16 +0000
+++ live-build/functions	2018-02-23 01:58:51 +0000
@@ -329,3 +329,34 @@
 	chroot "$CHROOT_ROOT" dpkg-divert --remove --local \
 		--rename /usr/bin/systemd-detect-virt
 }
+
+recreate_initramfs() {
+	# Regenerate the initramfs by running update-initramfs in the
+	# chroot at $1 and copying the generated initramfs
+	# around. Beware that this was written for a single use case
+	# (live-server) and may not work in all cases without
+	# tweaking...
+	# config/common must be sourced before calling this function.
+	CHROOT="$1"
+	# Start by cargo culting bits of lb_chroot_hacks:
+	if [ -n "$LB_INITRAMFS_COMPRESSION" ]; then
+		echo "COMPRESS=$LB_INITRAMFS_COMPRESSION" > "$CHROOT"/etc/initramfs-tools/conf.d/livecd-rootfs.conf
+	fi
+	chroot "$CHROOT" sh -c "${UPDATE_INITRAMFS_OPTIONS:-} update-initramfs -k all -t -u"
+	rm -rf "$CHROOT"/etc/initramfs-tools/conf.d/livecd-rootfs.conf
+	# Then bits of lb_binary_linux-image:
+	case "${LB_INITRAMFS}" in
+		casper)
+			DESTDIR="binary/casper"
+			;;
+
+		live-boot)
+			DESTDIR="binary/live"
+			;;
+
+		*)
+			DESTDIR="binary/boot"
+			;;
+	esac
+	mv "$CHROOT"/boot/initrd.img-* $DESTDIR
+}

=== modified file 'live-build/ubuntu-server/hooks/032-installer-squashfs.binary'
--- live-build/ubuntu-server/hooks/032-installer-squashfs.binary	2018-02-20 02:22:43 +0000
+++ live-build/ubuntu-server/hooks/032-installer-squashfs.binary	2018-02-23 01:58:51 +0000
@@ -22,6 +22,7 @@
 fi
 
 . config/functions
+. config/common
 
 SQUASH_ROOT=binary/boot/squashfs.dir
 OVERLAY_ROOT=binary/overlay
@@ -50,8 +51,10 @@
 # Install any requirements for the installer, for things we don't want
 # to see on the installed system
 chroot $SQUASH_ROOT apt-get update
-chroot $SQUASH_ROOT apt-get -y install user-setup
-chroot $SQUASH_ROOT apt-get -y install curtin
+chroot $SQUASH_ROOT apt-get -y install user-setup curtin lupin-casper
+
+# Installing casper means we need a new initramfs
+UPDATE_INITRAMFS_OPTIONS=CASPER_GENERATE_UUID=1 recreate_initramfs $SQUASH_ROOT
 
 # Don't let cloud-init run in the live session.
 touch $SQUASH_ROOT/etc/cloud/cloud-init.disabled