Merge lp:~robertliu/live-build/lb-sg-2.x into lp:~ce-infrastructure/live-build/lb-sg-2.x

[Robert Liu]

Now intel-microcode and amd64-microcode are the depends of linux-*-image-*, and they will cause the initrd become a composite file, i.e. contains two archives in single initrd. So we need to detect if the initrd is a composite one and split the archives accordingly.

This proposed method is originally from lsinitramfs, and rewrite to fulfill the needs of live-build

Please note we still need to modify a hook (binary_local-hooks/85recompress-initrd) of bugsy-config to avoid problems when generating initrd.lz for the final ISO image.

[Tim Chen (timchen119)]

LGTM, some comments below.

[lp:~robertliu/live-build/lb-sg-2.x]

1634. By Robert Liu on 2018-08-01

Move series to trusty

[Bin Li (binli)]

I will test it, I met issue on 4.15 and latest oem kernel.

[Bin Li (binli)]

@Robert,

 Do you know why 'IUCODE_TOOL_INITRAMFS=no' in /etc/default/intel-microcode can't work in latest kernel? I worked before, XD.

[Robert Liu (robertliu)]

@Bin,
In recent days, there is a similar updates from amd64-microcode package. You need to modify /etc/default/amd64-microcode as well.

[Robert Liu (robertliu)]

Here is what I did in bugsy-config. It does the same procedure to extract and repack the initrd.
https://git.launchpad.net/~oem-solutions-engineers/carson/+git/carson-sanbernardino-xenial-amd64/commit/?id=9051f058b3d91205d5949327c43e3f5f8f7807af

[Bin Li (binli)]

Robert,

 Got it, thanks a lot!

[Bin Li (binli)]

I tried in sutton project, the image could be built successfully, but when we install from U disk, we found kernel panic with the 1032-oem kernel.

Initramfs unpacking failed: junk in compressed archive.
...
VFS: unable to mount root fs on unknown-block(1,0)

I doubt it's related to the initramfs.

[Robert Liu (robertliu)]

@Bin,
Do you have this patch in your bugsy-config?
I'm sure that without the patch, you will meet boot issues.

https://git.launchpad.net/~oem-solutions-engineers/carson/+git/carson-sanbernardino-xenial-amd64/commit/?id=9051f058b3d91205d5949327c43e3f5f8f7807af

[Bin Li (binli)]

@Robert,

 No we don't use the bugsy-config one, from my understanding your patch for live-build fix this issue, and don't need use it in bugsy-config. XD

[Robert Liu (robertliu)]

@Bin,
If there is any task processing the initramfs, then we need to rewrite it to make it understand the new format. That's why I change the hook of bugsy-config. The hook is used to re-compress the initramfs into lzma format.

Unmerged revisions

1634. By Robert Liu on 2018-08-01

Move series to trusty

1633. By Robert Liu on 2018-08-01

* Add handling for composite initrd.
* Update Vcs-Browser and Vcs-Bzr

=== modified file 'debian/changelog'
--- debian/changelog	2017-12-22 03:42:55 +0000
+++ debian/changelog	2018-08-01 08:37:24 +0000
@@ -1,3 +1,10 @@
+live-build (2.0.12-2cesg35) trusty; urgency=medium
+
+  * Add handling for composite initrd.
+  * Update Vcs-Browser and Vcs-Bzr
+
+ -- Robert Liu <[e-mail]>  Tue, 31 Jul 2018 17:04:10 +0800
+
 live-build (2.0.12-2cesg34) precise; urgency=medium
 
   * Support kernel mount option in precise, trusty, artful build machine.

=== modified file 'debian/control'
--- debian/control	2011-05-30 18:47:54 +0000
+++ debian/control	2018-08-01 08:37:24 +0000
@@ -7,8 +7,8 @@
 Build-Depends: debhelper (>= 7.0.50~)
 Standards-Version: 3.9.1
 Homepage: http://live.debian.net/devel/live-build/
-Vcs-Browser: http://bazaar.launchpad.net/~oem-solutions-releng/live-build/lb-sg-2.x/files
-Vcs-Bzr: lp:~oem-solutions-releng/live-build/lb-sg-2.x
+Vcs-Browser: https://launchpad.net/~ce-infrastructure/live-build/lb-sg-2.x
+Vcs-Bzr: lp:~ce-infrastructure/live-build/lb-sg-2.x
 
 Package: live-build
 Architecture: all

=== modified file 'scripts/build/lb_binary_disk'
--- scripts/build/lb_binary_disk	2011-06-15 21:13:12 +0000
+++ scripts/build/lb_binary_disk	2018-08-01 08:37:24 +0000
@@ -83,6 +83,20 @@
 	> binary/.disk/archive_trace
 fi
 
+# Read bytes out of a file, checking that they are valid hex digits
+readhex()
+{
+	dd < "$1" bs=1 skip="$2" count="$3" 2> /dev/null | \
+		LANG=C grep -E "^[0-9A-Fa-f]{$3}\$"
+}
+
+# Check for a zero byte in a file
+checkzero()
+{
+	dd < "$1" bs=1 skip="$2" count=1 2> /dev/null | \
+		LANG=C grep -q -z '^$'
+}
+
 if [ "${LB_LINUX_PACKAGES}" != "none" ]
 then
 	case "${LB_INITRAMFS}" in
@@ -92,7 +106,38 @@
 				mkdir -p uuid
 				cd uuid
 
-				zcat "../${INITRD}" | cpio --quiet -id conf/uuid.conf
+				# the following codes is from lsinitramfs originally
+				# copy here to identify if the initrd is a composite
+				#
+				# There may be a prepended uncompressed archive.  cpio
+				# won't tell us the true size of this so we have to
+				# parse the headers and padding ourselves.  This is
+				# very roughly based on linux/lib/earlycpio.c
+				offset=0
+				while true; do
+					if checkzero "../${INITRD}" $offset; then
+						offset=$((offset + 4))
+						continue
+					fi
+					magic="$(readhex "../${INITRD}" $offset 6)" || break
+					test $magic = 070701 || test $magic = 070702 || break
+					namesize=0x$(readhex "../${INITRD}" $((offset + 94)) 8)
+					filesize=0x$(readhex "../${INITRD}" $((offset + 54)) 8)
+					offset=$(((offset + 110)))
+					offset=$(((offset + $namesize + 3) & ~3))
+					offset=$(((offset + $filesize + 3) & ~3))
+				done
+
+				if [ $offset -ne 0 ]; then
+					# Extract main archive
+					subarchive=$(mktemp ${TMPDIR:-/var/tmp}/initrd_XXXXXX)
+					dd < "../${INITRD}" bs="$offset" skip=1 2> /dev/null \
+						> $subarchive
+					zcat $subarchive | cpio --quiet -id conf/uuid.conf
+					rm -f $subarchive
+				else
+					zcat "../${INITRD}" | cpio --quiet -id conf/uuid.conf
+				fi
 
 				if [ -e conf/uuid.conf ]
 				then