Merge lp:~mwhudson/livecd-rootfs/live-server-gzip-initramfs into lp:livecd-rootfs
- live-server-gzip-initramfs
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 1657 |
Proposed branch: | lp:~mwhudson/livecd-rootfs/live-server-gzip-initramfs |
Merge into: | lp:livecd-rootfs |
Diff against target: |
56 lines (+14/-3) 2 files modified
debian/changelog (+7/-0) live-build/auto/config (+7/-3) |
To merge this branch: | bzr merge lp:~mwhudson/livecd-rootfs/live-server-gzip-initramfs |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Steve Langasek | Approve | ||
Balint Reczey | Pending | ||
Review via email: mp+341326@code.launchpad.net |
Commit message
Description of the change
I'm fairly sure this will make the boot faster in KVM at least but does anyone have any ideas how to quantify it? Will it be replicated on real hardware?
- 1640. By Michael Hudson-Doyle
-
Set INITRAMFS_
COMPRESSION to gzip for live-server builds to speed up the
boot. (LP: #1750873)
Michael Hudson-Doyle (mwhudson) wrote : | # |
> Tagging Balint for a review, to speak to / quantify the tradeoffs here between
> gzip and lzma (compressed size + time to read initramfs from bootloader, vs.
> time to decompress in kernel).
Thanks.
> There's no support for this yet in livecd-rootfs + live-build
Or mkinitramfs, given that you need to pass -l to lz4 when compressing to get the kernel to understand the initrd.lz4.
> , but this seems
> like an ideal use case for lz4 instead - compressed initrd created once
> server-side, and uncompressed many times
> on each boot of the live image. But we're still a way out from being able to
> make such a change and shouldn't block on it for bionic.
I did try lz4 in my playing around and it did not compress as well as gzip:
-rw-r--r-- 1 root root 148M Mar 12 12:59 initrd
-rw-r--r-- 1 root root 55M Mar 12 13:00 initrd.gz.3
-rw-r--r-- 1 root root 52M Mar 12 13:00 initrd.gz.6
-rw-r--r-- 1 root root 52M Mar 12 13:00 initrd.gz.9
-r--r--r-- 1 root root 33M Mar 11 20:36 initrd.lz
-rw-r--r-- 1 root root 61M Mar 12 13:01 initrd.lz4.3
-rw-r--r-- 1 root root 60M Mar 12 13:01 initrd.lz4.6
-rw-r--r-- 1 root root 60M Mar 12 13:00 initrd.lz4.9
It may have been faster to decompress but it was 0.7s for gzip, 0.4s for lz4 vs 5.2s for lzma sort of numbers.
Getting support for lz4 initrds would seem to involve changing a lot of things (e.g. initrd_suffix in debian-cd) and generally doesn't seem worth it to me at this point in the cycle.
Does the kernel support zstd initrds yet? :)
> Also, aligning with my previous feedback to @rcj, we should really not be
> shadowing the live-build default compression here, but instead ensure
> $INITRAMFS_
> gzip; future, lz4?) DTRT.
So you'd prefer something along the lines of this:
=== modified file 'live-build/
--- live-build/
+++ live-build/
@@ -818,7 +818,7 @@
${KERNEL_
--initsystem none \
--bootloader "$BOOTLOADER" \
- --initramfs-
+ ${INITRAMFS_
--cache false \
${BOOTAPPEND_
$OPTS \
(live-build appears to default to gzip by my reading)? That makes sense but is it safe to land at this point of the cycle?
Steve Langasek (vorlon) wrote : | # |
On Tue, Mar 13, 2018 at 08:38:21PM -0000, Michael Hudson-Doyle wrote:
> > Also, aligning with my previous feedback to @rcj, we should really not be
> > shadowing the live-build default compression here, but instead ensure
> > $INITRAMFS_
> > gzip; future, lz4?) DTRT.
> So you'd prefer something along the lines of this:
> === modified file 'live-build/
> --- live-build/
> +++ live-build/
> @@ -818,7 +818,7 @@
> ${KERNEL_
> --initsystem none \
> --bootloader "$BOOTLOADER" \
> - --initramfs-
> + ${INITRAMFS_
> --cache false \
> ${BOOTAPPEND_
> $OPTS \
Yes, except that you also need to fix up the other image flavor cases to set
lzma where it's currently set.
> (live-build appears to default to gzip by my reading)? That makes sense
> but is it safe to land at this point of the cycle?
Provided we have appropriate code review wrt the above to ensure we are only
changing the value for the server live CD and not accidentally changing it
for any of the others, I think this is fine to land.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://
<email address hidden> <email address hidden>
Michael Hudson-Doyle (mwhudson) wrote : | # |
> On Tue, Mar 13, 2018 at 08:38:21PM -0000, Michael Hudson-Doyle wrote:
> > > Also, aligning with my previous feedback to @rcj, we should really not be
> > > shadowing the live-build default compression here, but instead ensure
> > > $INITRAMFS_
> (currently
> > > gzip; future, lz4?) DTRT.
>
> > So you'd prefer something along the lines of this:
>
> > === modified file 'live-build/
> > --- live-build/
> > +++ live-build/
> > @@ -818,7 +818,7 @@
> > ${KERNEL_
> > --initsystem none \
> > --bootloader "$BOOTLOADER" \
> > - --initramfs-
> > + ${INITRAMFS_
> "$INITRAMFS_
> > --cache false \
> > ${BOOTAPPEND_
> > $OPTS \
>
> Yes, except that you also need to fix up the other image flavor cases to set
> lzma where it's currently set.
Currently not set, I assume you mean? The case statement I am editing in the current MP has 22 cases, only 2 (or 3 in my MP) set INITRAMFS_
> > (live-build appears to default to gzip by my reading)? That makes sense
> > but is it safe to land at this point of the cycle?
>
> Provided we have appropriate code review wrt the above to ensure we are only
> changing the value for the server live CD and not accidentally changing it
> for any of the others, I think this is fine to land.
>
> --
> Steve Langasek Give me a lever long enough and a Free OS
> Debian Developer to set it on, and I can move the world.
> Ubuntu Developer http://
> <email address hidden> <email address hidden>
- 1641. By Michael Hudson-Doyle
-
Make lzma initramfs compression opt-in rather than opt-out, and do not
opt-in to it for live-server builds to speed up the boot. (LP: #1750873)
Steve Langasek (vorlon) wrote : | # |
On Wed, Mar 14, 2018 at 02:50:23AM -0000, Michael Hudson-Doyle wrote:
> > Yes, except that you also need to fix up the other image flavor cases to set
> > lzma where it's currently set.
> Currently not set, I assume you mean?
Yes, precisely :)
> The case statement I am editing in the current MP has 22 cases, only 2 (or
> 3 in my MP) set INITRAMFS_
> add INITRAMFS_
> I can do it if so...
I think it's probably appropriate to add this as a separate case statement
to avoid *quite* so much code duplication; but yes, we should explicitly
call out the products individually where we want to use lzma.
- 1642. By Michael Hudson-Doyle
-
reduce duplication somewhat
Michael Hudson-Doyle (mwhudson) wrote : | # |
> I think it's probably appropriate to add this as a separate case statement
> to avoid *quite* so much code duplication; but yes, we should explicitly
> call out the products individually where we want to use lzma.
Please have another look.
Steve Langasek (vorlon) wrote : | # |
One comment, otherwise LGTM
- 1643. By Michael Hudson-Doyle
-
do not set INITRAMFS_
COMPRESSION for projects that end up passing --initramfs=none to lb config
Steve Langasek (vorlon) : | # |
- 1644. By Michael Hudson-Doyle
-
merge trunk
Balint Reczey (rbalint) wrote : | # |
Zstd initrd support is not there yet in the kernel thus it is not easily backportable to older LTS releases. While lz4 compresses worse the speedup compared to gz is substantial and not all tools have to be changed before 18.04 GA.
I took @vorlon's "does not seem urgent" [1] as a soft-block before 18.04 GA but I would be happier to just add lz4 support and we can provide slightly bigger but faster-booting images.
[1] https:/
Preview Diff
1 | === modified file 'debian/changelog' |
2 | --- debian/changelog 2018-04-10 22:59:37 +0000 |
3 | +++ debian/changelog 2018-04-11 07:11:49 +0000 |
4 | @@ -1,3 +1,10 @@ |
5 | +livecd-rootfs (2.520) UNRELEASED; urgency=medium |
6 | + |
7 | + * Make lzma initramfs compression opt-in rather than opt-out, and do not |
8 | + opt-in to it for live-server builds to speed up the boot. (LP: #1750873) |
9 | + |
10 | + -- Michael Hudson-Doyle <michael.hudson@ubuntu.com> Tue, 13 Mar 2018 21:24:27 +1300 |
11 | + |
12 | livecd-rootfs (2.519) bionic; urgency=medium |
13 | |
14 | [ Robert C Jennings ] |
15 | |
16 | === modified file 'live-build/auto/config' |
17 | --- live-build/auto/config 2018-04-10 22:59:31 +0000 |
18 | +++ live-build/auto/config 2018-04-11 07:11:49 +0000 |
19 | @@ -323,6 +323,12 @@ |
20 | fi |
21 | |
22 | case $PROJECT in |
23 | + base|edubuntu|edubuntu-dvd|kubuntu|kubuntu-active|kubuntu-dvd|kubuntu-plasma5|lubuntu|lubuntu-next|mythbuntu|ubuntu|ubuntu-budgie|ubuntu-desktop-next|ubuntu-dvd|ubuntu-gnome|ubuntu-mate|ubuntu-netbook|ubuntukylin|ubuntustudio-dvd|xubuntu) |
24 | + INITRAMFS_COMPRESSION="lzma" |
25 | + ;; |
26 | +esac |
27 | + |
28 | +case $PROJECT in |
29 | ubuntu|ubuntu-dvd) |
30 | add_task install minimal standard ubuntu-desktop |
31 | LIVE_TASK='ubuntu-live' |
32 | @@ -559,7 +565,6 @@ |
33 | ;; |
34 | esac |
35 | OPTS="${OPTS:+$OPTS }--linux-packages=none --initramfs=none" |
36 | - INITRAMFS_COMPRESSION="none" |
37 | |
38 | # contains the framework definition |
39 | add_package install ubuntu-core-libs |
40 | @@ -655,7 +660,6 @@ |
41 | OPTS="${OPTS:+$OPTS }--hdd-label=cloudimg-rootfs" |
42 | OPTS="${OPTS:+$OPTS }--ext-resize-blocks=536870912 --ext-block-size=4096" |
43 | OPTS="${OPTS:+$OPTS }--ext-fudge-factor=15" |
44 | - INITRAMFS_COMPRESSION="gzip" |
45 | ;; |
46 | |
47 | *) |
48 | @@ -823,7 +827,7 @@ |
49 | ${KERNEL_FLAVOURS:+--linux-flavours "$KERNEL_FLAVOURS"} \ |
50 | --initsystem none \ |
51 | --bootloader "$BOOTLOADER" \ |
52 | - --initramfs-compression "${INITRAMFS_COMPRESSION:-lzma}" \ |
53 | + ${INITRAMFS_COMPRESSION:+--initramfs-compression "$INITRAMFS_COMPRESSION}" \ |
54 | --cache false \ |
55 | ${BOOTAPPEND_LIVE:+--bootappend-live "$BOOTAPPEND_LIVE"} \ |
56 | $OPTS \ |
Tagging Balint for a review, to speak to / quantify the tradeoffs here between gzip and lzma (compressed size + time to read initramfs from bootloader, vs. time to decompress in kernel).
There's no support for this yet in livecd-rootfs + live-build, but this seems like an ideal use case for lz4 instead - compressed initrd created once server-side, and uncompressed many times
on each boot of the live image. But we're still a way out from being able to make such a change and shouldn't block on it for bionic.
Also, aligning with my previous feedback to @rcj, we should really not be shadowing the live-build default compression here, but instead ensure $INITRAMFS_ COMPRESSION is unset for cases where live-build defaults (currently gzip; future, lz4?) DTRT.