unmkinitramfs fails with lz4 compressed initrds

Bug #1832108 reported by Dimitri John Ledkov
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
initramfs-tools (Debian)
Fix Released
Unknown
initramfs-tools (Ubuntu)
Fix Released
High
Unassigned
Cosmic
In Progress
High
Unassigned
Disco
Fix Released
High
Unassigned

Bug Description

[Impact]

 * Cannot unpack initrds compressed with lz4 due to changes in lz4

[Test Case]

$ sudo apt install initramfs-tools lz4 file
$ mkinitramfs -c lz4 -o foo.img
$ lsinitramfs foo.img
cpio: premature end of archive

[Regression Potential]

 * New lz4cat is more strict w.r.t. enforcing file name extensions, thus the fix is to feed a stream to lz4cat instead of asking it to open a file. The performance impact should be similar between both methods of reading file contents during ls/unmk-initramfs time.

[Other Info]

Original bug report:

unmkinitramfs fails with lz4 compressed initrds

Note:
$ lz4cat -t unmkinitramfs_Cz6Yl9
File extension doesn't match expected LZ4_EXTENSION (.lz4); will not process file: unmkinitramfs_Cz6Yl9

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package initramfs-tools - 0.133ubuntu8

---------------
initramfs-tools (0.133ubuntu8) eoan; urgency=medium

  * Switch back to lz4 by default.
  * Patch unmkinitramfs to cat possible lz4 archives first, as lz4 is
    particular about enforcing .lz4 file extensions when operating on
    files. LP: #1832108

 -- Dimitri John Ledkov <email address hidden> Mon, 10 Jun 2019 00:21:17 +0100

Changed in initramfs-tools (Ubuntu):
status: New → Fix Released
Revision history for this message
Balint Reczey (rbalint) wrote :

This seems to be working on Bionic:

root@bb-qemu:~# echo asd | lz4 > asd
root@bb-qemu:~# file asd
asd: LZ4 compressed data (v1.4+)
root@bb-qemu:~# lz4cat -t asd
root@bb-qemu:~# echo $?
0

Changed in initramfs-tools (Ubuntu Bionic):
status: New → Invalid
Changed in initramfs-tools (Ubuntu Cosmic):
status: New → Confirmed
Changed in initramfs-tools (Ubuntu Disco):
status: New → Confirmed
description: updated
Changed in initramfs-tools (Ubuntu Cosmic):
status: Confirmed → In Progress
Changed in initramfs-tools (Ubuntu Disco):
status: Confirmed → In Progress
Changed in initramfs-tools (Debian):
status: Unknown → New
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Dimitri, or anyone else affected,

Accepted initramfs-tools into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/initramfs-tools/0.131ubuntu19.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in initramfs-tools (Ubuntu Disco):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-disco
Revision history for this message
Brian Murray (brian-murray) wrote :

The cosmic upload is blocked as we are waiting on verification of bug 1778844.

Mathew Hodson (mhodson)
no longer affects: initramfs-tools (Ubuntu Bionic)
tags: added: regression-release
Changed in initramfs-tools (Ubuntu):
importance: Undecided → High
Changed in initramfs-tools (Ubuntu Cosmic):
importance: Undecided → High
Changed in initramfs-tools (Ubuntu Disco):
importance: Undecided → High
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Reproduce the problem:

# dpkg-query -W initramfs-tools lz4
initramfs-tools 0.131ubuntu19
lz4 1.8.3-1ubuntu1

# sed -i 's/gzip/lz4/' /etc/initramfs-tools/initramfs.conf

# update-initramfs -c -k test

# lsinitramfs /boot/initrd.img-test
cpio: premature end of archive

===

# sed 's/-updates/-proposed/' -i /etc/apt/sources.list
# apt update
# dpkg-query -W initramfs-tools lz4
initramfs-tools 0.131ubuntu19.1
lz4 1.8.3-1ubuntu1

# lsinitramfs /boot/initrd.img-test
...
all the files are listed

tags: added: verification-done-disco
removed: verification-needed-disco
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package initramfs-tools - 0.131ubuntu19.1

---------------
initramfs-tools (0.131ubuntu19.1) disco; urgency=medium

  * Patch unmkinitramfs to cat possible lz4 archives first, as lz4 is
    particular about enforcing .lz4 file extensions when operating on
    files. LP: #1832108

 -- Dimitri John Ledkov <email address hidden> Tue, 11 Jun 2019 16:17:51 +0100

Changed in initramfs-tools (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for initramfs-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Testing on cm3

switch to partitions #0, OK
mmc0(part 0) is current device
LOADBOOTENV
** Unable to read file uEnv.txt **
Running uenvcmd ...
ENVCMD
6644040 bytes read in 282 ms (22.5 MiB/s) (this is kernel)
2852487 bytes read in 123 ms (22.1 MiB/s) (this is initrd)
FAT: Misaligned buffer address (023b8687)
42049 bytes read in 14 ms (2.9 MiB/s)

2019-07-04T15:35:05.998027+0000 localhost kernel: Unpacking initramfs...
2019-07-04T15:35:05.998089+0000 localhost kernel: Freeing initrd memory: 2828K

(not sure if above is true gzip decompression speed of initramfs or not)

Watching telnet, from reboot until "Press enter to configure" took 41.84s (42.25s 41.50s).

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

$ file initrd.img
initrd.img: LZMA compressed data, streamed

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

 ls -latr initrd-lzma.img initrd-uncompressed.img initrd-gzip.img initrd-lz49l.img | sort
-rwxr-xr-x 1 xnox xnox 2852487 Jul 5 00:19 initrd-lzma.img
-rwxr-xr-x 1 xnox xnox 3889013 Jul 5 00:21 initrd-gzip.img
-rwxr-xr-x 1 xnox xnox 4466915 Jul 5 00:22 initrd-lz49l.img
-rwxr-xr-x 1 xnox xnox 7330304 Jul 5 00:20 initrd-uncompressed.img

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

uncompressed:

7330304 bytes read in 311 ms (22.5 MiB/s)
reboot time 39.39s
2019-07-04T16:54:38.050180+0000 localhost kernel: Unpacking initramfs...
2019-07-04T16:54:38.050253+0000 localhost kernel: Freeing initrd memory: 7200K

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

lz4

6644040 bytes read in 282 ms (22.5 MiB/s)
4466915 bytes read in 191 ms (22.3 MiB/s)
reboot time 39.49s
2019-07-04T16:54:38.012115+0000 localhost kernel: Unpacking initramfs...
2019-07-04T16:54:38.012178+0000 localhost kernel: Initramfs unpacking failed: Decoding failed
2019-07-04T16:54:38.012246+0000 localhost kernel: Freeing initrd memory: 4404K

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

gzip

6644040 bytes read in 283 ms (22.4 MiB/s)
3889013 bytes read in 167 ms (22.2 MiB/s)
reboot time 40.31s
2019-07-04T16:54:37.997158+0000 localhost kernel: Unpacking initramfs...
2019-07-04T16:54:37.997230+0000 localhost kernel: Freeing initrd memory: 3840K

Revision history for this message
Mario Limonciello (superm1) wrote :

@xnox,

Could you SRU this to bionic too? It becomes problematic to try to probe at an initramfs file from eoan when you're in bionic otherwise.

Changed in initramfs-tools (Debian):
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.