[Impact]
* Livecd-rootfs switches from using pxz to xz
* Pxz is installed at run-time instead of being a package dependency and pxz is in universe despite livecd-rootfs being in main. Both of those are undesired and fixing them makes livecd-rootfs easier to maintain following best practice and policy in Ubuntu.
[Test Case]
* Livecd-rootfs in Xenial uses (p)xz only for compressing raspi2 images.
To test the change install updated livecd-rootfs on armhf and create images setting the following environment variables: ARCH=armhf SUBARCH=raspi2 PROJECT=ubuntu-cpc.
Notice that the xz-compressed rapi2 image is still created.
* Test-build Xenial image and make sure they are not broken. A subset of those images are built in the autopkgtest suite.
[Regression Potential]
* It is a known and accepted regression that raspi2 image generation takes more wall-clock time after switching to xz because while Xenial's xz version does accept the -T4 option it does not compress in a parallel manner.
* The original change in Artful set -0 compression for xz which resulted in ~35% bigger compressed images. This regression is fixed in LP: #1732681 which is scheduled to be back-ported together with switching to xz.
* In case of this change causes regression it would manifest itself as failing build for raspi2, failing build for ubuntu-cpc images or increased raspi2 image size.
[Original bug report text]
Livecd-rootfs (main) 2.447 uses "pxz -T4" for xz compression, while pxz is not in main.
Xz 5.2 also supports multi-threaded compression thus xz could easily replace current pxz usage.
A quick test on compressing livecd.ubuntu-cpc.img (created by ubuntu-image) shows that there is very little difference in compressed file size while xz's -T4 brings substantial speedup in wall clock time:
$ cat test-xz
#!/bin/sh
TESTFILE=$1
echo "Compr.\tThreads\tWall.t\tUser.t\tSys.t\tRes.t" > times.txt
echo "Wall.t\tUser.t\tSys.t\tRes.t" > decompression-times.txt
echo "Size(kB)" > sizes.txt
for threads in 1 4; do
for compression in $(seq 0 9) $(seq -f '%0.fe' 0 9); do
/usr/bin/time -a -o times.txt -f "-$compression\t -T$threads\t%e\t%U\t%S\t%M" -- xz -T$threads -$compression -k $TESTFILE
/usr/bin/time -a -o decompression-times.txt -f "%e\t%U\t%S\t%M" -- xz -t $TESTFILE.xz
ls -k -s $TESTFILE.xz | cut -d' ' -f1 >> sizes.txt
rm $TESTFILE.xz
done
done
paste times.txt decompression-times.txt sizes.txt > time-size.txt
$ cat time-size.txt
Compr. Threads Wall.t User.t Sys.t Res.t Wall.t User.t Sys.t Res.t Size(kB)
-0 -T1 63.48 63.07 0.40 4932 1.25 1.16 0.08 2432 416324
-1 -T1 102.90 102.26 0.63 11064 1.53 1.26 0.14 3252 414624
-2 -T1 131.04 130.40 0.62 18544 1.43 1.27 0.09 4224 414616
-3 -T1 146.22 145.61 0.60 33992 1.43 1.24 0.12 6364 414616
-4 -T1 142.24 141.76 0.47 50488 1.34 1.21 0.10 6424 414584
-5 -T1 147.81 147.26 0.53 97616 1.37 1.22 0.10 10420 414564
-6 -T1 148.61 148.06 0.54 97608 1.35 1.23 0.10 10536 414564
-7 -T1 154.29 153.70 0.58 191800 1.44 1.22 0.14 18820 414564
-8 -T1 155.82 155.16 0.65 380336 1.53 1.27 0.13 35208 414564
-9 -T1 194.08 193.18 0.88 691128 1.59 1.31 0.13 67700 414444
-0e -T1 76.81 76.31 0.48 6296 1.27 1.21 0.05 2536 416256
-1e -T1 116.49 115.96 0.53 15068 1.28 1.20 0.07 3232 414568
-2e -T1 137.13 136.62 0.50 26892 1.30 1.20 0.10 4292 414564
-3e -T1 144.34 143.84 0.49 50556 1.31 1.21 0.09 6340 414564
-4e -T1 146.93 146.48 0.44 50412 1.30 1.20 0.10 6340 414568
-5e -T1 150.59 150.04 0.53 97552 1.32 1.22 0.10 10416 414564
-6e -T1 153.40 152.77 0.62 97564 1.32 1.22 0.09 10404 414568
-7e -T1 157.10 156.46 0.62 191956 1.35 1.26 0.08 18548 414568
-8e -T1 160.36 159.66 0.69 380248 1.46 1.25 0.13 35040 414568
-9e -T1 198.32 197.42 0.88 691256 1.49 1.19 0.14 67648 414448
-0 -T4 21.97 86.83 0.40 24896 1.28 1.22 0.05 2448 416368
-1 -T4 35.73 140.12 0.45 73456 1.29 1.22 0.06 3136 414640
-2 -T4 38.15 149.36 0.54 141044 1.30 1.21 0.08 4160 414624
-3 -T4 40.74 153.95 0.55 276336 1.30 1.22 0.08 6216 414620
-4 -T4 40.02 152.20 0.63 341980 1.38 1.23 0.11 6352 414588
-5 -T4 41.68 153.74 1.10 676656 1.49 1.27 0.10 10336 414564
-6 -T4 42.70 157.64 1.19 677144 1.48 1.25 0.13 10436 414564
-7 -T4 47.70 157.42 2.24 1330432 1.60 1.32 0.08 18480 414560
-8 -T4 74.56 178.48 13.32 1676392 1.69 1.34 0.10 35000 414560
Command terminated by signal 9 Command exited with non-zero status 1 0
-9 -T4 19.60 41.90 5.28 1656776 0.01 0.00 0.00 2076 416296
-0e -T4 30.23 117.70 0.56 30596 1.29 1.22 0.06 2444 414584
-1e -T4 37.16 144.83 0.51 90304 1.31 1.21 0.10 3272 414572
-2e -T4 38.73 151.84 0.50 174124 1.32 1.24 0.08 4296 414568
-3e -T4 40.78 156.10 0.65 342012 1.33 1.27 0.05 6432 414572
-4e -T4 42.03 161.22 0.62 342108 1.33 1.24 0.09 6396 414564
-5e -T4 43.10 160.05 1.08 676732 1.47 1.30 0.07 10448 414568
-6e -T4 43.17 159.25 1.31 676584 1.55 1.27 0.12 10328 414564
-7e -T4 48.12 162.80 2.09 1330552 1.60 1.29 0.12 18596 414568
-8e -T4 77.13 182.30 13.98 1673832 1.69 1.34 0.10 34920 0
Command terminated by signal 9 Command exited with non-zero status 1
-9e -T4 18.92 44.75 4.78 1662540 0.02 0.00 0.00 2196
This bug was fixed in the package livecd-rootfs - 2.448
---------------
livecd-rootfs (2.448) artful; urgency=medium
[ Balint Reczey ]
* Depend on packages in main instead of installing them at runtime
* Depend on vmdk-stream-
* Use xz -T4 -0 instead of pxz (LP: #1701132)
Supported xz versions before 5.2 also accept the -T4 option but ignore
it.
* Depend on xz-utils instead of pxz
[ Mathieu Trudel-Lapierre ]
* auto/config: enable Debug::
dependency bugs in the livefs is a little clearer.
[ Robert C Jennings ]
* live-build/
-- Brian Murray <email address hidden> Mon, 24 Jul 2017 11:06:24 -0700