debian-cd

Merge lp:~jibel/debian-cd/support_for_multilayer_images into lp:~ubuntu-cdimage/debian-cd/ubun3

  1. support_for_multilayer_images
  2. Merge into ubun3
Proposed by Jean-Baptiste Lallement
Status: Merged
Merged at revision: 2020
Proposed branch: lp:~jibel/debian-cd/support_for_multilayer_images
Merge into: lp:~ubuntu-cdimage/debian-cd/ubun3
Diff against target: 308 lines (+78/-54)
6 files modified
Makefile (+8/-1)
tools/add_live_filesystem (+30/-35)
tools/boot/disco/boot-amd64 (+12/-10)
tools/boot/disco/boot-arm64 (+9/-7)
tools/boot/disco/boot-s390x (+3/-1)
tools/boot/disco/common.sh (+16/-0)
To merge this branch: bzr merge lp:~jibel/debian-cd/support_for_multilayer_images
Reviewer Review Type Date Requested Status
Steve Langasek Approve
Review via email: mp+359228@code.launchpad.net

Commit message

Support for multi layers images

- Do not hardcode build artifact names.
- Download build artifacts to their own subdirectory for multilayer images
- Keep backward compatibility with "flat" images
- Add boot option layerfs-path for images that support casper to boot multilayer images

To post a comment you must log in.
lp:~jibel/debian-cd/support_for_multilayer_images updated
2007. By Adam Conrad

[xnox, r=adconrad] server-live: fixup s390x iso boot

2008. By Steve Langasek

Merge lp:~mwhudson/debian-cd/kernel-bits

2009. By Steve Langasek

Merge lp:~mwhudson/debian-cd/kernel-bits

Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

Merged trunk.

Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

Merged lp:~jibel/debian-cd/rename_artefacts_made_generic :
* Makefile, tools/add_live_filesystem: Generised renaming of build artefacts
* tools/add_live_filesystem: Use system's unmkinitramfs if version supports multipart initrd. Fallback to locally installed otherwise

lp:~jibel/debian-cd/support_for_multilayer_images updated
2010. By Adam Conrad

bionic: DEBVERSION=18.04.2, Release->Beta

2011. By Adam Conrad

bionic: BACKPORT_KERNEL="hwe-18.04"

2012. By Adam Conrad

Forward-port r1965 to all newer series.

Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

This last update adds the option layerfs-path to boot a multi-layer image.

lp:~jibel/debian-cd/support_for_multilayer_images updated
2013. By Adam Conrad

bionic: Beta->Release

2014. By Adam Conrad

bionic: Add HWE bootloader bits to live-server.

2015. By Adam Conrad

bionic: Release->Beta

2016. By Łukasz Zemczak

xenial: DEBVERSION=16.04.6

2017. By Łukasz Zemczak

trusty: DEBVERSION=14.04.6

Revision history for this message
Steve Langasek (vorlon) wrote :

This needs rebased against current trunk - in particular, revno 2014 introduces changes to add handling of hwe squashfs in addition to generic, which I noticed because the squashfs-generic handling in your branch doesn't handle the other possible kernel flavors that I expected it to.

review: Needs Fixing
lp:~jibel/debian-cd/support_for_multilayer_images updated
2018. By Steve Langasek

Prune dead code: we never produce artifacts called vmlinuz*.efi.signed with recent versions of livecd-rootfs, just expect our input filename to be sensible

2019. By Steve Langasek

Merge lp:~xnox/debian-cd/server-live-ppc64le

2020. By Jean-Baptiste Lallement

Rebased on trunk r2014

2021. By Jean-Baptiste Lallement

- Removed possible unecessary double-spaces on kernel command line
- fixed typo in comment

2022. By Jean-Baptiste Lallement

Do not ignore failures from ln

2023. By Jean-Baptiste Lallement

Be more on strict on squashfs-generic extension match

2024. By Jean-Baptiste Lallement

Revert unmkinitramfs specialization

Revision history for this message
Didier Roche-Tolomelli (didrocks) :
Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

Thanks for the review.

The branch has been rebased on trunk and updated according to your comments.
We tested a build or ubuntu-desktop.

Revision history for this message
Steve Langasek (vorlon) wrote :

thanks, only minor changes requested.

review: Needs Fixing
lp:~jibel/debian-cd/support_for_multilayer_images updated
2025. By Jean-Baptiste Lallement

* Explain non-obvious file naming in file renaming code
* Changed globing of *.squashfs-generic

Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote :

New pushed version should address previous comments.

Revision history for this message
Steve Langasek (vorlon) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Makefile'
2--- Makefile 2018-02-23 13:07:01 +0000
3+++ Makefile 2019-03-12 08:32:44 +0000
4@@ -252,7 +252,11 @@
5 ifeq ($(PROJECT),kubuntu-mobile)
6 CDBASE = $(CODENAME)-mobile-$(FULLARCH)
7 else
8+ ifeq ($(SUBPROJECT),canary)
9+ CDBASE = $(CODENAME)-desktop-canary-$(FULLARCH)
10+ else
11 CDBASE = $(CODENAME)-desktop-$(FULLARCH)
12+ endif
13 endif
14 endif
15 endif
16@@ -1105,7 +1109,10 @@
17 fi; \
18 done
19 ifeq ($(LIVE_FILESYSTEM),1)
20- -cp -a $(LIVEIMAGES)/$(FULLARCH).manifest $(OUT)/$(call CDBASE,$$n).manifest
21+ -for p in $(LIVEIMAGES)/$(FULLARCH)*.manifest; do \
22+ f=`basename $$p`; \
23+ cp -a $$p $(OUT)/$(call CDBASE,$$n).`echo $$f|cut -d. -f2-`; \
24+ done
25 -if [ -e $(LIVEIMAGES)/$(FULLARCH).manifest-remove ]; then \
26 cp -a $(LIVEIMAGES)/$(FULLARCH).manifest-remove $(OUT)/$(call CDBASE,$$n).manifest-remove; \
27 elif [ -e $(LIVEIMAGES)/$(FULLARCH).manifest-desktop ]; then \
28
29=== modified file 'tools/add_live_filesystem'
30--- tools/add_live_filesystem 2019-02-14 06:07:23 +0000
31+++ tools/add_live_filesystem 2019-03-12 08:32:44 +0000
32@@ -10,40 +10,36 @@
33 mkdir -p "$BDIR/CD1/$DIR"
34 if [ -e "$LIVEIMAGES/$FULLARCH.cloop" ]; then
35 ln "$LIVEIMAGES/$FULLARCH.cloop" "$BDIR/CD1/$DIR/filesystem.cloop"
36-elif [ -e "$LIVEIMAGES/$FULLARCH.squashfs" ]; then
37- ln "$LIVEIMAGES/$FULLARCH.squashfs" "$BDIR/CD1/$DIR/filesystem.squashfs"
38- if [ -e "$LIVEIMAGES/$FULLARCH.size" ]; then
39- ln "$LIVEIMAGES/$FULLARCH.size" "$BDIR/CD1/$DIR/filesystem.size"
40- fi
41- if [ -e "$LIVEIMAGES/$FULLARCH.squashfs.gpg" ]; then
42- ln "$LIVEIMAGES/$FULLARCH.squashfs.gpg" "$BDIR/CD1/$DIR/filesystem.squashfs.gpg"
43- fi
44- if [ -e "$LIVEIMAGES/$FULLARCH.maas-rack.squashfs" ]; then
45- mkdir -p "$BDIR/CD1/$DIR/maas/"
46- ln "$LIVEIMAGES/$FULLARCH.maas-rack.squashfs" "$BDIR/CD1/$DIR/maas/maas-rack.squashfs"
47- fi
48- if [ -e "$LIVEIMAGES/$FULLARCH.maas-region.squashfs" ]; then
49- mkdir -p "$BDIR/CD1/$DIR/maas/"
50- ln "$LIVEIMAGES/$FULLARCH.maas-region.squashfs" "$BDIR/CD1/$DIR/maas/maas-region.squashfs"
51- fi
52- if [ -e "$LIVEIMAGES/$FULLARCH.modules.squashfs-generic" ]; then
53- mkdir -p "$BDIR/CD1/$DIR/extras/"
54- ln "$LIVEIMAGES/$FULLARCH.modules.squashfs-generic" "$BDIR/CD1/$DIR/extras/modules.squashfs-generic"
55- fi
56- if [ -e "$LIVEIMAGES/$FULLARCH.modules.squashfs-generic-hwe" ]; then
57- mkdir -p "$BDIR/CD1/$DIR/extras/"
58- ln "$LIVEIMAGES/$FULLARCH.modules.squashfs-generic-hwe" "$BDIR/CD1/$DIR/extras/modules.squashfs-generic-hwe"
59- fi
60-
61- if [ -e "$LIVEIMAGES/$FULLARCH.installer.squashfs" ]; then
62- ln "$LIVEIMAGES/$FULLARCH.installer.squashfs" "$BDIR/CD1/$DIR/installer.squashfs"
63- if [ -e "$LIVEIMAGES/$FULLARCH.installer.size" ]; then
64- ln "$LIVEIMAGES/$FULLARCH.installer.size" "$BDIR/CD1/$DIR/installer.size"
65- fi
66- if [ -e "$LIVEIMAGES/$FULLARCH.installer.squashfs.gpg" ]; then
67- ln "$LIVEIMAGES/$FULLARCH.installer.squashfs.gpg" "$BDIR/CD1/$DIR/installer.squashfs.gpg"
68- fi
69- fi
70+elif [ -n "$(ls $LIVEIMAGES/$FULLARCH*.squashfs 2>/dev/null)" ]; then
71+ for ext in squashfs squashfs.gpg modules.squashfs-generic modules.squashfs-generic-hwe size manifest; do
72+ for origfilename in $(ls $LIVEIMAGES/$FULLARCH*.$ext 2>/dev/null); do
73+ # Strip fullarch from file basename and optionally prepend "filesystem."
74+ # For instance:
75+ # <arch>.filename.<ext> -> filename.<ext>
76+ # <arch>.<ext> -> filesystem.<ext>
77+ origfilename="$(basename ${origfilename})"
78+ filename=${origfilename#$FULLARCH.}
79+ if [ "$filename" = "$ext" ]; then
80+ filename="filesystem.$filename"
81+ fi
82+
83+ # Copy in squashfs corresponding subdirectory for multi-layer with sublayers system
84+ case $filename in
85+ maas*)
86+ filepath="maas/$filename"
87+ ;;
88+ *.modules.squashfs-*)
89+ filepath="extras/$filename"
90+ ;;
91+ *)
92+ filepath="$filename"
93+ ;;
94+ esac
95+
96+ mkdir -p $BDIR/CD1/$DIR/$(dirname "$filepath")
97+ ln "$LIVEIMAGES/$origfilename" "$BDIR/CD1/$DIR/$filepath"
98+ done
99+ done
100 else
101 echo "No filesystem for $FULLARCH!" >&2
102 fi
103@@ -57,7 +53,6 @@
104 fi
105 done
106 done
107-ln "$LIVEIMAGES/$FULLARCH.manifest" "$BDIR/CD1/$DIR/filesystem.manifest" || true
108 if [ -e "$LIVEIMAGES/$FULLARCH.manifest-remove" ]; then
109 ln "$LIVEIMAGES/$FULLARCH.manifest-remove" "$BDIR/CD1/$DIR/filesystem.manifest-remove" || true
110 elif [ -e "$LIVEIMAGES/$FULLARCH.manifest-desktop" ]; then
111
112=== modified file 'tools/boot/disco/boot-amd64'
113--- tools/boot/disco/boot-amd64 2019-03-08 16:59:14 +0000
114+++ tools/boot/disco/boot-amd64 2019-03-12 08:32:44 +0000
115@@ -440,23 +440,25 @@
116 liveparams=
117 esac
118
119+ LAYERFS_PATH_OPT=$(add_layerfs_path_opt)
120+
121 cat >> boot$N/isolinux/txt.cfg <<EOF
122 label live
123 menu label $livelabel
124 kernel $CASPER_KERNEL
125- append $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams initrd=$CASPER_INITRD $QUIET $SPLASH ---
126+ append $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams$LAYERFS_PATH_OPT initrd=$CASPER_INITRD $QUIET $SPLASH ---
127 EOF
128 cat >> $CDDIR/boot/grub/grub.cfg <<EOF
129 menuentry "${livelabel#^}" {
130 set gfxpayload=keep
131- linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams $QUIET $SPLASH ---
132+ linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams$LAYERFS_PATH_OPT $QUIET $SPLASH ---
133 initrd $CASPER_INITRD
134 }
135 EOF
136 cat >> $CDDIR/boot/grub/loopback.cfg <<EOF
137 menuentry "${livelabel#^}" {
138 set gfxpayload=keep
139- linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams iso-scan/filename=\${iso_path} $QUIET $SPLASH ---
140+ linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams$LAYERFS_PATH_OPT iso-scan/filename=\${iso_path} $QUIET $SPLASH ---
141 initrd $CASPER_INITRD
142 }
143 EOF
144@@ -465,18 +467,18 @@
145 label live-install
146 menu label ^Install $HUMANPROJECT
147 kernel $CASPER_KERNEL
148- append $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper only-ubiquity initrd=$CASPER_INITRD $QUIET $SPLASH ---
149+ append $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$LAYERFS_PATH_OPT only-ubiquity initrd=$CASPER_INITRD $QUIET $SPLASH ---
150 EOF
151 cat >> $CDDIR/boot/grub/grub.cfg <<EOF
152 menuentry "Install $HUMANPROJECT" {
153 set gfxpayload=keep
154- linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper only-ubiquity $QUIET $SPLASH ---
155+ linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$LAYERFS_PATH_OPT only-ubiquity $QUIET $SPLASH ---
156 initrd $CASPER_INITRD
157 }
158 EOF
159 cat >> $CDDIR/boot/grub/loopback.cfg <<EOF
160 menuentry "Install $HUMANPROJECT" {
161- linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper only-ubiquity iso-scan/filename=\${iso_path} $QUIET $SPLASH ---
162+ linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$LAYERFS_PATH_OPT only-ubiquity iso-scan/filename=\${iso_path} $QUIET $SPLASH ---
163 initrd $CASPER_INITRD
164 }
165 EOF
166@@ -556,7 +558,7 @@
167 cat >> $CDDIR/boot/grub/${!kp}grub.cfg <<EOF
168 menuentry "OEM install (for manufacturers)" {
169 set gfxpayload=keep
170- linux $CASPER_KERNEL $KERNEL_PARAMS ${DEFAULT_PRESEED/$PREFIX_PRESEED/$PREFIX_PRESEED${!kp}} boot=casper only-ubiquity quiet splash oem-config/enable=true ---
171+ linux $CASPER_KERNEL $KERNEL_PARAMS ${DEFAULT_PRESEED/$PREFIX_PRESEED/$PREFIX_PRESEED${!kp}} boot=casper$LAYERFS_PATH_OPT only-ubiquity quiet splash oem-config/enable=true ---
172 initrd $CASPER_INITRD
173 }
174 EOF
175@@ -640,18 +642,18 @@
176 label check
177 menu label ^Check disc for defects
178 kernel $CASPER_KERNEL
179- append $KERNEL_PARAMS boot=casper integrity-check initrd=$CASPER_INITRD quiet splash ---
180+ append $KERNEL_PARAMS boot=casper$LAYERFS_PATH_OPT integrity-check initrd=$CASPER_INITRD quiet splash ---
181 EOF
182 cat >> $CDDIR/boot/grub/grub.cfg <<EOF
183 menuentry "Check disc for defects" {
184 set gfxpayload=keep
185- linux $CASPER_KERNEL $KERNEL_PARAMS boot=casper integrity-check quiet splash ---
186+ linux $CASPER_KERNEL $KERNEL_PARAMS boot=casper$LAYERFS_PATH_OPT integrity-check quiet splash ---
187 initrd $CASPER_INITRD
188 }
189 EOF
190 cat >> $CDDIR/boot/grub/loopback.cfg <<EOF
191 menuentry "Check disc for defects" {
192- linux $CASPER_KERNEL $KERNEL_PARAMS boot=casper integrity-check iso-scan/filename=\${iso_path} quiet splash ---
193+ linux $CASPER_KERNEL $KERNEL_PARAMS boot=casper$LAYERFS_PATH_OPT integrity-check iso-scan/filename=\${iso_path} quiet splash ---
194 initrd $CASPER_INITRD
195 }
196 EOF
197
198=== modified file 'tools/boot/disco/boot-arm64'
199--- tools/boot/disco/boot-arm64 2018-10-30 13:12:35 +0000
200+++ tools/boot/disco/boot-arm64 2019-03-12 08:32:44 +0000
201@@ -163,6 +163,8 @@
202 EOF
203 fi
204
205+LAYERFS_PATH_OPT=$(add_layerfs_path_opt)
206+
207 if [ "$CDIMAGE_LIVE" = 1 ]; then
208 if [ "$PROJECT" = kubuntu ] || [ "$PROJECT" = kubuntu-plasma5 ]; then
209 livelabel="^Start $HUMANPROJECT"
210@@ -174,14 +176,14 @@
211 cat >> $CDDIR/boot/grub/grub.cfg <<EOF
212 menuentry "${livelabel#^}" {
213 set gfxpayload=keep
214- linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams quiet splash ---
215+ linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams$LAYERFS_PATH_OPT quiet splash ---
216 initrd $CASPER_INITRD
217 }
218 EOF
219 cat >> $CDDIR/boot/grub/loopback.cfg <<EOF
220 menuentry "${livelabel#^}" {
221 set gfxpayload=keep
222- linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams iso-scan/filename=\${iso_path} quiet splash ---
223+ linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$liveparams$LAYERFS_PATH_OPT iso-scan/filename=\${iso_path} quiet splash ---
224 initrd $CASPER_INITRD
225 }
226 EOF
227@@ -189,13 +191,13 @@
228 cat >> $CDDIR/boot/grub/grub.cfg <<EOF
229 menuentry "Install $HUMANPROJECT" {
230 set gfxpayload=keep
231- linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper only-ubiquity quiet splash ---
232+ linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$LAYERFS_PATH_OPT only-ubiquity quiet splash ---
233 initrd $CASPER_INITRD
234 }
235 EOF
236 cat >> $CDDIR/boot/grub/loopback.cfg <<EOF
237 menuentry "Install $HUMANPROJECT" {
238- linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper only-ubiquity iso-scan/filename=\${iso_path} quiet splash ---
239+ linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$LAYERFS_PATH_OPT only-ubiquity iso-scan/filename=\${iso_path} quiet splash ---
240 initrd $CASPER_INITRD
241 }
242 EOF
243@@ -230,7 +232,7 @@
244 cat >> $CDDIR/boot/grub/grub.cfg <<EOF
245 menuentry "OEM install (for manufacturers)" {
246 set gfxpayload=keep
247- linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper only-ubiquity quiet splash oem-config/enable=true ---
248+ linux $CASPER_KERNEL $KERNEL_PARAMS $DEFAULT_PRESEED boot=casper$LAYERFS_PATH_OPT only-ubiquity quiet splash oem-config/enable=true ---
249 initrd $CASPER_INITRD
250 }
251 EOF
252@@ -268,13 +270,13 @@
253 cat >> $CDDIR/boot/grub/grub.cfg <<EOF
254 menuentry "Check disc for defects" {
255 set gfxpayload=keep
256- linux $CASPER_KERNEL $KERNEL_PARAMS boot=casper integrity-check quiet splash ---
257+ linux $CASPER_KERNEL $KERNEL_PARAMS boot=casper$LAYERFS_PATH_OPT integrity-check quiet splash ---
258 initrd $CASPER_INITRD
259 }
260 EOF
261 cat >> $CDDIR/boot/grub/loopback.cfg <<EOF
262 menuentry "Check disc for defects" {
263- linux $CASPER_KERNEL $KERNEL_PARAMS boot=casper integrity-check iso-scan/filename=\${iso_path} quiet splash ---
264+ linux $CASPER_KERNEL $KERNEL_PARAMS boot=casper$LAYERFS_PATH_OPT integrity-check iso-scan/filename=\${iso_path} quiet splash ---
265 initrd $CASPER_INITRD
266 }
267 EOF
268
269=== modified file 'tools/boot/disco/boot-s390x'
270--- tools/boot/disco/boot-s390x 2018-12-23 18:12:29 +0000
271+++ tools/boot/disco/boot-s390x 2019-03-12 08:32:44 +0000
272@@ -53,9 +53,11 @@
273 cp -lf "$DI_DIR/generic/initrd.off" $imagedir
274 cp -lf "$DI_DIR/generic/initrd.siz" $imagedir
275
276+LAYERFS_PATH_OPT=$(add_layerfs_path_opt)
277+
278 # For server-live images, use casper's initrd and kernel instead
279 if [ "$CDIMAGE_LIVE" = 1 ]; then
280- CMDLINE="boot=casper --- quiet"
281+ CMDLINE="boot=casper$LAYERFS_PATH_OPT --- quiet"
282 mv $CDDIR/casper/filesystem.initrd-generic $imagedir/initrd.ubuntu
283 mv $CDDIR/casper/filesystem.kernel-generic $imagedir/kernel.ubuntu
284 echo "$CMDLINE" > $imagedir/parmfile.ubuntu
285
286=== modified file 'tools/boot/disco/common.sh'
287--- tools/boot/disco/common.sh 2018-10-30 13:12:35 +0000
288+++ tools/boot/disco/common.sh 2019-03-12 08:32:44 +0000
289@@ -158,3 +158,19 @@
290 echo -n "$NEW_OPT " >> $OPTS_FILE
291 fi
292 }
293+
294+# for multi-layer images, add a kernel command line option to set layerfs-path
295+# and select the leaf filesystem to determine the list of layers to merge in
296+# an overlay and boot from.
297+add_layerfs_path_opt() {
298+ case $PROJECT in
299+ ubuntu)
300+ if [ "$SUBPROJECT" = "canary" ]; then
301+ echo ' layerfs-path=minimal.standard.live.squashfs'
302+ return
303+ fi
304+ ;;
305+ *)
306+ ;;
307+ esac
308+}

Subscribers

People subscribed via source and target branches