Merge lp:~rafaelfolco/cirros/cirros into lp:cirros

Proposed by Rafael Folco on 2016-09-20
Status: Needs review
Proposed branch: lp:~rafaelfolco/cirros/cirros
Merge into: lp:cirros
Diff against target: 1072 lines (+1046/-0) (has conflicts)
5 files modified
bin/build-release.OTHER (+265/-0)
bin/bundle.OTHER (+348/-0)
bin/grab-grub-ieee (+100/-0)
bin/part2disk.OTHER (+284/-0)
doc/README-powerpc.txt.OTHER (+49/-0)
Conflict adding files to bin.  Created directory.
Conflict because bin is not versioned, but has versioned children.  Versioned directory.
Contents conflict in bin/build-release
Contents conflict in bin/bundle
Contents conflict in bin/part2disk
Conflict adding files to doc.  Created directory.
Conflict because doc is not versioned, but has versioned children.  Versioned directory.
Contents conflict in doc/README-powerpc.txt
To merge this branch: bzr merge lp:~rafaelfolco/cirros/cirros
Reviewer Review Type Date Requested Status
Scott Moser 2016-09-20 Pending
Review via email: mp+306211@code.launchpad.net

Description of the change

Add support for Grub2 on Power architectures.

Power bootable qcow2 images require Grub to be installed on a PreP Boot partition. This patch enables CirrOS to boot from Grub2/PreP.

To post a comment you must log in.

Unmerged revisions

381. By Rafael Folco on 2016-09-19

Code polishing and Minor fixes

380. By Rafael Folco on 2016-09-01

Download first, build br-source first arch

379. By Rafael Folco on 2016-08-31

Add Grub2 support to Power

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added directory 'bin'
2=== added file 'bin/build-release.OTHER'
3--- bin/build-release.OTHER 1970-01-01 00:00:00 +0000
4+++ bin/build-release.OTHER 2016-09-20 12:52:40 +0000
5@@ -0,0 +1,265 @@
6+#!/bin/bash
7+#
8+# Authors: Scott Moser <smoser@canonical.com>
9+# Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
10+# Rafael Folco <rfolco@br.ibm.com>
11+#
12+# This program is free software: you can redistribute it and/or modify
13+# it under the terms of the GNU General Public License as published by
14+# the Free Software Foundation, version 3 of the License.
15+#
16+# This program is distributed in the hope that it will be useful,
17+# but WITHOUT ANY WARRANTY; without even the implied warranty of
18+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
19+# GNU General Public License for more details.
20+#
21+# You should have received a copy of the GNU General Public License
22+# along with this program. If not, see <http://www.gnu.org/licenses/>.
23+
24+[ $# -ge 1 ] || { echo "must give version"; exit 1; }
25+
26+source "${0%/*}/common-functions.sh"
27+
28+VER=$1
29+daily=false
30+if [ "$1" = "daily" ]; then
31+ VER="d$(date +%y%m%d)"
32+ daily=true
33+fi
34+shift
35+pre=cirros-$VER
36+BR_VER="2015.05"
37+ARCHES="${ARCHES:-i386 x86_64 arm powerpc aarch64 ppc64 ppc64le}"
38+KVER="4.4.0-28.47" # Ubuntu 16.04
39+GVER="2.02~beta2-36ubuntu8"
40+ME=$(readlink -f "$0")
41+MY_D=${ME%/*}
42+PATH=${MY_D}:$PATH
43+
44+set -e
45+set -o pipefail
46+
47+OUT="$PWD/../build-$VER"
48+LOGFILE="$OUT/date.txt"
49+export TMPDIR="$OUT/tmp"
50+mkdir -p "$OUT" "$TMPDIR"
51+
52+msg() {
53+ echo "$@" >> "$LOGFILE"
54+ echo "$@"
55+}
56+logevent() {
57+ # logevent(msg, [from])
58+ # log the message in $1 and time since $2, defaulting to last call.
59+ local up delta msg="$1" ref="$2" human=""
60+ up=${SECONDS}
61+ if [ "$ref" = "-" ]; then
62+ ref=""
63+ elif [ "$ref" = "" ]; then
64+ ref="${_LAST_UP}"
65+ fi
66+ if [ -n "$ref" ]; then
67+ delta="$(($up-$ref))"
68+ sec2human "$delta" tshort && human=${_RET}
69+ fi
70+ msg "$(date -R)" "$msg" "${delta:+Took ${human}. [${delta}s]}"
71+ _LAST_UP=$up
72+ _RET=${_LAST_UP}
73+}
74+
75+build_arch() {
76+ local arch="" quiet=false cmd="" log=""
77+ if [ "$1" = "quiet" ]; then
78+ quiet=true
79+ shift
80+ fi
81+ arch="$1"
82+ log="${OUT}/build-$arch.log"
83+ cmd=( make ARCH=$arch "OUT_D=$OUT/build/$arch"
84+ ${CCACHE_D:+"BR2_CCACHE_DIR=${CCACHE_D}/$arch"} )
85+
86+ logevent "start $arch" -
87+ if $quiet; then
88+ error "building $arch into $log"
89+ time "${cmd[@]}" > "$log" 2>&1
90+ else
91+ time "${cmd[@]}" 2>&1 | tee "$log"
92+ fi
93+ ret=$?
94+ logevent "finish $arch [ret=$ret]"
95+ return $ret
96+}
97+
98+get_grub_format() {
99+ local arch=$1
100+ case "$arch" in
101+ powerpc|ppc*) format=ieee;;
102+ *) format=efi;;
103+ esac
104+ echo $format
105+}
106+
107+get_img_size() {
108+ local arch=$1
109+ case "$arch" in
110+ powerpc|ppc*|aarch64) size=96M;;
111+ *) size="";;
112+ esac
113+ echo $size
114+}
115+
116+if [ "$1" = "build_arch" ]; then
117+ shift
118+ build_arch "$@"
119+ exit
120+elif [ $# -gt 1 ]; then
121+ fail "confused by $# arguments: $*"
122+fi
123+
124+# really just here to check that VER is a tag
125+# or source code checkout would fail
126+if ! $daily; then
127+ revno=$(bzr tags -r "tag:$VER") || fail "$VER: not a tag in $PWD."
128+ revno=$(echo "$revno" | awk '{print $2}')
129+fi
130+
131+logevent "begin" -
132+tstart=${_RET}
133+
134+logevent "start download" -
135+rm -f download
136+mkdir -p ../download
137+ln -snf ../download download
138+brtgz="buildroot-${BR_VER}.tar.gz"
139+dl "http://buildroot.uclibc.org/downloads/$brtgz" "download/$brtgz"
140+logevent "end download"
141+
142+logevent "start unpack" -
143+rm -Rf "buildroot-${BR_VER}"
144+rm -f buildroot
145+tar -xvf download/buildroot-${BR_VER}.tar.gz
146+ln -snf buildroot-${BR_VER} buildroot
147+
148+# we do not do this here, so that we're not dependent on the
149+# cvs working (which wont work through http_proxy) and also
150+# to have revision controlled information in that file.
151+#./bin/mkcabundle > src/etc/ssl/certs/ca-certificates.crt
152+
153+( cd buildroot && QUILT_PATCHES="$PWD/../patches-buildroot" quilt push -a )
154+
155+echo "$VER" > "src/etc/cirros/version"
156+logevent "end unpack"
157+
158+# Stage 1: DOWNLOAD
159+logevent "start br-source" -
160+make ARCH=${ARCHES%% *} br-source
161+logevent "end br-source"
162+
163+for arch in ${ARCHES}; do
164+ format=$(get_grub_format "$arch")
165+ mkdir -p "$OUT/stage/$arch"
166+
167+ # grab kernel
168+ logevent "start kernel download" -
169+ grab-kernels "$KVER" $arch
170+ logevent "end kernel download"
171+
172+ # grab grub
173+ logevent "start grub $arch-$format download" -
174+ grab-grub-${format} "$GVER" $arch
175+ logevent "end grub $arch-$format download"
176+done
177+
178+# STAGE 2: BUILD
179+jobs_flag=""
180+parallel=true
181+case "${CIRROS_PARALLEL:-none}" in
182+ none) parallel=false;;
183+ 0|true) :;;
184+ [0-9]|[0-9][0-9]) jobs_flag="--jobs=${CIRROS_PARALLEL}";;
185+ auto) command -v parallel >/dev/null || parallel=false;;
186+ *) fail "unknown value for CIRROS_PARALLEL=$CIRROS_PARALLEL";;
187+esac
188+
189+if $parallel; then
190+ parallel --ungroup ${jobs_flag} \
191+ "$0" "$VER" build_arch quiet {} ::: ${ARCHES}
192+else
193+ for arch in ${ARCHES}; do
194+ build_arch "$arch"
195+ done;
196+fi
197+
198+# STAGE 3: BUNDLE
199+for arch in ${ARCHES}; do
200+ # bundle rootfs, kernel & grub
201+ format=$(get_grub_format "$arch")
202+ size=$(get_img_size "$arch")
203+ logevent "start bundling $arch" -
204+ sudo ./bin/bundle -v ${size:+--size=$size} --arch="$arch" \
205+ "$OUT/build/$arch/rootfs.tar" \
206+ ./download/kernel-$arch.tar.gz ./download/grub-${format}-$arch.tar.gz \
207+ "$OUT/stage/$arch";
208+ logevent "finish bundling $arch"
209+done
210+
211+sudo chown -R $USER:$USER "$OUT/stage"
212+
213+mkdir -p "$OUT/release"
214+
215+#srctgz="$OUT/release/cirros-$VER-source.tar.gz"
216+#bzr export -r "tag:$VER" --format=tgz --root="cirros-$VER" "$srctgz"
217+#echo "wrote source tgz: $srctgz"
218+if ! $daily; then
219+ ( srcd="$PWD" && tmpd=$(mktemp -d) && cd "$tmpd" &&
220+ bzr branch "$srcd" -r tag:$VER cirros-$VER &&
221+ rm -Rf cirros-$VER/.bzr &&
222+ echo "$VER" > "cirros-$VER/src/etc/cirros/version" &&
223+ tar cvzf - cirros-$VER ) > "$OUT/release/cirros-$VER-source.tar.gz"
224+fi
225+
226+rm -f "$OUT/stage"/*/"$pre"*
227+for arch in ${ARCHES}; do
228+ p=$pre-$arch
229+ ( cd "$OUT/stage/$arch" &&
230+ ln kernel $p-vmlinuz && ln kernel $p-kernel &&
231+ ln initramfs $p-initrd && ln initramfs $p-initramfs &&
232+ ln part.img $p-rootfs.img &&
233+ ln blank.img $p-blank.img &&
234+ ln disk.img $p-disk.img &&
235+ ln filesys.tar.gz $p-lxc.tar.gz &&
236+ true
237+ );
238+done
239+
240+logevent "start populating release" -
241+for arch in ${ARCHES}; do
242+ p=$pre-$arch
243+ ( cd "$OUT/stage/$arch" &&
244+ cp $p-kernel $p-initramfs $p-lxc.tar.gz "$OUT/release/" &&
245+ gzip -9 -c $p-rootfs.img > $OUT/release/$p-rootfs.img.gz ) &&
246+ ( cd "$OUT/stage/$arch" &&
247+ tar cvzf - $p-blank.img $p-vmlinuz $p-initrd
248+ ) > "$OUT/release/$p-uec.tar.gz"
249+ cp "$OUT/stage/$arch/$p-disk.img" "$OUT/release/$p-disk.img"
250+done
251+
252+mkdir -p "$OUT/release/buildroot_rootfs"
253+for arch in ${ARCHES}; do
254+ t="$OUT/release/buildroot_rootfs/buildroot-$VER-$arch.tar"
255+ cp "$OUT/build/$arch/rootfs.tar" "$t"
256+ gzip --force -9 "$t"
257+done
258+
259+chmod u+rwX,go+rX -R "$OUT/release/"*
260+
261+sumfiles=$(cd "$OUT/release" && for f in *; do
262+ [ -f "$f" -a "$f" != MD5SUMS ] && echo "$f"; done)
263+( cd "$OUT/release" && md5sum $sumfiles > MD5SUMS )
264+
265+logevent "finish populate release" -
266+msg "output in $OUT/release"
267+msg "entire process took $SECONDS seconds"
268+logevent "finished" "$tstart"
269+
270+# vi: tabstop=4 expandtab
271
272=== added file 'bin/bundle.OTHER'
273--- bin/bundle.OTHER 1970-01-01 00:00:00 +0000
274+++ bin/bundle.OTHER 2016-09-20 12:52:40 +0000
275@@ -0,0 +1,348 @@
276+#!/bin/bash
277+#
278+# Authors: Scott Moser <smoser@canonical.com>
279+# Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
280+# Rafael Folco <rfolco@br.ibm.com>
281+#
282+# This program is free software: you can redistribute it and/or modify
283+# it under the terms of the GNU General Public License as published by
284+# the Free Software Foundation, version 3 of the License.
285+#
286+# This program is distributed in the hope that it will be useful,
287+# but WITHOUT ANY WARRANTY; without even the implied warranty of
288+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
289+# GNU General Public License for more details.
290+#
291+# You should have received a copy of the GNU General Public License
292+# along with this program. If not, see <http://www.gnu.org/licenses/>.
293+#
294+
295+source "${0%/*}/common-functions.sh"
296+
297+TEMP_D=""
298+UMOUNT=""
299+DEF_SIZE=${DEF_SIZE:-32M}
300+DEF_MODULES="acpiphp e1000 ne2k-pci 8139cp pcnet32 ip_tables"
301+
302+Usage() {
303+ cat <<EOF
304+Usage: ${0##*/} rootfs.tar kpkg.deb output_dir
305+ [re]Bundle a buildroot rootfs into a mini-cloud image
306+
307+ options:
308+ -s | --size S resize image to size (default: ${DEF_SIZE})
309+ --arch A prepare for arch A
310+
311+ Example:
312+ ${0##*/} rootfs.tar linux-image-*-virtuaal*.deb build-output/
313+EOF
314+}
315+cleanup() {
316+ [ -z "${UMOUNT}" ] || umount "${UMOUNT}"
317+ [ -z "${TEMP_D}" -o ! -d "${TEMP_D}" ] || rm -Rf "${TEMP_D}"
318+}
319+xrsync() {
320+ rsync --archive --xattrs --hard-links --acls --sparse "$@"
321+}
322+
323+short_opts="hs:v"
324+long_opts="arch:,initrd-busybox:,help,size:,verbose"
325+getopt_out=$(getopt --name "${0##*/}" \
326+ --options "${short_opts}" --long "${long_opts}" -- "$@") &&
327+ eval set -- "${getopt_out}" ||
328+ bad_Usage
329+
330+topdir=$(cd "${0%/*}/.." && pwd)
331+size=${DEF_SIZE}
332+FS_LABEL="cirros-rootfs"
333+fs_type="ext3"
334+arch=""
335+
336+while [ $# -ne 0 ]; do
337+ cur=${1}; next=${2};
338+ case "$cur" in
339+ -h|--help) Usage; exit 0;;
340+ -s|--size) size=${next}; shift;;
341+ --arch) arch=${next}; shift;;
342+ -v|--verbose) DEBUG=$((${DEBUG}+1));;
343+ --) shift; break;;
344+ esac
345+ shift;
346+done
347+
348+[ $# -eq 4 ] || bad_Usage "must give rootfs.tar, kernel pkg, grub-{efi|ieee}.tar, out_dir"
349+rootfs_in=${1}
350+kpkg_in=${2}
351+grub_in=${3}
352+out_d_in=${4}
353+
354+PATH="$topdir/bin:$PATH"
355+src_dir="${topdir}/src"
356+src_symlinks="${topdir}/symlinks.list"
357+makedevs_list="${topdir}/makedevs.list"
358+fixup_fs="${topdir}/fixup-fs"
359+xgrubd="$topdir/grubd"
360+
361+[ "$(id -u)" = "0" ] || fail "sorry... must be root"
362+
363+[ -d "${src_dir}" ] || fail "no source dir ${src_d}"
364+
365+TEMP_D=$(mktemp -d "${TMPDIR:-/tmp}/.${0##*/}.XXXXXX") ||
366+ fail "failed to make tempd"
367+trap cleanup EXIT
368+
369+mkdir -p "${out_d_in}" && out_d=$(readlink -f "${out_d_in}") &&
370+ rootfs=$(readlink -f "${rootfs_in}") &&
371+ kpkg=$(readlink -f "${kpkg_in}") ||
372+ fail "failed to get full path for input"
373+
374+out_partimg="${out_d}/part.img"
375+out_diskimg="${out_d}/disk.img"
376+out_kernel="${out_d}/kernel"
377+out_initramfs="${out_d}/initramfs"
378+out_diskimg="${out_d}/disk.img"
379+out_blankimg="${out_d}/blank.img"
380+out_filesys_lxc="${out_d}/filesys.tar.gz"
381+inter_d="$out_d/intermediate"
382+
383+mp="${TEMP_D}/mnt"
384+kernel_d="${TEMP_D}/kernel"
385+kern_list_full="${TEMP_D}/kernel.files.full"
386+kern_files="${TEMP_D}/kernel.files"
387+kern_modules="${TEMP_D}/kernel.files.modules"
388+overlay_d="${TEMP_D}/overlay"
389+initramfs_d="${TEMP_D}/initramfs"
390+initramfs="${TEMP_D}/initramfs.img"
391+stage_d="$TEMP_D/staging"
392+
393+kernel_tar="$inter_d/kernel.tar"
394+overlay_tar="$inter_d/overlay.tar"
395+filesys_tar="$inter_d/filesys.tar"
396+
397+modfile="$src_dir/etc/modules"
398+if [ -f "$modfile" ]; then
399+ MODULES=$("$src_dir/etc/init.d/load-modules" parse_modules \
400+ "$modfile" "$arch") ||
401+ fail "failed to read modules"
402+ MODULES=$(echo "$MODULES" | sed 's, .*,,' | tr '\n' ' ')
403+else
404+ MODULES=${DEF_MODULES}
405+fi
406+debug 1 "modules: $MODULES"
407+
408+prepare-grub "$xgrubd" ||
409+ fail "failed to get grub binary"
410+
411+mkdir -p "${mp}" "${kernel_d}" "${overlay_d}" \
412+ "${initramfs_d}" "$inter_d" "$stage_d" ||
413+ fail "failed to make temp dirs"
414+
415+debug 1 "creating filesystem in ${out_partimg}"
416+rm -f "$out_partimg"
417+truncate "--size=${size}" "${out_partimg}" ||
418+ fail "failed to create ${out_partimg} of size ${size}"
419+
420+out=$("mkfs.${fs_type}" -F "${out_partimg}" -L "${FS_LABEL}" 2>&1) ||
421+ fail "failed to make filesystem of type ${fs_type}: ${out}"
422+
423+cp "$out_partimg" "$out_blankimg" ||
424+ fail "failed to to copy blank partition image"
425+
426+debug 1 "preparing kernel overlay"
427+# creating kernel tarball
428+case "$kpkg_in" in
429+ *.deb)
430+ dpkg -x "${kpkg_in}" "${kernel_d}" ||
431+ fail "failed extracting kernel deb package in ${kpkg_in}"
432+ ;;
433+ *.tar*)
434+ tar -C "$kernel_d" -xf "$kpkg_in" ||
435+ fail "failed extracting kernel tarball in ${kpkg_in}"
436+ ;;
437+ *) fail "unknown file format for $kpkg_in";;
438+esac
439+
440+( cd "${kernel_d}" && find * -type f ) > "${kern_list_full}" ||
441+ fail "failed to extract kernel to ${kernel_d}"
442+
443+kver=""
444+for x in "$kernel_d/lib/modules"/*; do
445+ [ -d "$x/kernel" ] || continue
446+ [ -z "$kver" ] ||
447+ fail "2 or more things looked like kernels in lib/modules of $kpkg_in"
448+ kver="${x##*/}"
449+done
450+[ -n "$kver" ] ||
451+ fail "failed to find kernel version. no lib/modules/* ?"
452+
453+depmod -a --basedir "${kernel_d}" "${kver}" ||
454+ fail "failed to run depmod"
455+
456+mdep="${kernel_d}/lib/modules/${kver}/modules.dep"
457+for x in ${MODULES}; do
458+ grep -q "/${x}.ko" "${mdep}" ||
459+ { error "WARNING: no ${x} in kernel package!"; continue; }
460+ awk -F: '$1 ~ mat {
461+ sub(":","",$1)
462+ printf("%s/%s\n",p,$1)
463+ leng=split($0,deps," ")
464+ x=2 # strange, but 0 contains nothing, 1 contains first field (with :)
465+ while ( x<=leng ) {
466+ printf("%s/%s\n", p, deps[x]);
467+ x++
468+ }
469+ }' mat="/${x}.ko$" p="lib/modules/${kver}" "${mdep}"
470+done > "${kern_modules}"
471+sort -u "${kern_modules}" > "${kern_files}"
472+vmlinuz=$( cd "${kernel_d}" && [ -f boot/vmlinu?-* ] &&
473+ echo boot/vmlinu?-* ) && echo "${vmlinuz}" >> "${kern_files}" &&
474+ ln -sf "$vmlinuz" "$kernel_d/vmlinuz" && echo "vmlinuz" >> "$kern_files" ||
475+ fail "no kernel (boot/vmlinuz-*) found in ${kpkg_in}"
476+echo "boot/config-$kver" >> "$kern_files"
477+
478+tar -C "${kernel_d}" -cpf - \
479+ --files-from "${kern_files}" > "${kernel_tar}" ||
480+ fail "failed to collect kernel files"
481+
482+debug 1 "preparing source overlay from ${src_dir}"
483+xrsync "${src_dir}/" "${overlay_d}" ||
484+ fail "failed to copy source dir"
485+
486+chown -R 0:0 "${overlay_d}" || fail "failed to chown files in overlay"
487+
488+if [ -f "${src_symlinks}" ]; then
489+ ( cd "${overlay_d}" &&
490+ while read src target; do
491+ { [ -d "${target%/*}" ] || mkdir -p "${target%/*}"; } ||
492+ { error "could not create ${target%/*}"; exit 1; }
493+ ln -sf "${src}" "${target}" || exit 1
494+ done < "${src_symlinks}"
495+ ) || fail "failed to create symlinks"
496+fi
497+if [ -f "${makedevs_list}" ]; then
498+ xmakedevs "$makedevs_list" "$overlay_d" ||
499+ fail "failed to makedevs on overlay"
500+fi
501+
502+( cd "$overlay_d" && tar -cpf - * ) > "$overlay_tar" ||
503+ fail "failed to make overlay_tar"
504+
505+debug 1 "populating staging directory"
506+tar -C "$stage_d" -xpf - < "$rootfs_in" ||
507+ fail "failed to extract rootfs_tar"
508+tar -C "$stage_d" -xpf - < "$overlay_tar" ||
509+ fail "failed to extract overlay_tar"
510+
511+if [ -x "${fixup_fs}" ]; then
512+ "${fixup_fs}" "${stage_d}" ||
513+ fail "failed to fixup filesystem"
514+fi
515+
516+( cd "$stage_d" && tar -Scpzf - -- * ) > "$out_filesys_lxc" ||
517+ fail "failed to create filesys tarball"
518+
519+tar -C "$stage_d" -xpf - < "$kernel_tar" ||
520+ fail "failed to extract kernel_tar"
521+tar -C "$stage_d" -xpf - < "$xgrubd/bootgrub.tar" ||
522+ fail "failed to extract bootgrub"
523+
524+depmod -a --basedir "$stage_d" "${kver}" ||
525+ fail "failed to run depmod for kver ${kver} in output"
526+
527+debug 1 "creating initramfs"
528+xrsync "$stage_d/" "$initramfs_d" ||
529+ fail "failed to copy to initramfs_d"
530+rm -Rf "$initramfs_d/vmlinuz" "$initramfs_d/boot" ||
531+ fail "failed to remove files in initramfs staging dir"
532+
533+( cd "$initramfs_d" && find . | cpio --quiet -o -H newc |
534+ gzip -9 ) > "$initramfs"
535+
536+# Here we create /boot/initrd.img-${kver}.
537+# That is the initramfs that is put inside of disk images.
538+# Since the disk images have all the modules available in the
539+# filesystem, we put no modules in this initramfs. Having them
540+# in both places would just waste space.
541+#
542+# That means that the kernel must have builtin drivers for
543+# any block devices. Another path to accomplishing the same
544+# space savings is what cloud-initramfs-copymods does.
545+# The solution there is that if there is no /lib/modules/<kver>
546+# inside the target filesystem, then the initramfs mount binds
547+# its /lib/modules/<kver> into the target.
548+rm -Rf "$initramfs_d/lib/modules/$kver" ||
549+ fail "failed to remove lib/modules for mini initramfs"
550+
551+( cd "$initramfs_d" && find . | cpio --quiet -o -H newc |
552+ gzip -9 ) > "${initramfs}.smaller"
553+
554+cp "${initramfs}.smaller" "$stage_d/boot/initrd.img-${kver}" &&
555+ ln -s "boot/initrd.img-${kver}" "${stage_d}/initrd.img" ||
556+ fail "failed to copy initramfs to stage dir"
557+
558+debug 1 "packing clean kernel_tar"
559+tar -C "$stage_d" -cpf - \
560+ vmlinuz initrd.img boot/ lib/modules/ > "$kernel_tar" ||
561+ fail "failed to create clean kerne_tar"
562+
563+( cd "$stage_d" && tar -cpf - * ) > "$filesys_tar" ||
564+ fail "failed to create filesys_tar"
565+
566+debug 1 "populating image"
567+mount -o loop "${out_partimg}" "${mp}" && UMOUNT=${mp} ||
568+ fail "failed to mount ${out_partimg} loopback"
569+
570+tar -C "$mp" -xpf - < "$filesys_tar" ||
571+ fail "failed to populate mount point"
572+
573+umount "${mp}" && UMOUNT="" ||
574+ fail "failed to unmount ${out_partimg}"
575+
576+cp "${kernel_d}/${vmlinuz}" "${out_kernel}" ||
577+ fail "failed to copy kernel to ${out_kernel}"
578+
579+{ [ -z "${out_initramfs}" ] || cp "${initramfs}" "${out_initramfs}"; } ||
580+ fail "failed to copy initramfs to ${out_initramfs}"
581+
582+debug 1 "fixing grub entry in partimg"
583+tmp_part="$TEMP_D/part.img.disk"
584+cp "$out_partimg" "$tmp_part" &&
585+ mount -o loop "$tmp_part" "$mp" && UMOUNT="$mp" ||
586+ fail "failed to mount $tmp_part"
587+sed -i 's/(hd0)/(hd0,0)/' "$mp/boot/grub/menu.lst" ||
588+ fail "failed to edit /boot/grub/menu.lst in image"
589+umount "$mp" && UMOUNT="" ||
590+ fail "failed to unmount partimg"
591+
592+case $arch in
593+ i386|x86_64) grub_options="--grub1 --grub-efi";;
594+ arm|aarch64) grub_options="--grub-efi";;
595+ powerpc|ppc*) grub_options="--grub-ieee";;
596+ *) grub_options=;;
597+esac
598+
599+debug 1 "creating disk image"
600+out=$(PATH=$xgrubd:$PATH part2disk $grub_options "$tmp_part" \
601+ "$grub_in" "$arch" "$out_diskimg.raw" 2>&1) ||
602+ fail "failed to create disk image: $out"
603+qemu-img convert -O qcow2 -c "$out_diskimg.raw" "$out_diskimg" ||
604+ fail "failed to convert disk image"
605+rm -f "$out_diskimg.raw" "$tmp_part"
606+
607+if [ -n "${SUDO_USER}" ]; then
608+ u=${SUDO_USER}
609+ g=$(id -g "${u}") || g=${u}
610+ chown "${u}:${g}" -R "$out_d" ||
611+ fail "failed to grant ownership of ${u}:${g} to ${u}:${g}"
612+fi
613+
614+echo "wrote ${out_partimg}"
615+echo "wrote ${out_diskimg}"
616+echo "wrote ${out_kernel}"
617+echo "wrote ${out_initramfs}"
618+echo "wrote ${out_blankimg}"
619+echo "wrote ${out_filesys_lxc}"
620+
621+exit 0
622+
623+# vi: tabstop=4 expandtab
624
625=== added file 'bin/grab-grub-ieee'
626--- bin/grab-grub-ieee 1970-01-01 00:00:00 +0000
627+++ bin/grab-grub-ieee 2016-09-20 12:52:40 +0000
628@@ -0,0 +1,100 @@
629+#!/bin/bash
630+#
631+# Copyright (C) 2016 IBM Corp.
632+#
633+# Author: Rafael Folco <rfolco@br.ibm.com>
634+#
635+# This program is free software: you can redistribute it and/or modify
636+# it under the terms of the GNU General Public License as published by
637+# the Free Software Foundation, version 3 of the License.
638+#
639+# This program is distributed in the hope that it will be useful,
640+# but WITHOUT ANY WARRANTY; without even the implied warranty of
641+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
642+# GNU General Public License for more details.
643+#
644+# You should have received a copy of the GNU General Public License
645+# along with this program. If not, see <http://www.gnu.org/licenses/>.
646+#
647+
648+source "${0%/*}/common-functions.sh"
649+
650+burl="https://launchpad.net/ubuntu/+archive/primary/+files/"
651+outdir="./download"
652+def_arches="ppc64le powerpc ppc64"
653+
654+[ "$1" = "--outdir" ] && { outdir="$1" && shift; }
655+gver="$1"
656+shift
657+
658+if [ $# -eq 0 ]; then
659+ set -- ${def_arches}
660+fi
661+
662+[ -d "$outdir" ] || mkdir -p "$outdir" ||
663+ fail "failed mkdir $outdir"
664+
665+[ -n "$gver" ] || fail "must give grub version"
666+
667+deb2tar() {
668+
669+ local deb="$1" tar="$2"
670+
671+ t=$(dirname "$deb")
672+ tdir=$(mktemp -d "$t/.XXXXXX") || return
673+
674+ debug 1 "creating $out in tempdir at $tdir"
675+
676+ mkdir -p "$tdir/deb"
677+ dpkg -x "$deb" "$tdir/deb" || fail "failed to unpack $deb"
678+
679+ gbin="grub"
680+ gmod="boot cat configfile echo ext2 halt loadenv minicmd normal part_gpt
681+ reboot search search_fs_uuid search_fs_file search_label serial sleep
682+ test linux"
683+
684+ debug 2 "grub-mkimage: creating grub powerpc-ieee1275 image"
685+ grub-mkimage --directory "${tdir}/deb/usr/lib/grub/powerpc-ieee1275" \
686+ --prefix "(ieee1275/disk,gpt1)/boot/grub" \
687+ --output "$tdir/${gbin}" \
688+ --format "powerpc-ieee1275" \
689+ --compression "none" $gmod
690+
691+ rm -rf "$tdir/deb"
692+ debug 2 "creating grub.tar.gz"
693+ tar -C "$tdir/" -czf "$tdir/grub.tar.gz" $gbin &&
694+ mv "$tdir/grub.tar.gz" "$tar" || {
695+ error "failed creating tarball";
696+ rm -Rf "$tdir"
697+ return 1;
698+ }
699+ rm -Rf "$tdir"
700+ return 0
701+
702+}
703+
704+for arch in "$@"; do
705+ case "$arch" in
706+ ppc64le)
707+ debarch='ppc64el'
708+ ;;
709+ powerpc|ppc64)
710+ debarch='powerpc'
711+ ;;
712+ *) debug 1 "no grub-ieee1275 for $arch";;
713+ esac
714+
715+ # e.g. https://launchpad.net/ubuntu/+archive/primary/+files/grub-ieee1275-bin_2.02~beta2-36ubuntu8_ppc64el.deb
716+ deb="grub-ieee1275-bin_${gver}_${debarch}.deb"
717+ if [ ! -f "${outdir}/${deb}" ]; then
718+ dl $burl/$deb "$outdir/$deb" || fail "failed dl $burl/$deb"
719+ fi
720+ tar="grub-ieee-${gver}-${arch}.tar.gz"
721+ deb2tar "$outdir/$deb" "$outdir/$tar" ||
722+ fail "failed deb2tar from $deb to $tar"
723+
724+ error "wrote $outdir/${tar}"
725+ ln -sf "${tar}" "$outdir/grub-ieee-${arch}.tar.gz" ||
726+ fail "failed symlink for $outdir/grub-ieee-${arch}.tar.gz"
727+done
728+# vi: ts=4 expandtab
729
730=== added file 'bin/part2disk.OTHER'
731--- bin/part2disk.OTHER 1970-01-01 00:00:00 +0000
732+++ bin/part2disk.OTHER 2016-09-20 12:52:40 +0000
733@@ -0,0 +1,284 @@
734+#!/bin/bash
735+# part2disk - wrap a partition image in a disk image
736+#
737+# Copyright (C) 2010 Canonical Ltd.
738+#
739+# Authors: Scott Moser <smoser@canonical.com>
740+# Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
741+# Rafael Folco <rfolco@br.ibm.com>
742+#
743+# This program is free software: you can redistribute it and/or modify
744+# it under the terms of the GNU General Public License as published by
745+# the Free Software Foundation, version 3 of the License.
746+#
747+# This program is distributed in the hope that it will be useful,
748+# but WITHOUT ANY WARRANTY; without even the implied warranty of
749+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
750+# GNU General Public License for more details.
751+#
752+# You should have received a copy of the GNU General Public License
753+# along with this program. If not, see <http://www.gnu.org/licenses/>.
754+
755+source "${0%/*}/common-functions.sh"
756+
757+DEF_SECTOR_SIZE=512
758+base_d=$(dirname $(readlink -f "${0}"))
759+PATH="${PATH}:${base_d}"
760+
761+getsize() {
762+ local fname="$1" kname="" size=""
763+ if [ -b "${fname}" ]; then
764+ kname=$(readlink -f "${fname}") &&
765+ size=$(awk '$4 == kn { print $3 * 1024 }' \
766+ "kn=${kname##*/}" /proc/partitions) &&
767+ [ -n "${size}" ] || {
768+ error "failed to read size of ${fname} from /proc/partitions";
769+ return 1;
770+ }
771+ else
772+ size=$(stat --format "%s" "${fname}") || {
773+ error "failed to get size of ${fname}"
774+ return 1;
775+ }
776+ fi
777+ _RET="$size"
778+}
779+
780+Usage() {
781+ cat <<EOF
782+Usage: ${0##*/} [options] partition-image grub-{efi|ieee}-tarball architecture disk-image
783+
784+ Create disk image 'disk-image' with 'partition-image' in a partition
785+ inside it. Add grub if requested.
786+
787+ options:
788+ -G | --grub install grub to disk image mbr
789+ | --grub1 install grub1 to disk image mbr
790+ | --grub-efi install EFI grub to disk image
791+ | --grub-ieee install ieee1275 grub to disk image prep partition
792+ -s | --size S create the disk image of size 'S'.
793+ default is large enough to fit partition-image
794+ -v | --verbose increase verbosity
795+EOF
796+}
797+human2bytes() {
798+ # converts size suitable for input to resize2fs to bytes
799+ # s:512 byte sectors, K:kilobytes, M:megabytes, G:gigabytes
800+ # none: block size of the image
801+ local input=${1} defunit=${2:-1024}
802+ local unit count;
803+ case "$input" in
804+ *s) count=${input%s}; unit=512;;
805+ *K) count=${input%K}; unit=1024;;
806+ *M) count=${input%M}; unit=$((1024*1024));;
807+ *G) count=${input%G}; unit=$((1024*1024*1024));;
808+ *) count=${input} ; unit=${2:-1024};;
809+ esac
810+ _RET=$((${count}*${unit}))
811+}
812+
813+cleanup() {
814+ umount "$TMP/efi"
815+ umount "$TMP/rootfs"
816+ kpartx -d "$dimg"
817+}
818+
819+trap cleanup EXIT
820+
821+short_opts="b:c:Ghs:v"
822+long_opts="grub-ieee,grub-efi,grub1,grub,help,size:,verbose"
823+getopt_out=$(getopt --name "${0##*/}" \
824+ --options "${short_opts}" --long "${long_opts}" -- "$@") &&
825+ eval set -- "${getopt_out}" ||
826+ bad_Usage
827+
828+ssize=${DEF_SECTOR_SIZE}
829+size_in=""
830+grub_ptnum=1
831+grub=0
832+grub1=0
833+grub_efi=0
834+grub_ieee=0
835+while [ $# -ne 0 ]; do
836+ cur=${1}; next=${2};
837+ case "$cur" in
838+ -G|--grub)
839+ grub=1
840+ ;;
841+ -G|--grub1)
842+ grub1=1
843+ ;;
844+ --grub-efi)
845+ grub_efi=1
846+ ptype="ef00" # EFI System
847+ ;;
848+ --grub-ieee)
849+ grub_ieee=1
850+ ptype="4100" # PowerPC PreP Boot
851+ ;;
852+ -h|--help) Usage; exit 0;;
853+ -s|--size) size_in=$2; shift;;
854+ -v|--verbose) DEBUG=$((${DEBUG}+1));;
855+ --) shift; break;;
856+ esac
857+ shift;
858+done
859+
860+[ $# -eq 4 ] || bad_Usage "must supply partition image, grub-{efi|ieee} tarball, architecture and output file"
861+
862+pimg=${1}
863+grub_in=${2}
864+arch=${3}
865+dimg=${4}
866+
867+{ [ ${grub} -eq 0 ] || phelper=$(command -v part2disk-grubhelper); } ||
868+ fail "no part2disk-grubhelper in PATH"
869+[ $grub1 -eq 0 ] || command -v grub >/dev/null || fail "no 'grub' in path";
870+
871+[ -f "${pimg}" -o -b "${pimg}" ] || fail "${pimg}: not a file or block device"
872+
873+getsize "$pimg" ||
874+ fail "failed to get size of $pimg"
875+pimg_s="$_RET"
876+
877+front_pad=$((12*1024*1024))
878+tot_size=$(($front_pad+$pimg_s))
879+tot_size_sectors=$(($tot_size/$ssize))
880+
881+if [ -n "${size_in}" ]; then
882+ human2bytes "${size_in}" 1 || fail "failed to convert ${size_in} to bytes"
883+ size=${_RET}
884+else
885+ size=$tot_size_sectors
886+fi
887+
888+if [ -e "$dimg" ]; then
889+ getsize "$dimg" ||
890+ fail "failed to get size of $dimg"
891+ dimg_s="$_RET"
892+else
893+ dimg_s="$size"
894+fi
895+
896+if [ "${dimg_s}" -lt "$size" ]; then
897+ fail "size of $dimg ($dimg_s) not large enough to fit $size"
898+fi
899+
900+debug 1 "input is ${pimg_s} bytes."
901+debug 1 "target is ${tot_size} bytes."
902+
903+debug 2 "create target image"
904+dd if=/dev/zero of="${dimg}" bs=$ssize count=$tot_size_sectors \
905+ 2>/dev/null ||
906+ fail "failed to write to ${dimg}"
907+
908+debug 2 "create partitions"
909+sgdisk -n 15:2048:+8M -t 15:${ptype} -N 1 $dimg
910+loop_nr=`kpartx -av $dimg | head -n1 | cut -d" " -f3 | cut -d"p" -f2`
911+
912+debug 2 "parted $dimg print"
913+parted $dimg print
914+
915+# copy partition image. this writes $pimg bytes even if that is
916+# not divivisble by $ssize
917+debug 2 "copying ${pimg} to partition in ${dimg}"
918+dd if="$pimg" of=/dev/mapper/loop${loop_nr}p1 conv=notrunc \
919+ 2>/dev/null ||
920+ fail "failed to write ${pimg} into ${dimg}"
921+
922+if [ ${grub} -ne 0 ]; then
923+ debug 2 "invoking part2disk-grubhelper ${dimg}"
924+ sudo "${phelper}" "${dimg}" ||
925+ fail "part2disk-grubhelper ${dimg} failed"
926+fi
927+
928+if [ $grub1 -ne 0 ]; then
929+ debug 2 "installing grub"
930+ grub --no-floppy --batch <<EOF
931+device (hd0) $dimg
932+root (hd0,0)
933+setup (hd0)
934+quit
935+EOF
936+fi
937+
938+if [ $grub_efi -ne 0 ]; then
939+ case $arch in
940+ x86_64) efiarch=x64;;
941+ aarch64) efiarch=aa64;;
942+ i386) efiarch=ia32;;
943+ arm) efiarch=arm;;
944+ *) fail "arch $arch without EFI grub"
945+ esac
946+ TMP=$(mktemp -d)
947+ install -d $TMP/tar $TMP/efi $TMP/rootfs
948+
949+ mkfs.vfat /dev/mapper/loop${loop_nr}p15
950+ mount /dev/mapper/loop${loop_nr}p15 $TMP/efi
951+ install -d $TMP/efi/EFI/BOOT $TMP/efi/EFI/ubuntu
952+ tar -xf $grub_in -C $TMP/efi/EFI/BOOT
953+
954+ mount /dev/mapper/loop${loop_nr}p1 $TMP/rootfs
955+ kernelfile=$(cd $TMP/rootfs/boot; ls vmlinu* | head -n1)
956+ initrdfile=$(cd $TMP/rootfs/boot; ls initrd.img-* | head -n1)
957+
958+
959+ cat >$TMP/efi/EFI/ubuntu/grub.cfg <<EOF
960+set default=0
961+set timeout=1
962+menuentry 'CirrOS' {
963+ set root='hd0,gpt1'
964+ if [ x$feature_platform_search_hint = xy ]; then
965+ search --no-floppy --label --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci1,gpt1 cirros-rootfs
966+ else
967+ search --no-floppy --label --set=root cirros-rootfs
968+ fi
969+
970+ linux /boot/${kernelfile} LABEL=cirros-rootfs ro
971+ initrd /boot/${initrdfile}
972+}
973+EOF
974+
975+ umount "$TMP/efi"
976+ umount "$TMP/rootfs"
977+fi
978+
979+set -x
980+
981+# grub-ieee1275 (powerpc)
982+if [ $grub_ieee -ne 0 ]; then
983+ TMP=$(mktemp -d)
984+ install -d $TMP/rootfs $TMP/tar
985+
986+ mkfs.vfat /dev/mapper/loop${loop_nr}p15
987+ tar -xf $grub_in -C $TMP/tar
988+ dd if=$TMP/tar/grub of=/dev/mapper/loop${loop_nr}p15
989+
990+ mount /dev/mapper/loop${loop_nr}p1 $TMP/rootfs
991+ kernelfile=$(cd $TMP/rootfs/boot; ls vmlinu* | head -n1)
992+ initrdfile=$(cd $TMP/rootfs/boot; ls initrd.img-* | head -n1)
993+
994+ cat >$TMP/rootfs/boot/grub/grub.cfg <<EOF
995+set default=0
996+set timeout=1
997+menuentry 'CirrOS' {
998+ set root='ieee1275/disk,gpt1'
999+
1000+ if [ x$feature_platform_search_hint = xy ]; then
1001+ search --no-floppy --label --set=root --hint-ieee1275=ieee1275/disk,gpt1 cirros-rootfs
1002+ else
1003+ search --no-floppy --label --set=root cirros-rootfs
1004+ fi
1005+
1006+ linux /boot/${kernelfile} LABEL=cirros-rootfs ro
1007+ initrd /boot/${initrdfile}
1008+}
1009+EOF
1010+
1011+ umount "$TMP/rootfs"
1012+fi
1013+
1014+
1015+kpartx -d "$dimg"
1016+error "wrote to ${dimg}"
1017+# vi: ts=4 noexpandtab
1018
1019=== added directory 'doc'
1020=== added file 'doc/README-powerpc.txt.OTHER'
1021--- doc/README-powerpc.txt.OTHER 1970-01-01 00:00:00 +0000
1022+++ doc/README-powerpc.txt.OTHER 2016-09-20 12:52:40 +0000
1023@@ -0,0 +1,49 @@
1024+
1025+CirrOS PowerPC images
1026+=====================
1027+
1028+ * Use the build-release script w/ daily option
1029+ $ export ARCHES="ppc64le powerpc ppc64"; ./bin/build-release daily
1030+
1031+ This will build rootfs, grab kernels and grub, install grub w/ modules on
1032+ prep partition, and bundle image. The *-disk.img is a qcow2 bootable image
1033+ and contains a grub2 installed on a PreP Boot partition.
1034+
1035+ * Quick test
1036+ $ img=cirros-{YYMMDD}-ppc64le-disk.img; qemu-system-ppc64le -echr 0x05 \
1037+ -machine pseries-2.5,accel=kvm,usb=off -cpu host -m 512 -nographic \
1038+ $img
1039+
1040+ * Add the image to OpenStack
1041+ $ openstack image create --disk-format qcow2 --container-format bare \
1042+ --public --property os_command_line="console=hvc0 console=tty0" \
1043+ --property arch=ppc64le --property hypervisor_type=kvm \
1044+ --property hw_disk_bus=scsi --property hw_scsi_model=virtio-scsi \
1045+ --property hw_cdrom_bus=scsi cirros_qcow2 \
1046+ < cirros-{YYMMDD}-ppc64le-disk.img
1047+
1048+ NOTE: ppc64le/ppc64 images have been tested extensively on the OpenStack CI
1049+ using the daily builds - http://download.cirros-cloud.net/daily/ - and works
1050+ well with virtio drivers (network and disk).
1051+
1052+Currently known working on PowerKVM:
1053+
1054+$ { echo '#!/bin/sh'; echo 'echo HELLO WORLD'; } > user-data
1055+$ cloud-localds seed.img user-data
1056+
1057+## usb=off is required
1058+## -net nic and -drive get default (ibmveth and ibmvscsi) drivers
1059+$ qemu-system-ppc64 -machine pseries,accel=kvm,usb=off -m 1G \
1060+ -name foo -M pseries \
1061+ -device spapr-vscsi \
1062+ -net nic -net user \
1063+ -display none -nographic \
1064+ -drive file=disk1.img \
1065+ -kernel "$1" ${2:+-initrd "$2"} \
1066+ -append "root=/dev/sda console=hvc0"
1067+
1068+Also known working is adding 'blank.img' as a '-drive'.
1069+
1070+Historical NOTE(smoser) about virtio:
1071+What is not working is virtio. I have not successfully gotten virtio network
1072+or virtio disk working.

Subscribers

People subscribed via source and target branches