Merge lp:~dpigott/lava-master-image-scripts/move-partitioning-to-first-boot into lp:lava-master-image-scripts

Proposed by Dave Pigott
Status: Merged
Approved by: Dave Pigott
Approved revision: 61
Merged at revision: 59
Proposed branch: lp:~dpigott/lava-master-image-scripts/move-partitioning-to-first-boot
Merge into: lp:lava-master-image-scripts
Diff against target: 469 lines (+211/-145)
8 files modified
NEWS (+2/-1)
README (+5/-22)
contrib/lava-partition-disk (+0/-118)
contrib/preEnv.txt (+0/-1)
lava-create-master (+12/-2)
master-rootfs-extras/bin/lava-partition-disk (+184/-0)
master-rootfs-extras/lib/lava/common (+1/-0)
master-rootfs-extras/lib/lava/lava-master-init (+7/-1)
To merge this branch: bzr merge lp:~dpigott/lava-master-image-scripts/move-partitioning-to-first-boot
Reviewer Review Type Date Requested Status
Michael Hudson-Doyle (community) Approve
Review via email: mp+105655@code.launchpad.net

Description of the change

Get rid of the contrib, second stage code, and make it all happen on first boot. Re-write and tidy the partitioning code. Add a new option to lava-create-master-image to take in testrootfs size as a command line parameter.

Also put the image size back to 1GB. The 3GB was unnecessary, since we deploy onto testrootfs, which is usually 4G and in the LAVA farm is always 8GB.

To post a comment you must log in.
Revision history for this message
Zygmunt Krynicki (zyga) wrote :

450 -LAVA_PKGS="wget dosfstools u-boot-tools bzip2 python ntpdate"
451 +LAVA_PKGS="wget dosfstools u-boot-tools bzip2 python ntpdate parted"

Do we need parted? We use fdisk, we also need partprobe and mkfs.ext4 (they may be already contained in other packages)

Otherwise it looks fine on first read +2

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

I tested this branch and it seems to work. Nice :)

review: Approve
Revision history for this message
Dave Pigott (dpigott) wrote :

> 450 -LAVA_PKGS="wget dosfstools u-boot-tools bzip2 python ntpdate"
> 451 +LAVA_PKGS="wget dosfstools u-boot-tools bzip2 python ntpdate parted"
>
> Do we need parted? We use fdisk, we also need partprobe and mkfs.ext4 (they
> may be already contained in other packages)
>
> Otherwise it looks fine on first read +2

It was purely because of partprobe. Wasn't aware of another package that contains it.

61. By Dave Pigott

Add a little more commentary

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'NEWS'
--- NEWS 2012-05-04 11:46:04 +0000
+++ NEWS 2012-05-15 08:44:18 +0000
@@ -1,11 +1,12 @@
12012.05:12012.05:
2 2
3 * Unreleased3 * Unreleased
4 * Increase the size of the default image to 3G4 * Increase the size of the default image to 3G - now back at 1G
5 * Add support for partitioning SD cards via both native mmc interface and5 * Add support for partitioning SD cards via both native mmc interface and
6 USB adapters6 USB adapters
7 * Rename run-beagle-master-in-qemu.sh to boot-snaphot-in-qemu.sh and allow7 * Rename run-beagle-master-in-qemu.sh to boot-snaphot-in-qemu.sh and allow
8 the user to point it at any image pathname8 the user to point it at any image pathname
9 * Add partitioning on boot up rather than as a seperate phase
910
102u12.04:112u12.04:
1112
1213
=== modified file 'README'
--- README 2012-05-03 09:53:27 +0000
+++ README 2012-05-15 08:44:18 +0000
@@ -32,25 +32,8 @@
32from the Linaro website. Once prepared the caches "vanilla" image can be reused32from the Linaro website. Once prepared the caches "vanilla" image can be reused
33over and over. The tool creates a single file <board>-master.img.33over and over. The tool creates a single file <board>-master.img.
3434
35This image is not complete: it lacks the special partitioning layout required by35The first boot of a device with this card is special. It will initialize the
36LAVA dispatcher. To finish the process copy the image to an SD card as instructed36master image, including partitioning and labeling, and will likely take a few
37by the tool and run:37extra minutes to finish. The interactive serial console will be activated after
3838this process has finished. Once this process is done it is *not recommended* to
39 $ ./contrib/lava-partition-disk <board type> <device path> <boot mount>39clone the SD card anymore.
40
41Replace <board type> with one of: beagle, imx53, origen, panda, snowball
42or vexpress.
43
44Replace <device path> with the name of the system device that represents your
45SD card, typically this is /dev/sdX or /dev/mmcblkX (with X being a number or
46letter representing subsequent devices in your system).
47
48Replace <boot mount> with the mount point for the boot partition, typically on
49Ubuntu this will be /media/boot.
50
51This step will be integrated into the tool in the future. At this time the card
52is now ready and can be cloned (like a template). The first boot of a device
53with this card is special. It will initialize the master image and will likely
54take a few extra minutes to finish. The interactive serial console will be
55activated after this process has finished. Once this process is done it is *not
56recommended* to clone the SD card anymore.
5740
=== removed directory 'contrib'
=== removed file 'contrib/lava-partition-disk'
--- contrib/lava-partition-disk 2012-05-14 08:38:47 +0000
+++ contrib/lava-partition-disk 1970-01-01 00:00:00 +0000
@@ -1,118 +0,0 @@
1#!/bin/bash
2
3show_usage() {
4 echo "Usage: $0 <board type> <device path> <boot mount> {testrootfs partition size}"
5 echo "Where <board typei> is one of: beagle, imx53, origen, panda,"
6 echo " snowball or vexpress"
7 echo
8 echo "<device path> is the path to your sd card (e.g. /dev/sdb)"
9 echo "<boot mount> is where the boot partition of the sd card is (e.g. /media/boot"
10 echo
11 echo "The testrootfs partition size is optional and defauts to 4G."
12 echo "Make sure you specify the \"G\" in uppercase"
13}
14
15if [ "$3" = "" ]
16then
17 show_usage
18 exit 1
19fi
20
21PRE_ENV=
22
23case $1 in
24 beagle)
25 PRE_ENV=Y
26 SCHEME=2part;;
27
28 imx53)
29 SCHEME=3part;;
30
31 origen)
32 SCHEME=3part;;
33
34 panda)
35 PRE_ENV=Y
36 SCHEME=2part;;
37
38 snowball)
39 SCHEME=2part;;
40
41 vexpress)
42 SCHEME=2part;;
43
44 *)
45 echo "Board type \"$1\" not recognised"
46 exit 1;;
47esac
48
49DEVICE=$2
50
51BOOT_MOUNT=$3
52
53if [ "$4" = "" ]
54then
55 # SIZE is the size of the testrootfs partition. On an 8G sd you can fit
56 # 4G - put it up to 8G on a 16GB. The rest is allocated to sdcard for
57 # android
58 SIZE="4G"
59else
60 SIZE=$4
61fi
62
63if [ "$PRE_ENV" = "Y" ]
64then
65 cp ./contrib/preEnv.txt $BOOT_MOUNT
66 if [ "$?" != "0" ]
67 then
68 echo "Failed to copy preEnv.txt. Is the disk read only?"
69 exit 1
70 fi
71fi
72
73# origen and imx53 have a different original layout with 3 partitions
74# all others are the same (so far)
75
76if [ $SCHEME = "2part" ]
77then
78 TESTBOOT_PART=3
79 TESTROOTFS_PART=5
80 SDCARD_PART=6
81 LAST_BLOCK=2
82else
83 TESTBOOT_PART=5
84 TESTROOTFS_PART=6
85 SDCARD_PART=7
86 LAST_BLOCK=3
87fi
88
89if [[ $DEVICE =~ 'mmc' ]];
90then
91 PARTITION_PREFIX="p"
92else
93 PARTITION_PREFIX=""
94fi
95
96FIRST_BLOCK_START=`sudo fdisk -S 63 -H 255 -lc $DEVICE | grep "${DEVICE}${PARTITION_PREFIX}${LAST_BLOCK}" | awk '{print $3 + 1}'`
97
98if [ "$FIRST_BLOCK_START" = "" ]
99then
100 echo "Device specification error: $DEVICE"
101 exit 1
102fi
103
104if [ $SCHEME = "2part" ]
105then
106 EXTENDED_BLOCK_START=`echo $FIRST_BLOCK_START | awk '{print $1 + 131072}'`
107 FDISK_SCRIPT="n\np\n3\n$FIRST_BLOCK_START\n+64M\nt\n3\nc\nn\ne\n$EXTENDED_BLOCK_START\n\nn\n\n+$SIZE\nn\n\n\nt\n6\nc\np\nw\n"
108else
109 FDISK_SCRIPT="n\ne\n$FIRST_BLOCK_START\n\nn\n\n+64M\nt\n5\nc\nn\n\n+$SIZE\nn\n\n\nt\n7\nc\np\nw\n"
110fi
111
112echo -e $FDISK_SCRIPT | sudo fdisk -S 63 -H 255 -c $DEVICE
113
114sudo partprobe $DEVICE
115sudo mkfs.vfat -n testboot "${DEVICE}${PARTITION_PREFIX}${TESTBOOT_PART}"
116sudo mkfs.vfat -n sdcard "${DEVICE}${PARTITION_PREFIX}${SDCARD_PART}"
117sudo mkfs.ext3 -L testrootfs "${DEVICE}${PARTITION_PREFIX}${TESTROOTFS_PART}"
118sync
1190
=== removed file 'contrib/preEnv.txt'
--- contrib/preEnv.txt 2012-05-03 09:52:40 +0000
+++ contrib/preEnv.txt 1970-01-01 00:00:00 +0000
@@ -1,1 +0,0 @@
1bootdelay=10
20
=== modified file 'lava-create-master'
--- lava-create-master 2012-05-14 08:38:47 +0000
+++ lava-create-master 2012-05-15 08:44:18 +0000
@@ -16,6 +16,10 @@
16# When non-empty the tool will stop just before jumping into the rootfs16# When non-empty the tool will stop just before jumping into the rootfs
17export DEBUG=17export DEBUG=
1818
19# Used during partitioning on first boot - size of the testrootsfs partition
20# defaults to 4G (4 GigaBytes)
21export ROOTFS_SIZE="4G"
22
19# TODO: change to BOARD23# TODO: change to BOARD
20dev=24dev=
2125
@@ -58,7 +62,7 @@
5862
59show_usage() {63show_usage() {
60 # Display the usage line64 # Display the usage line
61 echo "Usage: lava-create-master [--rebuild-vanilla] [--verbose] (--help | --version | DEVICE) [ --output|-o IMAGE]"65 echo "Usage: lava-create-master [--rebuild-vanilla] [--verbose] (--help | --version | DEVICE) [ --output|-o IMAGE] [--rootfs-size|-r SIZE]"
62}66}
6367
6468
@@ -73,6 +77,7 @@
73 printf -- " %-9s - %s\n" "--verbose" "Display additional messages"77 printf -- " %-9s - %s\n" "--verbose" "Display additional messages"
74 printf -- " %-9s - %s\n" "--output" "Save the output as IMAGE"78 printf -- " %-9s - %s\n" "--output" "Save the output as IMAGE"
75 printf -- " %-9s - %s\n" "--rebuild-vanilla" "Always rebuild vanilla image"79 printf -- " %-9s - %s\n" "--rebuild-vanilla" "Always rebuild vanilla image"
80 printf -- " %-9s - %s\n" "--rootfs-size" "Size of the testrootfs partition (default 4G)"
76 echo81 echo
77 echo "supported devices:"82 echo "supported devices:"
78 echo83 echo
@@ -119,6 +124,10 @@
119 OUTPUT_IMAGE="$2"124 OUTPUT_IMAGE="$2"
120 shift 2125 shift 2
121 ;;126 ;;
127 --rootfs-size|-r)
128 ROOTFS_SIZE="$2"
129 shift 2
130 ;;
122 *)131 *)
123 if [ -n "$dev" ]; then132 if [ -n "$dev" ]; then
124 show_usage133 show_usage
@@ -335,7 +344,7 @@
335 --dev $LMC_DEV \344 --dev $LMC_DEV \
336 --rootfs ext4 \345 --rootfs ext4 \
337 --image-file $CACHE_DIR/pristine-images/$dev-vanilla.img \346 --image-file $CACHE_DIR/pristine-images/$dev-vanilla.img \
338 --image-size 3G \347 --image-size 1G \
339 --align-boot-part \348 --align-boot-part \
340 --binary $CACHE_DIR/image-pieces/$ROOTFS \349 --binary $CACHE_DIR/image-pieces/$ROOTFS \
341 --hwpack $CACHE_DIR/image-pieces/$HWPACK \350 --hwpack $CACHE_DIR/image-pieces/$HWPACK \
@@ -408,6 +417,7 @@
408 sed \417 sed \
409 -e "s!@LAVA_VERSION@!$LAVA_VERSION!" \418 -e "s!@LAVA_VERSION@!$LAVA_VERSION!" \
410 -e "s!@LAVA_URL@!$LAVA_URL!" \419 -e "s!@LAVA_URL@!$LAVA_URL!" \
420 -e "s!@LAVA_ROOTFS_SIZE@!$ROOTFS_SIZE!" \
411 -i \421 -i \
412 $MASTER_MNT/lib/lava/common422 $MASTER_MNT/lib/lava/common
413423
414424
=== added file 'master-rootfs-extras/bin/lava-partition-disk'
--- master-rootfs-extras/bin/lava-partition-disk 1970-01-01 00:00:00 +0000
+++ master-rootfs-extras/bin/lava-partition-disk 2012-05-15 08:44:18 +0000
@@ -0,0 +1,184 @@
1#!/bin/bash
2
3# By default we build a 4GByte partition. Specifiy on the command line for
4# anything different
5ROOTFS_SIZE="4G"
6
7show_usage() {
8 echo "Usage: $0 [--rootfs-size|-r SIZE]"
9}
10
11show_help() {
12 show_usage
13 echo
14 echo "available options:"
15 echo
16 printf -- " %-9s - %s\n" "--help" "Show this message"
17 printf -- " %-9s - %s\n" "--rootfs-size" "Size of the testrootfs partition (default 4G)"
18 echo
19 echo "See: $LAVA_URL for more information"
20}
21
22parse_argv() {
23 while test -n "$1"; do
24 case "$1" in
25 --help)
26 show_help
27 exit 0
28 ;;
29 --rootfs-size|-r)
30 ROOTFS_SIZE=$2;
31 shift 2
32 ;;
33 *)
34 show_usage
35 echo "Too many arguments, see --help for details"
36 exit 1
37 ;;
38 esac
39 done
40}
41
42# Basically, there are just enough differences between boards that you need
43# to specify which schemes and actions need to take place.
44#
45# PRE_ENV: means that we can put a preEnv.txt in the boot partition and it
46# will be read on boot and used, so we get a default bootdelay of 10s
47#
48# SCHEME: Basically describes the topology of the disk at the start. Some
49# boards have 2 partitions, others 3: TODO Make it sense which one, and even
50# detect that the card doesn't conform (e.g. already partitioned)
51#
52# DEVICE: Fairly self explanatory - the device that will be partitioned
53set_device() {
54 DEVICE_TYPE=`lava-device-info --device-type`
55
56 case $DEVICE_TYPE in
57 lava:beagle)
58 PRE_ENV=Y
59 SCHEME=2part
60 DEVICE="/dev/mmcblk0"
61 ;;
62 lava:imx53)
63 PRE_ENV=N
64 SCHEME=3part
65 DEVICE="/dev/mmcblk0"
66 ;;
67 lava:origen)
68 PRE_ENV=N
69 SCHEME=3part
70 DEVICE="/dev/mmcblk0"
71 ;;
72 lava:panda)
73 PRE_ENV=Y
74 SCHEME=2part
75 DEVICE="/dev/mmcblk0"
76 ;;
77 lava:snowball)
78 PRE_ENV=N
79 SCHEME=2part
80 DEVICE="/dev/mmcblk1"
81 ;;
82 lava:vexpress)
83 PRE_ENV=N
84 SCHEME=2part
85 DEVICE="/dev/mmcblk0"
86 ;;
87 *)
88 echo "Board type \"$1\" not supported"
89 exit 1
90 ;;
91 esac
92
93 # origen and imx53 have a different original layout with 3 partitions
94 # all others are the same (so far). All we're doing is defining the
95 # partition numbers for the partitions we're going to add
96
97 if [ $SCHEME = "2part" ]
98 then
99 TESTBOOT_PART=3
100 TESTROOTFS_PART=5
101 SDCARD_PART=6
102 LAST_BLOCK=2
103 else
104 TESTBOOT_PART=5
105 TESTROOTFS_PART=6
106 SDCARD_PART=7
107 LAST_BLOCK=3
108 fi
109
110 if [[ $DEVICE =~ 'mmc' ]];
111 then
112 PARTITION_PREFIX="p"
113 else
114 PARTITION_PREFIX=""
115 fi
116
117}
118
119
120write_preenv() {
121 mkdir -p /mnt/boot
122 mount /dev/disk/by-label/boot /mnt/boot
123 echo "bootdelay=10" >> /mnt/boot/preEnv.txt
124 if [ "$?" != "0" ]; then
125 echo "Failed to copy preEnv.txt. Is the disk read only?"
126 exit 1
127 fi
128 umount /mnt/boot
129}
130
131# Basically, we need the following scheme:
132# /dev/disk/by-label/boot - the master boot image
133# /dev/disk/by-label/rootfs - the master root filing system
134# /dev/disk/by-label/testboot - the deployed test boot image will go here
135# /dev/disk/by-label/testrootfs - the deployed test rootfs will go here
136# /dev/disk/by-label/sdcard - the Android sdcard partition goes here
137#
138# so first we partition the disk, if it's 2part, then 3 will be the testboot
139# and 4 will be an extended partition containing 5 as the testrootfs and 6
140# as sdcard.
141# For 3part then 4 will be an extended partition containing 5/testboot,
142# 6/testrootfs and 7/sdcard
143partition_disk() {
144 FIRST_BLOCK_START=`fdisk -S 63 -H 255 -lc $DEVICE | grep "${DEVICE}${PARTITION_PREFIX}${LAST_BLOCK}" | awk '{print $3 + 1}'`
145
146 if [ "$FIRST_BLOCK_START" = "" ]
147 then
148 echo "Device specification error: $DEVICE"
149 exit 1
150 fi
151
152 if [ $SCHEME = "2part" ]
153 then
154 # 131072 = 65536 * 2 - fdisk works in 512K blocks, and we want a 64M
155 # partition for testboot.
156 EXTENDED_BLOCK_START=`echo $FIRST_BLOCK_START | awk '{print $1 + 131072}'`
157 FDISK_SCRIPT="n\np\n3\n$FIRST_BLOCK_START\n+64M\nt\n3\nc\nn\ne\n$EXTENDED_BLOCK_START\n\nn\n\n+$ROOTFS_SIZE\nn\n\n\nt\n6\nc\np\nw\n"
158 else
159 FDISK_SCRIPT="n\ne\n$FIRST_BLOCK_START\n\nn\n\n+64M\nt\n5\nc\nn\n\n+$ROOTFS_SIZE\nn\n\n\nt\n7\nc\np\nw\n"
160 fi
161
162 echo -e $FDISK_SCRIPT | fdisk -S 63 -H 255 -c $DEVICE
163}
164
165# Do the partition labelling
166make_fs() {
167 partprobe $DEVICE
168 mkfs.vfat -n testboot "${DEVICE}${PARTITION_PREFIX}${TESTBOOT_PART}"
169 mkfs.vfat -n sdcard "${DEVICE}${PARTITION_PREFIX}${SDCARD_PART}"
170 mkfs.ext3 -L testrootfs "${DEVICE}${PARTITION_PREFIX}${TESTROOTFS_PART}"
171 sync
172}
173
174main() {
175 parse_argv "$@"
176 set_device
177 if [ "$PRE_ENV" = "Y" ]; then
178 write_preenv
179 fi
180 partition_disk
181 make_fs
182}
183
184main "$@"
0185
=== modified file 'master-rootfs-extras/lib/lava/common'
--- master-rootfs-extras/lib/lava/common 2012-03-22 17:46:56 +0000
+++ master-rootfs-extras/lib/lava/common 2012-05-15 08:44:18 +0000
@@ -1,6 +1,7 @@
1# Replaced by lava-create-master1# Replaced by lava-create-master
2LAVA_VERSION=@LAVA_VERSION@2LAVA_VERSION=@LAVA_VERSION@
3LAVA_URL=@LAVA_URL@3LAVA_URL=@LAVA_URL@
4LAVA_ROOTFS_SIZE=@LAVA_ROOTFS_SIZE@
45
5# Persistent device configuration file6# Persistent device configuration file
6LAVA_DEVICE_CONF=/var/lib/lava/device.conf7LAVA_DEVICE_CONF=/var/lib/lava/device.conf
78
=== modified file 'master-rootfs-extras/lib/lava/lava-master-init'
--- master-rootfs-extras/lib/lava/lava-master-init 2012-05-14 07:05:18 +0000
+++ master-rootfs-extras/lib/lava/lava-master-init 2012-05-15 08:44:18 +0000
@@ -14,7 +14,7 @@
14LAVA_FIRST_BOOT_LOG=/var/log/lava/first-boot.log14LAVA_FIRST_BOOT_LOG=/var/log/lava/first-boot.log
1515
16# Set of packages to install on first boot16# Set of packages to install on first boot
17LAVA_PKGS="wget dosfstools u-boot-tools bzip2 python ntpdate"17LAVA_PKGS="wget dosfstools u-boot-tools bzip2 python ntpdate parted"
1818
19DEVICE_TYPE=`lava-device-info --device-type`19DEVICE_TYPE=`lava-device-info --device-type`
2020
@@ -54,6 +54,12 @@
54 echo "Unable to synchronize time" >&254 echo "Unable to synchronize time" >&2
55 return 155 return 1
56 fi56 fi
57 # Do disk partitioning
58 echo "Partitioning disk to LAVA format" >> "$LAVA_FIRST_BOOT_LOG"
59 if ! lava-partition-disk -r $LAVA_ROOTFS_SIZE >> "$LAVA_FIRST_BOOT_LOG" 2>&1; then
60 echo "Unable to partition disk" >&2
61 return 1
62 fi
57 if [ "$DEVICE_TYPE" = "lava:panda" ]; then63 if [ "$DEVICE_TYPE" = "lava:panda" ]; then
58 echo "Applying sysctl fix for streaming errors on Panda" >> "$LAVA_FIRST_BOOT_LOG"64 echo "Applying sysctl fix for streaming errors on Panda" >> "$LAVA_FIRST_BOOT_LOG"
59 echo vm.min_free_kbytes = 32768 >> /etc/sysctl.conf65 echo vm.min_free_kbytes = 32768 >> /etc/sysctl.conf

Subscribers

People subscribed via source and target branches