Merge lp:~liuyq0307/lava-dispatcher/test-bootloader into lp:lava-dispatcher

[Yongqin Liu]

This modification will support to use the boot loader of the test images when we can load the test boot loader from the master boot loader

And by this mechanism, we can resolve the problem of booting omapzoom and aosp panda build in lava

have tested with the panda-stable build from https://android-build.linaro.org/builds/~linaro-android/panda-jb-gcc47-tilt-stable-blob/#build=51

and aosp build from my local on my local lava instance

[Andy Doan (doanac)]

This approach makes me a bit nervous. We are basically chain-loading boot loaders. Its really cool, but I don't know if its safe and worth the additional cost of code maintenence. If we really can't get around boot loaders, maybe we should put some specific u-boot version on some boards in the lab and tag them, so certain builds can use them?

[Zygmunt Krynicki (zyga)]

W dniu 10.09.2012 19:49, Andy Doan pisze:
> This approach makes me a bit nervous. We are basically chain-loading boot loaders. Its really cool, but I don't know if its safe and worth the additional cost of code maintenence. If we really can't get around boot loaders, maybe we should put some specific u-boot version on some boards in the lab and tag them, so certain builds can use them?

Chain loading bootloaders basically tests something else than the boot
loader booting as the first thing on a given hardware.

I'd say it's interesting but the value is limited. Having said that I'm
not really opposed to it.

PS: Testing my canonical.com account while I get my linaro.net account ready

Thanks
ZK

[Yongqin Liu (liuyq0307)]

> This approach makes me a bit nervous. We are basically chain-loading boot
> loaders. Its really cool, but I don't know if its safe and worth the
> additional cost of code maintenence.
If possible, can we ask some u-boot experienced guys, if this is ok in theory?
If this is ok, I actually like to use the test boot loader to load the test image.

> If we really can't get around boot
> loaders, maybe we should put some specific u-boot version on some boards in
> the lab and tag them, so certain builds can use them?
This is ok I think, what we need is just two version master images.
one version for aosp-panda and omapzoom-panda builds, and one for others.

[Loïc Minier (lool)]

On Fri, Sep 07, 2012, Yongqin Liu wrote:
> + if change_test_bootloader_delay and change_test_bootloader_delay == "1":
> + sed_cmd = 'sed -i s/bootdelay=0/bootdelay=9/ /mnt/lava/boot/u-boot.bin'

sed-ing binary files does scare me; other options I can think of:
* unpack + sed + repack boot.scr
* sed uEnv.txt
* configure our bootloaders to use lava.txt or lava.scr file if present

Another angle would be to provide such a set of features in our Linaro
images, wrapped under some firmware management scripts; e.g. a
set-boot-delay script that we'd call from LAVA, but this is a lot of
integration work for a relatively weird use case. We could also patch
u-boot to detect that it's running under LAVA and do something special,
but I don't have a specific idea on what do to there.

--
Loïc Minier

[Yongqin Liu (liuyq0307)]

On 11 September 2012 20:01, Loïc Minier <email address hidden> wrote:
> On Fri, Sep 07, 2012, Yongqin Liu wrote:
>> + if change_test_bootloader_delay and change_test_bootloader_delay == "1":
>> + sed_cmd = 'sed -i s/bootdelay=0/bootdelay=9/ /mnt/lava/boot/u-boot.bin'
>
> sed-ing binary files does scare me;
yes, this is the ways I don't like too.
I have tried to use preEnv.txt on the same partition of testboot,
but it does not make sense.

> other options I can think of:
> * unpack + sed + repack boot.scr
> * sed uEnv.txt
> * configure our bootloaders to use lava.txt or lava.scr file if present
you mean modify the u-boot source?

> Another angle would be to provide such a set of features in our Linaro
> images, wrapped under some firmware management scripts; e.g. a
> set-boot-delay script that we'd call from LAVA, but this is a lot of
> integration work for a relatively weird use case. We could also patch
> u-boot to detect that it's running under LAVA and do something special,
> but I don't have a specific idea on what do to there.
>
> --
> Loïc Minier
>
> https://code.launchpad.net/~liuyq0307/lava-dispatcher/test-bootloader/+merge/123225
> You are the owner of lp:~liuyq0307/lava-dispatcher/test-bootloader.

--
Thanks,
Yongqin Liu
---------------------------------------------------------------
#mailing list
<email address hidden>
http://lists.linaro.org/mailman/listinfo/linaro-android
<email address hidden>
http://lists.linaro.org/pipermail/linaro-validation

[Loïc Minier (lool)]

On Tue, Sep 11, 2012, Yongqin Liu wrote:
> you mean modify the u-boot source?

Yes; maybe this is too intrusive, but I felt it was more elegant and
robust; I list other options though

--
Loïc Minier

=== modified file 'lava_dispatcher/client/base.py'
--- lava_dispatcher/client/base.py	2012-08-23 15:33:00 +0000
+++ lava_dispatcher/client/base.py	2012-09-07 07:34:35 +0000
@@ -449,6 +449,9 @@
         """Reboot the system to the test android image."""
         self._boot_linaro_android_image()
         self.in_test_shell(timeout=900)
+        # ensure we are root (AOSP is not root upon initial login)
+        # and make PS1 changed after run su command
+        self.proc.sendline('su')
         self.proc.sendline("export PS1=\"root@linaro: \"")
         self.proc.expect(self.tester_str, timeout=120)
         #TODO: set up proxy

=== modified file 'lava_dispatcher/client/master.py'
--- lava_dispatcher/client/master.py	2012-08-28 04:56:24 +0000
+++ lava_dispatcher/client/master.py	2012-09-07 07:34:35 +0000
@@ -24,7 +24,6 @@
 import os
 import re
 import shutil
-import subprocess
 import time
 import traceback
 import atexit
@@ -132,8 +131,16 @@
     session.run('mkdir -p /mnt/boot')
     session.run('mount /dev/disk/by-label/testboot /mnt/boot')
     _deploy_tarball_to_board(session, bootfs, '/mnt/boot')
+
+    ##make the test uboot to delay 9 seconds
+    change_test_bootloader_delay = session._client.device_option(
+                                             "change_test_bootloader_delay")
+    if  change_test_bootloader_delay and change_test_bootloader_delay == "1":
+        sed_cmd = 'sed -i s/bootdelay=0/bootdelay=9/ /mnt/lava/boot/u-boot.bin'
+        session.run(sed_cmd)
     session.run('umount /mnt/boot')
 
+
 def _deploy_linaro_android_testboot(session, boottbz2, pkgbz2=None):
     logging.info("Deploying test boot filesystem")
     session.run('umount /dev/disk/by-label/testboot', failok=True)
@@ -149,6 +156,16 @@
 
     _recreate_uInitrd(session)
 
+    ##make the test uboot to delay 9 seconds
+    change_test_bootloader_delay = session._client.device_option(
+                                             "change_test_bootloader_delay")
+    if  change_test_bootloader_delay and change_test_bootloader_delay == "1":
+        sed_cmd = 'sed -i s/bootdelay=0/bootdelay=9/ /mnt/lava/boot/u-boot.bin'
+        session.run(sed_cmd)
+
+    session.run('umount /mnt/lava/boot')
+
+
 def _update_uInitrd_partitions(session, rc_filename):
     # Original android sdcard partition layout by l-a-m-c
     sys_part_org = session._client.device_option("sys_part_android_org")
@@ -186,14 +203,24 @@
 
     # The mount partitions have moved from init.rc to init.partitions.rc
     # For backward compatible with early android build, we updatep both rc files
-    _update_uInitrd_partitions(session, 'init.rc')
-    _update_uInitrd_partitions(session, 'init.partitions.rc')
+    # For omapzoom and aosp the operation for mounting partitions are
+    # in init.omap4pandaboard.rc files
+    possible_partitions_files = string_to_list(
+                                    session._client.config.get(
+                                        'possible_partitions_files'))
+
+    for f in possible_partitions_files:
+        if session.is_file_exist(f):
+            _update_uInitrd_partitions(session, f)
+            # Will update the PS1 in init.rc in the following
+            # So will not cat the file here
+            if f != 'init.rc':
+                session.run("cat %s" % f, failok=True)
 
     session.run(
         'sed -i "/export PATH/a \ \ \ \ export PS1 root@linaro: " init.rc')
 
     session.run("cat init.rc")
-    session.run("cat init.partitions.rc", failok=True)
 
     session.run(
         'cpio -i -t -F ramdisk.cpio | cpio -o -H newc | \
@@ -852,9 +879,8 @@
             index = self.proc.expect(
                 ['.+', pexpect.EOF, pexpect.TIMEOUT], timeout=1, lava_no_logging=1)
 
-    def _enter_uboot(self):
-        interrupt_boot_prompt = self.device_option('interrupt_boot_prompt')
-        if self.proc.expect(interrupt_boot_prompt) != 0:
+    def _enter_uboot(self, interrupt_prompt):
+        if self.proc.expect(interrupt_prompt) != 0:
             raise Exception("Faile to enter uboot")
 
         interrupt_boot_command = self.device_option('interrupt_boot_command')
@@ -876,17 +902,62 @@
     def _boot_linaro_android_image(self):
         self._boot(string_to_list(self.config.get('boot_cmds_android')))
 
+    def _get_key_value(self, key, default_key, escape=False):
+        value = self.device_option(key)
+        if not value:
+            value = self.device_option(default_key)
+
+        if escape:
+            return re.escape(value)
+        else:
+            return value
+
+    @property
+    def master_loader_interrupt_prompt(self):
+        return self._get_key_value('master_interrupt_boot_prompt',
+                                            'interrupt_boot_prompt')
+
+    @property
+    def test_loader_interrupt_prompt(self):
+        return self._get_key_value('test_interrupt_boot_prompt',
+                                            'interrupt_boot_prompt')
+
+    @property
+    def master_bootloader_prompt(self):
+        return self._get_key_value('master_bootloader_prompt',
+                                            'bootloader_prompt',
+                                            escape=True)
+
+    @property
+    def test_bootloader_prompt(self):
+        return self._get_key_value('test_bootloader_prompt',
+                                            'bootloader_prompt',
+                                            escape=True)
+
+    def _load_test_bootlader(self, cmds):
+        for line in range(0, len(cmds)):
+            self.proc.sendline(cmds[0])
+            self.proc.sendline(cmds[line])
+            if line < len(cmds) - 1 :
+                self.proc.expect(self.master_bootloader_prompt, timeout=300)
+
     def _boot(self, boot_cmds):
         try:
             self.soft_reboot()
-            self._enter_uboot()
         except:
             logging.exception("_enter_uboot failed")
             self.hard_reboot()
-            self._enter_uboot()
+
+        ## enter the master boot loader to load the test boot loader
+        self._enter_uboot(self.master_loader_interrupt_prompt)
+        load_test_bootloader_cmds = string_to_list(
+                                self.config.get('load_test_bootloader_cmds'))
+        if load_test_bootloader_cmds:
+            self._load_test_bootlader(load_test_bootloader_cmds)
+            self._enter_uboot(self.test_loader_interrupt_prompt)
+
         self.proc.sendline(boot_cmds[0])
-        bootloader_prompt = re.escape(self.device_option('bootloader_prompt'))
         for line in range(1, len(boot_cmds)):
-            self.proc.expect(bootloader_prompt, timeout=300)
+            self.proc.expect(self.test_bootloader_prompt, timeout=300)
             self.proc.sendline(boot_cmds[line])
 

=== modified file 'lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf	2012-08-08 22:31:12 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf	2012-09-07 07:34:35 +0000
@@ -120,3 +120,24 @@
 git_url_disablesuspend_sh = "http://android.git.linaro.org/gitweb?p=device/linaro/common.git;a=blob_plain;f=disablesuspend.sh;hb=refs/heads/linaro-ics"
 # how long the disablesuspend script should take to complete
 #disablesuspend_timeout = 240
+
+# the u-boot command for loading the test bootloader from the master boot loader
+# Will use the u-boot of the master image to load the test image if not specified
+load_test_bootloader_cmds = 
+
+# This is used to when extend the bootdelay in u-boot.bin to 9 
+# when it is 0 default
+change_test_bootloader_delay = 
+
+# The message to interrupt into the master boot loader
+master_interrupt_boot_prompt = 
+# The prompt of the master boot loader
+master_bootloader_prompt = 
+# The message to interrupt into the test boot loader
+test_interrupt_boot_prompt =
+# The prompt of the test boot loader
+test_bootloader_prompt = 
+
+# the possible partition files need to be modified for android
+possible_partitions_files = init.partitions.rc,
+    init.rc

=== modified file 'lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf	2012-08-24 19:35:21 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf	2012-09-07 07:34:35 +0000
@@ -27,4 +27,12 @@
     vram=48M omapfb.vram=0:24M'",
     boot
 
+change_test_bootloader_delay = 1
+load_test_bootloader_cmds = fatload mmc 0:3 0x80E80000 u-boot.bin,
+    go 0x80E80000
+
 android_binary_drivers = http://192.168.1.21/LAVA_HTTP/android-binaries/panda-drivers.tgz
+
+possible_partitions_files = init.partitions.rc,
+    init.omap4pandaboard.rc,
+    init.rc