Merge lp:~liuyq0307/lava-dispatcher/fix-1195536 into lp:lava-dispatcher

[Yongqin Liu]

1. fix the bug in _find_and_copy function,
     we should use the root to genertate the src path
2. rename the file variable to file_name,
     since file is a keyword of python

[Yongqin Liu (liuyq0307)]

One thing I am not sure is that if I should change the usage of _copy_needed_files_from_partition method in deploy_android method of lava_dispatcher/device/fastmodel.py file.

it is called as such in the deploy_linaro method:
self._copy_needed_files_from_directory(odir)
self._copy_needed_files_from_partition(self.config.boot_part, 'rtsm')
self._copy_needed_files_from_partition(self.config.root_part, 'boot')

but called like this in the deploy_android method:
self._copy_needed_files_from_partition(self.config.boot_part, '')

if we add following line to the deploy_android method, I guess it will work too.
self._copy_needed_files_from_partition(self.config.boot_part, 'rtsm')

but do we really need to specify the second parameter?

[Tyler Baker (tyler-baker)]

Thanks for fixing these issue I introduced :)

However the issue still persists:

http://www.nwdrone.com/scheduler/job/1331

review: Approve

[lp:~liuyq0307/lava-dispatcher/fix-1195536]

633. By Yongqin Liu on 2013-06-28

remove the directory of axf file path specified for simulator_axf_files
based on the thought like this:
we do not have much files in the boot partition,
and there will be no files with the same base name under different directory

634. By Yongqin Liu on 2013-06-28

change for copy necessary uImage and uInitrd files

635. By Yongqin Liu on 2013-06-28

fix the error about parameter of _copy_first_find_from_list

636. By Yongqin Liu on 2013-06-28

fix some other bugs

[Tyler Baker (tyler-baker)]

Still in the process of testing these changes again.

One small issue I've encountered:

- simulator_kernel = schema.StringOption(default=None)
- simulator_initrd = schema.StringOption(default=None)
+ simulator_kernel_files = schema.ListOption(default=None)
+ simulator_initrd_files = schema.ListOption(default=None)

review: Needs Fixing

[Tyler Baker (tyler-baker)]

With the changes mentioned above, I confirmed this fix is now working properly.

[Tyler Baker (tyler-baker)]

We should also update the device-types for the following:

rtsm_foundation-armv8

rtsm_ve-armv8

[lp:~liuyq0307/lava-dispatcher/fix-1195536]

637. By Yongqin Liu on 2013-07-01

fix the bug of definition of simulator_kernel_files and simulator_initrd_files

[Yongqin Liu (liuyq0307)]

Hi, Tyler

I updated the definition of simulator_initrd_files and simulator_kernel_files according to your comments.
Please check that.

> We should also update the device-types for the following:
>
> rtsm_foundation-armv8
>
> rtsm_ve-armv8
For the above comments, sorry, I am not very clear about what to do.
Why should we update these two device types.

They don't need to specify the simulator_initrd_files and simulator_kernel_files now.

[lp:~liuyq0307/lava-dispatcher/fix-1195536]

638. By Yongqin Liu on 2013-07-01

fix bug of use of simulator_kernel and simulator_initrd

[Tyler Baker (tyler-baker)]

> Hi, Tyler
>
> I updated the definition of simulator_initrd_files and simulator_kernel_files
> according to your comments.
> Please check that.
>
> > We should also update the device-types for the following:
> >
> > rtsm_foundation-armv8
> >
> > rtsm_ve-armv8
> For the above comments, sorry, I am not very clear about what to do.
> Why should we update these two device types.
>
> They don't need to specify the simulator_initrd_files and
> simulator_kernel_files now.

At this point they do not. However in the future they will need to ;) We can do this work now or later, I just thought it might be nice to have them updated now.

[Tyler Baker (tyler-baker)]

review: Approve

[Zygmunt Krynicki (zyga)]

File is not a keyword in python.

It is okay to use, it might be confusing sometimes though

Thanks
ZK
2 lip 2013 17:08, "Tyler Baker" <email address hidden> napisał(a):

> Review: Approve
>
>
> --
>
> https://code.launchpad.net/~liuyq0307/lava-dispatcher/fix-1195536/+merge/171954
> You are subscribed to branch lp:lava-dispatcher.
>

=== modified file 'lava_dispatcher/config.py'
--- lava_dispatcher/config.py	2013-06-20 15:37:54 +0000
+++ lava_dispatcher/config.py	2013-07-01 10:49:37 +0000
@@ -85,8 +85,8 @@
     simulator_version_command = schema.StringOption()
     simulator_command = schema.StringOption()
     simulator_axf_files = schema.ListOption()
-    simulator_kernel = schema.StringOption(default=None)
-    simulator_initrd = schema.StringOption(default=None)
+    simulator_kernel_files = schema.ListOption(default=None)
+    simulator_initrd_files = schema.ListOption(default=None)
     simulator_dtb = schema.StringOption(default=None)
     simulator_uefi = schema.StringOption(default=None)
     simulator_boot_wrapper = schema.StringOption(default=None)
 
=== modified file 'lava_dispatcher/default-config/lava-dispatcher/device-types/rtsm_ve-a15x1-a7x1.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/rtsm_ve-a15x1-a7x1.conf	2013-06-19 16:22:43 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/rtsm_ve-a15x1-a7x1.conf	2013-07-01 10:49:37 +0000
@@ -45,17 +45,23 @@
            sendline 2
 
 simulator_axf_files =
-	img.axf
-	linux-system-ISW.axf
-	rtsm/linux-system-semi.axf
-
-simulator_kernel = vmlinuz.*
-simulator_initrd = initrd.*
+    img.axf
+    linux-system-ISW.axf
+    linux-system-semi.axf
+
+simulator_kernel_files =
+    uImage
+    vmlinuz.*
+
+simulator_initrd_files =
+    uInitrd
+    initrd.*
+
 simulator_dtb = rtsm_ve-ca15x1-ca7x1.dtb
 simulator_uefi = uefi_rtsm_ve-ca15.bin
 
 license_file = 8224@localhost
-sim_bin = /opt/arm/RTSM_A15-A7x14_VE/Linux64_RTSM_VE_Cortex-A15x4-A7x4/RTSM_VE_Cortex-A15x4-A7x4
+sim_bin = /opt/arm/RTSM_A15-A7x14_VE/Linux64_RTSM_VE_Cortex-A15x1-A7x1/RTSM_VE_Cortex-A15x1-A7x1
 android_adb_port = 6555
 
 simulator_version_command = %(sim_bin)s --version | grep "Fast Models" | sed 's/Fast Models \[//' | sed 's/\]//'
 
=== modified file 'lava_dispatcher/default-config/lava-dispatcher/device-types/rtsm_ve-a15x4-a7x4.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/rtsm_ve-a15x4-a7x4.conf	2013-06-19 16:22:43 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/rtsm_ve-a15x4-a7x4.conf	2013-07-01 10:49:37 +0000
@@ -45,12 +45,18 @@
            sendline 2
 
 simulator_axf_files =
-	img.axf
-	linux-system-ISW.axf
-	rtsm/linux-system-semi.axf
-
-simulator_kernel = vmlinuz.*
-simulator_initrd = initrd.*
+    img.axf
+    linux-system-ISW.axf
+    linux-system-semi.axf
+
+simulator_kernel_files =
+    uImage
+    vmlinuz.*
+
+simulator_initrd_files =
+    uInitrd
+    initrd.*
+
 simulator_dtb = rtsm_ve-ca15x4-ca7x4.dtb
 simulator_uefi = uefi_rtsm_ve-ca15.bin
 
 
=== modified file 'lava_dispatcher/device/fastmodel.py'
--- lava_dispatcher/device/fastmodel.py	2013-06-25 10:50:02 +0000
+++ lava_dispatcher/device/fastmodel.py	2013-07-01 10:49:37 +0000
@@ -23,10 +23,8 @@
 import cStringIO
 import logging
 import os
-import shutil
 import stat
 import subprocess
-import re
 
 import lava_dispatcher.device.boot_options as boot_options
 
@@ -48,7 +46,6 @@
     ensure_directory,
     extract_targz,
     DrainConsoleOutput,
-    string_to_list,
     )
 
 
@@ -93,24 +90,30 @@
             subdir = os.path.join(mntdir, subdir)
             self._copy_needed_files_from_directory(subdir)
 
+    def _copy_first_find_from_list(self, subdir, odir, file_list):
+        f_path = None
+        for fname in file_list:
+            f_path = self._find_and_copy(subdir, odir, fname)
+            if f_path:
+                break
+
+        return f_path
+
     def _copy_needed_files_from_directory(self, subdir):
         odir = os.path.dirname(self._sd_image)
         if self._bootloader == 'u_boot':
             # Extract the bootwrapper from the image
-            for fname in self.config.simulator_axf_files:
-                if self._axf is None:
-                    self._axf = self._find_and_copy(
-                                   subdir, odir, fname)
-                else:
-                    break
+            if self.config.simulator_axf_files and self._axf is None:
+                self._axf = self._copy_first_find_from_list(subdir, odir,
+                                            self.config.simulator_axf_files)
             # Extract the kernel from the image
-            if self.config.simulator_kernel and self._kernel is None:
-                self._kernel = self._find_and_copy(
-                                   subdir, odir, self.config.simulator_kernel)
+            if self.config.simulator_kernel_files and self._kernel is None:
+                self._kernel = self._copy_first_find_from_list(subdir, odir,
+                                            self.config.simulator_kernel_files)
             # Extract the initrd from the image
-            if self.config.simulator_initrd and self._initrd is None:
-                self._initrd = self._find_and_copy(
-                                   subdir, odir, self.config.simulator_initrd)
+            if self.config.simulator_initrd_files and self._initrd is None:
+                self._initrd = self._copy_first_find_from_list(subdir, odir,
+                                            self.config.simulator_initrd_files)
             # Extract the dtb from the image
             if self.config.simulator_dtb and self._dtb is None:
                 self._dtb = self._find_and_copy(
@@ -128,13 +131,13 @@
                 raise RuntimeError('No AXF found, %r' %
                                    self.config.simulator_axf_files)
             # Kernel is needed only for b.L models
-            if self._kernel is None and self.config.simulator_kernel:
+            if self._kernel is None and self.config.simulator_kernel_files:
                 raise RuntimeError('No KERNEL found, %r' %
-                                   self.config.simulator_kernel)
+                                   self.config.simulator_kernel_files)
             # Initrd is needed only for b.L models
-            if self._initrd is None and self.config.simulator_initrd:
+            if self._initrd is None and self.config.simulator_initrd_files:
                 raise RuntimeError('No INITRD found, %r' %
-                                   self.config.simulator_initrd)
+                                   self.config.simulator_initrd_files)
             # DTB is needed only for b.L models
             if self._dtb is None and self.config.simulator_dtb:
                 raise RuntimeError('No DTB found, %r' %
 
=== modified file 'lava_dispatcher/device/target.py'
--- lava_dispatcher/device/target.py	2013-06-19 21:49:04 +0000
+++ lava_dispatcher/device/target.py	2013-07-01 10:49:37 +0000
@@ -158,14 +158,14 @@
     def _find_and_copy(self, rootdir, odir, pattern, name=None):
         dest = None
         for root, dirs, files in os.walk(rootdir):
-            for file in files:
-                if re.match(pattern, file):
+            for file_name in files:
+                if re.match(pattern, file_name):
                     if name:
                         dest = os.path.join(odir, name)
                     else:
-                        dest = os.path.join(odir, file)
+                        dest = os.path.join(odir, file_name)
                     if rootdir != odir:
-                        src = os.path.join(rootdir, file)
+                        src = os.path.join(root, file_name)
                         shutil.copyfile(src, dest)
                         return dest
                     else: