Merge lp:~liuyq0307/lava-android-test/v8-skia into lp:lava-android-test

[Yongqin Liu]

update v8 and skia test

v8: change to use d8 command in case there is no v8shell command.
    because the v8shell command is change to d8 form JB version

skia: add option for the time parameter, so that we can pass this number
    from android build page to make it not spend too much time

[lp:~liuyq0307/lava-android-test/v8-skia]

233. By Yongqin Liu on 2013-01-22

update 0xbench to accept specified option from android-build page

1. test to be run can be specified as "--ez <test> true", where "<test>"
   can be math/2d/3d/vm/native
2. the time for waiting finish can be specified from android-build
   as option like "-t 3600", but this must be specified with the test to be run,
   otherwise there will be no test to run automatically

234. By Yongqin Liu on 2013-01-22

fix the problem of timeout option -t

1. -t is accepted by am start command, so we changed to use -timeout
2. pass the $OPTIONS to android_0xbenchmark_wait.py

[Yongqin Liu (liuyq0307)]

Also add modification for 0xbench test:

update 0xbench to accept specified option from android-build page

1. test to be run can be specified as "--ez <test> true", where "<test>"
   can be math/2d/3d/vm/native
2. the time for waiting finish can be specified from android-build
   as option like "-timeout 3600", but this must be specified with the test to be run,
   otherwise there will be no test to run automatically

[lp:~liuyq0307/lava-android-test/v8-skia]

235. By Yongqin Liu on 2013-01-22

adjust --ez autorun true to the begin of the am comand

[Andy Doan (doanac)]

v8.py has:

154 -ADB_SHELL_STEPS = ['"cd /data/benchmark/v8; v8shell run.js"']
155 +adb_shell = ('"'
156 + 'cd /data/benchmark/v8;'
157 + 'if which v8shell 2>/dev/null 1>/dev/null;'
158 + 'then v8shell run.js; '
159 + 'else d8 run.js; fi'
160 + '"')
161 +ADB_SHELL_STEPS = [adb_shell]

that's a little awkward, since shell steps can be an array this could be broken up into multiple strings

[Yongqin Liu (liuyq0307)]

> v8.py has:
>
> 154 -ADB_SHELL_STEPS = ['"cd /data/benchmark/v8; v8shell run.js"']
> 155 +adb_shell = ('"'
> 156 + 'cd /data/benchmark/v8;'
> 157 + 'if which v8shell 2>/dev/null 1>/dev/null;'
> 158 + 'then v8shell run.js; '
> 159 + 'else d8 run.js; fi'
> 160 + '"')
> 161 +ADB_SHELL_STEPS = [adb_shell]
>
> that's a little awkward,
Yes, I know this is not good.
but we need all the shell steps in one string.
then we can run it as adb shell "shell steps".
if we change it to ADB_SHELL_STEPS = [steps1, steps2,..], this will not work.

> since shell steps can be an array this could be
> broken up into multiple strings
sorry, not understand clearly here.
in which case it will be broken up?

[lp:~liuyq0307/lava-android-test/v8-skia]

236. By Yongqin Liu on 2013-01-22

revert the change on lava_android_test/provider.py

[Andy Doan (doanac)]

review: Approve

=== modified file 'lava_android_test/test_definitions/0xbench.py'
--- lava_android_test/test_definitions/0xbench.py	2012-06-25 17:23:05 +0000
+++ lava_android_test/test_definitions/0xbench.py	2013-01-22 16:33:27 +0000
@@ -34,6 +34,9 @@
 
 curdir = os.path.realpath(os.path.dirname(__file__))
 
+# "-timeout timeout" must be specified with at least one test
+# can not only specified the "-timeout timeout" as option
+DEFAULT_OPTIONS = '--ez math true --ez 2d true --ez 3d true --ez vm true'
 INSTALL_STEPS_HOST_POST = [
         ("python %s/android-0xbenchmark/android_0xbenchmark_modify_path.py"
          " $(SERIAL)") % curdir]
@@ -43,11 +46,10 @@
          " $(SERIAL)") % curdir]
 RUN_STEPS_ADB_SHELL = ['logcat -c',
         ("am start -n org.zeroxlab.zeroxbenchmark/"
-         "org.zeroxlab.zeroxbenchmark.Benchmark --ez math true"
-         " --ez 2d true --ez 3d true --ez vm true --ez autorun true")]
+         "org.zeroxlab.zeroxbenchmark.Benchmark --ez autorun true $(OPTIONS)")]
 RUN_STEPS_HOST_POST = [
-        'python %s/android-0xbenchmark/android_0xbenchmark_wait.py $(SERIAL)' %
-         curdir]
+        'python %s/android-0xbenchmark/android_0xbenchmark_wait.py '
+        '$(SERIAL) $(OPTIONS)' % curdir]
 
 
 class ZeroXBenchmarkTestParser(lava_android_test.testdef.AndroidTestParser):
@@ -82,4 +84,5 @@
 parser = ZeroXBenchmarkTestParser()
 testobj = lava_android_test.testdef.AndroidTest(testname="0xbench",
             installer=inst, runner=run, parser=parser,
-             org_ouput_file=os.path.join(save_dir, '0xBenchmark.bundle'))
+            org_ouput_file=os.path.join(save_dir, '0xBenchmark.bundle'),
+            default_options=DEFAULT_OPTIONS)

=== modified file 'lava_android_test/test_definitions/android-0xbenchmark/android_0xbenchmark_wait.py'
--- lava_android_test/test_definitions/android-0xbenchmark/android_0xbenchmark_wait.py	2012-06-16 13:36:48 +0000
+++ lava_android_test/test_definitions/android-0xbenchmark/android_0xbenchmark_wait.py	2013-01-22 16:33:27 +0000
@@ -24,15 +24,29 @@
 
 from lava_android_test.utils import stop_at_pattern
 
-if len(sys.argv) == 1:
-    adb_cmd = "adb"
-else:
+adb_cmd = "adb"
+# here assumes that there is no serial number will start with '-'
+# and the options passed are start with '-' as first option
+if len(sys.argv) > 1 and (not sys.argv[1].startswith('-')):
     adb_cmd = "adb -s %s" % (sys.argv[1])
 
+timeout = 2400
+for index in range(1, len(sys.argv)):
+    arg = sys.argv[index]
+    if arg == '-timeout' and \
+       (index + 1 < len(sys.argv)) and \
+       sys.argv[index + 1]:
+        try:
+            timeout = int(sys.argv[index + 1])
+        except ValueError:
+            pass
+        finally:
+            break
+
 logcat_cmd = '%s logcat' % (adb_cmd)
 pattern = "Displayed org.zeroxlab.zeroxbenchmark/.Report"
 
-if not stop_at_pattern(command=logcat_cmd, pattern=pattern, timeout=2400):
+if not stop_at_pattern(command=logcat_cmd, pattern=pattern, timeout=timeout):
     print "0xbench Test: TIMEOUT Fail"
     sys.exit(1)
 

=== modified file 'lava_android_test/test_definitions/skia.py'
--- lava_android_test/test_definitions/skia.py	2012-06-28 09:52:29 +0000
+++ lava_android_test/test_definitions/skia.py	2013-01-22 16:33:27 +0000
@@ -31,13 +31,15 @@
 
 test_name = 'skia'
 
+DEFAULT_OPTIONS = '1000'
+
 INSTALL_STEPS_ADB_PRE = []
 # Skia can do many more benchmarks, but it becomes almost too much data
 # to make a nice chart for. The -match limits the ones we run
 ADB_SHELL_STEPS = ['logcat -c',
-                 'skia_bench -repeat 1000 -timers w -config 565 -match bitmap',
-                 'skia_bench -repeat 1000 -timers w -config 565 -match rects',
-                 'skia_bench -repeat 1000 -timers w -config 565 -match repeat',
+                 'skia_bench -repeat $(OPTIONS) -timers w -config 565 -match bitmap',
+                 'skia_bench -repeat $(OPTIONS) -timers w -config 565 -match rects',
+                 'skia_bench -repeat $(OPTIONS) -timers w -config 565 -match repeat',
                   'logcat -d -s "skia:*"']
 
 
@@ -83,4 +85,5 @@
 testobj = lava_android_test.testdef.AndroidTest(testname=test_name,
                                     installer=inst,
                                     runner=run,
-                                    parser=parser)
+                                    parser=parser,
+                                    default_options=DEFAULT_OPTIONS)

=== modified file 'lava_android_test/test_definitions/v8.py'
--- lava_android_test/test_definitions/v8.py	2012-06-25 17:37:01 +0000
+++ lava_android_test/test_definitions/v8.py	2013-01-22 16:33:27 +0000
@@ -33,7 +33,13 @@
 test_name = 'v8'
 
 INSTALL_STEPS_ADB_PRE = []
-ADB_SHELL_STEPS = ['"cd /data/benchmark/v8; v8shell run.js"']
+adb_shell = ('"'
+             'cd /data/benchmark/v8;'
+             'if which v8shell 2>/dev/null 1>/dev/null;'
+             'then  v8shell run.js; '
+             'else d8 run.js; fi'
+             '"')
+ADB_SHELL_STEPS = [adb_shell]
 PATTERN = "^(?P<test_case_id>.*?):\s+(?P<measurement>\d+)\s*$"
 
 inst = lava_android_test.testdef.AndroidTestInstaller(