Merge lp:~liuyq0307/lava-android-test/methanol into lp:lava-android-test

[Yongqin Liu]

add test for methanol
update according to the review comment
Add the support for running on chrome browser

[Yongqin Liu (liuyq0307)]

change to use BaseHTTPServer to server the web pages and cgi server

[Yongqin Liu (liuyq0307)]

This branch has been tested on staging. Please see the following jobs:
http://staging.validation.linaro.org/scheduler/job/25406
http://staging.validation.linaro.org/scheduler/job/25407

[Andy Doan (doanac)]

On 08/09/2012 08:45 AM, Yongqin Liu wrote:
> === added directory 'lava_android_test/test_definitions/methanol'
> === added file 'lava_android_test/test_definitions/methanol.py'
> --- lava_android_test/test_definitions/methanol.py 1970-01-01 00:00:00 +0000
> +++ lava_android_test/test_definitions/methanol.py 2012-08-09 11:16:19 +0000
<snip>
> + def real_parse(self, result_filename=None, output_filename='methanol_result.json',
> + test_name=''):
> + """Parse test output to gather results
> + Use the pattern specified when the class was instantiated to look
> + through the results line-by-line and find lines that match it.
> + Results are then stored in self.results. If a fixupdict was supplied
> + it is used to convert test result strings to a standard format.
> + """

Seems like this comment should go in the parent's implementation rather
than this one.

> === added file 'lava_android_test/test_definitions/methanol/methanol.sh'
> --- lava_android_test/test_definitions/methanol/methanol.sh 1970-01-01 00:00:00 +0000
> +++ lava_android_test/test_definitions/methanol/methanol.sh 2012-08-09 11:16:19 +0000

<snip>

> +#the default ip or domain used by the client to access the server
> +domain_ip='192.168.1.10'

Why should we have to have a default. We can figure out our own IP
address. Actually the python server you wrote figures it out, so maybe
we should just get it from the file it creates for us.

> +########################################################
> +###### NOT MODIFY SOURCE OF BELOW #####
> +########################################################
> +#android_dir="/sdcard/methanol"
Remove this line if its not needed anymore

<snip>
> +
> +function deploy(){
> + cur_path=`pwd`
> + target_dir=`mktemp -u --tmpdir=${cur_path} methanol-XXX`
> + git clone "${methanol_git}" "${target_dir}"
> + if [ $? -ne 0 ];then
> + echo "Faile to clone the methanol source from ${methanol_git}"
typo "Faile" should be "Failed"
> + cleanup
> + exit 1
> + fi

> +function test_methanol(){
> + if [ -n "$1" ]; then
> + test_type="-${1}"
> + else
> + test_type=""
> + fi
> +
> + wait_minutes=${2-1}
> +
> + mkdir -p/tmp/methanol/
> + sudo chmod -R 777 /tmp/methanol

Do we really need /tmp/methanol? this is just something we'll create and
never clean up.

> === modified file 'lava_android_test/testdef.py'
> --- lava_android_test/testdef.py 2012-06-28 09:52:29 +0000
> +++ lava_android_test/testdef.py 2012-08-09 11:16:19 +0000

> @@ -410,6 +414,25 @@
> Results are then stored in self.results. If a fixupdict was supplied
> it is used to convert test result strings to a standard format.
> """
> +
> + self.real_parse(result_filename=result_filename,
> + output_filename=output_filename, test_name=test_name)
> +
> + self.fixresults(self.fixupdict)
> + if self.appendall:
> + self.appendtoall(self.appendall)
> + self.fixmeasurements()
> + self.fixids(test_name=test_name)
> +
> + def real_parse(self, result_filename='stdout.log',
> + ...

[Yongqin Liu (liuyq0307)]

> > +#the default ip or domain used by the client to access the server
> > +domain_ip='192.168.1.10'
>
> Why should we have to have a default. We can figure out our own IP
> address. Actually the python server you wrote figures it out, so maybe
> we should just get it from the file it creates for us.
Sorry, could you show me how to figure out the available IP?
Using the ifconfig command to get the IP of eth0 is possible, but if the server has more than 2 IPs,
then we can't know which one is the right.
and if we pass 0.0.0.0, then all the IP address will be opened,
and this will cause security problems I think.

> > + mkdir -p/tmp/methanol/
> > + sudo chmod -R 777 /tmp/methanol
>
> Do we really need /tmp/methanol? this is just something we'll create and
> never clean up.
we can't pass a path(including /) to the save_methanol_data.py when using CGIHTTPServer as the server.
so we passed only the file name and get the result file from the default directory /tmp/methanol.
I delete the above two sentence, but the "/tmp/methanol" directory will still exist and won't be deleted.

> > + def real_parse(self, result_filename='stdout.log',
> > + output_filename='stdout.log', test_name=''):
> > + """Parse test output to gather results
> > +
> > + Use the pattern specified when the class was instantiated to look
> > + through the results line-by-line and find lines that match it.
> > + Results are then stored in self.results. If a fixupdict was
> supplied
> > + it is used to convert test result strings to a standard format.
> > + """
>
> There are a few other places in code that could use this function. I
> think: tjbench.py, skia.py, and 0xbench.py. We should fix those up as well.
yes, they are the same type, but here I'd like to leave them as they are now, not to touch them here.
we can update them when we need to update them in the future using this function,
and use this function when add new similar type test.

Thanks,
Yongqin Liu.

[Andy Doan (doanac)]

On 08/10/2012 10:18 PM, Yongqin Liu wrote:
>>> > >+#the default ip or domain used by the client to access the server
>>> > >+domain_ip='192.168.1.10'
>> >
>> >Why should we have to have a default. We can figure out our own IP
>> >address. Actually the python server you wrote figures it out, so maybe
>> >we should just get it from the file it creates for us.
> Sorry, could you show me how to figure out the available IP?
> Using the ifconfig command to get the IP of eth0 is possible, but if the server has more than 2 IPs,
> then we can't know which one is the right.
> and if we pass 0.0.0.0, then all the IP address will be opened,
> and this will cause security problems I think.
>

I think 0.0.0.0 is okay, we have firewalls to prevent access to the port
it will open.

[Michael Hudson-Doyle (mwhudson)]

Andy Doan <email address hidden> writes:

> On 08/10/2012 10:18 PM, Yongqin Liu wrote:
>>>> > >+#the default ip or domain used by the client to access the server
>>>> > >+domain_ip='192.168.1.10'
>>> >
>>> >Why should we have to have a default. We can figure out our own IP
>>> >address. Actually the python server you wrote figures it out, so maybe
>>> >we should just get it from the file it creates for us.
>> Sorry, could you show me how to figure out the available IP?
>> Using the ifconfig command to get the IP of eth0 is possible, but if the server has more than 2 IPs,
>> then we can't know which one is the right.
>> and if we pass 0.0.0.0, then all the IP address will be opened,
>> and this will cause security problems I think.
>>
>
> I think 0.0.0.0 is okay, we have firewalls to prevent access to the port
> it will open.

That still doesn't help the test know which IP address to give to the
browser on the DUT...

Cheers,
mwh

[lp:~liuyq0307/lava-android-test/methanol]

203. By Yongqin Liu on 2012-08-13

redirect the log of the simple server to a file

204. By Yongqin Liu on 2012-08-13

change the way to no login for chrome and decrease the wait time for svg and example

205. By Yongqin Liu on 2012-08-15

change to use option to receive the ip address

206. By Yongqin Liu on 2012-08-15

add default option in methanol.py

207. By Yongqin Liu on 2012-08-15

add the option to the test_id in result bundle

208. By Yongqin Liu on 2012-08-15

fix for twice cleanup and kill chrome for the first time

209. By Yongqin Liu on 2012-08-16

increase the wait for for fire.html and fire-svg.html on chrome

Unmerged revisions

=== modified file 'MANIFEST.in'
--- MANIFEST.in	2012-07-10 06:34:06 +0000
+++ MANIFEST.in	2012-08-16 10:38:20 +0000
@@ -13,4 +13,5 @@
 include lava_android_test/test_definitions/pm-qa/*
 include lava_android_test/test_definitions/shells/*
 include lava_android_test/test_definitions/hostshells/*
+include lava_android_test/test_definitions/methanol/*
 

=== modified file 'lava_android_test/commands.py'
--- lava_android_test/commands.py	2012-08-07 02:28:47 +0000
+++ lava_android_test/commands.py	2012-08-16 10:38:20 +0000
@@ -751,6 +751,14 @@
                                              serial)
     if test_id:
         bundle['test_runs'][0]['test_id'] = test_id
+    else:
+        attrs = bundle['test_runs'][0].get('attributes')
+        if attrs:
+            run_options = attrs.get('run_options')
+            if run_options:
+                test_id = '%s(%s)' % (bundle['test_runs'][0]['test_id'],
+                                      run_options)
+                bundle['test_runs'][0]['test_id'] = test_id
 
     test_tmp.parse(result_id)
     stdout_text = adb.read_file(os.path.join(resultdir,

=== added directory 'lava_android_test/test_definitions/methanol'
=== added file 'lava_android_test/test_definitions/methanol.py'
--- lava_android_test/test_definitions/methanol.py	1970-01-01 00:00:00 +0000
+++ lava_android_test/test_definitions/methanol.py	2012-08-16 10:38:20 +0000
@@ -0,0 +1,66 @@
+# Copyright (c) 2012 Linaro
+
+# Author: Linaro Validation Team <[e-mail]>
+#
+# This file is part of LAVA Android Test.
+#
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+The Methanol is a page load benchmarking engine.
+The benchmark engine measures the overall page layouting and rendering time of the browser.
+Only several test cases are attached to the engine, because of licensing issue.
+
+**URL:** https://gitorious.org/methanol
+
+**Default options:** None
+"""
+
+import os
+import json
+
+import lava_android_test.testdef
+
+from lava_android_test.config import get_config
+
+curdir = os.path.realpath(os.path.dirname(__file__))
+config = get_config()
+
+result_dir = config.resultsdir_android
+RUN_STEPS_HOST_PRE = ["bash %s/methanol/methanol.sh $(SERIAL) $(OPTIONS)" % curdir]
+
+class MethanolTestParser(lava_android_test.testdef.AndroidTestParser):
+
+    def real_parse(self, result_filename=None, output_filename='methanol_result.json',
+               test_name=''):
+        """Parse test output to gather results
+        Use the pattern specified when the class was instantiated to look
+        through the results line-by-line and find lines that match it.
+        Results are then stored in self.results.  If a fixupdict was supplied
+        it is used to convert test result strings to a standard format.
+        """
+        with open(output_filename) as stream:
+            test_results_data = stream.read()
+            test_results_json = json.loads(test_results_data)
+            self.results['test_results'] = test_results_json
+
+
+inst = lava_android_test.testdef.AndroidTestInstaller()
+run = lava_android_test.testdef.AndroidTestRunner(steps_host_pre=RUN_STEPS_HOST_PRE)
+parser = MethanolTestParser()
+testobj = lava_android_test.testdef.AndroidTest(testname="methanol",
+            installer=inst, runner=run, parser=parser,
+            org_ouput_file='/data/local/methanol/methanol_result.json',
+            default_options='-b DEFAULT -d 192.168.1.10')

=== added file 'lava_android_test/test_definitions/methanol/methanol.sh'
--- lava_android_test/test_definitions/methanol/methanol.sh	1970-01-01 00:00:00 +0000
+++ lava_android_test/test_definitions/methanol/methanol.sh	2012-08-16 10:38:20 +0000
@@ -0,0 +1,317 @@
+#!/bin/bash
+# Copyright (c) 2012 Linaro
+
+# Author: Linaro Validation Team <[e-mail]>
+#
+# This file is part of LAVA Android Test.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+#the default ip or domain used by the client to access the server
+DEFAULT_DOMAIN_IP='192.168.1.10'
+DEFAULT_BROWSER='DEFAULT'
+
+########################################################
+######        NOT MODIFY SOURCE OF BELOW           #####
+########################################################
+THIS_FILE="$0"
+methanol_git="git://gitorious.org/methanol/methanol.git"
+chrome_apk_url="http://testdata.validation.linaro.org/chrome/Chrome-latest.apk"
+server_settings_file="/etc/lava/web_server/settings.conf"
+result_dir_android="/data/local/methanol"
+declare -a RESULTS=();
+methanol_url="/"
+report_url="/cgi/save_methanol_data.py"
+report_res_dir="/tmp/methanol"
+target_dir=""
+server_pid=""
+
+ADB_OPTION=""
+domain_ip="${DEFAULT_DOMAIN_IP}"
+browser="${DEFAULT_BROWSER}"
+
+function parse_arg(){
+    serial=""
+    while test -n "$1"; do
+        case "$1" in
+            --browser|-b)
+                if [ "x$2" = "x" ]; then
+                    echo "Error: $1 requires an argument which should be DEFAULT or CHROME"
+                    exit 1
+                fi
+                browser="$2"
+                if [ "X${browser}" != "XDEFAULT" ] && [ "X${browser}" != "XCHROME" ]; then
+                    echo "Error: $1 requires an argument  which should be DEFAULT or CHROME"
+                    exit 1
+                fi
+                shift 2
+                ;;
+            --domain|-d)
+                if [ "x$2" = "x" ]; then
+                    echo "Error: $1 requires an argument"
+                    exit 1
+                fi
+                domain_ip="$2"
+                shift 2
+                ;;
+            --help|-h)
+                show_usage
+                exit 0
+                ;;
+            *)
+                if [ -n "${serial}" ]; then
+                    show_usage
+                    echo "Too many arguments, see --help for details"
+                    exit 1
+                else
+                    serial="$1"
+                    shift
+                fi
+                ;;
+        esac
+    done
+
+    if [ -n "${serial}" ]; then
+        ADB_OPTION="-s ${serial}"
+    fi
+}
+
+function show_usage(){
+    echo "`basename ${THIS_FILE}` [serial_no] [-b DEFAULT|CHROME] [-d domain_ip]"
+}
+
+function patch_sources(){
+    src_root_dir=${1}
+    if [ -z ${src_root_dir} ]; then
+        return
+    fi
+
+    if [ \! -d ${src_root_dir} ]; then
+        return
+    fi
+
+    ## The following test case cannot be on android browser, so comment them out
+    ## Patch the engine.js
+    sed -i "s% + results;% + results.replace\(new RegExp('/','gm'), '_'\);%" ${src_root_dir}/engine.js
+
+    ##Patch svg.js
+    sed -i s%\"svg/anim/earth.svg\",%\"svg/anim/earth.svg\"/*,%    ${src_root_dir}/svg.js
+    sed -i s%\"svg/anim/svg.svg\"%\"svg/anim/svg.svg\"*/%    ${src_root_dir}/svg.js
+
+    ##Patch smp.js
+    sed -i s%\"smp/3d-terrain-demo/single/index.html\",%//\"smp/3d-terrain-demo/single/index.html\",%   ${src_root_dir}/smp.js
+    sed -i s%\"smp/3d-terrain-demo/worker/index.html\",%//\"smp/3d-terrain-demo/worker/index.html\",%   ${src_root_dir}/smp.js
+    sed -i s%\"smp/fire-on-water/worker/index.html\",%//\"smp/fire-on-water/worker/index.html\",%   ${src_root_dir}/smp.js
+}
+
+function deploy(){
+
+    if [ "${browser}" = "CHROME" ]; then
+        echo "wget --progress=dot -e dotbytes=1M -np -l 10 --no-check-certificate ${chrome_apk_url} -O ./Chrome-latest.apk"
+        wget --progress=dot -e dotbytes=1M -np -l 10 --no-check-certificate ${chrome_apk_url} -O ./Chrome-latest.apk
+        if [ $? -ne 0 ]; then 
+            echo "Failed to download the chrome apk file from ${chrome_apk_url}."
+            cleanup
+            exit 1
+        fi
+        
+        adb ${ADB_OPTION} uninstall com.android.chrome
+
+        echo "adb ${ADB_OPTION} install ./Chrome-latest.apk"
+        adb ${ADB_OPTION} install ./Chrome-latest.apk
+        if [ $? -ne 0 ]; then 
+            echo "Failed to install the Chrome browser application."
+            cleanup
+            rm -f ./Chrome-latest.apk
+            exit 1
+        fi
+        rm -f ./Chrome-latest.apk
+        adb ${ADB_OPTION} shell am start com.android.chrome/com.google.android.apps.chrome.Main
+        sleep 10
+        f_preferences='/data/data/com.android.chrome/shared_prefs/com.android.chrome_preferences.xml'
+        f_preferences_base=`basename ${f_preferences}`
+        owner_grp=`adb ${ADB_OPTION} shell ls -l ${f_preferences}|cut -d \  -f2`
+        if [ -z "${owner_grp}" ]; then
+            echo "Failed to get the user/group infromation of chrome preferences file."
+            cleanup
+            rm -f ./Chrome-latest.apk
+            exit 1
+        fi
+        adb ${ADB_OPTION} pull ${f_preferences} ./${f_preferences_base}
+        sed -i '/<map>/ a\<boolean name="first_run_flow" value="true" />' ./${f_preferences_base}
+        adb ${ADB_OPTION} push ./${f_preferences_base} ${f_preferences}
+        adb ${ADB_OPTION} shell chown ${owner_grp}:${owner_grp} ${f_preferences}
+        chrome_pid=`adb ${ADB_OPTION} shell ps |grep -P 'chrome\s*$'|tr -s ' '|cut -d \  -f2`
+        if [ -n "${chrome_pid}" ]; then
+            adb ${ADB_OPTION} shell kill ${chrome_pid}
+        fi
+        rm -f ./${f_preferences_base}
+    fi
+
+    cur_path=`pwd`
+    target_dir=`mktemp -u --tmpdir=${cur_path} methanol-XXX`
+    git clone "${methanol_git}" "${target_dir}"
+    if [ $? -ne 0 ];then
+        echo "Failed to clone the methanol source from ${methanol_git}"
+        cleanup
+        exit 1
+    fi
+
+    #patch just because some test can not be run on android
+    patch_sources "${target_dir}"
+
+    url_file=`mktemp -u --tmpdir=${cur_path} url-XXX`
+    nohup python `dirname $0`/start_server.py "${domain_ip}" "${target_dir}" "${url_file}" &>server.log &
+    server_pid=$!
+    sleep 5
+    domain_protocol=`cat ${url_file}`
+    if [ -z "${domain_protocol}" ]; then
+        echo "Cannot get the url of the temporary created server."
+        echo "Failed to deploy the temporary server"
+        cleanup
+        exit 1
+    fi
+}
+
+function check_url(){
+    if [ -n "${report_url}" ]; then
+        wget -q "${domain_protocol}/${report_url}" -O /dev/null
+        if [ $? -ne 0 ]; then
+            echo "The report url(${domain_protocol}/${report_url}) cannot be accessed"
+            echo "Please put the save_methanol_data.py to the cgi-bin directory"
+            echo "of your web server, and make sure it is accessible."
+            cleanup
+            exit 1
+        fi
+    fi
+
+    if [ -n "${methanol_url}" ]; then
+        wget -q "${domain_protocol}/${methanol_url}" -O /dev/null
+        if [ $? -ne 0 ]; then
+            echo "The url(${domain_protocol}/${methanol_url}) cannot be accessed"
+            echo "Please clone the methanol directory to local via following command"
+            echo "    git clone ${methanol_git}"
+            echo "and copy the entire directory to some place of your web server"
+            echo "and make sure it is accessible."
+            cleanup
+            exit 1
+        fi
+    else
+        echo "Please speecify the methanol url that will be used for test."
+        cleanup
+        exit 1
+    fi
+}
+
+function wait_result(){
+    if [ -n "$1" ]; then
+        file_path="$1"
+    else
+        return 0
+    fi
+    wait_minutes=${2-1}
+
+    for (( i=1; i<=${wait_minutes}; i++ )); do
+        sleep 60
+        if [ -f "${file_path}" ]; then
+            return 0
+        fi
+    done
+    return 1
+}
+
+function test_methanol(){
+    if [ -n "$1" ]; then
+        test_type="-${1}"
+    else
+        test_type=""
+    fi
+
+    wait_minutes=${2-1}
+
+    result_file=`mktemp -u --tmpdir=${report_res_dir} fire${test_type}-XXX.json`
+    res_basename=`basename ${result_file}`
+    test_url="${domain_protocol}/${methanol_url}/fire${test_type}.html"
+    if [ -n "${report_url}" ]; then
+        test_url="${test_url}?reportToUrl=${report_url}%3Fsave2file=${res_basename}"
+    fi
+    
+    component_default="com.android.browser/.BrowserActivity"
+    component_chrome=" com.android.chrome/com.google.android.apps.chrome.Main"
+    if [ "${browser}" = "CHROME" ]; then
+        component=${component_chrome}
+    else
+        component=${component_default}
+    fi
+    echo "adb ${ADB_OPTION} shell am start -a android.intent.action.VIEW -d ${test_url} -n ${component}"
+    adb ${ADB_OPTION} shell "am start -a android.intent.action.VIEW -d ${test_url} -n ${component}"
+    wait_result "${result_file}" ${wait_minutes}
+    if [ $? -eq 0 ]; then
+        cur_path=`pwd`
+        cp -uvf ${result_file}  ${cur_path}/${res_basename}
+        echo "result_file=${cur_path}/${res_basename}"
+        RESULTS[${#RESULTS[@]}]="${cur_path}/${res_basename}"
+
+        rm -f ${result_file}
+    else
+        echo "Failed to get the test result of fire${test_type}.html"
+        #cleanup
+        #exit 1
+    fi
+}
+
+function cleanup(){
+    echo "DO CLEAN UP"
+    rm -fr methanol_result.json "${RESULTS[@]}"
+    if [ -n "${server_pid}" ]; then
+        kill -9 ${server_pid}
+    fi
+    if [ -n "${target_dir}" ]; then
+        rm -fr "${target_dir}"
+    fi
+    if [ "${browser}" = "CHROME" ]; then
+        adb ${ADB_OPTION} uninstall com.android.chrome
+    fi
+}
+
+function main(){
+    parse_arg "$@"
+    
+    trap cleanup EXIT
+
+    deploy
+
+    check_url
+    echo `date`: starts to test fire.html
+    test_methanol "" 30 
+    echo `date`: starts to test fire-svg.html
+    test_methanol "svg" 150
+    echo `date`: starts to test fire-smp.html
+    test_methanol "smp" 100
+    echo `date`: all tests completed
+
+    echo "Merge results of file: ${RESULTS[@]}"
+    `dirname $0`/methanol_merge_results.py methanol_result.json "${RESULTS[@]}"
+    if [ $? -eq 0 ]; then
+        adb ${ADB_OPTION} shell mkdir ${result_dir_android}
+        adb ${ADB_OPTION} push methanol_result.json "${result_dir_android}/methanol_result.json"
+        for f in "${RESULTS[@]}"; do
+            adb ${ADB_OPTION} push "${f}" "${result_dir_android}"
+        done
+        echo "The result is also push to android: ${result_dir_android}/${res_basename}"
+    else
+        echo "Failed to merge the results"
+    fi
+}
+main "$@"

=== added file 'lava_android_test/test_definitions/methanol/methanol_merge_results.py'
--- lava_android_test/test_definitions/methanol/methanol_merge_results.py	1970-01-01 00:00:00 +0000
+++ lava_android_test/test_definitions/methanol/methanol_merge_results.py	2012-08-16 10:38:20 +0000
@@ -0,0 +1,81 @@
+#!/usr/bin/python
+# Copyright (c) 2012 Linaro
+
+# Author: Linaro Validation Team <[e-mail]>
+#
+# This file is part of LAVA Android Test.
+#
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import re
+import string
+import sys
+
+import simplejson as json
+
+if len(sys.argv) < 3:
+    basename = os.path.basename(sys.argv[0])
+    print 'Please specified the merge target file and source files like:'
+    print '\t %s target-result-file source-file1 source-file2 ...'
+    sys.exit(1)
+
+target_file = sys.argv[1]
+test_results = []
+for f in sys.argv[2:]:
+    if not os.path.exists(f):
+        print "The file(%s) does not exist" % f
+        continue
+
+    with open(f) as stream:
+        f_basename = os.path.basename(f)
+        last_hyphen_index = string.rfind(f_basename, '-')
+        if last_hyphen_index != -1:
+            file_id = f_basename[:last_hyphen_index]
+        else:
+            file_id = ''
+
+        jobdata = stream.read()
+        results_data = json.loads(jobdata)
+        for res in  results_data:
+            test_case_id = res.get('test_case_id')
+            average = res.get('average')
+            avg_dev = res.get('average_deviate')
+            if file_id and test_case_id == 'summary':
+                test_case_id = '%s-summary' % file_id
+            test_case_id = test_case_id.replace('/', '_')
+            badchars = "[^a-zA-Z0-9\._-]"
+            test_case_id = re.sub(badchars, "", test_case_id.replace(" ", "_"))
+            test_results.append({'test_case_id': '%s_avg' % test_case_id,
+                                 'result': 'pass',
+                                 'measurement': average,
+                                 'units': 'ms'})
+            test_results.append({'test_case_id': '%s_avg_dev' % test_case_id,
+                                 'result': 'pass',
+                                 'measurement': avg_dev,
+                                 'units': '%'})
+
+
+with open(target_file, 'w') as fd:
+    indent = ' ' * 2
+    separators = (', ', ': ')
+    json.dump(test_results, fd,
+               use_decimal=True,
+               indent=indent,
+               separators=separators,
+               sort_keys=False)
+
+print "The result has been merged in file: %s" % target_file
+sys.exit(0)

=== added file 'lava_android_test/test_definitions/methanol/start_server.py'
--- lava_android_test/test_definitions/methanol/start_server.py	1970-01-01 00:00:00 +0000
+++ lava_android_test/test_definitions/methanol/start_server.py	2012-08-16 10:38:20 +0000
@@ -0,0 +1,62 @@
+#/usr/bin/python
+# Copyright (c) 2012 Linaro
+
+# Author: Linaro Validation Team <[e-mail]>
+#
+# This file is part of LAVA Android Test.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import sys
+import CGIHTTPServer
+import BaseHTTPServer
+
+### check parameter
+if len(sys.argv) < 3:
+    print 'Please spsecify the ip and directory like this:'
+    print '    %s domain-or-ip directory-path url-file' % (
+                                                   os.path.basename(__file__))
+    sys.exit(1)
+
+domain = sys.argv[1]
+directory = sys.argv[2]
+if len(sys.argv) == 4:
+    url_file = sys.argv[3]
+else:
+    url_file = ''
+
+## change to that directory
+old_dir = os.getcwd()
+os.chdir(directory)
+
+## set the server configuration before start
+cgi_handler = CGIHTTPServer.CGIHTTPRequestHandler
+cgi_handler.cgi_directories.append('/cgi')
+httpd = BaseHTTPServer.HTTPServer((domain, 0), cgi_handler)
+url = '%s://%s:%s/' % ('http', httpd.socket.getsockname()[0],
+                        httpd.socket.getsockname()[1])
+
+## out put the url information to console and file for other script reference
+print "serving at url=", url
+if url_file:
+    with open(url_file, 'w') as stream:
+        stream.write(url)
+        print 'The information of url also have been wirtten into file(%s)' % (
+                                    url_file)
+
+try:
+    httpd.serve_forever()
+finally:
+    os.chdir(old_dir)

=== modified file 'lava_android_test/testdef.py'
--- lava_android_test/testdef.py	2012-06-28 09:52:29 +0000
+++ lava_android_test/testdef.py	2012-08-16 10:38:20 +0000
@@ -131,7 +131,7 @@
             output = self.adb.run_adb_cmd('shell cat %s' % optionfile)[1]
             bundle['test_runs'][0]['attributes']['install_options'] = output[0]
 
-    def _savetestdata(self, analyzer_assigned_uuid):
+    def _savetestdata(self, analyzer_assigned_uuid, run_options=""):
         config = get_config()
         TIMEFORMAT = '%Y-%m-%dT%H:%M:%SZ'
         bundle = {
@@ -142,7 +142,7 @@
                 'analyzer_assigned_date':
                         self.runner.starttime.strftime(TIMEFORMAT),
                 'time_check_performed': False,
-                'attributes':{},
+                'attributes':{'run_options': run_options},
                 'test_id': self.testname,
                 'test_results':[],
                 'attachments':[],
@@ -176,7 +176,7 @@
         self.runner.run(self.resultsdir, run_options=run_options)
         self._copyorgoutputfile(self.resultsdir)
         self._screencap(self.resultsdir)
-        self._savetestdata(str(uuid4()))
+        self._savetestdata(str(uuid4()), run_options=run_options)
         result_id = os.path.basename(self.resultsdir)
         print("ANDROID TEST RUN COMPLETE: Result id is '%s'" % result_id)
         os.chdir(self.origdir)
@@ -319,6 +319,10 @@
                     cmd = cmd.replace('$(OPTIONS)', option)
                 else:
                     cmd = cmd.replace('$(OPTIONS)', '')
+                if resultsdir is not None:
+                    cmd = cmd.replace('$(RESULTDIR)', resultsdir)
+                else:
+                    cmd = cmd.replace('$(RESULTDIR)', '')
                 cmd = cmd.strip()
                 ret_code = self.adb.run_adb_shell_for_test(cmd,
                                                            stdoutlog,
@@ -410,6 +414,23 @@
         Results are then stored in self.results.  If a fixupdict was supplied
         it is used to convert test result strings to a standard format.
         """
+
+        self.real_parse(result_filename=result_filename,
+              output_filename=output_filename, test_name=test_name)
+
+        self.fixresults(self.fixupdict)
+        if self.appendall:
+            self.appendtoall(self.appendall)
+        self.fixmeasurements()
+        self.fixids(test_name=test_name)
+
+    def real_parse(self, result_filename='stdout.log',
+              output_filename='stdout.log', test_name=''):
+        """Using the pattern to do the real parse operation
+        
+        generate the test_results elements from the result file by parsing 
+        with the pattern specified.
+        """
         if not self.pattern:
             return
 
@@ -448,11 +469,6 @@
                 if data.get('result') is None:
                     data['result'] = test_ok and 'pass' or 'fail'
                 self.results['test_results'].append(data)
-        self.fixresults(self.fixupdict)
-        if self.appendall:
-            self.appendtoall(self.appendall)
-        self.fixmeasurements()
-        self.fixids(test_name=test_name)
 
     def append(self, testid, entry):
         """Appends a dict to the test_results entry for a specified testid
@@ -593,6 +609,11 @@
             cmd = cmd.replace('$(OPTIONS)', option)
         else:
             cmd = cmd.replace('$(OPTIONS)', '')
+        if resultsdir is not None:
+            cmd = cmd.replace('$(RESULTDIR)', resultsdir)
+        else:
+            cmd = cmd.replace('$(RESULTDIR)', '')
+
         cmd = cmd.strip()
         rc, output = adb.run_cmd_host(cmd, quiet=False)
         if rc:
@@ -610,6 +631,10 @@
             cmd = cmd.replace('$(OPTIONS)', option)
         else:
             cmd = cmd.replace('$(OPTIONS)', '')
+        if resultsdir is not None:
+            cmd = cmd.replace('$(RESULTDIR)', resultsdir)
+        else:
+            cmd = cmd.replace('$(RESULTDIR)', '')
         cmd = cmd.strip()
         rc, output = adb.run_adb_cmd(cmd, quiet=False)
         if rc: