Merge lp:~liuyq0307/lava-android-test/cts-update into lp:lava-android-test

[Yongqin Liu]

Update the CTS test packages
In cts_prepare.sh there are some steps need to be done before the CTS test.
But have not found the right method to do this.
And it seems not affect the execution of CTS test.

There are the possibilities that some test cases will fail because of the lack of these settings.
If this is ture, then let first keep this information in some place.
When we find the ways to do the settings, delete the related information that time.

From now the possible methods I can think is to do the automatic settings by monkeyrunner script or instruments, but it seems not much flexible. When we add other settings later, these scripts will need update again.:(

But here first please review the entire follow of CTS test, the above problem should only be limited to the cts_prepare.sh file.

[lp:~liuyq0307/lava-android-test/cts-update]

195. By Yongqin Liu on 2012-06-28

add adb reconnect for adb command

196. By Yongqin Liu on 2012-06-28

modify the bug that not pass quiet parameter

197. By Yongqin Liu on 2012-06-28

decrease the sleep time after disconnect, and add status before the reconnect

198. By Yongqin Liu on 2012-06-28

modify the bug about return value

199. By Yongqin Liu on 2012-06-28

merge with trunk

[Andy Doan (doanac)]

adb.py:

149 + def conncect(self):

can we fix the spelling to "connect"

[lp:~liuyq0307/lava-android-test/cts-update]

200. By Yongqin Liu on 2012-07-02

modify the type conncect of connect

[Yongqin Liu (liuyq0307)]

> adb.py:
>
> 149 + def conncect(self):
>
> can we fix the spelling to "connect"

Sorry, have updated.
Please review again.

[Andy Doan (doanac)]

A question on cts.py:

338 +RUN_STEPS_HOST_PRE = ['python %s/cts/cts_wrapper.py $(SERIAL)' % curdir]

The "python" command should work as expected in our old pip/virtualenv style deployment. Will this work in our new buildout style? In other words, when I run:

 . /srv/lava/instances/staging/bin/activate
 which python

I get "/usr/bin/python". I don't know for sure if this will pick up the lava-android-test python path. It via inheriting an envionrment from the parent process. You may want to double-check though.

Other wise, seems good.

[Michael Hudson-Doyle (mwhudson)]

Andy Doan <email address hidden> writes:

> A question on cts.py:
>
> 338 +RUN_STEPS_HOST_PRE = ['python %s/cts/cts_wrapper.py $(SERIAL)' % curdir]
>
> The "python" command should work as expected in our old pip/virtualenv style deployment. Will this work in our new buildout style? In other words, when I run:
>
> . /srv/lava/instances/staging/bin/activate
> which python
>
> I get "/usr/bin/python". I don't know for sure if this will pick up
> the lava-android-test python path.

I don't think it will. We can get buildout to create a 'python'
interpreter in the style of virtualenv though.

> It via inheriting an envionrment from the parent process. You may want
> to double-check though.
>
> Other wise, seems good.

[Michael Hudson-Doyle (mwhudson)]

> I don't think it will. We can get buildout to create a 'python'
> interpreter in the style of virtualenv though.

I've done this now.

=== modified file 'lava_android_test/adb.py'
--- lava_android_test/adb.py	2012-05-23 03:50:55 +0000
+++ lava_android_test/adb.py	2012-07-02 01:59:17 +0000
@@ -17,11 +17,12 @@
 # You should have received a copy of the GNU General Public License
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 
-import threading
 import os
 import re
 import subprocess
 import tempfile
+import threading
+import time
 
 from Queue import Queue
 from lava_android_test.config import get_config
@@ -42,10 +43,23 @@
     target_dir = config.tempdir_android
 
     def __init__(self, serial=None, quiet=True):
+        self.cmdExecutor = CommandExecutor(quiet)
         if serial is not None:
             self.serial = serial
             self.adb = 'adb -s %s' % serial
-        self.cmdExecutor = CommandExecutor(quiet)
+        else:
+            self.serial = self.get_serial()
+
+    def get_serial(self):
+        if not self.serial:
+            serial_ary = self.run_cmd_host('adb get-serialno')[1]
+            serial = serial_ary[0].strip()
+            if not serial or serial == 'unknown':
+                return ''
+            else:
+                return serial
+        else:
+            return self.serial
 
     def push(self, source=None, target=None):
         if source is None:
@@ -209,58 +223,106 @@
             os.remove(tmpfile_name)
         return data
 
-    def get_shellcmdoutput(self, cmd=None):
+    def get_shellcmdoutput(self, cmd=None, quiet=True):
+        return self.get_shellcmdoutput_with_stderr(cmd=cmd, quiet=True)[0:2]
+
+    def run_adb_cmd(self, cmd, quiet=True):
+        return self.run_adb_cmd_with_stderr(cmd=cmd, quiet=quiet)[0:2]
+
+    def run_cmd_host(self, cmd, quiet=True):
+        return self.run_cmd_host_with_stderr(cmd, quiet=quiet)[0:2]
+
+    def get_shellcmdoutput_with_stderr(self, cmd=None, quiet=True):
         if cmd is None:
             return None
-        cmd = '%s shell %s' % (self.adb, cmd)
-        return self.run_cmd_host(cmd)
-
-    def run_cmd_host(self, cmd, quiet=True):
-        result = self.cmdExecutor.run(cmd, quiet)
-        return (result.returncode, result.stdout)
-
-    def run_adb_cmd(self, cmd, quiet=True):
-        cmd = '%s %s' % (self.adb, cmd)
-        result = self.cmdExecutor.run(cmd, quiet)
-        return (result.returncode, result.stdout)
-
-    def devices(self):
-        return self.run_cmd_host('%s devices' % self.adb)
+        return self.run_adb_cmd_with_stderr(cmd='shell %s' % cmd, quiet=quiet)
+
+    def run_adb_cmd_with_stderr(self, cmd, quiet=True):
+        if not self.isDeviceConnected():
+            print ("Reconnect adb connection of device(%s) "
+                   "for running command[%s]") % (self.get_serial(), cmd)
+            if not self.reconnect():
+                raise Exception('Failed to connect the device(%s)' % (
+                                                          self.get_serial()))
+        return self.run_cmd_host_with_stderr(cmd='%s %s' % (self.adb, cmd),
+                                              quiet=quiet)
+
+    def run_cmd_host_with_stderr(self, cmd, quiet=True):
+        result = self.cmdExecutor.run(cmd, quiet=quiet)
+        return (result.returncode, result.stdout, result.stderr)
 
     def run_adb_shell_for_test(self, cmd, stdoutlog=None,
                                stderrlog=None, quiet=False):
-        cmd = '%s shell %s' % (self.adb, cmd)
-        result = self.cmdExecutor.run(cmd, quiet)
-        if result.returncode != 0:
-            return result.returncode
-        self.push_stream_to_device(result.stdout, stdoutlog)
-        self.push_stream_to_device(result.stderr, stderrlog)
-        return result.returncode
+        (ret_code, stdout, stderr) = self.get_shellcmdoutput_with_stderr(
+                                                             cmd=cmd,
+                                                             quiet=quiet)
+        if ret_code != 0:
+            return ret_code
+        self.push_stream_to_device(stdout, stdoutlog)
+        self.push_stream_to_device(stderr, stderrlog)
+        return ret_code
 
     def push_stream_to_device(self, stream_lines, path):
+        if self.serial:
+            android_info = 'android(%s)' % self.serial
+        else:
+            android_info = 'android'
+
+        if not self.isDeviceConnected():
+            if not self.reconnect():
+                raise Exception('Failed to pull file(%s) to %s, '
+                                'because the device is not connected' % (
+                                                         path, android_info))
         basename = os.path.basename(path)
         tmp_path = os.path.join(config.tempdir_host, basename)
         if self.exists(path):
             retcode = self.pull(path, tmp_path)
             if retcode != 0:
                 raise Exception(
-                    'Failed to pull file(%s)stdout to android %s' % path)
+                    'Failed to pull file(%s) to %s' % (path, android_info))
 
         with open(tmp_path, 'a') as tmp_fd:
             tmp_fd.writelines(stream_lines)
             tmp_fd.close()
 
         if self.push(tmp_path, path)[1] is None:
-            raise Exception('Failed to push stdout to android %s' % path)
+            raise Exception(
+                    'Failed to pull file(%s) to %s' % (path, android_info))
         os.remove(tmp_path)
 
+    def devices(self):
+        return self.run_cmd_host('%s devices' % self.adb)
+
     def isDeviceConnected(self):
-        status, lines = self.run_cmd_host('%s get-state' % self.adb)
+        lines = self.run_cmd_host('%s get-state' % self.adb)[1]
         for line in lines:
             if 'device' in line:
                 return True
         return False
 
+    def connect(self):
+        if self.serial:
+            self.run_cmd_host('adb connect %s' % self.serial, quiet=False)
+            return self.isDeviceConnected()
+        return False
+
+    def disconnect(self):
+        if self.serial:
+            self.run_cmd_host('adb disconnect %s' % self.serial, quiet=False)
+            return not self.isDeviceConnected()
+        return False
+
+    def reconnect(self):
+        for i in range(1, 5):
+            print "LAVA: try to reconnect the device(%s) %i/5 times" % (
+                                                               self.serial, i)
+            if self.disconnect():
+                time.sleep(2)
+                if self.connect():
+                    return True
+            time.sleep(5)
+        return False
+
 
 class CommandExecutor(object):
     def __init__(self, quiet=True):

=== modified file 'lava_android_test/commands.py'
--- lava_android_test/commands.py	2012-06-28 09:52:29 +0000
+++ lava_android_test/commands.py	2012-07-02 01:59:17 +0000
@@ -152,13 +152,7 @@
         return self.adb.exists(test_dir)
 
     def get_device_serial(self):
-        if not self.args.serial:
-            serial_ary = ADB().run_cmd_host('adb get-serialno')[1]
-            serial = serial_ary[0].strip()
-            if not serial or serial == 'unknown':
-                return ''
-            else:
-                return serial
+        return ADB(self.args.serial).get_serial()
 
     def assertDeviceIsConnected(self):
         if not self.adb.isDeviceConnected():
@@ -169,7 +163,12 @@
                 raise Exception("No device found")
 
     def invoke(self):
-        self.adb = ADB(self.args.serial)
+        serial = self.get_device_serial()
+        if not serial:
+            raise LavaCommandError("No device attached")
+        self.serial = serial
+        self.adb = ADB(self.serial)
+
         try:
             self.assertDeviceIsConnected()
         except Exception as err:
@@ -388,7 +387,6 @@
 
     def invoke_sub(self):
 
-        config = get_config()
         test_name = 'custom'
         ADB_SHELL_STEPS = []
         STEPS_HOST_PRE = []
@@ -404,7 +402,7 @@
             file_url = self.args.command_file
             urlpath = urlparse.urlsplit(file_url).path
             file_name = os.path.basename(urlpath)
-            target_path = os.path.join(config.installdir_android,
+            target_path = os.path.join(self.config.installdir_android,
                                      test_name, file_name)
             STEPS_HOST_PRE = ["wget %s -O %s" % (file_url, file_name)]
             STEPS_ADB_PRE = ["push %s %s" % (file_name, target_path)]
@@ -490,25 +488,13 @@
                                  " bundle and finally save the complete bundle"
                                  " to the  specified FILE."))
 
-    def invoke(self):
-        config = get_config()
-        if self.args.serial:
-            serial = self.args.serial
-        else:
-            serial = self.get_device_serial()
-            if not serial:
-                raise LavaCommandError("No device attached")
-        self.adb = ADB(serial)
-        try:
-            self.assertDeviceIsConnected()
-        except Exception as err:
-            raise LavaCommandError(err)
+    def invoke_sub(self):
 
         if not utils.check_command_exist('monkeyrunner'):
             raise LavaCommandError('The command monkeyrunner can not be found')
 
         if self.args.repo_type == 'git':
-            target_dir = mkdtemp(prefix='git_repo', dir=config.tempdir_host)
+            target_dir = mkdtemp(prefix='git_repo', dir=self.config.tempdir_host)
             os.chmod(target_dir, 0755)
             GitRepository(self.args.url).checkout(target_dir)
         else:
@@ -524,7 +510,7 @@
 
         tip_msg = ("Run monkeyrunner scripts in following url on device(%s):"
                        "\n\turl=%s") % (
-                       self.args.serial,
+                       self.serial,
                        self.args.url)
 
         self.say_begin(tip_msg)
@@ -537,7 +523,7 @@
             if len(test_case_id) > 50:
                 test_case_id = '%s...' % (test_case_id[:50])
             try:
-                sub_bundle = self.run_monkeyrunner_test(script, serial,
+                sub_bundle = self.run_monkeyrunner_test(script, self.serial,
                                                          test_case_id)
                 test_result = {"test_case_id": test_case_id,
                                "result": 'pass'}
@@ -570,7 +556,6 @@
         self.say_end(tip_msg)
 
     def run_monkeyrunner_test(self, script, serial, test_case_id=None):
-        config = get_config()
 
         inst = AndroidTestInstaller()
         run = AndroidTestRunner(steps_host_pre=[
@@ -590,11 +575,11 @@
         ##The png files here are generated to the host by the monkeyrunner
         ##monkeyrunner is run on host, not on the target
         bundle = {}
-        org_png_file_list = utils.find_files(config.tempdir_host,
+        org_png_file_list = utils.find_files(self.config.tempdir_host,
                                              '.%s' % 'png')
         result_id = test.run(quiet=self.args.quiet)
         if self.args.output:
-            cur_all_png_list = utils.find_files(config.tempdir_host,
+            cur_all_png_list = utils.find_files(self.config.tempdir_host,
                                                 '.%s' % 'png')
             new_png_list = set(cur_all_png_list).difference(org_png_file_list)
             test_id = 'monkeyrunner(%s)' % (test_case_id)

=== modified file 'lava_android_test/test_definitions/android-0xbenchmark/android_0xbenchmark_wait.py'
--- lava_android_test/test_definitions/android-0xbenchmark/android_0xbenchmark_wait.py	2011-10-31 06:40:33 +0000
+++ lava_android_test/test_definitions/android-0xbenchmark/android_0xbenchmark_wait.py	2012-07-02 01:59:17 +0000
@@ -19,10 +19,11 @@
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
-import pexpect
 import sys
 import time
 
+from lava_android_test.utils import stop_at_pattern
+
 if len(sys.argv) == 1:
     adb_cmd = "adb"
 else:
@@ -30,16 +31,10 @@
 
 logcat_cmd = '%s logcat' % (adb_cmd)
 pattern = "Displayed org.zeroxlab.zeroxbenchmark/.Report"
-try:
-    proc = pexpect.spawn(logcat_cmd, logfile=sys.stdout)
-    id = proc.expect([pattern, pexpect.EOF], timeout=2400)
-    if id == 0:
-        proc.sendcontrol('C')
-except pexpect.TIMEOUT:
+
+if not stop_at_pattern(command=logcat_cmd, pattern=pattern, timeout=2400):
     print "0xbench Test: TIMEOUT Fail"
     sys.exit(1)
-finally:
-    proc.sendcontrol('C')
 
 time.sleep(3)
 sys.exit(0)

=== modified file 'lava_android_test/test_definitions/cts.py'
--- lava_android_test/test_definitions/cts.py	2012-06-25 17:23:05 +0000
+++ lava_android_test/test_definitions/cts.py	2012-07-02 01:59:17 +0000
@@ -33,7 +33,7 @@
 
 curdir = os.path.realpath(os.path.dirname(__file__))
 
-RUN_STEPS_HOST_PRE = ['/bin/bash %s/cts/cts_wrapper.sh $(SERIAL)' % curdir]
+RUN_STEPS_HOST_PRE = ['python %s/cts/cts_wrapper.py $(SERIAL)' % curdir]
 
 inst = lava_android_test.testdef.AndroidTestInstaller()
 run = lava_android_test.testdef.AndroidTestRunner(

=== added file 'lava_android_test/test_definitions/cts/cts_list_result_wrapper.sh'
--- lava_android_test/test_definitions/cts/cts_list_result_wrapper.sh	1970-01-01 00:00:00 +0000
+++ lava_android_test/test_definitions/cts/cts_list_result_wrapper.sh	2012-07-02 01:59:17 +0000
@@ -0,0 +1,26 @@
+#!/bin/bash
+# Copyright (C) 2012 Linaro Limited
+
+# Author: Linaro Validation Team <[e-mail]>
+#
+# This file is part of LAVA Android Test.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+#http://source.android.com/compatibility/downloads.html
+
+echo "./android-cts/tools/cts-tradefed l r"
+./android-cts/tools/cts-tradefed l r |tee cts_list_results.log
+
+exit 0
\ No newline at end of file

=== added file 'lava_android_test/test_definitions/cts/cts_prepare.sh'
--- lava_android_test/test_definitions/cts/cts_prepare.sh	1970-01-01 00:00:00 +0000
+++ lava_android_test/test_definitions/cts/cts_prepare.sh	2012-07-02 01:59:17 +0000
@@ -0,0 +1,130 @@
+#!/bin/bash
+# Copyright (C) 2012 Linaro Limited
+
+# Author: Linaro Validation Team <[e-mail]>
+#
+# This file is part of LAVA Android Test.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+#http://source.android.com/compatibility/downloads.html
+
+cts_pkg="android-cts-4.0.3_r3-linux_x86-arm.zip"
+media_pkg="android-cts-media-1.0.zip"
+site_url="https://dl.google.com/dl/android/cts/"
+#site_url="http://192.168.1.127/images/cts/"
+#export http_proxy=http://localhost:3128
+
+cts_pkg_url="${site_url}${cts_pkg}"
+media_pkg_url="${site_url}${media_pkg}"
+
+ADB_OPTION=""
+SERIAL=""
+if [ "x${1}" != "x" ]; then
+    ADB_OPTION="-s ${1}"
+    SERIAL="${1}"
+fi
+ADB_CMD="adb ${ADB_OPTION}"
+
+
+function download_unzip(){
+    if [ -z "$1" ] || [ -z "$2" ]; then
+        return
+    fi
+    url="${1}"
+    pkg="${2}"
+    
+    echo "wget --connect-timeout=30 -S --progress=dot -e dotbytes=2M ${url} -O ${pkg}"
+    wget -c -t 20 --connect-timeout=30 -S --progress=dot -e dotbytes=2M "${url}" -O ${pkg}
+    if [ $? -ne 0 ]; then
+        echo "Failed to get the package ${url}"
+        exit 1
+    fi
+    echo "unzip ${pkg}"
+    unzip ${pkg}
+    if [ $? -ne 0 ]; then
+        echo "Faild to unzip the package "
+        exit 1
+    fi
+}
+
+function main(){
+    rm -fr ${cts_pkg} ${media_pkg} android-cts
+    download_unzip "${cts_pkg_url}" ${cts_pkg}
+    download_unzip "${media_pkg_url}" ${media_pkg}
+
+    echo "${ADB_CMD} shell mkdir /mnt/sdcard/test"
+    ${ADB_CMD} shell mkdir /mnt/sdcard/test
+    chmod +x copy_media.sh
+    echo "copy_media.sh all ${ADB_OPTION}"
+    /bin/bash ./copy_media.sh all ${ADB_OPTION}
+	
+    #1. Your phone should be running a user build (Android 4.0 and later) from source.android.com
+    #2. Please refer to this link on the Android developer site and set up your device accordingly.
+    #3. Make sure that your device has been flashed with a user build (Android 4.0and later) before you run CTS.
+    ####Step 1~3 is done by deployment
+	
+    #4. You need to ensure the Text To Speech files are installed on the device. 
+    #   You can check via Settings > Speech synthesis > Install voice data 
+    #   before running CTS tests. 
+    #   (Note that this assumes you have Android Market installed on the device, 
+    #   if not you will need to install the files manually via adb)
+    ##TODO don't know how to do this yet
+	
+    #5. Make sure the device has a SD card plugged in and the card is empty. 
+    #   Warning: CTS may modify/erase data on the SD card plugged in to the device.
+    #6. Do a factory data reset on the device (Settings > SD Card & phone storage >Factory data reset). 
+    #   Warning: This will erase all user data from the phone.
+    #7. Make sure no lock pattern is set on the device (Settings > Security > Screen Lock should be 'None').
+    #8. Make sure the "USB Debugging" development option is checked (Settings >Developer options > USB debugging).
+    #9. Make sure Settings > Developer options > Stay Awake is checked
+    #10. Make sure Settings > Developer options > Allow mock locations is checked
+    ####Step 5~10 is done by deployment
+	
+    #11. Make sure device is connected to a functioning Wi-Fi network (Settings > WiFi)
+    ${ADB_CMD} shell am start -a android.intent.action.MAIN -n com.android.settings/.Settings
+    ${ADB_CMD} shell service call wifi 13 i32 1
+    sleep 5
+	
+    #12. Make sure the device is at the home screen at the start of CTS (Press the home button).
+    ${ADB_CMD} shell input keyevent 3
+    sleep 3
+	
+    #13. While a device is running tests, it must not be used for any other tasks.
+    #14. Do not press any keys on the device while CTS is running. 
+    #    Pressing keys or touching the screen of a test device will interfere with the running tests and may lead to test failures.
+    #####Steps 13~14 should be the ok because nobody will operation the test target.
+	
+    #15. Set up accessibility tests:
+    echo "${ADB_CMD} install -r android-cts/repository/testcases/CtsDelegatingAccessibilityService.apk"
+    ${ADB_CMD} install -r android-cts/repository/testcases/CtsDelegatingAccessibilityService.apk
+    if [ $? -ne 0 ]; then
+        echo "Faild to install CtsDelegatingAccessibilityService.apk"
+        exit 1
+    fi
+    ##TODO On the device, enable Settings > Accessibility > DelegatingAccessibility Service
+	
+    #16. Set up device administration tests:
+    echo "${ADB_CMD} install -r android-cts/repository/testcases/CtsDeviceAdmin.apk"
+    ${ADB_CMD} install -r android-cts/repository/testcases/CtsDeviceAdmin.apk
+    if [ $? -ne 0 ]; then
+        echo "Faild to install CtsDeviceAdmin.apk"
+        exit 1
+    fi
+    ##TODO On the device, enable Settings > Security > Device Administrators >android.deviceadmin.cts.CtsDeviceAdmin* settings
+	
+	exit 0
+}
+
+main "$@"

=== added file 'lava_android_test/test_definitions/cts/cts_redirect.sh'
--- lava_android_test/test_definitions/cts/cts_redirect.sh	1970-01-01 00:00:00 +0000
+++ lava_android_test/test_definitions/cts/cts_redirect.sh	2012-07-02 01:59:17 +0000
@@ -0,0 +1,24 @@
+#!/bin/bash
+# Copyright (C) 2012 Linaro Limited
+
+# Author: Linaro Validation Team <[e-mail]>
+#
+# This file is part of LAVA Android Test.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+output_file=${1} && shift
+eval "$@" &> ${output_file} &
+echo $!
+exit 0

=== renamed file 'lava_android_test/test_definitions/cts/cts_wrapper.sh' => 'lava_android_test/test_definitions/cts/cts_run_wrapper.sh'
--- lava_android_test/test_definitions/cts/cts_wrapper.sh	2012-02-10 18:16:42 +0000
+++ lava_android_test/test_definitions/cts/cts_run_wrapper.sh	2012-07-02 01:59:17 +0000
@@ -20,62 +20,7 @@
 
 #http://source.android.com/compatibility/downloads.html
 
-ADB_CMD="adb"
-if [ "x${1}" != "x" ]; then
-    ADB_CMD="${ADB_CMD} -s ${1}"
-fi
-
-cts_url_txt='https://wiki.linaro.org/TestDataLinkPage?action=AttachFile&do=get&target=android-cts-url.txt'
-mix_zip_name='mix.zip'
-android_cts_name='android-cts.zip'
-
-rm -fr android-cts.zip android-cts
-
-echo "wget -i ${cts_url_txt} -O ${mix_zip_name}"
-wget -i "${cts_url_txt}" -O ${mix_zip_name}
-if [ $? -ne 0 ]; then
-    echo "Failed to get the android-cts packages"
-    exit 1
-fi
-echo "tail --lines=+2 ${mix_zip_name}>${android_cts_name}"
-tail --lines=+2 ${mix_zip_name}>${android_cts_name}
-if [ $? -ne 0 ]; then
-    echo "Failed to get the android-cts packages from downloaded packages"
-    exit 1
-fi
-
-echo "unzip ${android_cts_name}"
-unzip ${android_cts_name}
-if [ $? -ne 0 ]; then
-    echo "Faild to unzip the android-cts packages"
-    exit 1
-fi
-
-echo "${ADB_CMD} install -r android-cts/repository/testcases/CtsDeviceAdmin.apk"
-${ADB_CMD} install -r android-cts/repository/testcases/CtsDeviceAdmin.apk
-if [ $? -ne 0 ]; then
-    echo "Faild to install CtsDeviceAdmin.apk"
-    exit 1
-fi
-
-RET_CODE=0
-#test_str='--package android.admin'
-test_str='--plan AppSecurity'
-test_str='--plan CTS'
-if [ "x${1}" != "x" ]; then
-    echo "./android-cts/tools/cts-tradefed run cts --serial ${1} ${test_str}|tee cts_output.log"
-    ./android-cts/tools/cts-tradefed run cts --serial ${1} ${test_str}|tee cts_output.log
-    RET_CODE=$?
-else
-    echo "./android-cts/tools/cts-tradefed run cts ${test_str}|tee cts_output.log"
-    ./android-cts/tools/cts-tradefed run cts ${test_str}|tee cts_output.log
-    RET_CODE=$?
-fi
-rm -f tee cts_output.log
-if [ ${RET_CODE} -ne 0 ]; then
-    echo "Faild to run cts for test (${test_str})"
-### comment it so that we can get the test result that has been executed
-### otherwise we won't get any test result about cts
-#    exit 1
-fi
-exit 0
+echo ./android-cts/tools/cts-tradefed "$@"
+./android-cts/tools/cts-tradefed "$@" |tee -a cts_output.log
+
+exit 0
\ No newline at end of file

=== added file 'lava_android_test/test_definitions/cts/cts_wrapper.py'
--- lava_android_test/test_definitions/cts/cts_wrapper.py	1970-01-01 00:00:00 +0000
+++ lava_android_test/test_definitions/cts/cts_wrapper.py	2012-07-02 01:59:17 +0000
@@ -0,0 +1,174 @@
+#!/usr/bin/python
+
+# Copyright (c) 2012 Linaro
+
+# Author: Linaro Validation Team <[e-mail]>
+#
+# This file is part of LAVA Android Test.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import re
+import sys
+from lava_android_test.adb import ADB
+from lava_android_test.utils import stop_at_pattern
+
+adb = ADB(sys.argv[1])
+curdir = os.path.realpath(os.path.dirname(__file__))
+
+
+def get_not_executed():
+    list_result_path = os.path.join(curdir, 'cts_list_result_wrapper.sh')
+    list_result_cmd = "bash %s" % list_result_path
+
+    pattern = 'CTS          unknown'
+    if not stop_at_pattern(command=list_result_cmd,
+                            pattern=pattern, timeout=5):
+        print "Failed to list the cts result for device(%s)" % adb.get_serial()
+
+    with open('cts_list_results.log') as fd:
+        #0        17237  126   0     2012.06.23_03.31.49  CTS        unknown
+        pattern = ("\s*\d+\s+\d+\s+\d+\s+(?P<no_executed>\d+)"
+                   "\s+.+CTS\s+unknown\s*$")
+        pat = re.compile(pattern)
+        for line in fd.readlines():
+            match = pat.search(line)
+            if not match:
+                continue
+            return match.groupdict()['no_executed']
+        return 0
+
+
+def prepare_cts():
+    cts_prepare_path = os.path.join(curdir, 'cts_prepare.sh')
+    cts_prepare_cmd = "bash %s" % cts_prepare_path
+    if not stop_at_pattern(command="%s %s" % (cts_prepare_cmd,
+                                              adb.get_serial()),
+                           timeout=18000):
+        print "Preapration for CTS test times out"
+        return False
+    return True
+
+
+def run_cts_with_plan(cts_cmd=None):
+    pattern = "Time:"
+    plan_command = '--plan CTS'
+    if cts_cmd:
+        plan_command = "%s %s" % (cts_cmd, plan_command)
+    if not stop_at_pattern(command=plan_command, pattern=pattern,
+                            timeout=36000):
+        print "CTS test times out"
+        return False
+
+    return True
+
+
+def run_cts_continue(cts_cmd=None):
+    pattern = "Time:"
+    continue_command = '--continue-session 0'
+    if cts_cmd:
+        continue_command = "%s %s" % (cts_cmd, continue_command)
+
+    while True:
+        number_of_not_executed = get_not_executed()
+        if number_of_not_executed and int(number_of_not_executed) > 0:
+            print ('Reconnect the adb connection before continuing '
+                   'the CTS on device(%s)') % adb.get_serial()
+            if not adb.reconnect():
+                print "Faile to reconnect the adb connection of device(%s)" % (
+                                                             adb.get_serial())
+                break
+
+            print "Continue the uncompleted CTS test on device(%s)" % (
+                                                           adb.get_serial())
+
+            if not stop_at_pattern(command=continue_command,
+                                   pattern=pattern,
+                                   timeout=36000):
+                print "CTS test times out"
+        else:
+            break
+
+
+def collect_log(command=None, output_file=None):
+    if command and output_file:
+        print 'Redirect the output of command[%s] to file[%s]' % (command,
+                                                                  output_file)
+        cmd = 'bash %s %s "%s"' % (os.path.join(curdir, 'cts_redirect.sh'),
+                                 output_file, command)
+        stdout = adb.run_cmd_host(cmd)[1]
+        if stdout:
+            return stdout[0].strip()
+
+    return None
+
+
+def collect_logs():
+
+    kmsg = {'command':
+                    'adb -s %s shell cat /proc/kmsg' % (adb.get_serial()),
+            'output_file': 'kmsg.log'}
+
+    logcat = {'command':
+                'adb -s %s logcat -c; adb -s %s logcat -v time' % (
+                                    adb.get_serial(), adb.get_serial()),
+              'output_file': 'logcat.log'}
+
+    ## define all the logs need to be collected
+    logs = [kmsg, logcat]
+    for log in logs:
+        pid = collect_log(command=log.get('command'),
+                         output_file=log.get('output_file'))
+        if pid:
+            log['pid'] = pid
+    return logs
+
+
+def print_log(logs=[]):
+    for log in logs:
+        log_file = log.get('output_file')
+        if log_file:
+            with open(log_file) as log_fd:
+                print '=========Log file [%s] starts=========>>>>>' % log_file
+                for line in log_fd.readlines():
+                    print line.rstrip()
+                print '<<<<<=========Log file [%s] ends=========' % log_file
+
+
+def main():
+    run_wrapper_path = os.path.join(curdir, 'cts_run_wrapper.sh')
+    run_wrapper_cmd = "bash %s" % run_wrapper_path
+    run_wrapper_cmd = '%s run cts --serial %s' % (run_wrapper_cmd,
+                                                      adb.get_serial())
+
+    logs = collect_logs()
+    if not prepare_cts():
+        sys.exit(1)
+    try:
+        run_cts_with_plan(run_wrapper_cmd)
+        run_cts_continue(run_wrapper_cmd)
+    finally:
+        for log in logs:
+            pid = log.get('pid')
+            if pid:
+                adb.run_cmd_host('kill -9 %s' % pid)
+
+        print_log(logs)
+
+    sys.exit(0)
+
+
+if __name__ == '__main__':
+    main()

=== modified file 'lava_android_test/utils.py'
--- lava_android_test/utils.py	2012-05-23 06:46:47 +0000
+++ lava_android_test/utils.py	2012-07-02 01:59:17 +0000
@@ -14,9 +14,11 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 import re
 import os
+import pexpect
 import shutil
 import subprocess
 import sys
+import time
 import urllib2
 import urlparse
 
@@ -174,3 +176,27 @@
     for f in flist:
         if os.path.exists(f):
             os.unlink(f)
+
+
+def stop_at_pattern(command=None, pattern=None, timeout=-1):
+    if not command:
+        return
+
+    if not pattern:
+        response = [pexpect.EOF]
+    else:
+        response = [pattern, pexpect.EOF]
+
+    result = True
+    proc = pexpect.spawn(command, logfile=sys.stdout)
+    try:
+        match_id = proc.expect(response, timeout=timeout)
+        if match_id == 0:
+            time.sleep(5)
+    except pexpect.TIMEOUT:
+        result = False
+    finally:
+        proc.sendcontrol('C')
+        proc.sendline('')
+
+    return result