Merge lp:~psivaa/ubuntu-test-cases/mako-recoveries into lp:ubuntu-test-cases/touch

[Para Siva]

To make sure that we recover the device fully in cases of 'detect_device' failures.

This is to fix a couple of issues:
1. In instances like [1], where 'adb shell' works but for some reason, phablet_utils.detect_device is failing. The fix is to perform a full-recovery, in such a case too. [2] indicates that the full flashing being carried out in cases where 'INFO:root:Device detected as mako' is not present in the logs. Compare this to [2] where 'INFO:root:Device detected as mako' being present after 'job-runner/scripts/recover.py mako-08' call and the full recovery is not attempted.

2. To make sure that 'adb reboot bootloader' actually boots the device into 'fastboot' before attempting to flash the device.

[1]: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-mako-08/1626/console
[2]: http://s-jenkins.ubuntu-ci:8080/job/psivaa-touch-flash-mako-08/1/console
[3]: http://s-jenkins.ubuntu-ci:8080/job/psivaa-touch-flash-mako-08/2/console

[Joe Talbott (joetalbott)]

One very minor inline comment. Otherwise, from a code standpoint it looks fine to me.

[Paul Larson (pwlars)]

A few comments below. I'll leave it up to you whether you want to fix them, but I think it's worth considering.

review: Needs Fixing

[Para Siva (psivaa)]

Thanks Paul for the comments. I've replied to them. We could discuss this in the IRC as well.

[lp:~psivaa/ubuntu-test-cases/mako-recoveries]

414. By Para Siva on 2015-09-10

Fix commenting comment

[Francis Ginther (fginther)]

I understand plars concerns with these changes, but we need to make some changes to stop this devices from getting in a usable, but unflashable state and poisoning results. I'm comfortable with this approach as a first attempt to address the problem and accept that we may need to revert or revisit it.

Just one inline comment that I think should be addressed.

review: Needs Fixing

[lp:~psivaa/ubuntu-test-cases/mako-recoveries]

415. By Para Siva on 2015-09-11

Review comment fixes to only infinitely retry as opposed to retry 3 times

[Para Siva (psivaa)]

Thanks for the comments. rev 415 contains the fix for Francis's comments.

[Francis Ginther (fginther)]

I've tested this in a s-jenkins provision operation and boottest. Should be good to land.

review: Approve

=== modified file 'scripts/provision.sh'
--- scripts/provision.sh	2015-06-03 21:40:26 +0000
+++ scripts/provision.sh	2015-09-11 08:25:28 +0000
@@ -109,14 +109,22 @@
     if [ -n "${ANDROID_SERIAL}" ] ; then
         # Entering the bootloader should take < 10 seconds, add some
         # padding for device variance.
-        sleep 30
-        if ! fastboot devices | grep -q "${ANDROID_SERIAL}"; then
-            log "Device not in fastboot after adb reboot bootloader"
-            # After a failed 'reboot bootloader' attempt, a reboot
-            # is used to get the device back to a saner state.
-            adb reboot
-            return 1
-        fi
+        while true; do
+            sleep 30
+            if ! fastboot devices | grep -q "${ANDROID_SERIAL}"; then
+                log "Device not in fastboot after `adb reboot bootloader`"
+                # XXX psivaa: 20150910: No point in continuing
+                # if `adb reboot bootloader` fails,
+                # which appears to cause issues during testing.
+                # This fix is to improve from
+                # `adb reboot && return 1` to carry out infinite
+                # `adb reboot bootloader`
+                adb reboot bootloader
+            else
+                log "=========== Device in fastboot =========="
+                break
+            fi
+       done
     fi
     return 0
 }
@@ -135,10 +143,7 @@
 full_flash() {
     log "FLASHING DEVICE"
     DEVICE_TYPE=$(get-device-type)
-    # Use a 60 second retry loop for reboot_bootloader.
-    # If the attempt failed, it may take nearly 60 seconds to complete
-    # the reboot cycle to get the device back to a sane state.
-    retry 60 3 reboot_bootloader
+    reboot_bootloader
     RECOVERY=""
     # We need to distinguish between devices with no recovery images and
     # failures to download existing recovery images. Only krillin

=== modified file 'scripts/recover.py'
--- scripts/recover.py	2015-06-01 17:07:47 +0000
+++ scripts/recover.py	2015-09-11 08:25:28 +0000
@@ -8,6 +8,8 @@
 import time
 import yaml
 
+from phabletutils.environment import detect_device
+
 log = logging.getLogger()
 logging.basicConfig(level=logging.INFO)
 
@@ -77,6 +79,12 @@
     log.error("Fatal error marking {} offline".format(node))
 
 
+def get_device_type():
+    device_type = os.environ.get('DEVICE_TYPE')
+    if device_type is None:
+        device_type = detect_device(None)
+
+
 def recover(device_name):
     try:
         device = device_info.get_device(device_name)
@@ -87,6 +95,10 @@
     if state in ('device'):
         try:
             device.check_adb_shell()
+            # XXX psivaa 20150910 We've seen instances when
+            # detect_device throwing exceptions even when adb_shell working OK
+            # recover it fully in such a situation
+            get_device_type()
         except:
             # The device looks like it's available, but not responding
             return _full_recovery(device)