Merge lp:~fginther/ubuntu-touch-boottest/more-retries into lp:ubuntu-touch-boottest

[Francis Ginther]

Add more retries and a delay between attempts.

Add more retries and a delay between attempts.

Tested here:
http://d-jenkins.ubuntu-ci:8080/view/Wily/view/BootTest/job/fjg-boottest/56/console

Task:
https://trello.com/c/pTuLFEr2/117-boottest-improve-retry-logic

[lp:~fginther/ubuntu-touch-boottest/more-retries]

8. By Francis Ginther on 2015-06-25

Dump adt-run log file on failed tests.

[Evan (ev)]

Looks good.

review: Approve

[Para Siva (psivaa)]

+1, thanks for flake8 fix too.

review: Approve

=== modified file 'run-adt.py'
--- run-adt.py	2015-06-05 20:44:58 +0000
+++ run-adt.py	2015-06-25 22:04:35 +0000
@@ -10,9 +10,13 @@
 
 import subprocess
 import sys
+import time
 
 # Number of times to run adt-run before giving up.
-MAX_RUN_COUNT = 3
+MAX_RUN_COUNT = 6
+
+# Delay (in seconds) to increment between attempts
+RETRY_DELAY = 10
 
 # adt-run exit codes we should re-run the adt-run call on:
 # XXX plars: error 12 should be removed once it's properly detected as
@@ -22,9 +26,9 @@
 
 def main():
     arguments = sys.argv[1:]
-    for _ in range(MAX_RUN_COUNT):
+    for attempt in range(MAX_RUN_COUNT):
         with open('adt-run-stdout', 'wb') as stdout_file, \
-        open('adt-run-stderr', 'wb') as stderr_file:
+                open('adt-run-stderr', 'wb') as stderr_file:
             process = subprocess.Popen(
                 arguments,
                 stdout=stdout_file,
@@ -33,6 +37,8 @@
             returncode = process.wait()
             if returncode not in RETRY_CODES:
                 break
+            # Add a linear increasing delay between attempts
+            time.sleep(RETRY_DELAY * attempt)
     sys.exit(returncode)