Merge lp:~raharper/curtin/trunk.fix-vmtest-install-ok-check into lp:~curtin-dev/curtin/trunk

[Ryan Harper]

Modify how vmtests checks if a curtin install was successful.

1. we missed finding failed installs on precise/trusty due to non-systemd output. systemd prepends non-pid-1 process output to console with the calling process, so we had 'cloud-init[.*]', however on pre-systemd systems, that's not present. Fix this by dropping the cloud-init portion of the regular expression

2. Add in additional error messages we've seen during debugging of test cases

3. Modify the check to first see if we can find curtin's successful install message; if that's not found, then look for common errors.

[lp:~raharper/curtin/trunk.fix-vmtest-install-ok-check]

306. By Ryan Harper on 2015-11-18

vmtest: move install log checking to a function

Move install log checking logic to separate function.

307. By Ryan Harper on 2015-11-18

from trunk

[Scott Moser (smoser)]

I like this a lot, the only thing i hesitate on is jenkins running. we want to make sure console logs or artifacts there have everything we can to debug.

Diogo, could you run some and give us a pointer?

[Scott Moser (smoser)]

my comment above should have been on ryan's trunk.change-vmtest-default-output.
after revno 306 here, i like approve this MP.

review: Approve

=== modified file 'tests/vmtests/__init__.py'
--- tests/vmtests/__init__.py	2015-11-18 17:15:08 +0000
+++ tests/vmtests/__init__.py	2015-11-18 22:21:21 +0000
@@ -295,13 +295,12 @@
             logger.debug('Checking curtin install output for errors')
             with open(cls.install_log) as l:
                 install_log = l.read()
-            errors = re.findall(
-                '\[.*\]\ cloud-init.*:.*Installation\ failed', install_log)
-            if len(errors) > 0:
+            errmsg, errors = check_install_log(install_log)
+            if errmsg:
                 for e in errors:
-                    logger.debug(e)
-                logger.debug('Errors during curtin installer')
-                raise Exception('Errors during curtin installer')
+                    logger.error(e)
+                logger.error(errmsg)
+                raise Exception(cls.__name__ + ":" + errmsg)
             else:
                 logger.debug('Install OK')
         else:
@@ -432,6 +431,34 @@
                 self.assertIn(diskname, contents)
 
 
+def check_install_log(install_log):
+    # look if install is OK via curtin 'Installation ok"
+    # if we dont find that, scan for known error messages and report
+    # if we don't see any errors, fail with general error
+    errors = []
+    errmsg = None
+
+    # regexps expected in curtin output
+    install_pass = "Installation finished. No error reported."
+    install_fail = "({})".format("|".join([
+                   'Installation\ failed',
+                   'ImportError: No module named.*',
+                   'Unexpected error while running command',
+                   'E: Unable to locate package.*']))
+
+    install_is_ok = re.findall(install_pass, install_log)
+    if len(install_is_ok) == 0:
+        errors = re.findall(install_fail, install_log)
+        if len(errors) > 0:
+            for e in errors:
+                logger.error(e)
+            errmsg = ('Errors during curtin installer')
+        else:
+            errmsg = ('Failed to verify Installation is OK')
+
+    return errmsg, errors
+
+
 def get_apt_proxy():
     # get setting for proxy. should have same result as in tools/launch
     apt_proxy = os.environ.get('apt_proxy')