Merge ~canonical-kernel-team/+git/autotest-client-tests:phlin/bt-log-core into ~canonical-kernel-team/+git/autotest-client-tests:master

[Po-Hsu Lin]

KERNTT-464
Add support to read log from journalctl, so that the log check test can
be run on Ubuntu Core systems.

[Francis Ginther (fginther)]

Changes look reasonable. Have you been able to catch one of these issues on a core system to ensure that this test will correctly fail?

review: Approve

[Po-Hsu Lin (cypressyew)]

No, I use a Google n2d-standard-64 instance (which is expected to raise WARNING in syslog) with Focal to test the regex.

In syslog:
May 7 01:50:37 f-l-gen-s2-n2dstd64-u-boot kernel: [ 181.772052] nvme 0000:00:04.0: overflow 0x0008003edbbac000+262144 of DMA mask ffffffffffffffff bus mask 0
May 7 01:50:37 f-l-gen-s2-n2dstd64-u-boot kernel: [ 181.781746] ------------[ cut here ]------------
May 7 01:50:37 f-l-gen-s2-n2dstd64-u-boot kernel: [ 181.781750] WARNING: CPU: 58 PID: 3002 at kernel/dma/direct.c:35 report_addr+0x37/0x60

From journalctl:
May 07 01:50:37 f-l-gen-s2-n2dstd64-u-boot kernel: ------------[ cut here ]------------
May 07 01:50:37 f-l-gen-s2-n2dstd64-u-boot kernel: WARNING: CPU: 58 PID: 3002 at kernel/dma/direct.c:35 report_addr+0x37/0x60

This has been tested by removing the /var/log/syslog to force it to use journalctl:
02:08:33 DEBUG| Running 'journalctl -k > /tmp/journalctl-syslog'
02:08:33 INFO | Checking error message in /tmp/journalctl-syslog:
02:08:33 INFO | Scanning for pattern "kernel:( \[ *\d+\.\d+\])? BUG:.*"
02:08:33 INFO | PASSED, log clean.
02:08:33 INFO | Scanning for pattern "kernel:( \[ *\d+\.\d+\])? Oops:.*"
02:08:33 INFO | PASSED, log clean.
02:08:33 INFO | Scanning for pattern "kernel:( \[ *\d+\.\d+\])? kernel BUG at.*"
02:08:33 INFO | PASSED, log clean.
02:08:33 INFO | Scanning for pattern "kernel:( \[ *\d+\.\d+\])? WARNING:.*"
02:08:33 INFO | Pattern found. Matching lines as follows:
02:08:33 INFO | kernel: WARNING: CPU: 58 PID: 3002 at kernel/dma/direct.c:35 report_addr+0x37/0x60

I will amend the commit message to make it more clear.

[Po-Hsu Lin (cypressyew)]

Commit message amended to:
    KERNTT-464
    Add support to read log from journalctl, so that the log check test can
    be run on Ubuntu Core systems.

    The regex must be modified. Here is an example of WARNING taken from
    the syslog of a Google n2d-standard-64 instance:

      May 7 01:50:37 f-l-gen-s2-n2dstd64-u-boot kernel: [ 181.781750] WARNING: CPU: 58 PID: 3002 at kernel/dma/direct.c:35 report_addr+0x37/0x60

    Here is the output of journalctl on this instance:

      May 07 01:50:37 f-l-gen-s2-n2dstd64-u-boot kernel: WARNING: CPU: 58 PID: 3002 at kernel/dma/direct.c:35 report_addr+0x37/0x60

    I have this code test on this instance by removing /var/log/syslog to
    force it to use journalctl output and it's failing as expected:

      Running 'journalctl -k > /tmp/journalctl-syslog'
      Checking error message in /tmp/journalctl-syslog:
      Scanning for pattern "kernel:( \[ *\d+\.\d+\])? BUG:.*"
      PASSED, log clean.
      Scanning for pattern "kernel:( \[ *\d+\.\d+\])? Oops:.*"
      PASSED, log clean.
      Scanning for pattern "kernel:( \[ *\d+\.\d+\])? kernel BUG at.*"
      PASSED, log clean.
      Scanning for pattern "kernel:( \[ *\d+\.\d+\])? WARNING:.*"
      Pattern found. Matching lines as follows:
      kernel: WARNING: CPU: 58 PID: 3002 at kernel/dma/direct.c:35 report_addr+0x37/0x60

Applied and pushed, thanks!

diff --git a/ubuntu_boot/ubuntu_boot.py b/ubuntu_boot/ubuntu_boot.py
index 93068c6..2d813e0 100644
--- a/ubuntu_boot/ubuntu_boot.py
+++ b/ubuntu_boot/ubuntu_boot.py
@@ -28,14 +28,19 @@ class ubuntu_boot(test.test):
         else:
             logfile = '/var/log/syslog'
         patterns = [
-            'kernel: \[ *\d+\.\d+\] BUG:.*',
-            'kernel: \[ *\d+\.\d+\] Oops:.*',
-            'kernel: \[ *\d+\.\d+\] kernel BUG at.*',
-            'kernel: \[ *\d+\.\d+\] WARNING:.*'
+            'kernel:( \[ *\d+\.\d+\])? BUG:.*',
+            'kernel:( \[ *\d+\.\d+\])? Oops:.*',
+            'kernel:( \[ *\d+\.\d+\])? kernel BUG at.*',
+            'kernel:( \[ *\d+\.\d+\])? WARNING:.*'
         ]
         test_passed = True
-        print('Checking error message in {}:'.format(logfile))
+        if not os.path.exists(logfile):
+            # Hack for systems without syslog (Ubuntu Core)
+            logfile = '/tmp/journalctl-syslog'
+            utils.system('journalctl -k > {}'.format(logfile))
+
         if os.path.exists(logfile):
+            print('Checking error message in {}:'.format(logfile))
             with open(logfile) as f:
                 content = f.read()
                 for pat in patterns: