Merge ~canonical-kernel-team/+git/autotest-client-tests:phlin/fail-cpuset-hotplug into ~canonical-kernel-team/+git/autotest-client-tests:master

[Po-Hsu Lin]

cpuset_hotplug will hang on X-aws-4.4 VMs. This issue can be reproduced
all the way back to X-aws-1128, with current and previous LTP fork.

When this happens, it will cause failure to cpuset_memory as well and
eventually leads to an incomplete test report. Let's fail this test
directly on this kernel to get the test going.

This issue didn't emerge until we begin to run ubuntu_ltp_controllers on
Xenial in recent cycle.

Patch tested on aws@obruchev, and it's working as expected.

This issue itself is probably a won't fix on ESM kernels, but I think we can keep it as-is just for a reference.

[Po-Hsu Lin (cypressyew)]

I just found there are some exceptions on different VMs, will collect more data.

[Po-Hsu Lin (cypressyew)]

So from the data that I collected, a very quick solution to filter those failing VMs out is to check CPU <= 2, not sure whether this is the cause of the failure though.

[Po-Hsu Lin (cypressyew)]

os.cpu_count() <= 2 added to the if statement, it's ready to be reviewed now.

[Cory Todd (corytodd)]

Is the intent of raising TestError to force the user to hint the test?

[Po-Hsu Lin (cypressyew)]

Yes, so the test will be marked as failed directly, without being tested with the `runltp` executable.

One example is the fanotify07 test in ubuntu_ltp_syscalls, which is a test-blocking won't fix issue, the output looks like this:

15:40:54 INFO | START ubuntu_ltp_syscalls.fanotify07 ubuntu_ltp_syscalls.fanotify07 timestamp=1688485254 timeout=900 localtime=Jul 04 15:40:54
15:40:54 DEBUG| Persistent state client._record_indent now set to 2
15:40:54 DEBUG| Persistent state client.unexpected_reboot now set to ('ubuntu_ltp_syscalls.fanotify07', 'ubuntu_ltp_syscalls.fanotify07')
15:40:54 DEBUG| Waiting for pid 14676 for 900 seconds
15:40:54 ERROR| Exception escaping from test:
Traceback (most recent call last):
  File "/home/ubuntu/autotest/client/shared/test.py", line 411, in _exec
    _call_test_function(self.execute, *p_args, **p_dargs)
  File "/home/ubuntu/autotest/client/shared/test.py", line 823, in _call_test_function
    return func(*args, **dargs)
  File "/home/ubuntu/autotest/client/shared/test.py", line 291, in execute
    postprocess_profiled_run, args, dargs)
  File "/home/ubuntu/autotest/client/shared/test.py", line 212, in _call_run_once
    self.run_once(*args, **dargs)
  File "/home/ubuntu/autotest/client/tests/ubuntu_ltp_syscalls/ubuntu_ltp_syscalls.py", line 143, in run_once
    raise error.TestError("fanotify07 (lp:1775165) won't fix on T/X and blocking test to finish properly (lp:1944545), mark it as failed directly")
TestError: fanotify07 (lp:1775165) won't fix on T/X and blocking test to finish properly (lp:1944545), mark it as failed directly
15:40:54 ERROR| child process failed
15:40:54 DEBUG| Traceback (most recent call last):
15:40:54 DEBUG| File "/home/ubuntu/autotest/client/parallel.py", line 25, in fork_start
15:40:54 DEBUG| l()
15:40:54 DEBUG| File "/home/ubuntu/autotest/client/job.py", line 505, in
15:40:54 DEBUG| l = lambda: test.runtest(self, url, tag, args, dargs)
15:40:54 DEBUG| File "/home/ubuntu/autotest/client/test.py", line 125, in runtest
15:40:54 DEBUG| job.sysinfo.log_after_each_iteration)
15:40:54 DEBUG| File "/home/ubuntu/autotest/client/shared/test.py", line 913, in runtest
15:40:54 DEBUG| mytest._exec(args, dargs)
15:40:54 DEBUG| File "/home/ubuntu/autotest/client/shared/test.py", line 411, in _exec
15:40:54 DEBUG| _call_test_function(self.execute, *p_args, **p_dargs)
15:40:54 DEBUG| File "/home/ubuntu/autotest/client/shared/test.py", line 823, in _call_test_function
15:40:54 DEBUG| return func(*args, **dargs)
15:40:54 DEBUG| File "/home/ubuntu/autotest/client/shared/test.py", line 291, in execute
15:40:54 DEBUG| postprocess_profiled_run, args, dargs)
15:40:54 DEBUG| File "/home/ubuntu/autotest/client/shared/test.py", line 212, in _call_run_once
15:40:54 DEBUG| self.run_once(*args, **dargs)
15:40:54 DEBUG| File "/home/ubuntu/autotest/client/tests/ubuntu_ltp_syscalls/ubuntu_ltp_syscalls.py", line 143, in run_once
15:40:54 DEBUG| raise error.TestError("fanotify07 (lp:1775165) won't fix on T/X and blocking test to finish properly (lp:1944545), mark it as failed directly")
15:40:54 DEBUG| TestError: fanotify07 (lp:1...

[Cory Todd (corytodd)]

Oh cool, I like it! This will be super useful, nice work getting this all assembled with plenty of supporting evidence.

review: Approve

[Francis Ginther (fginther)]

review: Approve

[Sean Feole (sfeole)]

review: Approve

[Po-Hsu Lin (cypressyew)]

Applied and pushed, thanks!

diff --git a/ubuntu_ltp_controllers/ubuntu_ltp_controllers.py b/ubuntu_ltp_controllers/ubuntu_ltp_controllers.py
index 5c3fed9..7e0aa37 100644
--- a/ubuntu_ltp_controllers/ubuntu_ltp_controllers.py
+++ b/ubuntu_ltp_controllers/ubuntu_ltp_controllers.py
@@ -44,6 +44,7 @@ class ubuntu_ltp_controllers(test.test):
             'libtirpc-dev',
             'pkg-config',
             'quota',
+            'virt-what',
             'xfslibs-dev',
             'xfsprogs',
         ]
@@ -126,6 +127,11 @@ class ubuntu_ltp_controllers(test.test):
             print("Set timeout multiplier LTP_TIMEOUT_MUL=5 for memcontrol03")
             os.environ["LTP_TIMEOUT_MUL"] = '5'
 
+        # Special case to fail cpuset_hotplug on X-aws-4.4 VM (LP: #2026722)
+        if test_name == 'cpuset_hotplug' and re.search('4.4.0-.*aws', platform.uname()[2]):
+            if utils.system_output('virt-what', verbose=False) and os.cpu_count() <= 2:
+                raise error.TestError('cpuset_hotplug will hang on X-aws-4.4 VM (LP: #2026722)')
+
         cmd = '/opt/ltp/runltp -f /tmp/target -q -C /dev/null -l /dev/null -T /dev/null'
         print(utils.system_output(cmd, verbose=False))
         # /dev/loop# creation will be taken care by the runltp