Merge ~medicalwei/plainbox-provider-checkbox:fix-audio-test into plainbox-provider-checkbox:master

[Yao Wei]

Set XDG_RUNTIME_DIR before testing audio

This attempts to resolve issue LP:#1904707. By correctly setting XDG_RUNTIME_DIR before running audio test we can work around issue that the terminal environment after test involving reboot (invoked by pm_test.py) cannot connect to pulseaudio.

[jeremyszu (os369510)]

@Wei,

Would you please share your test result with/without your patch here?

619bf66... by Yao Wei on 2021-07-21

Add XDG_RUNTIME_DIR check in pactl_list.sh

[Yao Wei (medicalwei)]

I am not able to reproduce the original bug because I returned the machine where the bug occurs.

It seems the reboot is triggered by the fact that it has NVIDIA dGPU, and dGPU switching was the reboot criteria.

However, by manually doing `sudo -s` to root then `sudo -s -u user` to user account I can recreate the symptom:

--------------[ Running job 1 / 2. Estimated time left: 0:00:11 ]---------------
-------------------------[ audio/detect_sinks_sources ]-------------------------
ID: com.canonical.certification::audio/detect_sinks_sources
Category: com.canonical.plainbox::audio
... 8< -------------------------------------------------------------------------
Connection failure: Connection refused
pa_context_connect() failed: Connection refused
No available sources found
Connection failure: Connection refused
pa_context_connect() failed: Connection refused
No available sinks found
------------------------------------------------------------------------- >8 ---
Outcome: job failed
--------------[ Running job 2 / 2. Estimated time left: 0:00:10 ]---------------
--------------------[ audio/alsa_record_playback_automated ]--------------------
ID: com.canonical.certification::audio/alsa_record_playback_automated
Category: com.canonical.plainbox::audio
Job cannot be started because:
 - required dependency 'com.canonical.certification::audio/detect_sinks_sources' has failed
Outcome: job cannot be started

then at the same environment running the audio test with the patch, the audio tests are passed:

--------------[ Running job 2 / 3. Estimated time left: 0:00:11 ]---------------
-------------------------[ audio/detect_sinks_sources ]-------------------------
ID: com.canonical.certification::audio/detect_sinks_sources
Category: com.canonical.plainbox::audio
... 8< -------------------------------------------------------------------------
5alsa_input.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__hw_sofhdadsp__sourcemodule-alsa-card.cs16le 2ch 48000HzSUSPENDED
6alsa_input.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__hw_sofhdadsp_6__sourcemodule-alsa-card.cs16le 2ch 48000HzSUSPENDED
1alsa_output.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__hw_sofhdadsp_5__sinkmodule-alsa-card.cs16le 2ch 48000HzSUSPENDED
2alsa_output.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__hw_sofhdadsp_4__sinkmodule-alsa-card.cs16le 2ch 48000HzSUSPENDED
3alsa_output.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__hw_sofhdadsp_3__sinkmodule-alsa-card.cs16le 2ch 48000HzSUSPENDED
4alsa_output.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__hw_sofhdadsp__sinkmodule-alsa-card.cs16le 2ch 48000HzSUSPENDED
------------------------------------------------------------------------- >8 ---
Outcome: job passed
--------------[ Running job 3 / 3. Estimated time left: 0:00:10 ]---------------
--------------------[ audio/alsa_record_playback_automated ]--------------------
ID: com.canonical.certification::audio/alsa_record_playback_automated
Category: com.canonical.plainbox::audio
... 8< -------------------------------------------------------------------------
INFO:root:Using PulseAudio identifier 5 (alsa_input.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi...

[Jonathan Cave (jocave)]

Typically environment setup like should not need to happen in the job itself. It suggests that something else is going wrong during the running of the test plan. I gather there is some other context to this problem that other people are aware of so I will leave it to them to contribute.

review: Disapprove

[Sylvain Pineau (sylvain-pineau)]

The issue you're facing here is all because pm_test and its way to restart sessions:
https://git.launchpad.net/plainbox-provider-checkbox/tree/bin/pm_test.py#n288

When restarting checkbox after a reboot it opens a terminal and run sudo -u $USER basically. That does not preserve all the desktop env vars we used to get from a genuine desktop user session.

It's something that we do support via checkbox remote.

The checkbox-ng remote service takes care of that: https://git.launchpad.net/checkbox-ng/commit/plainbox/impl/session/remote_assistant.py?id=2938b1da085ae093fedc30114e91f0792133a409

We're planning to kill pm_test.py from all our jobs and push to adopt REMOTE for all QA testing.

So needless to fix anything at audio_test level, definitely not the right place. Please give remote a go, all com.canonical.certification::audio jobs should work just fine after a reboot of the DUT.

review: Disapprove

[Yao Wei (medicalwei)]

Here at SWE we mostly run checkbox locally. Are we pushing all local use case to remote?

Unmerged commits

619bf66... by Yao Wei on 2021-07-21

Add XDG_RUNTIME_DIR check in pactl_list.sh

4dff6ad... by Yao Wei on 2021-07-20

Set XDG_RUNTIME_DIR before testing audio

diff --git a/bin/audio_test.py b/bin/audio_test.py
index 4b3f89c..cd60f15 100755
--- a/bin/audio_test.py
+++ b/bin/audio_test.py
@@ -10,6 +10,7 @@ import re
 import subprocess
 import sys
 import time
+import os
 try:
     import gi
     gi.require_version('GLib', '2.0')
@@ -518,12 +519,18 @@ def process_arguments():
                     (one frequency/magnitude pair per line)""")
     return parser.parse_args()
 
+def set_xdg_runtime_dir():
+    if "XDG_RUNTIME_DIR" not in os.environ:
+        os.environ["XDG_RUNTIME_DIR"] = "/run/user/{uid}".format(uid=os.getuid())
 
 #
 def main():
     # Get arguments.
     args = process_arguments()
 
+    # Set XDG_RUNTIME_DIR if unset
+    set_xdg_runtime_dir()
+
     # Setup logging
     level = logging.INFO
     if args.debug:
diff --git a/bin/pactl_list.sh b/bin/pactl_list.sh
index a7abb87..2f26cc4 100755
--- a/bin/pactl_list.sh
+++ b/bin/pactl_list.sh
@@ -2,6 +2,10 @@
 
 EXIT_CODE=0
 
+if [ -z "$XDG_RUNTIME_DIR" ] ; then
+    export XDG_RUNTIME_DIR="/run/user/${UID}"
+fi
+
 for device in "sources" "sinks"
 do
     if ! pactl list $device short | grep -v -E "monitor|auto_null"