Merge ~alextu/plainbox-provider-checkbox:test-cpu-pc10-by-turbostate into plainbox-provider-checkbox:master

[Alex Tu]

    pick turbostate-long-idle-c10 test case from pc-sanity

    from https://git.launchpad.net/plainbox-provider-pc-sanity/tree/usr/share/plainbox-provider-checkbox/units/pc-sanity/pc-sanity-power-consumption.pxu

------

This test case is for power consumption checking.
The CPU package power state is expected to sleep deep to C10 in long idle otherwise it's mostly failed for e-star compliance.

[Alex Tu (alextu)]

After this test case been merged.
The plan is to add this case into the somerville auto plan[1] because Dell platforms need e-star compliances.

[1] https://git.launchpad.net/oem-qa-checkbox/tree/units/somerville-addon.pxu

[Ray Chen (ray.chen)]

Just question, from https://trello.com/c/3C0b5Nkt and other related cards, Is this test would also apply to another PC-OEM project?

[Alex Tu (alextu)]

This test job is an automatic test case and should be included into projects those need e-star compliance.

to avoid confusing, I moved the URL of trello card to commit comment instead(just for tracker).

[Jonathan Cave (jocave)]

This breaks checkbox job writing rules in such a myriad of ways that I think listing them all here would not be productive. Hard no from me.

review: Disapprove

[Alex Tu (alextu)]

thanks for review, I'm following the suggestion to refine this case : https://chat.canonical.com/canonical/pl/wg1csztgtjn7mk1a7r1111uy8o

[Alex Tu (alextu)]

I refactored the code to adopt the policy, please review again.

the passed case for reference:
https://paste.ubuntu.com/p/7jdVmVRjC8/

the failed case for reference:
https://paste.ubuntu.com/p/QrCZQzDRyW/

btw, this automatic test case is also added to cpu-cert-automated plan which will be nested by other auto plan.

[Sylvain Pineau (sylvain-pineau)]

Same here, too many violations (use of sudo, exit code not propagated, set -x instead of set -e, installation via apt, DISPLAY hardcoded, ...)

review: Disapprove

[Alex Tu (alextu)]

past the concern from reviewer here for reference:
1. installing linux-tools-"$(uname -r)" not work on the snap environment, so this case should be put into oem provider.
```
such requirement (linux-tools-5.6.0-1008-oem) will never work if checkbox is installed via snap. Or we need a new build of e.g cdts everytime a new kernel is published

at the same time making this kind of job deb only is not ideal
```
https://chat.canonical.com/canonical/pl/ctfx3gzd77gqxrdc7g4ass91sw

2. the sru pool as a wide range of cpu, some are really old. So, this new test case will always failed on those machines.
https://chat.canonical.com/canonical/pl/8wbj4h6wgjgtm8yf3he8fxb4hy

[Sylvain Pineau (sylvain-pineau)]

would be good to record the following setting to restore them after such test:

gsettings get org.gnome.desktop.screensaver lock-enabled
gsettings get org.gnome.desktop.session idle-delay

Once again I'm not entirely sure waiting 30s with running processes in the background can make the system going into C10

[Alex Tu (alextu)]

"Once again I'm not entirely sure waiting 30s with running processes in the background can make the system going into C10"
Ans : yes, that's the way we are currently counting on to catch the problematic CPU package which can not get into C10 during long idle. There's must be some blocking issue(Bug) if your machine can not get into long idle after set idle-delay to 10 then wait for 30s.

And I refined the logic to keep the idle-delay value and only restore it when there's a keeped one. BTW, lock-enable is not necessary during my testing.

For the concern of impact environment that checkbox run in snap,
per-talked with Rex that we should restrict this job to be only executed on classic environment to avoid impact the environment that running in snap.
Do we have a way to achieve it? by which 'requires:' value?

[Alex Tu (alextu)]

the message for test result for reference:
https://pastebin.ubuntu.com/p/zh5xVqnpN5/

log files in session folder:
ubuntu@dell-bto-focal-fossa-201810-26535:~$ cat /home/ubuntu/.cache/plainbox/sessions/checkbox-run-2020-08-25T03.26.21.session/CHECKBOX_DATA/turbostat-pkgpc-longidle.log | pastebinit
https://paste.ubuntu.com/p/4pf2j6GRGQ/

[Sylvain Pineau (sylvain-pineau)]

Have you seen https://code.launchpad.net/~kchsieh/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/390146 ?

[Alex Tu (alextu)]

I refactored this patch futher to cover both s2i and longidle.

Yes, KC discussed with me that we aligned that his case is to cover hardware capability by sys filesystem.

And mine is to cover the system level power consumption to make sure the system can pass e-star compliance.

Which is targeted to achieve this goal : https://docs.google.com/presentation/d/1KXBGWPg5_pVTzg-bxaE27PF7rzAdMS_esF-DO5k6woY/edit?ts=5f4d92fe#slide=id.g92ed89da92_0_14

[Alex Tu (alextu)]

ubuntu@dell-bto-focal-fossa-201810-26535:~$ DISPLAY=:0 checkbox-cli run com.canonical.certification::power-management/check-turbostat-long-idle-residency

https://pastebin.canonical.com/p/MDFXpnnvp5/

ubuntu@dell-bto-focal-fossa-201810-26535:~$ DISPLAY=:0 checkbox-cli run com.canonical.certification::power-management/check-turbostat-long-idle-residency

https://pastebin.canonical.com/p/ntzpZ5yMdR/

[Alex Tu (alextu)]

added more reviewers by following the conclusion of meeting https://docs.google.com/document/d/1Je8XLmuX5PS45wwyLk7c7JZtqj1lLk5vA7m2EU3aG24/edit#heading=h.8v5lfpwr404n

[Sylvain Pineau (sylvain-pineau)]

I still think your MR and KC's https://code.launchpad.net/~kchsieh/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/390146 have redundancies in many places.

I'm not sure which method works the best to ensure that system can go into idle. KC is using
`xset dpms force off` but your job instead relies on `gsettings set org.gnome.desktop.session idle-delay`.

Which one is good, idk. Again I'm only evaluating your MP from the code perspective and compliance with checkbox. I'm not a PM expert.

One last thing I'd like to know, from the blog post you both linked it seems idle can be reached if NO usb devices are connected. won't be the case for most of SRU systems. I don't see such recommendation either in your job desc.

review: Needs Fixing

[Sylvain Pineau (sylvain-pineau)]

grep for 'Unplug USB devices.' in https://01.org/blogs/qwang59/2018/how-achieve-s0ix-states-linux

[Alex Tu (alextu)]

As we explained that this this test case set higher bar to cover whole system behavior targeting to pass e-star compliance which all power states should stay stably in long idle and s2i.

The one from KC is just to check the capability of power state, say it's the 1st line checking hardware/driver capability. And mine is system stability.

So, KC's test can just counting on `xset dpms force off` to check the capability. And I choice `gsettings set org.gnome.desktop.session idle-delay` to simulate the system behavior.

I understand you have concern on the power management part, so I can invite HWE join this review.

For the 'Unplug USB devices.': yes, there might some problematic USB blocking power state. Because autosanity and QA will run this test without USB storage, so it's ok.
So, the plan would be land this test for autosnaiy, Canonical QA, ODM QA testing first. Then we think the solution before land it to SRU plan.

And thanks for the inline suggestion, I'll check it.

[You-Sheng Yang (vicamo)]

review: Needs Fixing

[Alex Tu (alextu)]

thanks for your review and replied inline.
I will fix them later.

[Alex Tu (alextu)]

another job which also need linux-tools package, we might can share the same way installing linux-tools : https://code.launchpad.net/~alexhung/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/391097

[You-Sheng Yang (vicamo)]

Didn't find something seriously wrong. Thank you.

review: Approve

[Jonathan Cave (jocave)]

Switching to work in progress. If the problems identified by the down voters are addressed then you may request reviews again.

[Alex Tu (alextu)]

refer to the experience of this test case on pc-sanity, the additional patches 77431a8 and 4613170 to make sure the test done on expected environment. They need additional resource from https://code.launchpad.net/~alextu/plainbox-provider-resource/+git/plainbox-provider-resource/+merge/395357

the 9438fae gives more instruction about the next step when this case failed.

[Alex Tu (alextu)]

Hi Sylvain,
Please help to review this MP, I mad it better base on pc-sanity experience.

The script checking turbostate result has been reviewed by Vicamo@HWE.

This test case is needed for the test upgrading oem kernel 5.6 to hwe
kernel 5.8 on machines in certification pool.

On Tue, Dec 15, 2020, 7:45 PM Alex Tu <email address hidden> wrote:

> Alex Tu has proposed merging
> ~alextu/plainbox-provider-checkbox:test-cpu-pc10-by-turbostate into
> plainbox-provider-checkbox:master.
>
> Commit message:
> pick turbostate-long-idle-c10 test case from pc-sanity
>
> from
> https://git.launchpad.net/plainbox-provider-pc-sanity/tree/usr/share/plainbox-provider-checkbox/units/pc-sanity/pc-sanity-power-consumption.pxu
>
> ------
>
> This test case is for power consumption checking.
> The CPU package power state is expected to sleep deep to C10 in long idle
> otherwise it's mostly failed for e-star compliance.
>
> Requested reviews:
> You-Sheng Yang (vicamo)
> Sylvain Pineau (sylvain-pineau)
> OEM Solutions Group: Engineers (oem-solutions-engineers)
> OEM Services QA (oem-qa)
> Pierre Equoy (pieq)
> StanleyHuang (stanley31)
> Jonathan Cave (jocave)
> Gene Li (genelicc)
> Rex Tsai (chihchun)
> jeremyszu (os369510)
> Checkbox Developers (checkbox-dev)
>
> For more details, see:
>
> https://code.launchpad.net/~alextu/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/388726
> --
> You are the owner of
> ~alextu/plainbox-provider-checkbox:test-cpu-pc10-by-turbostate.
>

[Jonathan Cave (jocave)]

As far as I'm concerned it's still not ok to have a job install packages within the checkbox run. The user or the provisioning mechanism of the device should do this if it is required.

review: Needs Fixing

[Alex Tu (alextu)]

If we don't permit checkbox to install linux-tools-"$(uname -r)", then it will be a limitation that users need to manually install it before testing.

so far, this restriction block:
 - this MP
 - https://code.launchpad.net/~alexhung/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/394155

But seems sylvain-pineau agreed that installation of linux-tools-"$(uname -r)" on another MP: https://code.launchpad.net/~alexhung/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/394155 .

Do we have a consistent guild line for the policy of checkbox test cases? Or wiki for the policies?

[Jonathan Cave (jocave)]

> If we don't permit checkbox to install linux-tools-"$(uname -r)", then it will be a limitation that users need to manually install it before testing.

Yes, personally I prefer this. In my opinion the set of packages available is a piece of system state that is recorded by a resource job at the start of a checkbox session and should not be modified by checkbox during a session. This maintains the possible ways of dependencies being available as:

1) They are part of the os image by default
2) a. they are installed as a deb package dependency if checkbox is installed by deb
   b. they are included in the snap package as a stage-package or snapcarft part if checkbox is installed by snap
3) They are explicitly added prior the checkbox run by the user / automated provisioning system etc

For me adding a fourth option of allowing jobs to add packages mid session breaks "integrity" of information that checkbox gathers and includes in the submissions.

[Maciej Kisielewski (kissiel)]

I'm marking this MR as "work in progress", the reviews are in and we need to find a consensus.

[Alex Tu (alextu)]

Sorry for late response. It's quilt busy lately.
By following jocave's suggestion, I removed the installation of linux-tools and the require for "package.name == 'checkbox-ng'".

Thanks for reviewing and please review the latest change.

This is a test result:
https://pastebin.canonical.com/p/tVHdXXtVJ8/

[Sylvain Pineau (sylvain-pineau)]

One small issue wit the test plan.

Review of the core script will take more time

review: Needs Fixing

[Sylvain Pineau (sylvain-pineau)]

Does the new version runs using checkbox remote?

Are you still planning to run this nested test plan as part of kernel SRU testing?

review: Needs Information

[Alex Tu (alextu)]

Yes, all the verification of this MP are on checkbox remote, and this is another one after removing 'device' from `power-turbostat-residency-automated`

Yes, the 1st step we are doing is always include this case during new machine enablement.
Then we can include it to Kernel SRU when we feel it's stable enough.

So, after this MP merged, we are able to ask ODM do the test on their machines to capture more data.

And SRU Jenkins script can also try this plan so that we can see how it goes on certification pool.

For the core script, I know it's not that easy for you review. So, I call help to You-Sheng Yang(Vicamo)@HWE and he approved that as well(see the comment above).

ea683da... by Alex Tu on 2021-04-09

removed non-necessary id from power-turbostat-residency-automated

[Alex Tu (alextu)]

the test result from checkbox remote
https://paste.ubuntu.com/p/TfJtSkX9Qz/

[Sylvain Pineau (sylvain-pineau)]

I've reviewed the execution from the pastebin link above.

com.canonical.certification::miscellanea/install_kernel_tools does not exist in our current providers. where does it come from?

Unfortunately, one of the first dependency failed: com.canonical.certification::power-management/cpu-low-power-idle

Was it expected? I'd prefer a run where all tests are passing of course.

Next a couple of remarks regarding the depends fields of your new jobs. Some don't have any where others can have up to 3 job requirements. I'd help to have a clear answer here as I'd expect gfxrc6 related jobs to be skipped too (I can be wrong of course). Could you confirm the expected behavior, with cpu-low-power-idle failed is it normal to still run residency tests?

review: Needs Information

[OEM Taipei Bot (oem-taipei-bot)]

[BOT]
$ cat plainbox-provider-checkbox-0.59.0-1-ea683da-in-linux-container-focal-summary.log
bootstrap-client-cert-desktop-20-04-automated FAIL stderr: unable to find nested part: com.canonical.certification::submission-cert-automated
https://oem-share.canonical.com/share/lyoncore/artifacts/plainbox-provider-checkbox-0.59.0-1-ea683da-in-linux-container-focal

[OEM Taipei Bot (oem-taipei-bot)]

Execute `curl -X POST http://10.102.135.31/api/v1/teams/self-contained/pipelines/plainbox-provider-checkbox/resources/merge-proposal-388726/check/webhook?webhook_token=merge-proposal-388726` within TW VPN to restart the test.
[autopkgtest]
$ cat plainbox-provider-checkbox-0.59.0-1-ea683da-in-linux-container-focal-summary.log
bootstrap-client-cert-desktop-20-04-automated FAIL stderr: unable to find nested part: com.canonical.certification::submission-cert-automated
https://oem-share.canonical.com/partners/lyoncore/share/artifacts/plainbox-provider-checkbox-0.59.0-1-ea683da-in-linux-container-focal

Unmerged commits

ea683da... by Alex Tu on 2021-04-09

removed non-necessary id from power-turbostat-residency-automated

38a2e06... by Alex Tu on 2021-04-05

follow the suggestion from reviewer.

https://code.launchpad.net/~alextu/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/388726/comments/1042378

0297d6d... by Alex Tu on 2020-12-20

changed the way to enter long idle by xset dpms

The original way to simulate system get into long idle without
interactive for a while works on local run or ssh run checkbox. But
failed on checkbox remote.

So far, looks xset dpms get the same result, so I give up the original
way and move to xset dkms instead.

06ae2f0... by Alex Tu on 2020-12-20

preserve the turbostate log for s2i

07b3e3f... by Alex Tu on 2020-12-19

changed the way using sleep.mem_sleep

c2244e5... by Alex Tu on 2020-12-15

giving more tips for the failure of turbostat case.

cherry-pick 8873f8f from plainbox-provider-pc-sanity

4613170... by Alex Tu on 2020-12-11

restric turbostate tests to system that supported.

and update the automatic plan as well

77431a8... by Alex Tu on 2020-12-11

to check one residency in one job

decfec2... by Alex Tu on 2020-09-15

refactor and fixed things according to reviewer's suggestion.

46cda33... by Alex Tu on 2020-09-14

remove xte and add a automated plan to cover all jobs

and update readme of check-turbostat-power-residency.sh

diff --git a/bin/check-turbostat-power-residency.sh b/bin/check-turbostat-power-residency.sh
new file mode 100755
index 0000000..627f804
--- /dev/null
+++ b/bin/check-turbostat-power-residency.sh
@@ -0,0 +1,165 @@
+#!/bin/bash
+
+#refer to https://01.org/blogs/qwang59/2020/linux-s0ix-troubleshooting
+set -e
+result="pass"
+command -v turbostat || exit 1
+
+TARGET_STATS="GFX%rc6,Pkg%pc2,Pkg%pc3,Pkg%pc6,Pkg%pc7,Pkg%pc8,Pkg%pc9,Pk%pc10,SYS%LPI"
+declare -A stats_p
+declare -A avg_criteria
+op_mode="short-idle"
+custom_avg_criteria="true"
+i=0
+for s in ${TARGET_STATS//,/ }; do
+    stats_p[$i]="$s"
+    avg_criteria["$s"]=0
+    i=$((i+1))
+done
+
+usage() {
+cat << EOF
+usage: $(basename "$0") options [--output-directory <target-folder-for-turbostat-log>] [--op-mode <expected-operation-mode-of-e-star>]
+
+This tool will run turbostat and check if the power state meet our requirement.
+Most of operations are need root permission.
+The generated turbostat log will be put in <target-folder-for-turbostat-log>/turbostat-<expected-operation-mode-of-e-star>.log"
+
+    -h|--help Print this message
+    --output-directory
+            Sepcify the path of folder that you want to store turbostat logs. The default one is /tmp
+    -f      Read external turbostat log instead of doing turbostat.; do not need root permission.
+
+            Please get log by this command:
+            turbostat --out external-log --show GFX%rc6,Pkg%pc2,Pkg%pc3,Pkg%pc6,Pkg%pc7,Pkg%pc8,Pkg%pc9,Pk%pc10,SYS%LPI
+            Then:
+              $(basename "$0") -f path-to-external-log
+
+    --op-mode The expected operation mode defined by e-star spec. The valuse could be short-idle, long-idle or sleep-mode
+            It will be appended to the file name of the generated turbostat log file. If value is sleep-mode, this
+            script will call turbostat with rtcwake to put system to s2i mode.
+            The default is short-idle.
+
+            Usage:
+              $(basename "$0") --op-mode  sleep-mode ; # Enter s2i then get turbostat value
+
+              $(basename "$0") --op-mode  long-idle ; # You execute $(basename "$0") during system in long-idle.
+
+    --stat  Check if stat matchs expected percentage.
+
+            Usage: $(basename "$0") --stat [stat:percentage]
+
+            State could be GFX%rc6, Pk%pc10 or SYS%LPI
+            e.g. $(basename "$0") --stat Pk%pc10:60 --stat SYS%LPI:70
+EOF
+exit 1
+}
+
+while [ $# -gt 0 ]
+do
+    case "$1" in
+        -h | --help)
+            usage 0
+            exit 0
+            ;;
+        --op-mode)
+            shift
+            op_mode="$1";
+            ;;
+        --folder)
+            shift
+            [ -d "$1" ] || (echo "[ERROR] not exists folder $1" && usage)
+            session_folder=$1;
+            ;;
+        -f)
+            shift
+            if [ -f "$1" ]; then
+                EX_FILE="$1";
+        else
+        echo"[ERROR] $1 is not there."
+        usage
+            fi
+            ;;
+        --stat)
+            shift
+            [ -z "${TARGET_STATS##*${1%%:*}*}" ] || (echo "[ERROR] illegle parameter $1" && usage)
+            avg_criteria["${1%%:*}"]="${1##*:}"
+            custom_avg_criteria="true"
+            ;;
+        *)
+        usage
+       esac
+       shift
+done
+
+if [ "$custom_avg_criteria" != "true" ]; then
+    avg_criteria["GFX%rc6"]=50
+    avg_criteria["Pk%pc10"]=80
+    avg_criteria["SYS%LPI"]=70
+fi
+
+[ -n "$session_folder" ] || session_folder="/tmp"
+
+STAT_FILE="$session_folder/turbostat-$op_mode.log"
+
+require_root(){
+   if [ "$(id -u)" != "0" ]; then
+       >&2 echo "[ERROR]need root permission"
+       usage
+   fi
+}
+
+if [ -n "$EX_FILE" ]; then
+    cp "$EX_FILE" "$STAT_FILE"
+elif [ "$op_mode" == "sleep-mode" ]; then
+    require_root
+    echo "[INFO] getting turbostat log. Please wait for 60s"
+    turbostat --out "$STAT_FILE" --show $TARGET_STATS rtcwake -m freeze -s 60
+else
+    require_root
+    echo "[INFO] getting turbostat log. Please wait for about 120s"
+    # turbostat take 3-4 secs per iteration
+    turbostat --num_iterations 30 --out "$STAT_FILE" --show $TARGET_STATS
+fi
+i=0
+while read -r avg; do
+    if [ "${avg_criteria[${stats_p[$i]}]}" != "0" ]; then
+        echo "[INFO] checking if ${stats_p[$i]}($avg%) >= ${avg_criteria[${stats_p[$i]}]}%"
+        if [ "$(bc <<< "$avg >= ${avg_criteria[${stats_p[$i]}]}")" == "1" ]; then
+            echo "Passed."
+        else
+            >&2 echo "Failed" "avg $i : $avg NOT >= ${avg_criteria[${stats_p[$i]}]} "
+            result="failed"
+        fi
+    fi
+    i=$((i+1));
+done< <(grep -v "[a-zA-Z]" "$STAT_FILE" | awk '
+BEGIN { max = 0 }
+{
+    if (NF > max) max = NF;
+    for (i = 1; i <= NF; i++)
+    {
+        a[i] = a[i] + $i;
+    }
+}
+END {
+    for (i = 1; i <= max; i++)
+    {
+        if (a[i] > 1 ) print a[i] / NR;
+        else print 0
+    }
+}
+')
+
+if [ "$result" != "pass" ]; then
+    echo "Your system not get Intel CPU PC10 or s0ix as expected, it could impact e-star compliance."
+    echo "If the following test cases not run, that means your CPU architure or kernel driver not support PC10 or s0ix, please open bugs for HWE"
+    echo " - power-management/cpu-low-power-idle"
+    echo " - power-management/system-low-power-idle executed and passed."
+    echo "Otherwise, please get PHM report to check which device blocking PC10 or s0ix and open bugs for HWE to check kernel modules"
+    echo ""
+    echo "reference:"
+    echo " - https://01.org/blogs/qwang59/2020/linux-s0ix-troubleshooting"
+    echo " - https://01.org/blogs/qwang59/2018/how-achieve-s0ix-states-linux"
+    exit 1
+fi
diff --git a/units/power-management/jobs.pxu b/units/power-management/jobs.pxu
index d2ef21b..b0c41da 100644
--- a/units/power-management/jobs.pxu
+++ b/units/power-management/jobs.pxu
@@ -451,3 +451,124 @@ _steps:
 _verification:
     Did the Ambient Light Sensor values change when you shaking your hands over the sensor? 
     Did the Screen backlight also changed?
+
+plugin: shell
+category_id: com.canonical.plainbox::power-management
+id: power-management/turbostat_long_idle_result
+estimated_duration: 90s
+user: root
+requires:
+ executable.name == 'turbostat'
+command:
+ echo "Saving cpu package state tracking in $PLAINBOX_SESSION_SHARE/turbostat-long-idle.log and package_cstate_show"
+ xset dpms force off
+ check-turbostat-power-residency.sh --op-mode long-idle --folder "$PLAINBOX_SESSION_SHARE" --stat Pk%pc10:0 > /dev/null
+ xset dpms force off
+_summary: Get tubostate result from long idle.
+_description:
+ get turbo state result from long idle so that we can know cpu pkg power status. The turbostat tool is suggested by HWE. Also provide package_state_show for debug reference.
+ This test will generate a turbostat log under "$PLAINBOX_SESSION_SHARE".
+
+plugin: shell
+category_id: com.canonical.plainbox::power-management
+id: power-management/check-turbostat-long-idle-cpu-residency
+requires:
+ lsb.release >= "20.04"
+estimated_duration: 2s
+depends:
+ power-management/cpu-low-power-idle
+ power-management/turbostat_long_idle_result
+command:
+ [ -f "$PLAINBOX_SESSION_SHARE"/idle-delay.orig ] && gsettings set org.gnome.desktop.session idle-delay $(cat "$PLAINBOX_SESSION_SHARE"/idle-delay.orig)
+ check-turbostat-power-residency.sh -f "$PLAINBOX_SESSION_SHARE"/turbostat-long-idle.log --stat Pk%pc10:70
+_summary: Check expected CPU power stat residency in long-idle.
+_description:
+ check expected CPU power stat residency in long idle by by turbostat which HWE suggested and also provide package_state_show for debug reference.
+ This test job is an automatic test case and should be included into projects those need e-star compliance.
+
+plugin: shell
+category_id: com.canonical.plainbox::power-management
+id: power-management/check-turbostat-long-idle-s0ix-residency
+requires:
+ lsb.release >= "20.04"
+estimated_duration: 2s
+depends:
+ power-management/cpu-low-power-idle
+ power-management/system-low-power-idle
+ power-management/turbostat_long_idle_result
+command:
+ [ -f "$PLAINBOX_SESSION_SHARE"/idle-delay.orig ] && gsettings set org.gnome.desktop.session idle-delay $(cat "$PLAINBOX_SESSION_SHARE"/idle-delay.orig)
+ check-turbostat-power-residency.sh -f "$PLAINBOX_SESSION_SHARE"/turbostat-long-idle.log --stat SYS%LPI:70
+_summary: Check expected s0ix power stat residency in long-idle.
+_description:
+ check expected s0ix power stat residency in long idle by by turbostat which HWE suggested and also provide package_state_show for debug reference.
+ This test job is an automatic test case and should be included into projects those need e-star compliance.
+
+plugin: shell
+category_id: com.canonical.plainbox::power-management
+id: power-management/check-turbostat-long-idle-gfxrc6-residency
+requires:
+ lsb.release >= "20.04"
+ device.driver == 'i915'
+estimated_duration: 2s
+depends:
+ power-management/turbostat_long_idle_result
+command:
+ [ -f "$PLAINBOX_SESSION_SHARE"/idle-delay.orig ] && gsettings set org.gnome.desktop.session idle-delay $(cat "$PLAINBOX_SESSION_SHARE"/idle-delay.orig)
+ check-turbostat-power-residency.sh -f "$PLAINBOX_SESSION_SHARE"/turbostat-long-idle.log --stat GFX%rc6:90
+_summary: Check expected GFXRC6 power stat residency in long-idle.
+_description:
+ check expected GFXRC6 power stat residency in long idle by by turbostat which HWE suggested and also provide package_state_show for debug reference.
+ This test job is an automatic test case and should be included into projects those need e-star compliance.
+
+plugin: shell
+category_id: com.canonical.plainbox::power-management
+id: power-management/check-turbostat-s2i-cpu-residency
+estimated_duration: 90s
+user: root
+requires:
+ lsb.release >= "20.04"
+ sleep.mem_sleep == 's2idle'
+depends:
+ power-management/cpu-low-power-idle
+command:
+ check-turbostat-power-residency.sh --folder "$PLAINBOX_SESSION_SHARE" --op-mode sleep-mode --stat Pk%pc10:70
+_summary: Check expected CPU power stat residency in suspend to idle(s2i).
+_description:
+ check expected CPU power stat residency in suspend to idle(s2i) by turbostat which HWE suggested and also provide package_state_show for debug reference.
+ This test job is an automatic test case and should be included into projects those need e-star compliance.
+
+plugin: shell
+category_id: com.canonical.plainbox::power-management
+id: power-management/check-turbostat-s2i-s0ix-residency
+estimated_duration: 90s
+user: root
+requires:
+ lsb.release >= "20.04"
+ sleep.mem_sleep == 's2idle'
+depends:
+ power-management/cpu-low-power-idle
+ power-management/system-low-power-idle
+command:
+ check-turbostat-power-residency.sh --folder "$PLAINBOX_SESSION_SHARE" --op-mode sleep-mode --stat SYS%LPI:70
+_summary: Check expected s0ix power stat residency in suspend to idle(s2i).
+_description:
+ check expected s0ix power stat residency in suspend to idle(s2i) by turbostat which HWE suggested and also provide package_state_show for debug reference.
+ This test job is an automatic test case and should be included into projects those need e-star compliance.
+
+plugin: shell
+category_id: com.canonical.plainbox::power-management
+id: power-management/check-turbostat-s2i-gfxrc6-residency
+estimated_duration: 90s
+user: root
+requires:
+ lsb.release >= "20.04"
+ device.driver == 'i915'
+ sleep.mem_sleep == 's2idle'
+ executable.name == 'turbostat'
+command:
+ check-turbostat-power-residency.sh --folder "$PLAINBOX_SESSION_SHARE" --op-mode sleep-mode --stat GFX%rc6:90
+_summary: Check expected GFXRC6 power stat residency in suspend to idle(s2i).
+_description:
+ check expected GFXRC6 power stat residency in suspend to idle(s2i) by turbostat which HWE suggested and also provide package_state_show for debug reference.
+ This test job is an automatic test case and should be included into projects those need e-star compliance.
diff --git a/units/power-management/test-plan.pxu b/units/power-management/test-plan.pxu
index 8391bca..ce5ce7d 100644
--- a/units/power-management/test-plan.pxu
+++ b/units/power-management/test-plan.pxu
@@ -86,3 +86,15 @@ _description: Manual power tests for Snappy Ubuntu Core devices
 include:
     power-management/poweroff-manual
     power-management/reboot-manual
+
+id: power-turbostat-residency-automated
+unit: test plan
+_name: Check turbostate residency (automated)
+_description:
+include:
+    power-management/check-turbostat-long-idle-cpu-residency
+    power-management/check-turbostat-long-idle-s0ix-residency
+    power-management/check-turbostat-long-idle-gfxrc6-residency
+    power-management/check-turbostat-s2i-cpu-residency
+    power-management/check-turbostat-s2i-s0ix-residency
+    power-management/check-turbostat-s2i-gfxrc6-residency