Merge ~baconyao/checkbox-limerick/+git/checkbox-provider-limerick:add-vck190-server-test-plan into ~checkbox-dev/checkbox-limerick/+git/checkbox-provider-limerick:master

Proposed by PeiYao Chang
Status: Merged
Approved by: Vic Liu
Approved revision: f9acdda76442fd60974122e12685ef736e8e2dd6
Merged at revision: 732a1a76ccf89610217711044ebe7658f9d2abfc
Proposed branch: ~baconyao/checkbox-limerick/+git/checkbox-provider-limerick:add-vck190-server-test-plan
Merge into: ~checkbox-dev/checkbox-limerick/+git/checkbox-provider-limerick:master
Diff against target: 126 lines (+120/-0)
1 file modified
units/vck190-test-plan-classic.pxu (+120/-0)
Reviewer Review Type Date Requested Status
Vic Liu Approve
Review via email: mp+439119@code.launchpad.net

Commit message

Add: A new test plan to support VCK190

1. Since there's no plan to enable the Core image, therefore, support
   the classic test plan only.
2. I simple copy the plan from other platform and do the following
   changes in the exclude part of vck190-limerick-automated-classic-22-04.
   - Remove com.intel.ipdt::ipdt.*
      - No IPDT provider in Checkbox anymore
   - Remove com.canonical.certification::firmware/fwts_.*
      - Based the Test Report, this job is passed
          - Report name: limerick-versal-classic-server-2204-x01-20221111-20-manual-automatedtest-report
   - Remove com.canonical.certification::tpm2.0_4.1.1.*
      - No tpm2.0_4.1.1.* test plan in Jammy anymore
3. List some limitations below
  - The camera interface is connected to PL and we don't need to test it
  - HDMI RX is not enabled by the OOB image
  - VCK190 will be a headless enablement
  - Versal devices do not include a GPU
  - Versal devices do not include a hardened VCU in the PL

To post a comment you must log in.
Revision history for this message
Vic Liu (zongminl) wrote (last edit ):

Please see my inline comments and fix them, thank you.
This is the requirement document for your reference: https://drive.google.com/file/d/1MRXokuAduoUupvHfcC32sgBlNg_GCMKt/view?usp=share_link

review: Needs Fixing
Revision history for this message
PeiYao Chang (baconyao) wrote :

> Please see my inline comments and fix them, thank you.
> This is the requirement document for your reference: https://drive.google.com/
> file/d/1MRXokuAduoUupvHfcC32sgBlNg_GCMKt/view?usp=share_link

Updated and add those limitations in commit message.

Revision history for this message
Vic Liu (zongminl) wrote :

Thank you for the fix

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/units/vck190-test-plan-classic.pxu b/units/vck190-test-plan-classic.pxu
2new file mode 100644
3index 0000000..27a4dd7
4--- /dev/null
5+++ b/units/vck190-test-plan-classic.pxu
6@@ -0,0 +1,120 @@
7+# Copyright 2023 Canonical Ltd.
8+# All rights reserved.
9+#
10+# Test plans and (optionally) jobs unique to the Ubuntu Classic Limerick VCK190 devices.
11+#
12+
13+id: vck190-limerick-classic-22-04
14+unit: test plan
15+_name: Full manual + automated tests for Ubuntu Classic Limerick VCK190
16+_description:
17+ Combined manual and automated test plans for Ubuntu Classic Limerick VCK190 devices.
18+include:
19+nested_part:
20+ vck190-limerick-manual-classic-22-04
21+ vck190-limerick-automated-classic-22-04
22+
23+id: vck190-limerick-manual-classic-22-04
24+unit: test plan
25+_name: Manual only QA tests for Ubuntu Classic Limerick VCK190
26+_description:
27+ Ubuntu Classic QA test plan for the Limerick VCK190 hardware.
28+ This test plan contains all of the tests that require manual
29+ control of device hardware or some other user input to complete.
30+estimated_duration: 3600
31+include:
32+nested_part:
33+ com.canonical.certification::client-cert-iot-server-22-04-manual
34+exclude:
35+ com.canonical.certification::audio.* # keep if audio is not supported
36+ com.canonical.certification::bluetooth.* # keep if bluetooth is not supported
37+ com.canonical.certification::camera.* # keep if camera is not supported
38+ com.canonical.certification::mediacard.* # keep if external storage cards are not supported
39+ com.canonical.certification::socketcan.* # keep if socketcan is not supported
40+ com.canonical.certification::usb-c.* # keep if usb-c is not supported
41+ com.canonical.certification::usb3.* # keep if USB3 is not supported
42+ com.canonical.certification::watchdog.* # keep if watchdog is not supported
43+ com.canonical.certification::wireless.* # keep if wifi is not supported
44+ com.canonical.certification::wireless/wifi_ap.* # keep if wifi is supported but wifi hotspot is not
45+ com.canonical.certification::wwan.* # keep if wwan/4G/5G is not supported
46+ com.canonical.certification::led/fn # keep if there's no function key LED
47+ com.canonical.certification::led/bluetooth # keep if there's no bluetooth LED
48+ com.canonical.certification::led/power-blink-suspend # keep if suspend is not supported, or if there's no suspend LED
49+ com.canonical.certification::led/serial # keep if there are no serial LEDs
50+ com.canonical.certification::after-suspend-audio.* # keep if post-suspend audio is not supported
51+ com.canonical.certification::after-suspend.*bluetooth.* # keep if post-suspend bluetooth is not supported
52+ com.canonical.certification::after-suspend-socketcan.* # keep if post-suspend socketcan is not supported
53+ com.canonical.certification::after-suspend-usb3.* # keep if post-suspend USB3 is not supported
54+ com.canonical.certification::after-suspend-usb-c.* # keep if post-suspend USB-C is not supported
55+ com.canonical.certification::after-suspend-wireless.* # keep if post-suspend wifi is not supported
56+ com.canonical.certification::after-suspend-wireless/wifi_ap.* # keep if post-suspend wifi hotspot is not supported
57+ com.canonical.certification::after-suspend.*wwan.* # keep if post-suspend wwan/4G/5G is not supported
58+ com.canonical.certification::ethernet/wol_S5_.*
59+ com.canonical.certification::ethernet/wol_S4_.*
60+ com.canonical.certification::monitor/dvi
61+ com.canonical.certification::monitor/dvi-to-vga
62+ com.canonical.certification::monitor/hdmi-to-vga
63+ com.canonical.certification::monitor/vga
64+ com.canonical.certification::monitor/displayport_hotplug
65+ com.canonical.certification::after-suspend-monitor/dvi
66+ com.canonical.certification::after-suspend-monitor/dvi-to-vga
67+ com.canonical.certification::after-suspend-monitor/hdmi-to-vga
68+ com.canonical.certification::after-suspend-monitor/vga
69+ com.canonical.certification::after-suspend-monitor/displayport_hotplug
70+
71+id: vck190-limerick-automated-classic-22-04
72+unit: test plan
73+_name: Automated only QA tests for Ubuntu Classic Limerick VCK190
74+_description:
75+ Ubuntu Classic QA test plan for the Limerick VCK190 hardware.
76+ This test plan contains all of the automated tests used to
77+ validate the Limerick VCK190 device.
78+include:
79+ com.canonical.certification::install/apt-get-gets-updates
80+bootstrap_include:
81+ limerick_sku
82+nested_part:
83+ dtb-automated
84+ limerick-socketcan-auto-local
85+ com.canonical.certification::client-cert-iot-server-22-04-automated
86+ after-suspend-limerick-socketcan-auto-local
87+ com.canonical.certification::after-suspend-networking-automated
88+exclude:
89+ com.canonical.certification::audio.* # keep if audio is not supported
90+ com.canonical.certification::bluetooth.* # keep if bluetooth is not supported
91+ com.canonical.certification::camera.* # keep if camera is not supported
92+ com.canonical.certification::docker.* # keep if docker is unsupported
93+ com.canonical.certification::networking/predictable_names # keep if ethernet is supported and only one ethernet port exists
94+ com.canonical.certification::gpio.* # keep if gpio is not supported
95+ com.canonical.certification::location.* # keep if GPS is not supported
96+ com.canonical.certification::mediacard.* # keep if external storage cards are not supported
97+ com.canonical.certification::socketcan.* # keep if socketcan is not supported
98+ com.canonical.certification::watchdog.* # keep if watchdog is not supported
99+ com.canonical.certification::wireless.* # keep if wifi is not supported
100+ com.canonical.certification::wireless/wifi_ap.* # keep if wifi is supported but wifi hotspot is not
101+ com.canonical.certification::wwan.* # keep if wwan/4G/5G is not supported
102+ com.canonical.certification::after-suspend-audio.* # keep if post-suspend audio is not supported
103+ com.canonical.certification::after-suspend.*bluetooth.* # keep if post-suspend bluetooth is not supported
104+ com.canonical.certification::after-suspend-gpio.* # keep if post-suspend gpio is not supported
105+ com.canonical.certification::after-suspend-location.* # keep if post-suspend GPS is not supported
106+ com.canonical.certification::after-suspend-wireless.* # keep if post-suspend wifi is not supported
107+ com.canonical.certification::after-suspend.*wwan.* # keep if post-suspend wwan/4G/5G is not supported
108+ com.canonical.certification::disk/storage_device_mtd.*
109+ com.canonical.certification::power-management/cold-reboot
110+ com.canonical.certification::power-management/post-cold-reboot
111+
112+id: vck190-limerick-stress-classic-22-04
113+unit: test plan
114+_name: Stress tests for Ubuntu Classic Limerick VCK190
115+_description:
116+ Ubuntu Classic QA test plan that includes all stress tests required for Limerick VCK190 devices
117+include:
118+nested_part:
119+ com.canonical.certification::stress-iperf3-automated # keep if ethernet is supported
120+ limerick-suspend-stress-test
121+ com.canonical.certification::client-cert-iot-server-22-04-stress
122+exclude:
123+ com.canonical.certification::stress-tests/suspend.* # keep if suspend is not supported
124+ com.canonical.certification::stress-tests/hibernate.* # keep if hibernate is not supported
125+ com.canonical.certification::cold-boot-loop-.*
126+ com.canonical.certification::disk/disk_stress_ng_mtd.*

Subscribers

People subscribed via source and target branches