Checkbox Limerick

Merge ~liaou3/checkbox-limerick/+git/checkbox-provider-limerick:modify_core_testplan into ~checkbox-dev/checkbox-limerick/+git/checkbox-provider-limerick:master

  1. Git
  2. lp:~liaou3/checkbox-limerick/+git/checkbox-provider-limerick
  3. modify_core_testplan
  4. Merge into master
Proposed by Vincent Liao
Status: Merged
Approved by: Vincent Liao
Approved revision: 853fddda52c2cf828d9488b9bf4746aef51406bc
Merged at revision: 631c746fe9355be867287e94cbc1b1d80dc9ff61
Proposed branch: ~liaou3/checkbox-limerick/+git/checkbox-provider-limerick:modify_core_testplan
Merge into: ~checkbox-dev/checkbox-limerick/+git/checkbox-provider-limerick:master
Diff against target: 368 lines (+98/-122)
3 files modified
units/kd240-test-plan.pxu (+30/-9)
units/kr260-test-plan.pxu (+34/-56)
units/kv260-test-plan.pxu (+34/-57)
Reviewer Review Type Date Requested Status
Vic Liu Approve
StanleyHuang Approve
Patrick Liu (community) Approve
Rick Wu Pending
PeiYao Chang Pending
Review via email: mp+452627@code.launchpad.net

Commit message

Modify kr260 and kv260 to fit the UC22 test plan

Description of the change

Since checkbox-limerick is outdated and hasn't been used for a long time, we have to modify all to fit the latest UC22 tests. And to follow our latest policy, I also removed all exclude jobs and used manifest to control which job to run instead.
This MR nearly add nothing, the main thing of this MR is to remove all exclude jobs.

To post a comment you must log in.
Revision history for this message
Patrick Liu (patliu) wrote :

+1

review: Approve
Revision history for this message
StanleyHuang (stanley31) wrote :

LGTM

review: Approve
Revision history for this message
Vic Liu (zongminl) wrote :

I've added some inline comments to kr260-test-plan.pxu, please also do the same changes on kv260-test-plan.pxu and kd240-test-plan.pxu

review: Needs Fixing
Revision history for this message
Vincent Liao (liaou3) wrote :

> I've added some inline comments to kr260-test-plan.pxu, please also do the
> same changes on kv260-test-plan.pxu and kd240-test-plan.pxu
@Vic Fixed!

Revision history for this message
Vic Liu (zongminl) wrote :

Thank you for the updated commit, but there are still a few to be fixed

review: Needs Fixing
Revision history for this message
Vincent Liao (liaou3) wrote :

> Thank you for the updated commit, but there are still a few to be fixed
Fixed!

Revision history for this message
Vic Liu (zongminl) wrote :

All good, please do git squash before you top-approve this MR

review: Approve
Revision history for this message
Vincent Liao (liaou3) wrote :

Squashed. We are good to go.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/units/kd240-test-plan.pxu b/units/kd240-test-plan.pxu
2index 1610e91..31d0e8c 100644
3--- a/units/kd240-test-plan.pxu
4+++ b/units/kd240-test-plan.pxu
5@@ -1,14 +1,14 @@
6 # Copyright 2023 Canonical Ltd.
7 # All rights reserved.
8 #
9-# Test plans and (optionally) jobs unique to the Ubuntu Core Limerick KD240 devices.
10+# Test plans and (optionally) jobs unique to the Ubuntu Core 22 Limerick KD240 devices.
11 #
12
13 id: Kd240-limerick
14 unit: test plan
15-_name: Full manual + automated tests for Ubuntu Core Limerick KD240
16+_name: Full manual + automated tests for Ubuntu Core 22 Limerick KD240
17 _description:
18- Combined manual and automated test plans for Ubuntu Core Limerick KD240 devices.
19+ Combined manual and automated test plans for Ubuntu Core 22 Limerick KD240 devices.
20 include:
21 nested_part:
22 Kd240-limerick-manual
23@@ -16,41 +16,62 @@ nested_part:
24
25 id: Kd240-limerick-manual
26 unit: test plan
27-_name: Manual only QA tests for Ubuntu Core Limerick KD240
28+_name: Manual only QA tests for Ubuntu Core 22 Limerick KD240
29 _description:
30- Ubuntu Core QA test plan for the Limerick KD240 hardware.
31+ Ubuntu Core 22 QA test plan for the Limerick KD240 hardware.
32 This test plan contains all of the tests that require manual
33 control of device hardware or some other user input to complete.
34 estimated_duration: 3600
35 include:
36+ com.canonical.certification::disk/encryption/check-fde-tpm # keep if FDE with TPM is enabled
37 com.canonical.certification::ubuntucore/kernel-failboot-.* # keep if kernel snap is not pc-kernel
38 nested_part:
39 device-connections-tp
40+ com.canonical.qa.ceoem::ce-oem-manual
41 com.canonical.certification::client-cert-iot-ubuntucore-22-manual
42+ com.canonical.qa.ceoem::after-suspend-ce-oem-manual
43 exclude:
44
45 id: Kd240-limerick-automated
46 unit: test plan
47-_name: Automated only QA tests for Ubuntu Core Limerick KD240
48+_name: Automated only QA tests for Ubuntu Core 22 Limerick KD240
49 _description:
50- Ubuntu Core QA test plan for the Limerick KD240 hardware.
51+ Ubuntu Core 22 QA test plan for the Limerick KD240 hardware.
52 This test plan contains all of the automated tests used to
53 validate the Limerick KD240 device.
54 include:
55 com.canonical.certification::image/model-grade
56+ com.canonical.certification::miscellanea/secure_boot_mode_.* # keep if secure boot is enabled
57+ com.canonical.certification::disk/encryption/detect # keep if FDE is enabledbootstrap_include:
58+bootstrap_include:
59+ limerick_sku
60 nested_part:
61 device-connections-tp
62+ dtb-automated
63+ limerick-thermal-sensor-automated
64+ com.canonical.certification::rtc-automated
65+ com.canonical.qa.ceoem::ce-oem-automated
66 com.canonical.certification::client-cert-iot-ubuntucore-22-automated
67+ after-suspend-limerick-thermal-sensor-automated
68+ com.canonical.certification::after-suspend-networking-automated
69+ com.canonical.qa.ceoem::after-suspend-ce-oem-automated
70+ com.canonical.certification::after-suspend-rtc-automated
71+ com.canonical.certification::after-suspend-socketcan-auto-local
72+ com.canonical.certification::after-suspend-socketcan-auto-remote
73 exclude:
74
75 id: Kd240-limerick-stress
76 unit: test plan
77-_name: Stress tests for Ubuntu Core Limerick KD240
78+_name: Stress tests for Ubuntu Core 22 Limerick KD240
79 _description:
80- Ubuntu Core QA test plan that includes all stress tests required for Limerick KD240 devices
81+ Ubuntu Core 22 QA test plan that includes all stress tests required for Limerick KD240 devices
82 include:
83 nested_part:
84 device-connections-tp
85 com.canonical.certification::stress-iperf3-automated # keep if ethernet is supported
86+ limerick-suspend-stress-test
87 com.canonical.certification::client-cert-iot-ubuntucore-22-stress
88 exclude:
89+ com.canonical.certification::stress-tests/suspend.* # replace the suspend stress test in cert plan with limerick-suspend-stress-test
90+ com.canonical.certification::cold-boot-loop-.* # LP:2017486
91+ com.canonical.certification::disk/disk_stress_ng_mtd.* # LP:1965257
92diff --git a/units/kr260-test-plan.pxu b/units/kr260-test-plan.pxu
93index a270380..49bfcc7 100644
94--- a/units/kr260-test-plan.pxu
95+++ b/units/kr260-test-plan.pxu
96@@ -1,14 +1,14 @@
97-# Copyright 2022 Canonical Ltd.
98+# Copyright 2023 Canonical Ltd.
99 # All rights reserved.
100 #
101-# Test plans and (optionally) jobs unique to the Ubuntu Core Limerick KR260 devices.
102+# Test plans and (optionally) jobs unique to the Ubuntu Core 22 Limerick KR260 devices.
103 #
104
105 id: kr260-limerick
106 unit: test plan
107-_name: Full manual + automated tests for Ubuntu Core Limerick KR260
108+_name: Full manual + automated tests for Ubuntu Core 22 Limerick KR260
109 _description:
110- Combined manual and automated test plans for Ubuntu Core Limerick KR260 devices.
111+ Combined manual and automated test plans for Ubuntu Core 22 Limerick KR260 devices.
112 include:
113 nested_part:
114 kr260-limerick-manual
115@@ -16,84 +16,62 @@ nested_part:
116
117 id: kr260-limerick-manual
118 unit: test plan
119-_name: Manual only QA tests for Ubuntu Core Limerick KR260
120+_name: Manual only QA tests for Ubuntu Core 22 Limerick KR260
121 _description:
122- Ubuntu Core QA test plan for the Limerick KR260 hardware.
123+ Ubuntu Core 22 QA test plan for the Limerick KR260 hardware.
124 This test plan contains all of the tests that require manual
125 control of device hardware or some other user input to complete.
126 estimated_duration: 3600
127 include:
128+ com.canonical.certification::disk/encryption/check-fde-tpm # keep if FDE with TPM is enabled
129 com.canonical.certification::ubuntucore/kernel-failboot-.* # keep if kernel snap is not pc-kernel
130 nested_part:
131 device-connections-tp
132- com.canonical.certification::client-cert-iot-ubuntucore-20-manual
133+ com.canonical.qa.ceoem::ce-oem-manual
134+ com.canonical.certification::client-cert-iot-ubuntucore-22-manual
135+ com.canonical.qa.ceoem::after-suspend-ce-oem-manual
136 exclude:
137- com.canonical.certification::audio.* # keep if audio is not supported
138- com.canonical.certification::bluetooth.* # keep if bluetooth is not supported
139- com.canonical.certification::camera.* # keep if camera is not supported
140- com.canonical.certification::mediacard.* # keep if external storage cards are not supported
141- com.canonical.certification::socketcan.* # keep if socketcan is not supported
142- com.canonical.certification::usb-c.* # keep if usb-c is not supported
143- com.canonical.certification::watchdog.* # keep if watchdog is not supported
144- com.canonical.certification::wireless.* # keep if wifi is not supported
145- com.canonical.certification::wireless/wifi_ap.* # keep if wifi is supported but wifi hotspot is not
146- com.canonical.certification::wwan.* # keep if wwan/4G/5G is not supported
147- com.canonical.certification::led/fn # keep if there's no function key LED
148- com.canonical.certification::led/bluetooth # keep if there's no bluetooth LED
149- com.canonical.certification::led/power-blink-suspend # keep if suspend is not supported, or if there's no suspend LED
150- com.canonical.certification::led/serial # keep if there are no serial LEDs
151- com.canonical.certification::after-suspend-audio.* # keep if post-suspend audio is not supported
152- com.canonical.certification::after-suspend.*bluetooth.* # keep if post-suspend bluetooth is not supported
153- com.canonical.certification::after-suspend-socketcan.* # keep if post-suspend socketcan is not supported
154- com.canonical.certification::after-suspend-usb-c.* # keep if post-suspend USB-C is not supported
155- com.canonical.certification::after-suspend-wireless.* # keep if post-suspend wifi is not supported
156- com.canonical.certification::after-suspend-wireless/wifi_ap.* # keep if post-suspend wifi hotspot is not supported
157- com.canonical.certification::after-suspend.*wwan.* # keep if post-suspend wwan/4G/5G is not supported
158
159 id: kr260-limerick-automated
160 unit: test plan
161-_name: Automated only QA tests for Ubuntu Core Limerick KR260
162+_name: Automated only QA tests for Ubuntu Core 22 Limerick KR260
163 _description:
164- Ubuntu Core QA test plan for the Limerick KR260 hardware.
165+ Ubuntu Core 22 QA test plan for the Limerick KR260 hardware.
166 This test plan contains all of the automated tests used to
167 validate the Limerick KR260 device.
168 include:
169 com.canonical.certification::image/model-grade
170+ com.canonical.certification::miscellanea/secure_boot_mode_.* # keep if secure boot is enabled
171+ com.canonical.certification::disk/encryption/detect # keep if FDE is enabled
172+bootstrap_include:
173+ limerick_sku
174 nested_part:
175 device-connections-tp
176- com.canonical.certification::client-cert-iot-ubuntucore-20-automated
177+ dtb-automated
178+ limerick-thermal-sensor-automated
179+ com.canonical.certification::rtc-automated
180+ com.canonical.qa.ceoem::ce-oem-automated
181+ com.canonical.certification::client-cert-iot-ubuntucore-22-automated
182+ after-suspend-limerick-thermal-sensor-automated
183+ com.canonical.certification::after-suspend-networking-automated
184+ com.canonical.qa.ceoem::after-suspend-ce-oem-automated
185+ com.canonical.certification::after-suspend-rtc-automated
186+ com.canonical.certification::after-suspend-socketcan-auto-local
187+ com.canonical.certification::after-suspend-socketcan-auto-remote
188 exclude:
189- com.intel.ipdt::ipdt.* # keep if project is an ARM platform
190- com.canonical.certification::audio.* # keep if audio is not supported
191- com.canonical.certification::bluetooth.* # keep if bluetooth is not supported
192- com.canonical.certification::camera.* # keep if camera is not supported
193- com.canonical.certification::docker.* # keep if docker is unsupported
194- com.canonical.certification::firmware/fwts_.* # keep if your device is an ARM platform
195- com.canonical.certification::gpio.* # keep if gpio is not supported
196- com.canonical.certification::location.* # keep if GPS is not supported
197- com.canonical.certification::mediacard.* # keep if external storage cards are not supported
198- com.canonical.certification::socketcan.* # keep if socketcan is not supported
199- com.canonical.certification::watchdog.* # keep if watchdog is not supported
200- com.canonical.certification::wireless.* # keep if wifi is not supported
201- com.canonical.certification::wireless/wifi_ap.* # keep if wifi is supported but wifi hotspot is not
202- com.canonical.certification::wwan.* # keep if wwan/4G/5G is not supported
203- com.canonical.certification::after-suspend-audio.* # keep if post-suspend audio is not supported
204- com.canonical.certification::after-suspend.*bluetooth.* # keep if post-suspend bluetooth is not supported
205- com.canonical.certification::after-suspend-gpio.* # keep if post-suspend gpio is not supported
206- com.canonical.certification::after-suspend-location.* # keep if post-suspend GPS is not supported
207- com.canonical.certification::after-suspend-wireless.* # keep if post-suspend wifi is not supported
208- com.canonical.certification::after-suspend.*wwan.* # keep if post-suspend wwan/4G/5G is not supported
209
210 id: kr260-limerick-stress
211 unit: test plan
212-_name: Stress tests for Ubuntu Core Limerick KR260
213+_name: Stress tests for Ubuntu Core 22 Limerick KR260
214 _description:
215- Ubuntu Core QA test plan that includes all stress tests required for Limerick KR260 devices
216+ Ubuntu Core 22 QA test plan that includes all stress tests required for Limerick KR260 devices
217 include:
218 nested_part:
219 device-connections-tp
220 com.canonical.certification::stress-iperf3-automated # keep if ethernet is supported
221- com.canonical.certification::client-cert-iot-ubuntucore-20-stress
222+ limerick-suspend-stress-test
223+ com.canonical.certification::client-cert-iot-ubuntucore-22-stress
224 exclude:
225- com.canonical.certification::stress-tests/suspend.* # keep if suspend is not supported
226- com.canonical.certification::stress-tests/hibernate.* # keep if hibernate is not supported
227+ com.canonical.certification::stress-tests/suspend.* # replace the suspend stress test in cert plan with limerick-suspend-stress-test
228+ com.canonical.certification::cold-boot-loop-.* # LP:2017486
229+ com.canonical.certification::disk/disk_stress_ng_mtd.* # LP:1965257
230diff --git a/units/kv260-test-plan.pxu b/units/kv260-test-plan.pxu
231index 8753037..d983353 100644
232--- a/units/kv260-test-plan.pxu
233+++ b/units/kv260-test-plan.pxu
234@@ -1,14 +1,14 @@
235-# Copyright 2022 Canonical Ltd.
236+# Copyright 2023 Canonical Ltd.
237 # All rights reserved.
238 #
239-# Test plans and (optionally) jobs unique to the Ubuntu Core Limerick KV260 devices.
240+# Test plans and (optionally) jobs unique to the Ubuntu Core 22 Limerick KV260 devices.
241 #
242
243 id: kv260-limerick
244 unit: test plan
245-_name: Full manual + automated tests for Ubuntu Core Limerick KV260
246+_name: Full manual + automated tests for Ubuntu Core 22 Limerick KV260
247 _description:
248- Combined manual and automated test plans for Ubuntu Core Limerick KV260 devices.
249+ Combined manual and automated test plans for Ubuntu Core 22 Limerick KV260 devices.
250 include:
251 nested_part:
252 kv260-limerick-manual
253@@ -16,85 +16,62 @@ nested_part:
254
255 id: kv260-limerick-manual
256 unit: test plan
257-_name: Manual only QA tests for Ubuntu Core Limerick KV260
258+_name: Manual only QA tests for Ubuntu Core 22 Limerick KV260
259 _description:
260- Ubuntu Core QA test plan for the Limerick KV260 hardware.
261+ Ubuntu Core 22 QA test plan for the Limerick KV260 hardware.
262 This test plan contains all of the tests that require manual
263 control of device hardware or some other user input to complete.
264 estimated_duration: 3600
265 include:
266+ com.canonical.certification::disk/encryption/check-fde-tpm # keep if FDE with TPM is enabled
267 com.canonical.certification::ubuntucore/kernel-failboot-.* # keep if kernel snap is not pc-kernel
268 nested_part:
269 device-connections-tp
270- com.canonical.certification::client-cert-iot-ubuntucore-20-manual
271+ com.canonical.qa.ceoem::ce-oem-manual
272+ com.canonical.certification::client-cert-iot-ubuntucore-22-manual
273+ com.canonical.qa.ceoem::after-suspend-ce-oem-manual
274 exclude:
275- com.canonical.certification::audio.* # keep if audio is not supported
276- com.canonical.certification::bluetooth.* # keep if bluetooth is not supported
277- com.canonical.certification::camera.* # keep if camera is not supported
278- com.canonical.certification::mediacard.* # keep if external storage cards are not supported
279- com.canonical.certification::socketcan.* # keep if socketcan is not supported
280- com.canonical.certification::usb-c.* # keep if usb-c is not supported
281- com.canonical.certification::watchdog.* # keep if watchdog is not supported
282- com.canonical.certification::wireless.* # keep if wifi is not supported
283- com.canonical.certification::wireless/wifi_ap.* # keep if wifi is supported but wifi hotspot is not
284- com.canonical.certification::wwan.* # keep if wwan/4G/5G is not supported
285- com.canonical.certification::led/fn # keep if there's no function key LED
286- com.canonical.certification::led/bluetooth # keep if there's no bluetooth LED
287- com.canonical.certification::led/power-blink-suspend # keep if suspend is not supported, or if there's no suspend LED
288- com.canonical.certification::led/serial # keep if there are no serial LEDs
289- com.canonical.certification::after-suspend-audio.* # keep if post-suspend audio is not supported
290- com.canonical.certification::after-suspend.*bluetooth.* # keep if post-suspend bluetooth is not supported
291- com.canonical.certification::after-suspend-socketcan.* # keep if post-suspend socketcan is not supported
292- com.canonical.certification::after-suspend-usb-c.* # keep if post-suspend USB-C is not supported
293- com.canonical.certification::after-suspend-wireless.* # keep if post-suspend wifi is not supported
294- com.canonical.certification::after-suspend-wireless/wifi_ap.* # keep if post-suspend wifi hotspot is not supported
295- com.canonical.certification::after-suspend.*wwan.* # keep if post-suspend wwan/4G/5G is not supported
296
297 id: kv260-limerick-automated
298 unit: test plan
299-_name: Automated only QA tests for Ubuntu Core Limerick KV260
300+_name: Automated only QA tests for Ubuntu Core 22 Limerick KV260
301 _description:
302- Ubuntu Core QA test plan for the Limerick KV260 hardware.
303+ Ubuntu Core 22 QA test plan for the Limerick KV260 hardware.
304 This test plan contains all of the automated tests used to
305 validate the Limerick KV260 device.
306 include:
307 com.canonical.certification::image/model-grade
308+ com.canonical.certification::miscellanea/secure_boot_mode_.* # keep if secure boot is enabled
309+ com.canonical.certification::disk/encryption/detect # keep if FDE is enabled
310+bootstrap_include:
311+ limerick_sku
312 nested_part:
313 device-connections-tp
314- com.canonical.certification::client-cert-iot-ubuntucore-20-automated
315+ dtb-automated
316+ limerick-thermal-sensor-automated
317+ com.canonical.certification::rtc-automated
318+ com.canonical.qa.ceoem::ce-oem-automated
319+ com.canonical.certification::client-cert-iot-ubuntucore-22-automated
320+ after-suspend-limerick-thermal-sensor-automated
321+ com.canonical.certification::after-suspend-networking-automated
322+ com.canonical.qa.ceoem::after-suspend-ce-oem-automated
323+ com.canonical.certification::after-suspend-rtc-automated
324+ com.canonical.certification::after-suspend-socketcan-auto-local
325+ com.canonical.certification::after-suspend-socketcan-auto-remote
326 exclude:
327- com.intel.ipdt::ipdt.* # keep if project is an ARM platform
328- com.canonical.certification::audio.* # keep if audio is not supported
329- com.canonical.certification::bluetooth.* # keep if bluetooth is not supported
330- com.canonical.certification::camera.* # keep if camera is not supported
331- com.canonical.certification::docker.* # keep if docker is unsupported
332- com.canonical.certification::firmware/fwts_.* # keep if your device is an ARM platform
333- com.canonical.certification::networking/predictable_names # keep if ethernet is supported and only one ethernet port exists
334- com.canonical.certification::gpio.* # keep if gpio is not supported
335- com.canonical.certification::location.* # keep if GPS is not supported
336- com.canonical.certification::mediacard.* # keep if external storage cards are not supported
337- com.canonical.certification::socketcan.* # keep if socketcan is not supported
338- com.canonical.certification::watchdog.* # keep if watchdog is not supported
339- com.canonical.certification::wireless.* # keep if wifi is not supported
340- com.canonical.certification::wireless/wifi_ap.* # keep if wifi is supported but wifi hotspot is not
341- com.canonical.certification::wwan.* # keep if wwan/4G/5G is not supported
342- com.canonical.certification::after-suspend-audio.* # keep if post-suspend audio is not supported
343- com.canonical.certification::after-suspend.*bluetooth.* # keep if post-suspend bluetooth is not supported
344- com.canonical.certification::after-suspend-gpio.* # keep if post-suspend gpio is not supported
345- com.canonical.certification::after-suspend-location.* # keep if post-suspend GPS is not supported
346- com.canonical.certification::after-suspend-wireless.* # keep if post-suspend wifi is not supported
347- com.canonical.certification::after-suspend.*wwan.* # keep if post-suspend wwan/4G/5G is not supported
348
349 id: kv260-limerick-stress
350 unit: test plan
351-_name: Stress tests for Ubuntu Core Limerick KV260
352+_name: Stress tests for Ubuntu Core 22 Limerick KV260
353 _description:
354- Ubuntu Core QA test plan that includes all stress tests required for Limerick KV260 devices
355+ Ubuntu Core 22 QA test plan that includes all stress tests required for Limerick KV260 devices
356 include:
357 nested_part:
358 device-connections-tp
359 com.canonical.certification::stress-iperf3-automated # keep if ethernet is supported
360- com.canonical.certification::client-cert-iot-ubuntucore-20-stress
361+ limerick-suspend-stress-test
362+ com.canonical.certification::client-cert-iot-ubuntucore-22-stress
363 exclude:
364- com.canonical.certification::stress-tests/suspend.* # keep if suspend is not supported
365- com.canonical.certification::stress-tests/hibernate.* # keep if hibernate is not supported
366+ com.canonical.certification::stress-tests/suspend.* # replace the suspend stress test in cert plan with limerick-suspend-stress-test
367+ com.canonical.certification::cold-boot-loop-.* # LP:2017486
368+ com.canonical.certification::disk/disk_stress_ng_mtd.* # LP:1965257

Subscribers

People subscribed via source and target branches