Merge ~kissiel/snappy-hwe-snaps/+git/engineering-tests:one-provider into ~snappy-hwe-team/snappy-hwe-snaps/+git/engineering-tests:master

Proposed by Maciej Kisielewski on 2017-08-09
Status: Merged
Approved by: Simon Fels on 2017-08-29
Approved revision: b5df7704bb79d4ac89aa65e1de1b88d58cad8cb6
Merged at revision: 85bcdb994f88ae4f4d84be1fb654fdee674eb5bc
Proposed branch: ~kissiel/snappy-hwe-snaps/+git/engineering-tests:one-provider
Merge into: ~snappy-hwe-team/snappy-hwe-snaps/+git/engineering-tests:master
Diff against target: 7192 lines (+3291/-149)
39 files modified
bin/run-bluez-tests (+1/-1)
bin/run-modem-manager-tests (+1/-1)
bin/run-network-manager-tests (+1/-1)
bin/run-pulseaudio-tests (+1/-1)
bin/run-tpm-tests (+1/-1)
bin/run-udisks2-tests (+1/-1)
bin/run-wifi-ap-tests (+1/-1)
bin/run-wireless-tools-tests (+1/-1)
com.canonical.se:engineering-tests/manage.py (+21/-0)
com.canonical.se:engineering-tests/units/alsa/base.pxu (+47/-0)
com.canonical.se:engineering-tests/units/alsa/device.pxu (+15/-0)
com.canonical.se:engineering-tests/units/alsa/playback.pxu (+1/-0)
com.canonical.se:engineering-tests/units/bluez/desktop.pxu (+142/-0)
com.canonical.se:engineering-tests/units/bluez/general.pxu (+302/-0)
com.canonical.se:engineering-tests/units/bluez/hid.pxu (+83/-0)
com.canonical.se:engineering-tests/units/bluez/setup.pxu (+140/-0)
com.canonical.se:engineering-tests/units/captive-redirect/captive-redirect.pxu (+102/-0)
com.canonical.se:engineering-tests/units/media-hub/media-hub-snap.pxu (+73/-0)
com.canonical.se:engineering-tests/units/media-hub/media-hub.pxu (+65/-0)
com.canonical.se:engineering-tests/units/modem-manager/modem-manager.pxu (+103/-0)
com.canonical.se:engineering-tests/units/network-manager/network-manager.pxu (+325/-0)
com.canonical.se:engineering-tests/units/network-manager/power_management.pxu (+31/-0)
com.canonical.se:engineering-tests/units/network-manager/snap.pxu (+101/-0)
com.canonical.se:engineering-tests/units/network-manager/wireless.pxu (+54/-0)
com.canonical.se:engineering-tests/units/pulseaudio/pulseaudio.pxu (+106/-0)
com.canonical.se:engineering-tests/units/snapweb/snapweb.pxu (+182/-0)
com.canonical.se:engineering-tests/units/test-plan.pxu (+9/-0)
com.canonical.se:engineering-tests/units/tpm/tpm.pxu (+135/-0)
com.canonical.se:engineering-tests/units/udisks2/udisks2.pxu (+319/-0)
com.canonical.se:engineering-tests/units/upower/00_general.pxu (+52/-0)
com.canonical.se:engineering-tests/units/upower/10_battery_powered.pxu (+129/-0)
com.canonical.se:engineering-tests/units/upower/20_ac_powered.pxu (+49/-0)
com.canonical.se:engineering-tests/units/wifi-ap/wifi-ap.pxu (+213/-0)
com.canonical.se:engineering-tests/units/wifi-connect/wifi.connect.pxu (+202/-0)
com.canonical.se:engineering-tests/units/wireless-tools/wireless-tools.pxu (+100/-0)
com.canonical.se:engineering-tests/units/wpa-supplicant/00_general.pxu (+99/-0)
com.canonical.se:engineering-tests/units/wpa-supplicant/10_network_manager.pxu (+74/-0)
dev/null (+0/-74)
snapcraft.yaml (+9/-67)
Reviewer Review Type Date Requested Status
Konrad Zapałowicz (community) continuous-integration 2017-08-09 Approve on 2017-08-29
Simon Fels 2017-08-18 Approve on 2017-08-29
Maciej Kisielewski (community) Resubmit on 2017-08-15
Review via email: mp+328813@code.launchpad.net

Description of the change

Merge all the providers into one provider.

In order to empower reuse of the tests from engineering-tests, it's easier to have them in one plainbox-provider. They are all _provided_ by the same repository, so it IMHO makes more sense to have them grouped like that. Other (minor) benefit is having a lot less parts in snapcraft.yaml.

From a user's perspective nothing should change. I modified all the 'apps' to reflect the change in namespaces.

For your reviewing pleasure I baked a snap with what's on this branch:
https://drive.google.com/open?id=0B5xSCnq1-ZISdGtZLWpubkFwdDA

Potential follow up:
1) I left the alsa-utils reference untouched. The tests from A-U repo could live here, referencing, and requiring the A-U snap. This would deduplicate quite a bit.
2) All the 'apps' use plainbox run with a regex from commandline. I think having clearly defined "Test Plans" could be easier to use, maintain and - most importantly - reused.
3) Having a mix of dedicated 'launchers' (in ./bin) and direct calls could be improved, especially as a part of 2)
4) Theoretically plainbox should not be called. checkbox-cli should be used instead. This doesn't have any major importance here, as the benefits wouldn't be used (auto re-running of automated tests, bootstrapping test plans, ...)

To post a comment you must log in.
Maciej Kisielewski (kissiel) wrote :

And the real diff has more deletion than additions!
3218 insertions(+), 3389 deletions(-)

Maciej Kisielewski (kissiel) wrote :

I added another series of changes, mostly related to moving alsa-related tests to this repo. This way alsa-utils will be cleaner, and engineering-tests wouldn't require (quite big) alsa-utils part.

review: Resubmit
Konrad Zapałowicz (kzapalowicz) wrote :

lgtm

review: Approve (code)
Jim Hodapp (jhodapp) wrote :

I'd like Simon's comments on this as he's the person who set up the original structure for these tests. If he's ok with it, then I am as well.

Simon Fels (morphis) wrote :

LGTM

review: Approve
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/bin/run-bluez-tests b/bin/run-bluez-tests
index fa2d908..dee47c0 100755
--- a/bin/run-bluez-tests
+++ b/bin/run-bluez-tests
@@ -1,5 +1,5 @@
1#!/bin/sh1#!/bin/sh
2export PATH="$PATH:$SNAP/usr/sbin"2export PATH="$PATH:$SNAP/usr/sbin"
3exec python3 $(which plainbox) \3exec python3 $(which plainbox) \
4 run -i '2016.com.canonical.bluez::bluez.*' \4 run -i '2017.com.canonical.se::bluez.*' \
5 "$@"5 "$@"
diff --git a/bin/run-modem-manager-tests b/bin/run-modem-manager-tests
index 02fea4b..3233da5 100755
--- a/bin/run-modem-manager-tests
+++ b/bin/run-modem-manager-tests
@@ -1,5 +1,5 @@
1#!/bin/sh1#!/bin/sh
2export PATH="$PATH:$SNAP/usr/sbin"2export PATH="$PATH:$SNAP/usr/sbin"
3exec python3 $(which plainbox) \3exec python3 $(which plainbox) \
4 run -i '2016.com.canonical.modem-manager::modem_manager.*' \4 run -i '2017.com.canonical.se::modem_manager.*' \
5 "$@"5 "$@"
diff --git a/bin/run-network-manager-tests b/bin/run-network-manager-tests
index 7f90af1..8d48253 100755
--- a/bin/run-network-manager-tests
+++ b/bin/run-network-manager-tests
@@ -1,5 +1,5 @@
1#!/bin/sh1#!/bin/sh
2export PATH="$PATH:$SNAP/usr/sbin"2export PATH="$PATH:$SNAP/usr/sbin"
3exec python3 $(which plainbox) \3exec python3 $(which plainbox) \
4 run -i '2016.com.canonical.network-manager::network_manager.*' \4 run -i '2017.com.canonical.se::network_manager.*' \
5 "$@"5 "$@"
diff --git a/bin/run-pulseaudio-tests b/bin/run-pulseaudio-tests
index 4e0b088..395e8eb 100755
--- a/bin/run-pulseaudio-tests
+++ b/bin/run-pulseaudio-tests
@@ -1,5 +1,5 @@
1#!/bin/sh1#!/bin/sh
2export PATH="$PATH:$SNAP/usr/sbin"2export PATH="$PATH:$SNAP/usr/sbin"
3exec python3 $(which plainbox) \3exec python3 $(which plainbox) \
4 run -i '2016.com.canonical.pulseaudio::pulseaudio.*' \4 run -i '2017.com.canonical.se::pulseaudio.*' \
5 "$@"5 "$@"
diff --git a/bin/run-tpm-tests b/bin/run-tpm-tests
index ea48ca1..5c08021 100755
--- a/bin/run-tpm-tests
+++ b/bin/run-tpm-tests
@@ -1,5 +1,5 @@
1#!/bin/sh1#!/bin/sh
2export PATH="$PATH:$SNAP/usr/sbin"2export PATH="$PATH:$SNAP/usr/sbin"
3exec python3 $(which plainbox) \3exec python3 $(which plainbox) \
4 run -i '2016.com.canonical.tpm::tpm.*' \4 run -i '2017.com.canonical.se::tpm.*' \
5 "$@"5 "$@"
diff --git a/bin/run-udisks2-tests b/bin/run-udisks2-tests
index 5d5c73e..73fac7a 100755
--- a/bin/run-udisks2-tests
+++ b/bin/run-udisks2-tests
@@ -1,5 +1,5 @@
1#!/bin/sh1#!/bin/sh
2export PATH="$PATH:$SNAP/usr/sbin"2export PATH="$PATH:$SNAP/usr/sbin"
3exec python3 $(which plainbox) \3exec python3 $(which plainbox) \
4 run -i '2017.com.canonical.udisks2::udisks2.*' \4 run -i '2017.com.canonical.se::udisks2.*' \
5 "$@"5 "$@"
diff --git a/bin/run-wifi-ap-tests b/bin/run-wifi-ap-tests
index 0d7dce0..f69ac83 100755
--- a/bin/run-wifi-ap-tests
+++ b/bin/run-wifi-ap-tests
@@ -1,5 +1,5 @@
1#!/bin/sh1#!/bin/sh
2export PATH="$PATH:$SNAP/usr/sbin"2export PATH="$PATH:$SNAP/usr/sbin"
3exec python3 $(which plainbox) \3exec python3 $(which plainbox) \
4 run -i '2016.com.canonical.wifi-ap::wifi_ap.*' \4 run -i '2017.com.canonical.se::wifi_ap.*' \
5 "$@"5 "$@"
diff --git a/bin/run-wireless-tools-tests b/bin/run-wireless-tools-tests
index 69f5769..491c041 100755
--- a/bin/run-wireless-tools-tests
+++ b/bin/run-wireless-tools-tests
@@ -1,5 +1,5 @@
1#!/bin/sh1#!/bin/sh
2export PATH="$PATH:$SNAP/usr/sbin"2export PATH="$PATH:$SNAP/usr/sbin"
3exec python3 $(which plainbox) \3exec python3 $(which plainbox) \
4 run -i '2016.com.canonical.wireless-tools::wireless-tools.*' \4 run -i '2017.com.canonical.se::wireless-tools.*' \
5 "$@"5 "$@"
diff --git a/bluez/manage.py b/bluez/manage.py
6deleted file mode 1007556deleted file mode 100755
index 78b0dbd..0000000
--- a/bluez/manage.py
+++ /dev/null
@@ -1,10 +0,0 @@
1#!/usr/bin/env python3
2from plainbox.provider_manager import setup, N_
3
4setup(
5 name='plainbox-provider-bluez',
6 namespace='2016.com.canonical.bluez',
7 version="1.0",
8 description=N_("Plainbox provider for the BlueZ snap"),
9 gettext_domain="2016_com_canonical_bluez",
10)
diff --git a/bluez/units/desktop.pxu b/bluez/units/desktop.pxu
11deleted file mode 1006440deleted file mode 100644
index 0075fa5..0000000
--- a/bluez/units/desktop.pxu
+++ /dev/null
@@ -1,142 +0,0 @@
1# Copyright (C) 2017 Canonical, Ltd.
2# All rights reserved.
3#
4# Written by:
5# Matteo Croce <matteo.croce@canonical.com>
6
7id: bluez/setup/enabled_on_boot_desktop
8category_id: bluez
9_description: Ensure that Bluetooth is enabled on boot
10plugin: manual
11_steps:
12 1. Power on the PC
13 2. If Bluetooth isn't enabled on first boot, enable it and reboot
14_verification:
15 1. Go to System Settings->Bluetooth
16 2. Check that Bluetooth kill-switch is "on"
17 3. Look at Bluetooth indicator: verify it is active and Bluetooth kill switch is "on"
18 4. Verify that Bluetooth is still enabled
19
20id: bluez/setup/discoverable_by_default_desktop
21category_id: bluez
22_description: Ensure that device is discoverable by default
23plugin: manual
24_steps:
25 1. Power on the PC
26 2. Goto System Settings->Bluetooth
27_verification:
28 In Ubuntu Touch devices the 'discoverable' option should be enabled by default,
29 in desktop devices it should not
30
31id: bluez/setup/bt_name_correct_desktop
32category_id: bluez
33_description: Ensure that the Bluetooth device name is correct
34plugin: manual
35_steps:
36 1. Power on the PC
37_verification:
38 1. Go to System Settings->Bluetooth
39 2. Verify that the Bluetooth device name is correct:
40 is set to the Ubuntu Touch device name, eg. could be BQ Aquaris E4.5 on Ubuntu Touch,
41 or to the hostname for desktop
42
43id: bluez/connection/device_cycle
44category_id: bluez
45_description: Verify that devices will reconnect after power cycle
46plugin: manual
47_steps:
48 1. Go to Unity Control Center
49 2. Click Search for devices
50 3. Put the gadget in pairing mode
51 4. Select the gadget from the list and click pair
52 5. Power off the device
53 6. Power on the device
54_verification:
55 1. Verify that the paired device will reconnect automatically,
56 eg. it appears bold in the list
57
58id: bluez/connection/device_disconnect
59category_id: bluez
60_description: Verify that the connected device can be disconnected
61depends: bluez/connection/device_cycle
62plugin: manual
63_steps:
64 1. Pair a device with a computer
65 2. Remove the pairing
66 3. Pair the device a second time
67_verification:
68 1. Verify that the device can connect again
69
70id: bluez/connection/pc_reboot
71category_id: bluez
72_description: Verify that devices will reconnect after reboot
73depends: bluez/connection/device_cycle
74plugin: manual
75_steps:
76 1. Pair a device with the computer
77 2. Reboot the computer
78_verification:
79 1. Verify that the paired device will reconnect after rebooting it
80
81id: bluez/connection/device_discard
82category_id: bluez
83_description: Verify that the paired device can be forgotten
84depends: bluez/connection/device_cycle
85plugin: manual
86_steps:
87 1. Pair a device with the computer
88 2. Remove the pairing
89_verification:
90 1. Verify that the paired device no longer appears in the devices list,
91 eg. it's NOT listed in bold in Unity Control Center
92
93id: bluez/connection/no_connect_discarded
94category_id: bluez
95_description: Verify that a discarded device don't connect
96depends: bluez/connection/device_cycle
97plugin: manual
98_steps:
99 1. Pair a device with the computer
100 2. Remove the pairing
101_verification:
102 1. Verify that the paired device no longer connects
103
104id: bluez/connection/device_readd
105category_id: bluez
106_description: Verify that a discarded device can be paired again
107depends: bluez/connection/device_cycle
108plugin: manual
109_steps:
110 1. Pair a device with the computer
111 2. Remove the pairing
112 4. Pair the device again
113_verification:
114 1. Ensure that the device can connect
115
116id: bluez/discoverable/pc_not_discoverable
117category_id: bluez
118_description: Verify that the PC can disable discovery
119plugin: manual
120_steps:
121 1. Toggle the PC Bluetooth visibility off
122_verification:
123 1. Ensure that the PC can't be discovered from any other device
124
125id: bluez/discoverable/pc_discoverable
126category_id: bluez
127_description: Verify that the PC can enable discovery
128plugin: manual
129_steps:
130 1. Toggle the visibility on
131 2. Do a Bluetooth scan from another device, eg. a smartphone
132_verification:
133 1. Ensure that the PC can be discovered from the other device
134
135id: bluez/setup/ensure_interface_connection_setup_desktop
136category_id: bluez
137_description: Ensure that the bluez interface is connected
138plugin: manual
139_verification:
140 Open a text editor and make sure you can type text from the HID keyboard
141 into the text box. If testing a mouse, ensure that the cursor moves when
142 moving your HID mouse.
diff --git a/bluez/units/general.pxu b/bluez/units/general.pxu
143deleted file mode 1006440deleted file mode 100644
index 18b8b52..0000000
--- a/bluez/units/general.pxu
+++ /dev/null
@@ -1,302 +0,0 @@
1# Copyright (C) 2017 Canonical, Ltd.
2# All rights reserved.
3#
4# Written by:
5# Jim Hodapp <jim.hodapp@canonical.com>
6# Matteo Croce <matteo.croce@canonical.com>
7
8unit: category
9id: bluez
10_name: BlueZ
11
12id: bluez/setup/enabled_on_boot
13category_id: bluez
14_description: Ensure that Bluetooth is enabled on boot
15plugin: manual
16_steps:
17 1. Power on the Device Under Test
18 2. If Bluetooth isn't enabled on first boot, enable it and reboot
19 3. Use bluetoothctl to check if there is an active adapter
20 .
21 $ bluetoothctl
22_verification:
23 1. Check that there is a Controller in the output, eg.
24 [NEW] Controller 00:10:20:30:40:50 pi [default]
25
26id: bluez/scan/start_inquiry
27category_id: bluez
28_description: Verify that found gadgets are reported during an ongoing device discovery
29plugin: manual
30_steps:
31 1. Turn Bluetooth on on a gadget and make it discoverable
32 2. Use bluetoothctl to start a device scan:
33 .
34 $ bluetoothctl
35 [bluetooth]# scan on
36 [bluetooth]# devices
37 .
38_verification:
39 You should see the gadget listed after the local adapter:
40 Device XX:XX:XX:XX:XX:XX My Device
41 [NEW] Device 04:52:C7:0D:0C:03 QC 35s
42
43id: bluez/scan/stop_inquiry
44category_id: bluez
45_description: Verify that gadgets are not found with scan disabled
46plugin: manual
47_steps:
48 1. Use bluetoothctl to start stop the device scan
49 .
50 $ bluetoothctl
51 [bluetooth]# scan off
52 .
53 2. Turn bluetooth on on a gadget and make it discoverable
54 3. Use bluetoothctl to get the devices list
55 .
56 $ bluetoothctl
57 [bluetooth]# devices
58 .
59_verification:
60 You should only see the local adapter in the list
61 Device XX:XX:XX:XX:XX:XX My Device
62
63id: bluez/pairing/can_be_enabled
64category_id: bluez
65_description: Verify that the device pairing works
66plugin: manual
67_steps:
68 1. Use bluetoothctl to enable pairing
69 .
70 $ bluetoothctl
71 [bluetooth]# pairable on
72 .
73 2. Pair the Device Under Test from a gadget
74_verification:
75 1. You should be able to do the pairing
76
77id: bluez/pairing/can_be_disabled
78category_id: bluez
79_description: Verify that the pairing can be disabled
80plugin: manual
81_steps:
82 1. Use bluetoothctl to disable pairing
83 .
84 $ bluetoothctl
85 [bluetooth]# pairable off
86 .
87 2. Pair the Device Under Test from a gadget
88_verification:
89 1. You shouldn't be able to do the pairing
90
91id: bluez/setup/discoverable_by_default
92category_id: bluez
93_description: Ensure that Device Under Test is discoverable by default
94plugin: manual
95_steps:
96 1. Type:
97 .
98 $ bluetoothctl
99 [bluetooth]# show
100 2. Scan for devices from a smartphone
101_verification:
102 1. Verify that the Device Under Test is discoverable by default, eg.
103 .
104 $ bluetoothctl
105 [bluetooth]# show
106 .
107 contains the "Discoverable: yes" line
108 2. Check that the Device Under Test can be discovered on the smartphone
109
110id: bluez/setup/bt_name_correct
111category_id: bluez
112_description: Ensure that the Bluetooth Device Under Test name is correct
113plugin: manual
114_steps:
115 1. Type:
116 .
117 $ bluetoothctl
118 [bluetooth]# show
119_verification:
120 1. Verify that the Bluetooth Device Under Test name is correct: equals to the Ubuntu Touch
121 device name, eg. could be BQ Aquaris E4.5 on Ubuntu Touch,
122 or to the hostname for desktop
123 .
124 Name: Ubuntu
125
126id: bluez/setup/bt_name_correct_from_gadget
127category_id: bluez
128_description: Ensure that the Bluetooth Device Under Test name advertise correctly
129plugin: manual
130_steps:
131 1. Scan for devices from a smartphone
132_verification:
133 1. Verify that the Bluetooth Device Under Test name is correct as seen from an external device
134
135id: bluez/connection/gadget_reconnect_after_power_cycle
136category_id: bluez
137_description: Verify that gadgets will reconnect after power cycle
138plugin: manual
139_steps:
140 1. Connect a gadget with the Device Under Test:
141 .
142 $ bluetoothctl
143 bluetooth]# scan on
144 Discovery started
145 [NEW] Device 01:02:03:04:05:06 device
146 [bluetooth]# pair 01:02:03:04:05:06
147 Pairing successful
148 [bluetooth]# trust 01:02:03:04:05:06
149 Trusted
150 [bluetooth]# connect 01:02:03:04:05:06
151 .
152 2. Power off the gadget
153 3. Power on the gadget
154_verification:
155 1. Verify that the paired gadget will reconnect automatically
156 .
157 $ bluetoothctl
158 bluetooth]# info 01:02:03:04:05:06
159 .
160 and check for "Connected: Yes"
161
162id: bluez/connection/reconnect_after_dut_reboot
163category_id: bluez
164_description: Verify that gadgets will reconnect after reboot
165plugin: manual
166depends: bluez/connection/gadget_reconnect_after_power_cycle
167_steps:
168 1. Pair a gadget with the Device Under Test
169 2. Reboot the Device Under Test
170_verification:
171 1. Verify that the paired gadget will reconnect after reboot
172
173id: bluez/connection/gadget_disconnect
174category_id: bluez
175_description: Verify that the connected gadget can be disconnected
176plugin: manual
177depends: bluez/connection/gadget_reconnect_after_power_cycle
178_steps:
179 1. Connect a gadget with the Device Under Test
180 2. Disconnect it with:
181 .
182 $ bluetoothctl
183 [bluetooth]# disconnect 01:02:03:04:05:06
184 .
185_verification:
186 1. Verify that the gadget is disconnected
187 .
188 $ bluetoothctl
189 bluetooth]# info 01:02:03:04:05:06
190 .
191 and check for "Connected: no"
192
193id: bluez/connection/gadget_can_be_forgotten
194category_id: bluez
195_description: Verify that the paired gadget can be forgotten
196plugin: manual
197depends: bluez/connection/gadget_reconnect_after_power_cycle
198_steps:
199 1. Pair a gadget with the Device Under Test
200 2. Remove the pairing
201 .
202 $ bluetoothctl
203 [bluetooth]# remove 01:02:03:04:05:06
204_verification:
205 1. Verify that the paired gadget no longer appears in the gadgets list
206 .
207 $ bluetoothctl
208 bluetooth]# info 01:02:03:04:05:06
209 .
210 and check for "Paired: no"
211
212id: bluez/connection/gadget_paired_again
213category_id: bluez
214_description: Verify that forgotten gadgets can be paired again
215plugin: manual
216depends: bluez/connection/gadget_can_be_forgotten
217_steps:
218 1. Pair a gadget with the Device Under Test
219 2. Remove the pairing
220 4. Pair the gadget a second time
221_verification:
222 1. Ensure that the gadget be paired again:
223 .
224 $ bluetoothctl
225 bluetooth]# info 01:02:03:04:05:06
226 .
227 and check for "Paired: yes"
228
229id: bluez/discoverable/dut_not_discoverable
230category_id: bluez
231_description: Verify that the Device Under Test can disable discovery
232plugin: manual
233_steps:
234 1. Toggle the Device Under Test Bluetooth visibility off:
235 .
236 $ bluetoothctl
237 bluetooth]# discoverable off
238 .
239_verification:
240 1. Ensure that the Device Under Test can't be discovered from any other gadget
241
242id: bluez/discoverable/dut_discoverable
243category_id: bluez
244_description: Verify that the Device Under Test can enable discovery
245plugin: manual
246_steps:
247 1. Toggle the Device Under Test Bluetooth visibility on
248 .
249 $ bluetoothctl
250 bluetooth]# discoverable on
251 .
252_verification:
253 1. Ensure that the Device Under Test can be discovered from other gadgets
254
255id: bluez/audio/a2dp_ubuntu_classic
256category_id: bluez
257plugin: manual
258depends: bluez/connection/pair_and_connect
259_summary: Stream audio to Bluetooth headset
260_purpose:
261 Verify that you can stream music to a Bluetooth headset via A2DP on Ubuntu
262 Classic. Note: Skip this test if not running Ubuntu Classic
263_steps:
264 1. Follow the previous tests to list, pair and connect your headset
265 Navigate to the Sound panel in System Settings, select your headset device
266 in the Output tab, and make sure that Mode is set to A2DP
267 Then press the Test Sound button and test each speaker
268_verification:
269 Verify that you can hear the test sound "Front left", "Front right" in each
270 speaker respectively.
271
272id: bluez/audio/a2dp_disconnect
273category_id: bluez
274plugin: manual
275depends: bluez/audio/a2dp_ubuntu_classic
276_summary: Disconnect Bluetooth headset
277_purpose:
278 Verify that you can disconnect Bluetooth headset while streaming and audio
279 playback switches back to the Device Under Test's primary speaker
280_steps:
281 1. Follow the previous tests to start streaming audio to the headset via
282 A2DP. Then execute the following command to disconnect the headset from
283 active A2DP streaming:
284 .
285 [<dut_name>]# disconnect <gadget_address>
286_verification:
287 Verify that you see output similar to the following:
288 Attempting to disconnect from 04:52:C7:0D:0C:03
289 [CHG] Device 04:52:C7:0D:0C:03 ServicesResolved: no
290 Successful disconnected
291 [CHG] Device 04:52:C7:0D:0C:03 Connected: no
292 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Key: 0x0110
293 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x40
294 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x0c
295 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x02
296 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x41
297 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x00
298 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x61
299 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x71
300 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0xcd
301 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x92
302 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x6c
diff --git a/bluez/units/hid.pxu b/bluez/units/hid.pxu
303deleted file mode 1006440deleted file mode 100644
index 394e361..0000000
--- a/bluez/units/hid.pxu
+++ /dev/null
@@ -1,83 +0,0 @@
1# Copyright (C) 2017 Canonical, Ltd.
2# All rights reserved.
3#
4# Written by:
5# Matteo Croce <matteo.croce@canonical.com>
6
7id: bluez/hid/pair_and_connect_hid
8category_id: bluez
9plugin: manual
10depends: bluez/setup/ensure_interface_connection_setup
11_summary: A Bluetooth HID gadget (e.g. keyboard, mouse) is connected and paired
12_purpose:
13 Check if your Bluetooth HID gadget of choice (e.g. keyboard, mouse) can be
14 paired to the host system.
15_steps:
16 1. Execute the following command to pair and connect your HID gadget to the
17 host system:
18 .
19 [bluetooth]# scan on
20 [bluetooth]# scan off
21 [bluetooth]# pair 01:02:03:04:05:06
22 [bluetooth]# connect 01:02:03:04:05:06
23 .
24_verification:
25 Open a text editor and make sure you can type text from the HID keyboard
26 into the text box. If testing a mouse, ensure that the cursor moves when
27 moving your HID mouse
28
29id: bluez/hid/bt_mouse_works
30category_id: bluez
31_description: Verify that Bluetooth mice do work
32plugin: manual
33depends: bluez/hid/pair_and_connect_hid
34_steps:
35 1. Scan for a BT mouse and connect it to the DUT
36_verification:
37 1. Verify that the paired mouse can move the pointer
38 2. Verify that the device is automatically connected at boot
39 3. Verify that the device is automatically connected after resume from sleep
40
41id: bluez/hid/bt_keyboard_works
42category_id: bluez
43_description: Verify that Bluetooth keyboards do work
44plugin: manual
45_steps:
46 1. Scan for a BT keyboard and connect it to the DUT, eg.
47 .
48 $ bluetoothctl
49 [bluetooth]# scan on
50 [bluetooth]# scan off
51 [bluetooth]# pair 01:02:03:04:05:06
52 Pairing successful
53 [bluetooth]# trust 01:02:03:04:05:06
54 [bluetooth]# connect 01:02:03:04:05:06
55 Trusted
56_verification:
57 1. Verify that the paired keyboard can write in a text editor
58 2. Verify that the device is automatically connected at boot
59 3. Verify that the device is automatically connected after resume from sleep
60 4. Press a key on the keyboard and power it off. Verify that the keystrokes are not transferred endlessly
61
62id: bluez/hid/ble_mouse_works
63category_id: bluez
64_description: Verify that BLE mices do work
65plugin: manual
66_steps:
67 1. Pair a BLE mouse with the DUT.
68_verification:
69 1. Verify that the paired mouse can move the pointer
70 2. Verify that the device is automatically connected at boot
71 3. Verify that the device is automatically connected after resume from sleep
72
73id: bluez/hid/ble_keyboard_works
74category_id: bluez
75_description: Verify that BLE keyboards do work
76plugin: manual
77_steps:
78 1. Pair a BLE keyboard with the DUT.
79_verification:
80 1. Verify that the paired keyboard can write in a text editor
81 2. Verify that the device is automatically connected at boot
82 3. Verify that the device is automatically connected after resume from sleep
83 4. Press a key on the keyboard and power it off. Verify that the keystrokes are not transferred endlessly
diff --git a/bluez/units/setup.pxu b/bluez/units/setup.pxu
84deleted file mode 1006440deleted file mode 100644
index 3060426..0000000
--- a/bluez/units/setup.pxu
+++ /dev/null
@@ -1,140 +0,0 @@
1# Copyright (C) 2017 Canonical, Ltd.
2# All rights reserved.
3#
4# Written by:
5# Matteo Croce <matteo.croce@canonical.com>
6
7id: bluez/setup/ensure_interface_connection_setup
8category_id: bluez
9_description: Ensure that the bluez interface is connected
10plugin: manual
11_steps:
12 1. Connect required bluez slot/plug
13 .
14 $ sudo snap connect bluez:client bluez:service
15 .
16_verification:
17 Verify that you see the following when running snap interfaces bluez:
18 bluez:service bluez:client
19
20id: bluez/snap/ensure_nonroot_cannot_use
21category_id: bluez
22plugin: manual
23depends: bluez/setup/ensure_interface_connection_setup
24_summary: bluetoothctl and obexctl cannot be run as non-root user
25_purpose:
26 Verify that the bluetoothctl and obexctl commands cannot be run as non-root
27_steps:
28 1. Start the bluetoothctl and obexctl commands as non-root user
29 .
30 $ bluetoothctl
31 $ obexctl
32 .
33_verification:
34 You should see outout similar to the following:
35 .
36 $ bluetoothctl
37 This script must be run as root
38 $ obexctl
39 This script must be run as root
40
41id: bluez/scan/get_all
42category_id: bluez
43plugin: manual
44depends: bluez/setup/ensure_interface_connection_setup
45_summary: All Bluetooth gadgets in range are listed
46_purpose:
47 Check if all powered on gadgets that are within range as listed.
48_steps:
49 1. Execute the following command to retrieve the list of all Bluetooth gadgets:
50 .
51 $ /snap/bin/bluez.bluetoothctl
52 [bluetooth]# scan on
53 .
54_verification:
55 Your expected Bluetooth gadget(s) is/are shown in the output list such as:
56 [NEW] Device 04:52:C7:0D:0C:03 QC 35s
57
58id: bluez/connection/pair_and_connect
59category_id: bluez
60plugin: manual
61depends: bluez/setup/ensure_interface_connection_setup
62_summary: A Bluetooth gadget of choice is paired and connected
63_purpose:
64 Check if your Bluetooth gadget of choice can be paired to the host system.
65_steps:
66 1. Execute the following command to pair and connect your gadget to the host
67 system using the address listed in the previous scan test:
68 .
69 [bluetooth]# pair <gadget_address>
70 [bluetooth]# connect <gadget_address>
71 .
72_verification:
73 You should see output similar to the following:
74 Attempting to pair with 04:52:C7:0D:0C:03
75 [CHG] Device 04:52:C7:0D:0C:03 Connected: yes
76 [CHG] Device 04:52:C7:0D:0C:03 Modalias: bluetooth:v009Ep400Cd0106
77 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00000000-deca-fade-deca-deafdecacaff
78 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001101-0000-1000-8000-00805f9b34fb
79 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001108-0000-1000-8000-00805f9b34fb
80 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110b-0000-1000-8000-00805f9b34fb
81 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110c-0000-1000-8000-00805f9b34fb
82 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110e-0000-1000-8000-00805f9b34fb
83 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000111e-0000-1000-8000-00805f9b34fb
84 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001200-0000-1000-8000-00805f9b34fb
85 [CHG] Device 04:52:C7:0D:0C:03 ServicesResolved: yes
86 [CHG] Device 04:52:C7:0D:0C:03 Paired: yes
87 Pairing successful
88 [CHG] Device 04:52:C7:0D:0C:03 ServicesResolved: no
89 [CHG] Device 04:52:C7:0D:0C:03 Connected: no
90 Attempting to connect to 04:52:C7:0D:0C:03
91 [CHG] Device 04:52:C7:0D:0C:03 Connected: yes
92 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00000000-deca-fade-deca-deafdecacaff
93 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001101-0000-1000-8000-00805f9b34fb
94 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001108-0000-1000-8000-00805f9b34fb
95 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110b-0000-1000-8000-00805f9b34fb
96 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110c-0000-1000-8000-00805f9b34fb
97 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110e-0000-1000-8000-00805f9b34fb
98 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000111e-0000-1000-8000-00805f9b34fb
99 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000112f-0000-1000-8000-00805f9b34fb
100 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001200-0000-1000-8000-00805f9b34fb
101 Connection successful
102 [CHG] Device 04:52:C7:0D:0C:03 ServicesResolved: yes
103
104id: bluez/snap/ensure_btmon_can_be started
105category_id: bluez
106plugin: manual
107depends: bluez/setup/ensure_interface_connection_setup
108_summary: btmon tools function
109_purpose:
110 Check that the btmon tool shipped as a part of the bluez snap can be used.
111_steps:
112 1. Start the btmon command
113 .
114 $ sudo btmon
115 .
116_verification:
117 You should see outout similar to the following:
118 .
119 $ sudo btmon
120 Bluetooth monitor ver 5.37
121 = New Index: 00:1A:7D:DA:71:08 (BR/EDR,USB,hci0) [hci0] 0.626494
122
123id: bluez/snap/ensure_hcidump_can_be started
124category_id: bluez
125plugin: manual
126depends: bluez/setup/ensure_interface_connection_setup
127_summary: btmon tools works
128_purpose:
129 Check that the hcidump tool shipped as a part of the bluez snap can be used.
130_steps:
131 1. Start the hcidump command
132 .
133 $ sudo hcidump
134 .
135_verification:
136 You should see outout similar to the following:
137 .
138 $ sudo hcidump
139 HCI sniffer - Bluetooth packet analyzer ver 5.37
140 device: hci0 snap_len: 1500 filter: 0xffffffff
diff --git a/captive-redirect/manage.py b/captive-redirect/manage.py
141deleted file mode 1007550deleted file mode 100755
index 56ea1c8..0000000
--- a/captive-redirect/manage.py
+++ /dev/null
@@ -1,10 +0,0 @@
1#!/usr/bin/env python3
2from plainbox.provider_manager import setup, N_
3
4setup(
5 name='plainbox-provider-captive-redirect',
6 namespace='2017.com.canonical.captive-redirect',
7 version="1.0",
8 description=N_("Plainbox provider for the Captive Redirect snap"),
9 gettext_domain="2017_com_canonical_captive_redirect",
10)
diff --git a/captive-redirect/units/captive-redirect.pxu b/captive-redirect/units/captive-redirect.pxu
11deleted file mode 1006440deleted file mode 100644
index d45f0e6..0000000
--- a/captive-redirect/units/captive-redirect.pxu
+++ /dev/null
@@ -1,102 +0,0 @@
1# Copyright (C) 2017 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Matteo Croce <matteo.croce@canonical.com>
6
7unit: category
8id: captive_redirect
9_name: Captive portal redirect
10
11id: captive_redirect/setup/ensure_interface_connection_setup
12category_id: captive_redirect
13_description: Ensure that the captive-redirect interfaces are connected
14plugin: manual
15_steps:
16 1. Install the captive-redirect snap via
17 .
18 $ snap install captive-redirect
19 .
20_verification:
21 Verify that you see the following when running snap interfaces:
22 :firewall-control captive-redirect
23
24id: captive_redirect/setup/default_disabled
25category_id: captive_redirect
26_description: Ensure that captive-redirect is disabled by default
27plugin: manual
28_steps:
29 1. Get the configuration with:
30 .
31 $ captive-redirect.config status
32 .
33 2. Ensure that there isn't any REDIRECT iptables chain:
34 .
35 $ sudo iptables-save |grep captive
36 .
37_verification:
38 1. Check that the redirect is disabled:
39 .
40 active: false
41 .
42 2. You shouldn't see any iptables rule
43
44id: captive_redirect/setup/iptables_rules_all_interfaces
45category_id: captive_redirect
46_description: Ensure that enabling captive-redirect sets up the correct iptables rules
47plugin: manual
48_steps:
49 1. Enable captive-redirect with:
50 .
51 $ sudo snap set captive-redirect active=true port=9876
52 .
53 2. Get the iptables rules with:
54 .
55 $ sudo iptables-save |grep captive
56 .
57_verification:
58 1. You should see iptables rules like:
59 .
60 :captive - [0:0]
61 -A PREROUTING -j captive
62 -A captive -p tcp -m multiport --dports 80,443 -j REDIRECT --to-ports 9876
63 .
64
65id: captive_redirect/setup/iptables_rules_one_interface
66category_id: captive_redirect
67_description: Ensure that enabling captive-redirect sets up iptables rules for a single interface
68plugin: manual
69_steps:
70 1. Enable captive-redirect with:
71 .
72 $ sudo snap set captive-redirect active=true port=9999 interfaces=tun4
73 .
74 2. Get the iptables rules with:
75 .
76 $ sudo iptables-save |grep captive
77 .
78_verification:
79 1. You should see iptables rules like:
80 .
81 :captive - [0:0]
82 -A PREROUTING -j captive
83 -A captive -i tun4 -p tcp -m multiport --dports 80,443 -j REDIRECT --to-ports 9999
84 .
85
86id: captive_redirect/functionality/redirect_works
87category_id: captive_redirect
88_description: Ensure that enabling captive-redirect really redirects the traffic
89plugin: manual
90_steps:
91 1. Install a sample web server with:
92 .
93 $ sudo snap install xkcd-webserver
94 .
95 2. Connect another device to the DUT (Device Under Test) and setup the DUT as default gateway
96 3. Enable the redirect with:
97 .
98 $ sudo snap set captive-redirect active=true port=80 interfaces=
99 .
100 4. Try to access an HTTP (avoid HTTPS, it will give a certificate error) web page from the device
101_verification:
102 1. You should see a blank page pointing to XKCD instead of the typed address
diff --git a/com.canonical.se:engineering-tests/manage.py b/com.canonical.se:engineering-tests/manage.py
103new file mode 1007550new file mode 100755
index 0000000..6a27365
--- /dev/null
+++ b/com.canonical.se:engineering-tests/manage.py
@@ -0,0 +1,21 @@
1#!/usr/bin/env python3
2from plainbox.provider_manager import setup, N_
3
4# You can inject other stuff here but please don't go overboard.
5#
6# In particular, if you need comprehensive compilation support to get
7# your bin/ populated then please try to discuss that with us in the
8# upstream project IRC channel #checkbox on irc.freenode.net.
9
10# NOTE: one thing that you could do here, that makes a lot of sense,
11# is to compute version somehow. This may vary depending on the
12# context of your provider. Future version of PlainBox will offer git,
13# bzr and mercurial integration using the versiontools library
14# (optional)
15
16setup(
17 name='2017.com.canonical.se:engineering-tests',
18 version="1.0",
19 description=N_("The com.canonical.se:engineering-tests provider"),
20 gettext_domain="com_canonical_se_engineering-tests",
21)
diff --git a/com.canonical.se:engineering-tests/units/alsa/base.pxu b/com.canonical.se:engineering-tests/units/alsa/base.pxu
0new file mode 10064422new file mode 100644
index 0000000..077a012
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/alsa/base.pxu
@@ -0,0 +1,47 @@
1# Copyright (c) 2017 Canonical Ltd.
2# All rights resevered.
3#
4# Written by:
5# Simon Fels <simon.fels@canonical.com>
6
7unit: category
8id: alsa_utils
9_name: Alsa utilities
10
11id: alsa_utils/setup/automatic_plug_connection
12category_id: alsa_utils
13_description:
14 Ensure that all interfaces needed by the alsa-utils snap are
15 automatically connected.
16plugin: shell
17command:
18 snap interfaces | grep -Pzq ":home +alsa-utils"
19 snap interfaces | grep -Pzq ":alsa +alsa-utils"
20estimated_duration: 1s
21flags: preserve-locale
22
23id: alsa_utils/setup/automatic_aliases
24category_id: alsa_utils
25_description:
26 Ensure that all aliases provided by the snap are automatically setup.
27plugin: shell
28command:
29 test -x /snap/bin/speaker-test
30 test -x /snap/bin/aseqnet
31 test -x /snap/bin/alsatplg
32 test -x /snap/bin/arecordmidi
33 test -x /snap/bin/amidi
34 test -x /snap/bin/alsaloop
35 test -x /snap/bin/aconnect
36 test -x /snap/bin/iecset
37 test -x /snap/bin/alsabat
38 test -x /snap/bin/aseqdump
39 test -x /snap/bin/aplaymidi
40 test -x /snap/bin/alsactl
41 test -x /snap/bin/amixer
42 test -x /snap/bin/aplay
43 test -x /snap/bin/alsaucm
44 test -x /snap/bin/alsamixer
45 test -x /snap/bin/arecord
46estimated_duration: 1s
47flags: preserve-locale
diff --git a/com.canonical.se:engineering-tests/units/alsa/device.pxu b/com.canonical.se:engineering-tests/units/alsa/device.pxu
0new file mode 10064448new file mode 100644
index 0000000..a3f7b34
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/alsa/device.pxu
@@ -0,0 +1,15 @@
1# FIXME: Convert this test back to an automated one. See LP #1665652
2id: alsa_utils/device/can_list_available_cards
3category_id: alsa_utils
4_description:
5 Validate that available ALSA sound cards can be listed with the
6 utilities the snap ships.
7plugin: manual
8_steps:
9 Execute the following command:
10 .
11 $ /snap/bin/aplay -L
12 .
13_verification:
14 Verify that the command prints out a list of available ALSA
15 devices.
diff --git a/com.canonical.se:engineering-tests/units/alsa/playback.pxu b/com.canonical.se:engineering-tests/units/alsa/playback.pxu
0new file mode 10064416new file mode 100644
index 0000000..d1a8083
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/alsa/playback.pxu
@@ -0,0 +1 @@
1# FIXME: Convert this test back to an automated one. See LP #1665652
diff --git a/com.canonical.se:engineering-tests/units/bluez/desktop.pxu b/com.canonical.se:engineering-tests/units/bluez/desktop.pxu
0new file mode 1006442new file mode 100644
index 0000000..0075fa5
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/bluez/desktop.pxu
@@ -0,0 +1,142 @@
1# Copyright (C) 2017 Canonical, Ltd.
2# All rights reserved.
3#
4# Written by:
5# Matteo Croce <matteo.croce@canonical.com>
6
7id: bluez/setup/enabled_on_boot_desktop
8category_id: bluez
9_description: Ensure that Bluetooth is enabled on boot
10plugin: manual
11_steps:
12 1. Power on the PC
13 2. If Bluetooth isn't enabled on first boot, enable it and reboot
14_verification:
15 1. Go to System Settings->Bluetooth
16 2. Check that Bluetooth kill-switch is "on"
17 3. Look at Bluetooth indicator: verify it is active and Bluetooth kill switch is "on"
18 4. Verify that Bluetooth is still enabled
19
20id: bluez/setup/discoverable_by_default_desktop
21category_id: bluez
22_description: Ensure that device is discoverable by default
23plugin: manual
24_steps:
25 1. Power on the PC
26 2. Goto System Settings->Bluetooth
27_verification:
28 In Ubuntu Touch devices the 'discoverable' option should be enabled by default,
29 in desktop devices it should not
30
31id: bluez/setup/bt_name_correct_desktop
32category_id: bluez
33_description: Ensure that the Bluetooth device name is correct
34plugin: manual
35_steps:
36 1. Power on the PC
37_verification:
38 1. Go to System Settings->Bluetooth
39 2. Verify that the Bluetooth device name is correct:
40 is set to the Ubuntu Touch device name, eg. could be BQ Aquaris E4.5 on Ubuntu Touch,
41 or to the hostname for desktop
42
43id: bluez/connection/device_cycle
44category_id: bluez
45_description: Verify that devices will reconnect after power cycle
46plugin: manual
47_steps:
48 1. Go to Unity Control Center
49 2. Click Search for devices
50 3. Put the gadget in pairing mode
51 4. Select the gadget from the list and click pair
52 5. Power off the device
53 6. Power on the device
54_verification:
55 1. Verify that the paired device will reconnect automatically,
56 eg. it appears bold in the list
57
58id: bluez/connection/device_disconnect
59category_id: bluez
60_description: Verify that the connected device can be disconnected
61depends: bluez/connection/device_cycle
62plugin: manual
63_steps:
64 1. Pair a device with a computer
65 2. Remove the pairing
66 3. Pair the device a second time
67_verification:
68 1. Verify that the device can connect again
69
70id: bluez/connection/pc_reboot
71category_id: bluez
72_description: Verify that devices will reconnect after reboot
73depends: bluez/connection/device_cycle
74plugin: manual
75_steps:
76 1. Pair a device with the computer
77 2. Reboot the computer
78_verification:
79 1. Verify that the paired device will reconnect after rebooting it
80
81id: bluez/connection/device_discard
82category_id: bluez
83_description: Verify that the paired device can be forgotten
84depends: bluez/connection/device_cycle
85plugin: manual
86_steps:
87 1. Pair a device with the computer
88 2. Remove the pairing
89_verification:
90 1. Verify that the paired device no longer appears in the devices list,
91 eg. it's NOT listed in bold in Unity Control Center
92
93id: bluez/connection/no_connect_discarded
94category_id: bluez
95_description: Verify that a discarded device don't connect
96depends: bluez/connection/device_cycle
97plugin: manual
98_steps:
99 1. Pair a device with the computer
100 2. Remove the pairing
101_verification:
102 1. Verify that the paired device no longer connects
103
104id: bluez/connection/device_readd
105category_id: bluez
106_description: Verify that a discarded device can be paired again
107depends: bluez/connection/device_cycle
108plugin: manual
109_steps:
110 1. Pair a device with the computer
111 2. Remove the pairing
112 4. Pair the device again
113_verification:
114 1. Ensure that the device can connect
115
116id: bluez/discoverable/pc_not_discoverable
117category_id: bluez
118_description: Verify that the PC can disable discovery
119plugin: manual
120_steps:
121 1. Toggle the PC Bluetooth visibility off
122_verification:
123 1. Ensure that the PC can't be discovered from any other device
124
125id: bluez/discoverable/pc_discoverable
126category_id: bluez
127_description: Verify that the PC can enable discovery
128plugin: manual
129_steps:
130 1. Toggle the visibility on
131 2. Do a Bluetooth scan from another device, eg. a smartphone
132_verification:
133 1. Ensure that the PC can be discovered from the other device
134
135id: bluez/setup/ensure_interface_connection_setup_desktop
136category_id: bluez
137_description: Ensure that the bluez interface is connected
138plugin: manual
139_verification:
140 Open a text editor and make sure you can type text from the HID keyboard
141 into the text box. If testing a mouse, ensure that the cursor moves when
142 moving your HID mouse.
diff --git a/com.canonical.se:engineering-tests/units/bluez/general.pxu b/com.canonical.se:engineering-tests/units/bluez/general.pxu
0new file mode 100644143new file mode 100644
index 0000000..18b8b52
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/bluez/general.pxu
@@ -0,0 +1,302 @@
1# Copyright (C) 2017 Canonical, Ltd.
2# All rights reserved.
3#
4# Written by:
5# Jim Hodapp <jim.hodapp@canonical.com>
6# Matteo Croce <matteo.croce@canonical.com>
7
8unit: category
9id: bluez
10_name: BlueZ
11
12id: bluez/setup/enabled_on_boot
13category_id: bluez
14_description: Ensure that Bluetooth is enabled on boot
15plugin: manual
16_steps:
17 1. Power on the Device Under Test
18 2. If Bluetooth isn't enabled on first boot, enable it and reboot
19 3. Use bluetoothctl to check if there is an active adapter
20 .
21 $ bluetoothctl
22_verification:
23 1. Check that there is a Controller in the output, eg.
24 [NEW] Controller 00:10:20:30:40:50 pi [default]
25
26id: bluez/scan/start_inquiry
27category_id: bluez
28_description: Verify that found gadgets are reported during an ongoing device discovery
29plugin: manual
30_steps:
31 1. Turn Bluetooth on on a gadget and make it discoverable
32 2. Use bluetoothctl to start a device scan:
33 .
34 $ bluetoothctl
35 [bluetooth]# scan on
36 [bluetooth]# devices
37 .
38_verification:
39 You should see the gadget listed after the local adapter:
40 Device XX:XX:XX:XX:XX:XX My Device
41 [NEW] Device 04:52:C7:0D:0C:03 QC 35s
42
43id: bluez/scan/stop_inquiry
44category_id: bluez
45_description: Verify that gadgets are not found with scan disabled
46plugin: manual
47_steps:
48 1. Use bluetoothctl to start stop the device scan
49 .
50 $ bluetoothctl
51 [bluetooth]# scan off
52 .
53 2. Turn bluetooth on on a gadget and make it discoverable
54 3. Use bluetoothctl to get the devices list
55 .
56 $ bluetoothctl
57 [bluetooth]# devices
58 .
59_verification:
60 You should only see the local adapter in the list
61 Device XX:XX:XX:XX:XX:XX My Device
62
63id: bluez/pairing/can_be_enabled
64category_id: bluez
65_description: Verify that the device pairing works
66plugin: manual
67_steps:
68 1. Use bluetoothctl to enable pairing
69 .
70 $ bluetoothctl
71 [bluetooth]# pairable on
72 .
73 2. Pair the Device Under Test from a gadget
74_verification:
75 1. You should be able to do the pairing
76
77id: bluez/pairing/can_be_disabled
78category_id: bluez
79_description: Verify that the pairing can be disabled
80plugin: manual
81_steps:
82 1. Use bluetoothctl to disable pairing
83 .
84 $ bluetoothctl
85 [bluetooth]# pairable off
86 .
87 2. Pair the Device Under Test from a gadget
88_verification:
89 1. You shouldn't be able to do the pairing
90
91id: bluez/setup/discoverable_by_default
92category_id: bluez
93_description: Ensure that Device Under Test is discoverable by default
94plugin: manual
95_steps:
96 1. Type:
97 .
98 $ bluetoothctl
99 [bluetooth]# show
100 2. Scan for devices from a smartphone
101_verification:
102 1. Verify that the Device Under Test is discoverable by default, eg.
103 .
104 $ bluetoothctl
105 [bluetooth]# show
106 .
107 contains the "Discoverable: yes" line
108 2. Check that the Device Under Test can be discovered on the smartphone
109
110id: bluez/setup/bt_name_correct
111category_id: bluez
112_description: Ensure that the Bluetooth Device Under Test name is correct
113plugin: manual
114_steps:
115 1. Type:
116 .
117 $ bluetoothctl
118 [bluetooth]# show
119_verification:
120 1. Verify that the Bluetooth Device Under Test name is correct: equals to the Ubuntu Touch
121 device name, eg. could be BQ Aquaris E4.5 on Ubuntu Touch,
122 or to the hostname for desktop
123 .
124 Name: Ubuntu
125
126id: bluez/setup/bt_name_correct_from_gadget
127category_id: bluez
128_description: Ensure that the Bluetooth Device Under Test name advertise correctly
129plugin: manual
130_steps:
131 1. Scan for devices from a smartphone
132_verification:
133 1. Verify that the Bluetooth Device Under Test name is correct as seen from an external device
134
135id: bluez/connection/gadget_reconnect_after_power_cycle
136category_id: bluez
137_description: Verify that gadgets will reconnect after power cycle
138plugin: manual
139_steps:
140 1. Connect a gadget with the Device Under Test:
141 .
142 $ bluetoothctl
143 bluetooth]# scan on
144 Discovery started
145 [NEW] Device 01:02:03:04:05:06 device
146 [bluetooth]# pair 01:02:03:04:05:06
147 Pairing successful
148 [bluetooth]# trust 01:02:03:04:05:06
149 Trusted
150 [bluetooth]# connect 01:02:03:04:05:06
151 .
152 2. Power off the gadget
153 3. Power on the gadget
154_verification:
155 1. Verify that the paired gadget will reconnect automatically
156 .
157 $ bluetoothctl
158 bluetooth]# info 01:02:03:04:05:06
159 .
160 and check for "Connected: Yes"
161
162id: bluez/connection/reconnect_after_dut_reboot
163category_id: bluez
164_description: Verify that gadgets will reconnect after reboot
165plugin: manual
166depends: bluez/connection/gadget_reconnect_after_power_cycle
167_steps:
168 1. Pair a gadget with the Device Under Test
169 2. Reboot the Device Under Test
170_verification:
171 1. Verify that the paired gadget will reconnect after reboot
172
173id: bluez/connection/gadget_disconnect
174category_id: bluez
175_description: Verify that the connected gadget can be disconnected
176plugin: manual
177depends: bluez/connection/gadget_reconnect_after_power_cycle
178_steps:
179 1. Connect a gadget with the Device Under Test
180 2. Disconnect it with:
181 .
182 $ bluetoothctl
183 [bluetooth]# disconnect 01:02:03:04:05:06
184 .
185_verification:
186 1. Verify that the gadget is disconnected
187 .
188 $ bluetoothctl
189 bluetooth]# info 01:02:03:04:05:06
190 .
191 and check for "Connected: no"
192
193id: bluez/connection/gadget_can_be_forgotten
194category_id: bluez
195_description: Verify that the paired gadget can be forgotten
196plugin: manual
197depends: bluez/connection/gadget_reconnect_after_power_cycle
198_steps:
199 1. Pair a gadget with the Device Under Test
200 2. Remove the pairing
201 .
202 $ bluetoothctl
203 [bluetooth]# remove 01:02:03:04:05:06
204_verification:
205 1. Verify that the paired gadget no longer appears in the gadgets list
206 .
207 $ bluetoothctl
208 bluetooth]# info 01:02:03:04:05:06
209 .
210 and check for "Paired: no"
211
212id: bluez/connection/gadget_paired_again
213category_id: bluez
214_description: Verify that forgotten gadgets can be paired again
215plugin: manual
216depends: bluez/connection/gadget_can_be_forgotten
217_steps:
218 1. Pair a gadget with the Device Under Test
219 2. Remove the pairing
220 4. Pair the gadget a second time
221_verification:
222 1. Ensure that the gadget be paired again:
223 .
224 $ bluetoothctl
225 bluetooth]# info 01:02:03:04:05:06
226 .
227 and check for "Paired: yes"
228
229id: bluez/discoverable/dut_not_discoverable
230category_id: bluez
231_description: Verify that the Device Under Test can disable discovery
232plugin: manual
233_steps:
234 1. Toggle the Device Under Test Bluetooth visibility off:
235 .
236 $ bluetoothctl
237 bluetooth]# discoverable off
238 .
239_verification:
240 1. Ensure that the Device Under Test can't be discovered from any other gadget
241
242id: bluez/discoverable/dut_discoverable
243category_id: bluez
244_description: Verify that the Device Under Test can enable discovery
245plugin: manual
246_steps:
247 1. Toggle the Device Under Test Bluetooth visibility on
248 .
249 $ bluetoothctl
250 bluetooth]# discoverable on
251 .
252_verification:
253 1. Ensure that the Device Under Test can be discovered from other gadgets
254
255id: bluez/audio/a2dp_ubuntu_classic
256category_id: bluez
257plugin: manual
258depends: bluez/connection/pair_and_connect
259_summary: Stream audio to Bluetooth headset
260_purpose:
261 Verify that you can stream music to a Bluetooth headset via A2DP on Ubuntu
262 Classic. Note: Skip this test if not running Ubuntu Classic
263_steps:
264 1. Follow the previous tests to list, pair and connect your headset
265 Navigate to the Sound panel in System Settings, select your headset device
266 in the Output tab, and make sure that Mode is set to A2DP
267 Then press the Test Sound button and test each speaker
268_verification:
269 Verify that you can hear the test sound "Front left", "Front right" in each
270 speaker respectively.
271
272id: bluez/audio/a2dp_disconnect
273category_id: bluez
274plugin: manual
275depends: bluez/audio/a2dp_ubuntu_classic
276_summary: Disconnect Bluetooth headset
277_purpose:
278 Verify that you can disconnect Bluetooth headset while streaming and audio
279 playback switches back to the Device Under Test's primary speaker
280_steps:
281 1. Follow the previous tests to start streaming audio to the headset via
282 A2DP. Then execute the following command to disconnect the headset from
283 active A2DP streaming:
284 .
285 [<dut_name>]# disconnect <gadget_address>
286_verification:
287 Verify that you see output similar to the following:
288 Attempting to disconnect from 04:52:C7:0D:0C:03
289 [CHG] Device 04:52:C7:0D:0C:03 ServicesResolved: no
290 Successful disconnected
291 [CHG] Device 04:52:C7:0D:0C:03 Connected: no
292 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Key: 0x0110
293 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x40
294 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x0c
295 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x02
296 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x41
297 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x00
298 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x61
299 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x71
300 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0xcd
301 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x92
302 [CHG] Device 04:52:C7:0D:0C:03 ManufacturerData Value: 0x6c
diff --git a/com.canonical.se:engineering-tests/units/bluez/hid.pxu b/com.canonical.se:engineering-tests/units/bluez/hid.pxu
0new file mode 100644303new file mode 100644
index 0000000..394e361
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/bluez/hid.pxu
@@ -0,0 +1,83 @@
1# Copyright (C) 2017 Canonical, Ltd.
2# All rights reserved.
3#
4# Written by:
5# Matteo Croce <matteo.croce@canonical.com>
6
7id: bluez/hid/pair_and_connect_hid
8category_id: bluez
9plugin: manual
10depends: bluez/setup/ensure_interface_connection_setup
11_summary: A Bluetooth HID gadget (e.g. keyboard, mouse) is connected and paired
12_purpose:
13 Check if your Bluetooth HID gadget of choice (e.g. keyboard, mouse) can be
14 paired to the host system.
15_steps:
16 1. Execute the following command to pair and connect your HID gadget to the
17 host system:
18 .
19 [bluetooth]# scan on
20 [bluetooth]# scan off
21 [bluetooth]# pair 01:02:03:04:05:06
22 [bluetooth]# connect 01:02:03:04:05:06
23 .
24_verification:
25 Open a text editor and make sure you can type text from the HID keyboard
26 into the text box. If testing a mouse, ensure that the cursor moves when
27 moving your HID mouse
28
29id: bluez/hid/bt_mouse_works
30category_id: bluez
31_description: Verify that Bluetooth mice do work
32plugin: manual
33depends: bluez/hid/pair_and_connect_hid
34_steps:
35 1. Scan for a BT mouse and connect it to the DUT
36_verification:
37 1. Verify that the paired mouse can move the pointer
38 2. Verify that the device is automatically connected at boot
39 3. Verify that the device is automatically connected after resume from sleep
40
41id: bluez/hid/bt_keyboard_works
42category_id: bluez
43_description: Verify that Bluetooth keyboards do work
44plugin: manual
45_steps:
46 1. Scan for a BT keyboard and connect it to the DUT, eg.
47 .
48 $ bluetoothctl
49 [bluetooth]# scan on
50 [bluetooth]# scan off
51 [bluetooth]# pair 01:02:03:04:05:06
52 Pairing successful
53 [bluetooth]# trust 01:02:03:04:05:06
54 [bluetooth]# connect 01:02:03:04:05:06
55 Trusted
56_verification:
57 1. Verify that the paired keyboard can write in a text editor
58 2. Verify that the device is automatically connected at boot
59 3. Verify that the device is automatically connected after resume from sleep
60 4. Press a key on the keyboard and power it off. Verify that the keystrokes are not transferred endlessly
61
62id: bluez/hid/ble_mouse_works
63category_id: bluez
64_description: Verify that BLE mices do work
65plugin: manual
66_steps:
67 1. Pair a BLE mouse with the DUT.
68_verification:
69 1. Verify that the paired mouse can move the pointer
70 2. Verify that the device is automatically connected at boot
71 3. Verify that the device is automatically connected after resume from sleep
72
73id: bluez/hid/ble_keyboard_works
74category_id: bluez
75_description: Verify that BLE keyboards do work
76plugin: manual
77_steps:
78 1. Pair a BLE keyboard with the DUT.
79_verification:
80 1. Verify that the paired keyboard can write in a text editor
81 2. Verify that the device is automatically connected at boot
82 3. Verify that the device is automatically connected after resume from sleep
83 4. Press a key on the keyboard and power it off. Verify that the keystrokes are not transferred endlessly
diff --git a/com.canonical.se:engineering-tests/units/bluez/setup.pxu b/com.canonical.se:engineering-tests/units/bluez/setup.pxu
0new file mode 10064484new file mode 100644
index 0000000..3060426
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/bluez/setup.pxu
@@ -0,0 +1,140 @@
1# Copyright (C) 2017 Canonical, Ltd.
2# All rights reserved.
3#
4# Written by:
5# Matteo Croce <matteo.croce@canonical.com>
6
7id: bluez/setup/ensure_interface_connection_setup
8category_id: bluez
9_description: Ensure that the bluez interface is connected
10plugin: manual
11_steps:
12 1. Connect required bluez slot/plug
13 .
14 $ sudo snap connect bluez:client bluez:service
15 .
16_verification:
17 Verify that you see the following when running snap interfaces bluez:
18 bluez:service bluez:client
19
20id: bluez/snap/ensure_nonroot_cannot_use
21category_id: bluez
22plugin: manual
23depends: bluez/setup/ensure_interface_connection_setup
24_summary: bluetoothctl and obexctl cannot be run as non-root user
25_purpose:
26 Verify that the bluetoothctl and obexctl commands cannot be run as non-root
27_steps:
28 1. Start the bluetoothctl and obexctl commands as non-root user
29 .
30 $ bluetoothctl
31 $ obexctl
32 .
33_verification:
34 You should see outout similar to the following:
35 .
36 $ bluetoothctl
37 This script must be run as root
38 $ obexctl
39 This script must be run as root
40
41id: bluez/scan/get_all
42category_id: bluez
43plugin: manual
44depends: bluez/setup/ensure_interface_connection_setup
45_summary: All Bluetooth gadgets in range are listed
46_purpose:
47 Check if all powered on gadgets that are within range as listed.
48_steps:
49 1. Execute the following command to retrieve the list of all Bluetooth gadgets:
50 .
51 $ /snap/bin/bluez.bluetoothctl
52 [bluetooth]# scan on
53 .
54_verification:
55 Your expected Bluetooth gadget(s) is/are shown in the output list such as:
56 [NEW] Device 04:52:C7:0D:0C:03 QC 35s
57
58id: bluez/connection/pair_and_connect
59category_id: bluez
60plugin: manual
61depends: bluez/setup/ensure_interface_connection_setup
62_summary: A Bluetooth gadget of choice is paired and connected
63_purpose:
64 Check if your Bluetooth gadget of choice can be paired to the host system.
65_steps:
66 1. Execute the following command to pair and connect your gadget to the host
67 system using the address listed in the previous scan test:
68 .
69 [bluetooth]# pair <gadget_address>
70 [bluetooth]# connect <gadget_address>
71 .
72_verification:
73 You should see output similar to the following:
74 Attempting to pair with 04:52:C7:0D:0C:03
75 [CHG] Device 04:52:C7:0D:0C:03 Connected: yes
76 [CHG] Device 04:52:C7:0D:0C:03 Modalias: bluetooth:v009Ep400Cd0106
77 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00000000-deca-fade-deca-deafdecacaff
78 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001101-0000-1000-8000-00805f9b34fb
79 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001108-0000-1000-8000-00805f9b34fb
80 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110b-0000-1000-8000-00805f9b34fb
81 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110c-0000-1000-8000-00805f9b34fb
82 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110e-0000-1000-8000-00805f9b34fb
83 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000111e-0000-1000-8000-00805f9b34fb
84 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001200-0000-1000-8000-00805f9b34fb
85 [CHG] Device 04:52:C7:0D:0C:03 ServicesResolved: yes
86 [CHG] Device 04:52:C7:0D:0C:03 Paired: yes
87 Pairing successful
88 [CHG] Device 04:52:C7:0D:0C:03 ServicesResolved: no
89 [CHG] Device 04:52:C7:0D:0C:03 Connected: no
90 Attempting to connect to 04:52:C7:0D:0C:03
91 [CHG] Device 04:52:C7:0D:0C:03 Connected: yes
92 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00000000-deca-fade-deca-deafdecacaff
93 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001101-0000-1000-8000-00805f9b34fb
94 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001108-0000-1000-8000-00805f9b34fb
95 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110b-0000-1000-8000-00805f9b34fb
96 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110c-0000-1000-8000-00805f9b34fb
97 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000110e-0000-1000-8000-00805f9b34fb
98 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000111e-0000-1000-8000-00805f9b34fb
99 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 0000112f-0000-1000-8000-00805f9b34fb
100 [CHG] Device 04:52:C7:0D:0C:03 UUIDs: 00001200-0000-1000-8000-00805f9b34fb
101 Connection successful
102 [CHG] Device 04:52:C7:0D:0C:03 ServicesResolved: yes
103
104id: bluez/snap/ensure_btmon_can_be started
105category_id: bluez
106plugin: manual
107depends: bluez/setup/ensure_interface_connection_setup
108_summary: btmon tools function
109_purpose:
110 Check that the btmon tool shipped as a part of the bluez snap can be used.
111_steps:
112 1. Start the btmon command
113 .
114 $ sudo btmon
115 .
116_verification:
117 You should see outout similar to the following:
118 .
119 $ sudo btmon
120 Bluetooth monitor ver 5.37
121 = New Index: 00:1A:7D:DA:71:08 (BR/EDR,USB,hci0) [hci0] 0.626494
122
123id: bluez/snap/ensure_hcidump_can_be started
124category_id: bluez
125plugin: manual
126depends: bluez/setup/ensure_interface_connection_setup
127_summary: btmon tools works
128_purpose:
129 Check that the hcidump tool shipped as a part of the bluez snap can be used.
130_steps:
131 1. Start the hcidump command
132 .
133 $ sudo hcidump
134 .
135_verification:
136 You should see outout similar to the following:
137 .
138 $ sudo hcidump
139 HCI sniffer - Bluetooth packet analyzer ver 5.37
140 device: hci0 snap_len: 1500 filter: 0xffffffff
diff --git a/com.canonical.se:engineering-tests/units/captive-redirect/captive-redirect.pxu b/com.canonical.se:engineering-tests/units/captive-redirect/captive-redirect.pxu
0new file mode 100644141new file mode 100644
index 0000000..d45f0e6
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/captive-redirect/captive-redirect.pxu
@@ -0,0 +1,102 @@
1# Copyright (C) 2017 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Matteo Croce <matteo.croce@canonical.com>
6
7unit: category
8id: captive_redirect
9_name: Captive portal redirect
10
11id: captive_redirect/setup/ensure_interface_connection_setup
12category_id: captive_redirect
13_description: Ensure that the captive-redirect interfaces are connected
14plugin: manual
15_steps:
16 1. Install the captive-redirect snap via
17 .
18 $ snap install captive-redirect
19 .
20_verification:
21 Verify that you see the following when running snap interfaces:
22 :firewall-control captive-redirect
23
24id: captive_redirect/setup/default_disabled
25category_id: captive_redirect
26_description: Ensure that captive-redirect is disabled by default
27plugin: manual
28_steps:
29 1. Get the configuration with:
30 .
31 $ captive-redirect.config status
32 .
33 2. Ensure that there isn't any REDIRECT iptables chain:
34 .
35 $ sudo iptables-save |grep captive
36 .
37_verification:
38 1. Check that the redirect is disabled:
39 .
40 active: false
41 .
42 2. You shouldn't see any iptables rule
43
44id: captive_redirect/setup/iptables_rules_all_interfaces
45category_id: captive_redirect
46_description: Ensure that enabling captive-redirect sets up the correct iptables rules
47plugin: manual
48_steps:
49 1. Enable captive-redirect with:
50 .
51 $ sudo snap set captive-redirect active=true port=9876
52 .
53 2. Get the iptables rules with:
54 .
55 $ sudo iptables-save |grep captive
56 .
57_verification:
58 1. You should see iptables rules like:
59 .
60 :captive - [0:0]
61 -A PREROUTING -j captive
62 -A captive -p tcp -m multiport --dports 80,443 -j REDIRECT --to-ports 9876
63 .
64
65id: captive_redirect/setup/iptables_rules_one_interface
66category_id: captive_redirect
67_description: Ensure that enabling captive-redirect sets up iptables rules for a single interface
68plugin: manual
69_steps:
70 1. Enable captive-redirect with:
71 .
72 $ sudo snap set captive-redirect active=true port=9999 interfaces=tun4
73 .
74 2. Get the iptables rules with:
75 .
76 $ sudo iptables-save |grep captive
77 .
78_verification:
79 1. You should see iptables rules like:
80 .
81 :captive - [0:0]
82 -A PREROUTING -j captive
83 -A captive -i tun4 -p tcp -m multiport --dports 80,443 -j REDIRECT --to-ports 9999
84 .
85
86id: captive_redirect/functionality/redirect_works
87category_id: captive_redirect
88_description: Ensure that enabling captive-redirect really redirects the traffic
89plugin: manual
90_steps:
91 1. Install a sample web server with:
92 .
93 $ sudo snap install xkcd-webserver
94 .
95 2. Connect another device to the DUT (Device Under Test) and setup the DUT as default gateway
96 3. Enable the redirect with:
97 .
98 $ sudo snap set captive-redirect active=true port=80 interfaces=
99 .
100 4. Try to access an HTTP (avoid HTTPS, it will give a certificate error) web page from the device
101_verification:
102 1. You should see a blank page pointing to XKCD instead of the typed address
diff --git a/com.canonical.se:engineering-tests/units/media-hub/media-hub-snap.pxu b/com.canonical.se:engineering-tests/units/media-hub/media-hub-snap.pxu
0new file mode 100644103new file mode 100644
index 0000000..24ad2bc
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/media-hub/media-hub-snap.pxu
@@ -0,0 +1,73 @@
1# Copyright (C) 2017 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Alfonso Sánchez-Beato <alfonso.sanchez-beato@canonical.com>
6
7id: media_hub/snap_classic/play_wav
8category_id: media_hub
9plugin: manual
10_summary: Confined media-hub can play wav files
11_purpose:
12 Check if media-hub properly plays wav files when confined. The scenario is
13 Unity8 on Classic.
14_steps:
15 1. Install snaps and perform needed interface connections
16 .
17 $ snap install media-hub
18 $ snap install --devmode mediaplayer-app
19 $ sudo snap connect mediaplayer-app:mpris media-hub:mpris
20 .
21 2. Start media-hub-server (needed until LP #1613420 is solved)
22 .
23 $ media-hub.media-hub-service
24 .
25 3. Download sample file
26 .
27 $ wget -P ~/Music/ http://freewavesamples.com/files/Kawai-K5000W-Buchla-C1.wav
28 .
29 4. Launch mediaplayer-app
30 .
31 $ /snap/bin/mediaplayer-app ~/Music/Kawai-K5000W-Buchla-C1.wav
32 .
33 5. Let the song play until the end
34 6. Restart playback after the song finishes
35 7. Pause, then press play again a few times
36 8. While the song is playing, move the slider back and forth in time
37 9. Pause the song, then move the slider again
38_verification:
39 Verify that in all cases the audio is played as expected.
40
41id: media_hub/snap_classic/play_mp3
42category_id: media_hub
43plugin: manual
44_summary: Confined media-hub can play an mp3
45_purpose:
46 Check if media-hub properly plays an mp3 when confined. The scenario is Unity8
47 on Classic.
48_steps:
49 1. Install snaps and perform needed interface connections
50 .
51 $ snap install media-hub
52 $ snap install --devmode mediaplayer-app
53 $ sudo snap connect mediaplayer-app:mpris media-hub:mpris
54 .
55 2. Start media-hub-server (needed until LP #1613420 is solved)
56 .
57 $ media-hub.media-hub-service
58 .
59 3. Download sample file
60 .
61 $ wget -P ~/Music/ http://www.stephaniequinn.com/Music/Canon.mp3
62 .
63 4. Launch mediaplayer-app
64 .
65 $ /snap/bin/mediaplayer-app ~/Music/Canon.mp3
66 .
67 5. Let the song play until the end
68 6. Restart playback after the song finishes
69 7. Pause, then press play again a few times
70 8. While the song is playing, move the slider back and forth in time
71 9. Pause the song, then move the slider again
72_verification:
73 Verify that in all cases the audio is played as expected.
diff --git a/com.canonical.se:engineering-tests/units/media-hub/media-hub.pxu b/com.canonical.se:engineering-tests/units/media-hub/media-hub.pxu
0new file mode 10064474new file mode 100644
index 0000000..bdeec69
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/media-hub/media-hub.pxu
@@ -0,0 +1,65 @@
1# Copyright 2017 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Alfonso Sánchez-Beato <alfonso.sanchez-beato@canonical.com>
6
7unit: category
8id: media_hub
9_name: media-hub
10
11id: media_hub/video/play_h264
12category_id: media_hub
13plugin: manual
14_summary: H.264 videos play as expected
15_purpose:
16 Check if media-hub plays properly H.264 videos
17_steps:
18 1. Download H.264 video
19 .
20 $ wget https://download.blender.org/durian/trailer/sintel_trailer-720p.mp4
21 .
22 2. Start media-hub-server
23 .
24 $ MIR_SOCKET=/run/user/$UID/mir_socket media-hub-server --desktop_file_hint=unity8
25 .
26 3. Start to play the video
27 .
28 $ MIR_SOCKET=/run/user/$UID/mir_socket QT_QPA_PLATFORM=ubuntumirclient DISPLAY=:0 /usr/bin/mediaplayer-app --desktop_file_hint=unity8 sintel_trailer-720p.mp4
29 .
30 4. Let the video play until the end
31 5. Re-start after the video finishes
32 6. Pause, then press play again a few times
33 7. While the video is playing, move the slider back and forth in time
34 8. Pause the video, then move the slider again
35 9. Re-size, maximize, then minimize the window several times
36_verification:
37 Verify that in all cases the video is played as expected.
38
39id: media_hub/video/play_mpeg4
40category_id: media_hub
41plugin: manual
42_summary: MPEG4 videos play as expected
43_purpose:
44 Check if media-hub plays properly MPEG4 videos
45_steps:
46 1. Download H.264 video
47 .
48 $ wget https://download.blender.org/durian/trailer/Sintel_Trailer.480p.DivX_Home_Theater.divx
49 .
50 2. Start media-hub-server
51 .
52 $ MIR_SOCKET=/run/user/$UID/mir_socket media-hub-server --desktop_file_hint=unity8
53 .
54 3. Start to play the video
55 .
56 $ MIR_SOCKET=/run/user/$UID/mir_socket QT_QPA_PLATFORM=ubuntumirclient DISPLAY=:0 /usr/bin/mediaplayer-app --desktop_file_hint=unity8 Sintel_Trailer.480p.DivX_Home_Theater.divx
57 .
58 4. Let the video play until the end
59 5. Re-start after the video finishes
60 6. Pause, then press play again a few times
61 7. While the video is playing, move the slider back and forth in time
62 8. Pause the video, then move the slider again
63 9. Re-size, maximize, then minimize the window several times
64_verification:
65 Verify that in all cases the video is played as expected.
diff --git a/com.canonical.se:engineering-tests/units/modem-manager/modem-manager.pxu b/com.canonical.se:engineering-tests/units/modem-manager/modem-manager.pxu
0new file mode 10064466new file mode 100644
index 0000000..3912ac2
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/modem-manager/modem-manager.pxu
@@ -0,0 +1,103 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Jim Hodapp <jim.hodapp@canonical.com>
6# Alfonso Sanchez-Beato <alfonso.sanchez-beato@canonical.com>
7
8unit: category
9id: modem_manager
10_name: Modem Manager
11
12id: modem_manager/setup/ensure_interface_connection_setup
13category_id: modem_manager
14_description: Ensure that the modem-manager interface is connected
15plugin: manual
16_steps:
17 1. Install network-manager if not already installed
18 .
19 $ sudo snap install network-manager
20 .
21 2. Connect required modem-manager/network-manager slots/plugs
22 .
23 $ sudo snap connect modem-manager:mmcli modem-manager:service
24 $ sudo snap connect network-manager:nmcli network-manager:service
25 $ sudo snap connect network-manager:modem-manager modem-manager:service
26 $ sudo snap connect network-manager:ppp ubuntu-core:ppp
27 .
28_verification:
29 Verify that you see the following when running snap interfaces:
30 modem-manager:service modem-manager:mmcli,network-manager:modem-manager
31 network-manager:service network-manager:nmcli
32 :ppp network-manager
33
34id: modem_manager/modems/get_all
35category_id: modem_manager
36plugin: manual
37depends: modem_manager/setup/ensure_interface_connection_setup
38_summary: All active modem configuration items are listed
39_purpose:
40 Check if the modem shows up as being recognized by modem-manager.
41_steps:
42 1. Execute the following command to retrieve the list of all detected modems:
43 .
44 $ sudo mmcli -L
45_verification:
46 Your expected modem is shown in the output list.
47
48id: modem_manager/modems/get_info
49category_id: modem_manager
50plugin: manual
51depends: modem_manager/setup/ensure_interface_connection_setup
52_summary: All modem information listed
53_purpose:
54 Check if the modem details can be listed.
55_steps:
56 1. Execute the following command to retrieve the list of all detected modems.
57 Note, the modem number is listed from the previous test:
58 .
59 $ sudo mmcli -m <modem_number>
60_verification:
61 Your desired modem detail is listed.
62
63id: modem_manager/modems/connect
64category_id: modem_manager
65plugin: manual
66depends: modem_manager/setup/ensure_interface_connection_setup
67_summary: Connect to a listed modem
68_purpose:
69 Connect to a listed modem and verify connection status
70_steps:
71 1. Execute the following commands to connect to a particular modem. First run
72 "sudo nmcli d" to find out what you need to put in <interface> (look for
73 TYPE=gsm, the interface will be the name in the DEVICE column). It will
74 probably be something that starts with "tty" or "cdc". Also, replace
75 <op_apn> by looking up the APN string for your cell provider at
76 http://www.apn-settings.com (e.g. "fast.t-mobile.com" for T-Mobile 4G).
77 Finally, "gsmconn" is an arbitrary connection name and can be set to
78 anything that makes sense for the connection type.
79 .
80 $ sudo nmcli c add type gsm ifname <interface> con-name gsmconn apn <op_apn>
81 $ sudo nmcli r wwan on
82 .
83 2. Check that the connection is up with "sudo nmcli d". It should show that
84 state is "connected". This can take some time depending on network
85 conditions and on the modem type.
86 .
87 3. Check which is the network interface for the cellular connection. For this,
88 fist run
89 .
90 $ sudo nmcli d show <interface>
91 .
92 (here <interface> is the one from point 1), check the IP4.ADDRESS field,
93 then run
94 .
95 $ ip address
96 .
97 and there you can find the network interface that owns that address.
98 .
99 4. Add a route to 8.8.8.8 with
100 .
101 $ sudo ip route add 8.8.8.8 dev <network-interface>
102_verification:
103 Verify that you have a modem connection by pinging 8.8.8.8.
diff --git a/com.canonical.se:engineering-tests/units/network-manager/network-manager.pxu b/com.canonical.se:engineering-tests/units/network-manager/network-manager.pxu
0new file mode 100644104new file mode 100644
index 0000000..27d9b8b
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/network-manager/network-manager.pxu
@@ -0,0 +1,325 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Lorn Potter <lorn.potter@canonical.com>
6# Simon Fels <simon.fels@canonical.com>
7
8unit: category
9id: network_manager
10_name: Network Manager
11
12id: network_manager/misc/check_for_crashes
13category_id: network_manager
14plugin: manual
15_summary: Verify that NetworkManager hasn't crashed in previous runs.
16_steps:
17 1. Power on a Desktop system, this tests does not apply elsewhere.
18 2. List content of /var/crash for crash dumps:
19 .
20 ls -l /var/crash
21 .
22_verification:
23 1. Check that /var/crash is empty.
24
25id: network_manager/ethernet/enable_support
26category_id: network_manager
27plugin: manual
28_summary: Ethernet support for NetworkManager needs to be manually enabled
29_purpose:
30 By default ethernet support is disabled in the network-manager snap. In order
31 to be able to get it managed through NetworkManager a few manual steps need to
32 be performed.
33_steps:
34 1. Create /etc/netplan/00-default-nm-renderer.yaml with the following content:
35 .
36 network:
37 renderer: NetworkManager
38 .
39 2. Drop any other files from /etc/netplan and reboot the device.
40_verification:
41 NOTE: The device may get another IP address assigned as before. Use a network
42 scanner or a monitor+keyboard to check which IP address it got assigned.
43 .
44 Once the device is rebooted you should be able to connect to it by using the
45 network connection offered on the ethernet port. When logged into the device
46 verify NetworkManager automatically connected the ethernet port:
47 .
48 $ network-manager.nmcli d
49 DEVICE TYPE STATE CONNECTION
50 eth0 ethernet connected eth0
51 [...]
52 .
53 The state of all ethernet devices with a plugged cable and a working network
54 behind should be 'connected'.
55
56id: network_manager/ethernet/all_ports_detected
57depends: network_manager/ethernet/enable_support
58category_id: network_manager
59plugin: manual
60_summary: NetworkManager should detect all available ethernet ports
61_purpose:
62 NetworkManager should detect all available ethernet ports by default
63 on a system and list them as usable devices.
64_steps:
65 1. List all available ethernet ports
66 $ ifconfig -a | grep -E '$eth'
67 $ ifconfig -a | grep -E '$en'
68 2. List all ethernet ports NetworkManager knows about
69 $ network-manager.nmcli d
70_verification:
71 Both steps should show the same list of ethernet network interfaces.
72
73id: network_manager/ethernet/auto_connected
74depends: network_manager/ethernet/enable_support
75category_id: network_manager
76plugin: manual
77_summary: NetworkManager should auto-connect any ethernet port with a link
78_purpose:
79 When a cable is plugged into an ethernet port NetworkManager should try
80 to automatically connect the port.
81 .
82 NOTE: This test assumes that the cable you plug into the port is connected
83 to a network which offers a DHCP server.
84_steps:
85 1. Ensure that there is no connection for the port you want to use for
86 testing. Using a serial console to perform this test is recommended.
87 2. Drop any existing connections listed by
88 $ network-manager.nmcli c show
89 which are meant for the ethernet port of the device with
90 $ network-manager.nmcli c delete <UUID>
91 3. Plug the cable into the ethernet port.
92_verification:
93 The ethernet port should be automatically connected by NetworkManager. This
94 can be verified with
95 $ network-manager.nmcli d | grep <ethernet device name>
96 eth0 ethernet connected <connection name>
97 Please note that the <connection name> can be different as its
98 automatically selected by NetworkManager. It can be "Wired Connection 1"
99 for example.
100
101id: network_manager/ethernet/auto_connect_existing_connection
102depends: network_manager/ethernet/enable_support
103category_id: network_manager
104plugin: manual
105_summary: Cable plug/unplug should give an existing connection
106_purpose:
107 When a cable is unplugged and plugged again the ethernet port should
108 be automatically reconnected.
109_steps:
110 1. Check if the ethernet port is currently connected
111 $ network-manager.nmcli d
112 2. Unplug the cable from the ethernet port and check the connection
113 $ network-manager.nmcli d
114 3. Plug the cable again and check the connection status.
115_verification:
116 Before the cable is unplugged the ethernet port should be marked
117 as connected. Once the cable is removed the connection status
118 should switch to unconnected. When plugged back in, it should be marked
119 again as connected.
120
121id: network_manager/ethernet/unmanaged
122category_id: network_manager
123plugin: manual
124depends: network_manager/ethernet/auto_connected
125_summary: Check if ethernet device is unmanaged.
126_purpose:
127 Check if the ethernet device is not managed by Network Manager.
128 .
129 NOTE: This only applies to devices where ethernet is not managed by
130 NetworkManager. Check if /etc/netplan/00-default-nm-renderer.yaml
131 exists. If it does not, then NetworkManager doesn't touch any
132 ethernet connection. If it exists, skip this test.
133_steps:
134 1. Execute the following command to retrieve the list of all detected connections.
135 Note, the AP number is listed from the previous test:
136 .
137 $ /snap/bin/network-manager.nmcli d | grep en
138_verification:
139 The network device is listed as 'unmanaged', as an example:
140 enxb827ebf6b919 ethernet unmanaged --
141
142id: network_manager/connections/get_all
143category_id: network_manager
144plugin: manual
145depends: network_manager/ethernet/auto_connected
146_summary: All active AP configuration items are listed
147_purpose:
148 Check if the AP shows up as being recognized by Network Manager.
149_steps:
150 1. Execute the following command to retrieve the list of all detected connections:
151 .
152 $ /snap/bin/network-manager.nmcli d wifi list
153_verification:
154 Your expected AP is shown in the output list for example:
155 pool1 Infra 6 54 Mbit/s 75 *** WPA2
156
157id: network_manager/connections/agent
158category_id: network_manager
159plugin: manual
160depends: network_manager/ethernet/auto_connected
161_summary: Register for secret agent
162_purpose:
163 Register for Network Manager's secret agent and verify status
164_steps:
165 1. Execute the following command to register secret agent:
166 .
167 $ /snap/bin/network-manager.nmcli a
168_verification:
169 Output should be similiar to this:
170 nmcli successfully registered as a Network Manager's secret agent.
171 nmcli successfully registered as a polkit agent.
172
173id: network_manager/connections/connect
174category_id: network_manager
175plugin: manual
176depends: network_manager/ethernet/auto_connected
177_summary: Connect to a listed AP
178_purpose:
179 Connect to a listed AP and verify connection status
180_steps:
181 1. Execute the following commands to connect to a particular AP.
182 .
183 $ /snap/bin/network-manager.nmcli dev wifi con "yourssid" password "yourpassword"
184 Note, replace yourssid with the ssid if the AP you will connect to, and replace yourpassword
185 with the actual password/passphrase.
186_verification:
187 Verify that you have a AP connection by issuing this command:
188 $ ifconfig | grep -A 2 wl
189 .
190 verifying that the interface has an IP address with output similiar to this:
191 wlp5s0 Link encap:Ethernet HWaddr 70:77:81:33:92:6b
192 inet addr:10.0.0.108 Bcast:10.0.0.255 Mask:255.255.255.0
193 inet6 addr: 2001:8003:6501:2000:c1b3:3359:d2a3:f2d1/64 Scope:Global
194
195id: network_manager/connections/get_info
196category_id: network_manager
197plugin: manual
198depends: network_manager/ethernet/auto_connected
199_summary: All AP information listed
200_purpose:
201 Check if the AP details can be listed.
202_steps:
203 1. Execute the following command to retrieve the list of all detected connections.
204 Note, the AP ssid is listed from the previous test:
205 .
206 $ /snap/bin/network-manager.nmcli -p -f general,ip4,ip6 c show <AP_SSID>
207_verification:
208 Your desired AP detail is listed and has ip4/ip6 address.
209 ===============================================================================
210 Activate connection details (1554e3b2-76e6-4adc-b986-215b7393f8d9)
211 ===============================================================================
212 GENERAL.NAME: pool1
213 GENERAL.UUID: 1554e3b2-76e6-4adc-b986-215b7393f8d9
214 GENERAL.DEVICES: wlp5s0
215 GENERAL.STATE: activated
216 GENERAL.DEFAULT: no
217 GENERAL.DEFAULT6: no
218 GENERAL.VPN: no
219 GENERAL.ZONE: --
220 GENERAL.DBUS-PATH: /org/freedesktop/NetworkManager/ActiveConnection/2
221 GENERAL.CON-PATH: /org/freedesktop/NetworkManager/Settings/3
222 GENERAL.SPEC-OBJECT: /org/freedesktop/NetworkManager/AccessPoint/0
223 GENERAL.MASTER-PATH: --
224 -------------------------------------------------------------------------------
225 IP4.ADDRESS[1]: 10.0.0.108/24
226 IP4.GATEWAY: 10.0.0.138
227 IP4.DNS[1]: 10.0.0.138
228 IP4.DOMAIN[1]: gateway
229 -------------------------------------------------------------------------------
230 IP6.ADDRESS[1]: 2001:8003:6501:2000:c1b3:3359:d2a3:f2d1/64
231 IP6.ADDRESS[2]: 2001:8003:6501:2000:7277:81ff:fe33:926b/64
232 IP6.ADDRESS[3]: fe80::7277:81ff:fe33:926b/64
233 IP6.GATEWAY: fe80::e2b9:e5ff:fe5a:91b0
234 IP6.DNS[1]: 2001:8003:6501:2000::1
235 IP6.DOMAIN[1]: gateway
236 -------------------------------------------------------------------------------
237
238id: network_manager/ethernet/device_supports_wol
239depends: network_manager/ethernet/enable_support
240category_id: network_manager
241plugin: manual
242_summary: Verify that the device under test has support for Wake-on-LAN
243_purpose:
244 Verify that the device you're running the tests on supports Wake-on-LAN. For
245 example, devices like the Raspberry PI (1-3) do not support Wake-on-LAN.
246_steps:
247 Verify that your device supports Wake-on-LAN. You can look into the specs
248 for your device, ask the vendor or search through the internet.
249 .
250 For example the Raspberry PI can't suspend to RAM which is discussed and
251 outlined in https://github.com/raspberrypi/linux/issues/1281 which also
252 makes Wake-on-LAN impossible.
253_verification:
254 If device does not support Wake-on-LAN please SKIP this test case and all
255 depending test cases will be skipped too. If Wake-on-LAN is supported mark
256 test as PASSED.
257
258id: network_manager/ethernet/uses_global_wol
259depends: network_manager/ethernet/device_supports_wol
260category_id: network_manager
261plugin: manual
262_summary: Ethernet connections are using global wake-on-lan setting
263_purpose:
264 Check if an ethernet connection created by NetworkManager automatically uses the
265 global default for wake-on-lan.
266 .
267 NOTE: This requires a system which is configured to not use netplan to
268 generate connections for NetworkManager. On such a system an ethernet
269 connection will always have wake-on-lan disabled as netplan does this
270 by default if not explicitly stated in the netplan configuration.
271_steps:
272 1. Find connection UUID for the ethernet port
273 $ network-manager.nmcli c show | grep eth0
274 2. Verify the connection uses the global wake-on-lan setting
275 $ network-manager.nmcli -m multiline -f 802-3-ethernet.wake-on-lan c show <UUID>
276_verification:
277 The output should look like this:
278 .
279 802-3-ethernet.wake-on-lan: 1 (default)
280
281id: network_manager/ethernet/has_wol_enabled_by_default
282depends: network_manager/ethernet/device_supports_wol
283category_id: network_manager
284plugin: manual
285_summary: Ethernet port has wake-on-lan enabled by default
286_purpose:
287 Check if the actual ethernet device has wake-on-lan correctly enabled when
288 its connection is maintained by NetworkManager.
289_steps:
290 1. Install the se-test-tools snap
291 $ snap install --devmode --edge se-test-tools
292 2. Check the ethernet port for the wake-on-lan setting (eth0 in this case)
293 $ sudo se-test-tools.ethtool eth0
294_verification:
295 The output should be similar to this:
296 Settings for eth0:
297 [...]
298 Supports Wake-on: pumbg
299 Wake-on: g
300 [...]
301 .
302 The relevant line is the "Wake-on:" one. It should have the value "g"
303 set which marks the port as waking up on the magic packet.
304
305id: network_manager/ethernet/wakes_up_with_wol
306depends: network_manager/ethernet/device_supports_wol
307category_id: network_manager
308plugin: manual
309_summary: Wake on LAN configured by NetworkManager can wake up the device
310_purpose:
311 Check that another system can wake up from S5 the SUT using the connected ethernet port.
312_steps:
313 1. Ensure there is an ethernet cable attached to one of the ports and it's configured via
314 NetworkManager. Check with
315 $ network-manager.nmcli d | grep eth0
316 2. Note the MAC address of the the eth0 port
317 $ ifconfig eth0
318 3. Power off the system by running
319 $ sudo poweroff
320 4. From another computer on the same network run the following command:
321 $ wakeonlan {mac}
322 If wakeonlan tool is not installed run:
323 $ sudo apt install wakeonlan
324_verification:
325 Did the SUT wake up from S5?
diff --git a/com.canonical.se:engineering-tests/units/network-manager/power_management.pxu b/com.canonical.se:engineering-tests/units/network-manager/power_management.pxu
0new file mode 100644326new file mode 100644
index 0000000..8ae9b78
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/network-manager/power_management.pxu
@@ -0,0 +1,31 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Simon Fels <simon.fels@canonical.com>
6
7id: network_manager/power_management/connections_come_back_after_resume
8category_id: network_manager
9plugin: manual
10_summary: Connections managed by NetworkManager should come back when device wakes up
11_purpose:
12 When a device goes to sleep NetworkManager disconnects all connections but
13 will reconnect them back when the device reawakens.
14_steps:
15 1. Verify the device is connectd to a WiFi or ethernet network. You can use
16 .
17 $ network-manager.nmcli d
18 .
19 to check if a wireless network interface is connected or not.
20 2. Suspend the device by calling
21 .
22 $ systemctl suspend
23 .
24 3. Wake up the device again by pressing its power button
25_verification:
26 When the device comes back from suspend after pressing the power button
27 verify with
28 .
29 $ network-manager.nmcli d
30 .
31 that the previously connected WiFi/ethernet network is again connected.
diff --git a/com.canonical.se:engineering-tests/units/network-manager/snap.pxu b/com.canonical.se:engineering-tests/units/network-manager/snap.pxu
0new file mode 10064432new file mode 100644
index 0000000..419901a
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/network-manager/snap.pxu
@@ -0,0 +1,101 @@
1# Copyright 2017 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Matteo Croce <matteo.croce@canonical.com>
6
7id: network_manager/snap/check_interfaces
8category_id: network_manager
9plugin: manual
10_summary: Verify that snap interfaces are correctly connected
11_steps:
12 1. Install the snap from the store:
13 .
14 $ snap install network-manager
15 .
16_verification:
17 1. Verify that the 'network-manager' snap has all required plug connected:
18 .
19 $ snap interfaces |grep network-manager
20 :network-setup-observe network-manager
21 :ppp network-manager
22 modem-manager:service modem-manager:mmcli,network-manager:modem-manager
23 network-manager:service network-manager:nmcli
24 .
25
26id: network_manager/wifi/periodic_scanning
27category_id: network_manager
28depends: network_manager/snap/check_interfaces
29plugin: manual
30_summary: Verify that periodic Wi-Fi scanning occurs
31_steps:
32 1. Run wpa_cli to ensure that periodic scanning is occurring:
33 .
34 $ sudo wpa_cli
35 wpa_cli v2.4
36 .
37 Selected interface 'wlp3s0'
38 .
39 Interactive mode
40 .
41 .
42 2. Get the scan results with:
43 .
44 > scan_results
45 .
46_verification:
47 1. You should see some output like this at least once every 120 seconds
48 in the wpa_cli output
49 .
50 <3>CTRL-EVENT-SCAN-STARTED
51 <3>CTRL-EVENT-SCAN-RESULTS
52 .
53 2. You should see the list of reachable Access Points, eg.
54 .
55 bssid / frequency / signal level / flags / ssid
56 12:34:56:78:89:ab 2437 -85 [WPA2-PSK-CCMP+TKIP][ESS] Home-AP
57
58
59id: network_manager/wifi/no_dbus_duplicate_signals
60category_id: network_manager
61depends: network_manager/snap/check_interfaces
62plugin: manual
63_summary: Ensure that dbus signals aren't duplicated
64_steps:
65 1. Run dbus-monitor:
66 .
67 dbus-monitor --system --profile type=signal,interface=org.freedesktop.DBus.Properties
68 .
69_verification:
70 1. Ensure that when scanning occurs, there aren't duplicate PropertiesChanged
71 signals sent for each AccessPoint object path.
72 There should be a single DBus PropertiesChanged signal for each /AccessPoint/X):
73 .
74 sig 1489573828 1805 :1.13 <none> /org/freedesktop/NetworkManager/AccessPoint/0 org.freedesktop.DBus.Properties PropertiesChanged
75 sig 1489573828 1806 :1.13 <none> /org/freedesktop/NetworkManager/AccessPoint/1 org.freedesktop.DBus.Properties PropertiesChanged
76 sig 1489573828 1807 :1.13 <none> /org/freedesktop/NetworkManager/AccessPoint/2 org.freedesktop.DBus.Properties PropertiesChanged
77 sig 1489573828 1808 :1.13 <none> /org/freedesktop/NetworkManager/AccessPoint/3 org.freedesktop.DBus.Properties PropertiesChanged
78 sig 1489573828 1809 :1.13 <none> /org/freedesktop/NetworkManager/AccessPoint/4 org.freedesktop.DBus.Properties PropertiesChanged
79
80
81id: network_manager/wifi/ap_expires
82category_id: network_manager
83depends: network_manager/snap/check_interfaces
84plugin: manual
85_summary: Ensure that AccessPoint entries are volatile
86_steps:
87 1. Bring up an AP (eg. from a smartphone)
88 2. Start wpa_cli and do a wireless scan:
89 .
90 $ wpa_cli
91 > scan
92 .
93 wait 10 seconds, and get the scan results.
94 .
95 > scan_results
96 .
97 3. Shutdown the AP on the smartphone
98 4. Wait 120 seconds
99 5. Retry the wireless scan
100_verification:
101 1. In the second scan the smartphone AP should not be present
diff --git a/com.canonical.se:engineering-tests/units/network-manager/wireless.pxu b/com.canonical.se:engineering-tests/units/network-manager/wireless.pxu
0new file mode 100644102new file mode 100644
index 0000000..52a84d2
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/network-manager/wireless.pxu
@@ -0,0 +1,54 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Simon Fels <simon.fels@canonical.com>
6
7id: network_manager/wireless/powersave_off_by_default
8category_id: network_manager
9plugin: manual
10_summary: NetworkManager should disable powersave by default
11_purpose:
12 The powersave configuration option should cause the mode on the
13 actual hardware device to be turned off by default.
14_steps:
15 Install wireless-tools snap and check that powersave is turned off
16 on the actual hardware wireless device.
17 $ snap install wireless-tools
18_verification:
19 The iw utility should return the following for the selected wireless device
20 $ sudo wireless-tools.iw dev wlan0 get power_save
21 Power save: off
22
23id: network_manager/wireless/powersave_config_default_off
24category_id: network_manager
25plugin: manual
26_summary:
27 The configuration option should be set to disable by default
28_purpose:
29 On installation the snap sets the option to a default value
30 which should always be the same.
31_steps:
32 Retrieve the configuration value via
33 $ snap get network-manager wifi.powersave
34_verification:
35 The returned value should be "disable".
36
37id: network_manager/wireless/powersave_can_be_enabled
38category_id: network_manager
39plugin: manual
40_summary:
41 Enabling powersave with the configuration option of the snap should turn it on.
42_purpose:
43 The network-manager snap uses a configure hook to control how
44 the WiFi powersave option is set. It should be possible to enable
45 the option on the actual hardware wireless device with it.
46_steps:
47 1. Enable powersave via the configure hook
48 $ snap set network-manager wifi.powersave=enable
49_verification:
50 Powersave should now be turned on for all wireless devices. This
51 can be verified with
52 $ snap install wireless-tools
53 $ sudo wireless-tools.iw dev wlan0 get power_save
54 Power save: on
diff --git a/com.canonical.se:engineering-tests/units/pulseaudio/pulseaudio.pxu b/com.canonical.se:engineering-tests/units/pulseaudio/pulseaudio.pxu
0new file mode 10064455new file mode 100644
index 0000000..43af155
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/pulseaudio/pulseaudio.pxu
@@ -0,0 +1,106 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Konrad Zapalowicz <konrad.zapalowicz@canonical.com>
6
7unit: category
8id: pulseaudio
9_name: PulseAudio
10
11id: pulseaudio/setup/ensure_interface_connection_setup
12category_id: pulseaudio
13_description: Ensure that the pulseaudio interface is auto-connected
14plugin: manual
15_steps:
16 1. List the interfaces
17 .
18 $ snap interfaces
19 .
20_verification:
21 Verify that you see the following:
22 pulseaudio:service pulseaudio:client
23 :network pulseaudio
24
25id: pulseaudio/pactl/ensure-commands-can-be-run-by-root
26category_id: pulseaudio
27_description: Ensure that the pulseaudio commands can be run only by root
28plugin: manual
29depends: pulseaudio/setup/ensure_interface_connection_setup
30_steps:
31 1. Try to run pactl
32 .
33 $ pulseaudio.pactl
34 .
35 2. Try to run paplay
36 .
37 $ pulseaudio.paplay
38 .
39 3. Try to run parec
40 .
41 $ pulseaudio.parec
42 .
43_verification:
44 Verify that each step ends with the following message:
45 This script must be run as root
46
47id: pulseaudio/pactl/ensure-modules-are-listed
48category_id: pulseaudio
49_description: Ensure that the correct modules are loaded
50plugin: manual
51depends: pulseaudio/setup/ensure_interface_connection_setup
52_steps:
53 1. List the loaded modules
54 .
55 $ sudo pulseaudio.pactl list short
56 .
57_verification:
58 Verify that you see output similar to following:
59 0 module-device-restore
60 1 module-stream-restore
61 2 module-card-restore
62 3 module-augment-properties
63 4 module-switch-on-port-available
64 5 module-udev-detect
65 ...
66 80 protocol-native.c pactl
67 0 alsa_card.pci-0000_00_1b.0 module-alsa-card.c
68 .
69 Verify that amongs the other the following modules are listed:
70 module-device-restore
71 module-stream-restore
72 module-card-restore
73 module-switch-on-port-available
74 module-rescue-streams
75 module-always-sink
76
77id: pulseaudio/parecord/ensure-audio-can-be-recorded
78category_id: pulseaudio
79_description: Ensure that it is possible to record audio
80plugin: manual
81depends: pulseaudio/setup/ensure_interface_connection_setup
82_steps:
83 1. Record an audio .wav file
84 .
85 $ sudo pulseaudio.parec ~/record.wav
86 .
87_verification:
88 Verify that the record.wav file has been created and it's size is
89 greater than zero. Verify that the recorded file can be played and
90 it is possible to hear the audio.
91
92id: pulseaudio/paplay/ensure-audio-can-be-played
93category_id: pulseaudio
94_description: Ensure that it is possible to play auio files
95plugin: manual
96depends: pulseaudio/setup/ensure_interface_connection_setup
97_steps:
98 1. Play an audio .wav file
99 .
100 $ sudo pulseaudio.paplay ~/file.wav
101 .
102_verification:
103 Verify that the audio is hearable through the speakers. In case the
104 speakers are not available verify that the sink status has changed
105 from SUSPEND to RUNNING [use the sudo pulseaudio.pactl list sinks
106 command for that]
diff --git a/com.canonical.se:engineering-tests/units/snapweb/snapweb.pxu b/com.canonical.se:engineering-tests/units/snapweb/snapweb.pxu
0new file mode 100644107new file mode 100644
index 0000000..8f9e070
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/snapweb/snapweb.pxu
@@ -0,0 +1,182 @@
1# Copyright 2017 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Roberto Mier Escandon <roberto.escandon@canonical.com>
6
7unit: category
8id: snapweb
9_name: SnapWeb
10
11id: snapweb/setup/ensure_interface_connection_setup
12category_id: snapweb
13_description: Ensure that the snapweb interfaces are auto-connected
14plugin: manual
15_steps:
16 1. List the interfaces
17 .
18 $ snap interfaces snapweb
19 .
20_verification:
21 Verify that you see the following:
22 Slot Plug
23 :network snapweb
24 :network-bind snapweb
25 :snapd-control snapweb
26 :timeserver-control snapweb
27
28id: snapweb/setup/ensure-http-service-is-up
29category_id: snapweb
30_description: Ensure that the http interface is up and running
31plugin: manual
32depends: snapweb/setup/ensure_interface_connection_setup
33_steps:
34 1. Open a browser in same device and type in address bar:
35 .
36 http://localhost:4200
37 .
38_verification:
39 Verify that url is accessible and a warning is shown about web access is
40 not secure/private (depending on the browser)
41
42id: snapweb/setup/ensure-https-service-is-up
43category_id: snapweb
44_description: Ensure that the snapweb web interface is up and running
45plugin: manual
46depends: snapweb/setup/ensure-http-service-is-up
47_steps:
48 1. Open a browser in same device and type in address bar:
49 .
50 http://localhost:4200
51 .
52 2. After warning alert showing that web access is not secure/private,
53 proceed to access localhost. In some browsers it is needed to confirm
54 a security exception.
55_verification:
56 Verify that url is accessible and browser shows a page with title
57 'Access Control', requesting a Snapweb Access Token to continue.
58 Verify that url address is now https://localhost:4201/access-control
59
60id: snapweb/setup/ensure-web-service-is-accessible-from-lan
61category_id: snapweb
62_description: Ensure that the snapweb web interface is accessible from lan
63plugin: manual
64depends: snapweb/setup/ensure-https-service-is-up
65_steps:
66 1. Open a browser in other device connected to same LAN than the one
67 hosting snapweb and type in address bar:
68 .
69 http://[hostname]:4200
70 .
71 2. After warning alert showing that web access is not secure/private,
72 proceed to access localhost. In some browsers it is needed to confirm
73 a security exception.
74_verification:
75 Verify that url is accessible and browser shows a page with title
76 'Access Control', requesting a Snapweb Access Token to continue.
77 Verify that url address is now https://[hostname]:4201/access-control
78
79id: snapweb/setup/ensure-token-access
80category_id: snapweb
81_description: Ensure ui allows access when entered a valid token generated from cli
82plugin: manual
83depends: snapweb/setup/ensure-https-service-is-up
84_steps:
85 1. Reboot the device. This step is only needed to be done just after
86 first installation. If snap was previously installed and this is
87 an update, you can go to step 2.
88 2. Open a terminal in the device and type:
89 .
90 $ sudo snapweb.generate-token
91 .
92 3. Copy result token to clipboard
93 4. Access browser and type:
94 .
95 https://localhost:4201
96 .
97 5. Paste clipboard token into Snapweb Access Token textbox
98_verification:
99 Verify that access is allowed and installed apps list is shown
100
101id: snapweb/snaps/installed-snaps
102category_id: snapweb
103_description: Verify installed snaps list
104plugin: manual
105depends: snapweb/setup/ensure-token-access
106_steps:
107 1. Access browser and type:
108 .
109 https://localhost:4201
110 .
111 2. In a device terminal type:
112 .
113 $ snap list
114 .
115_verification:
116 Verify that the list of 'Apps installed' in browser is the same
117 as the list shown in terminal, except snapweb itself that should
118 be only shown in terminal list
119
120id: snapweb/snaps/installable-snaps
121category_id: snapweb
122_description: Verify that snaps can be installed and filtered
123plugin: manual
124depends: snapweb/setup/ensure-token-access
125_steps:
126 1. Access browser and type:
127 .
128 https://localhost:4201
129 .
130 2. Click on last entry on Apps installed section named 'Get more apps'
131 or click on 'Store' upper tab
132 3. Click on the links over snaps to filter them
133 4. Search for certain snap name in upper search box
134_verification:
135 Verify that at the beginning there is a list of snaps that can be installed,
136 everyone with its icon, version, author and a button with label 'Install'.
137 Verify that the list can be filtered by pressing links over
138 Verify you get results when searching for a snap that it's in the list of
139 installables
140
141id: snapweb/snaps/install-snap
142category_id: snapweb
143_description: Ensure that a snap can be installed using web ui
144plugin: manual
145depends: snapweb/setup/ensure-token-access
146_steps:
147 1. Access browser and type:
148 .
149 https://localhost:4201
150 .
151 2. Click on 'Install' button of any available snap
152 3. Wait for snap to be installed and open a terminal
153 4. Type in terminal:
154 .
155 $ snap list
156 .
157_verification:
158 Verify that the snap requested to be installed is there in the terminal resultant list
159 Verify that button in step 2 changes its label to 'Installing' while installing
160 Verify that button of the instaled snap has now 'Remove' in its label
161
162id: snapweb/snaps/remove-snap
163category_id: snapweb
164_description: Ensure that a snap can be removed using web ui
165plugin: manual
166depends: snapweb/snaps/install-snap
167_steps:
168 1. Access browser and type:
169 .
170 https://localhost:4201
171 .
172 2. Click on 'Remove' button of installed snap
173 3. Wait for snap to be removed and open a terminal
174 4. Type in terminal:
175 .
176 $ snap list
177 .
178_verification:
179 Verify that the snap requested to be removed is not in the terminal resultant list
180 Verify that button in step 2 changes its label to 'Removing' while installing
181 Verify that button of the removed snap has now 'Install' in its label
182
diff --git a/com.canonical.se:engineering-tests/units/test-plan.pxu b/com.canonical.se:engineering-tests/units/test-plan.pxu
0new file mode 100644183new file mode 100644
index 0000000..a7f31d1
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/test-plan.pxu
@@ -0,0 +1,9 @@
1id: engineering-tests
2unit: test plan
3_name: System Enablement Engineering Tests
4_description:
5 Set of tests to verify functionality of the snaps produced by the
6 System Enablement team at Canonical.
7estimated_duration: 2h
8include:
9 2017.com.canonical.se::.*
diff --git a/com.canonical.se:engineering-tests/units/tpm/tpm.pxu b/com.canonical.se:engineering-tests/units/tpm/tpm.pxu
0new file mode 10064410new file mode 100644
index 0000000..b00bd89
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/tpm/tpm.pxu
@@ -0,0 +1,135 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Scott Sweeny <scott.sweeny@canonical.com>
6
7unit: category
8id: tpm1.2
9_name: TPM 1.2
10
11id: tpm1.2/setup/ensure_interface_connection_setup
12category_id: tpm1.2
13_description: Ensure that the tpm1.2 interfaces are connected
14plugin: manual
15_steps:
16 1. Connect required tpm1.2 slots/plugs
17 .
18 $ sudo snap connect tpm:network :network
19 $ sudo snap connect tpm:network-bind :network-bind
20 $ sudo snap connect tpm:tpm :tpm
21 .
22_verification:
23 Verify that you see the following when running snap interfaces:
24 :network tpm
25 :network-bind tpm
26 :tpm tpm
27
28id: tpm1.2/setup/get_version
29category_id: tpm1.2
30plugin: manual
31depends: tpm1.2/setup/ensure_interface_connection_setup
32_summary: Can get version information from the TPM chip
33_purpose:
34 Check that basic communication with the TPM chip is working and that version
35 information can be retrieved with the expected command.
36_steps:
37 1. Execute the following command to show the TPM version:
38 .
39 $ /snap/bin/tpm.version
40_verification:
41 Output is similar to below with all fields showing data:
42 .
43 TPM 1.2 Version Info:
44 Chip Version: 1.2.5.81
45 Spec Level: 2
46 Errata Revision: 3
47 TPM Vendor ID: WEC
48 Vendor Specific data: 0000
49 TPM Version: 01010000
50 Manufacturer Info: 57454300
51
52id: tpm1.2/setup/self_test
53category_id: tpm1.2
54plugin: manual
55depends: tpm1.2/setup/ensure_interface_connection_setup
56_summary: TPM self test runs successfully
57_purpose:
58 Check that the TPM self test runs successfully and returns no errors.
59_steps:
60 1. Execute the following command to run the TPM self test:
61 .
62 $ /snap/bin/tpm.selftest
63_verification:
64 Output is similar to below (actual values are not important) and no
65 error messages are shown:
66 .
67 TPM Test Results:
68 ff010205 51ffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
69 ffffffff ffffffff ffffffff ffff0000 00000000 00000000 00000000 00000000
70 00000000 00000000 00000000 00000000 00000000 00000000 00000000 0000
71
72
73id: tpm1.2/configuration/take_ownership
74category_id: tpm1.2
75plugin: manual
76depends: tpm1.2/setup/ensure_interface_connection_setup
77_summary: Taking ownership of the device is possible
78_purpose:
79 Check that you are able to take ownership of the device using TPM commands.
80_steps:
81 1.Execute the following command to take ownership of the TPM chip:
82 .
83 $ /snap/bin/tpm.takeownership
84 .
85_verification:
86 Verify that you are prompted for both the owner key and the SRK (storage
87 root key) and that no errors are returned.
88 .
89 NOTE: This command will only work if it has not been run before. You can
90 reset the TPM with:
91 .
92 $ /snap/bin/tpm.clear
93 .
94 Which will invalidate all existing keys and passwords and disable the chip.
95 Once this is done you must re-enable the TPM from the device's BIOS before
96 it can be used.
97
98id: tpm1.2/crypto/show_public_key
99category_id: tpm1.2
100plugin: manual
101depends: tpm1.2/setup/ensure_interface_connection_setup
102_summary: Showing the public endorsement key succeeds
103_purpose:
104 Check to make sure that the endorsement key pair exists and the public key
105 can be queried.
106_steps:
107 1. Execute the following command to see the public key:
108 .
109 $ /snap/bin/tpm.getpubek
110 .
111 You may see an error similar to:
112 .
113 Tspi_TPM_GetPubEndorsementKey failed: 0x00000008 - layer=tpm, code=0008 (8), The TPM target command has been disabled
114 .
115 This can be safely ignored as long as the public key data is displayed.
116_verification:
117 Verify that the public key data is shown, for example:
118 .
119 Public Endorsement Key:
120 Version: 01010000
121 Usage: 0x0002 (Unknown)
122 Flags: 0x00000000 (!VOLATILE, !MIGRATABLE, !REDIRECTION)
123 AuthUsage: 0x00 (Never)
124 Algorithm: 0x00000020 (Unknown)
125 Encryption Scheme: 0x00000012 (Unknown)
126 Signature Scheme: 0x00000010 (Unknown)
127 Public Key:
128 8b59fad6 83514128 d56e2aa0 5eef7cff ad23c90d 0dd9b1e2 31ef093d 72d947d8
129 e02fbe3a 3c235ff0 9c487973 40a28c9b 8b83aaaf dc1254ad 31a95410 811742e5
130 92c16823 6f663413 f7067d3a 9af09ccc 12944a13 51d32980 59a98740 4008ab0c
131 07b7d93b 9393691c 8ae7d84c 5fd4fcef 147934f1 09326225 d5d1b652 ab3e12ee
132 fbd1cfba b18abedc de2215cb 97006b92 839fd9a4 3158bc8f 028fe516 550905b3
133 e3eef1e1 9f8dd3ab 331cbe62 d32ce937 0c11ad41 1d2e50f3 c69c7cde cd0bf564
134 4b7b16f8 0885dc66 d4411aba a824c1f1 ea8aa743 5c62fad7 e9540b85 5c0dc7af
135 3c1615ae 56af18e4 770aa107 06ef274b 202e2e3e d4c280d8 5ca9e886 9d889e31
diff --git a/com.canonical.se:engineering-tests/units/udisks2/udisks2.pxu b/com.canonical.se:engineering-tests/units/udisks2/udisks2.pxu
0new file mode 100644136new file mode 100644
index 0000000..7667aa5
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/udisks2/udisks2.pxu
@@ -0,0 +1,319 @@
1# Copyright 2017 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Scott Sweeny <scott.sweeny@canonical.com>
6
7unit: category
8id: udisks2
9_name: UDisks2
10
11id: udisks2/setup/ensure_interface_connection_setup
12category_id: udisks2
13_description: Ensure that the udisks2 interfaces are connected
14plugin: manual
15_steps:
16 1. Run the following:
17 $ sudo snap interfaces udisks2
18 .
19_verification:
20 Verify that you see the following:
21 Slot Plug
22 :mount-observe udisks2
23 :network-bind udisks2
24 udisks2:service udisks2:client
25 - udisks2:hardware-observe
26
27id: udisks2/client/can_communicate_with_udisksd
28category_id: udisks2
29plugin: manual
30depends: udisks2/setup/ensure_interface_connection_setup
31_summary: udisksctl can communicate with udisksd
32_purpose:
33 Makes sure udisksctl can communicate with udisksd
34_steps:
35 1. Run a simple udisksctl command
36 .
37 $ sudo udisks2.udisksctl status
38 .
39_verification:
40 Verify that the command returns a list of connected storage devices and no
41 errors.
42
43id: udisks2/service/can_receive_hotplug_events
44category_id: udisks2
45plugin:manual
46depends: udisks2/client/can_communicate_with_udisksd
47_summary: udisks can see when a drive is plugged in
48_purpose:
49 Makes sure that udisks gets the hotplug event when a USB drive is
50 plugged in
51_steps:
52 1. Start the udisks client and have it watch for new events
53 .
54 $ sudo udisks2.udisksctl monitor
55 .
56 2. Plug in a USB storage device and wait for activity from the client
57 .
58 3. Unplug the USB storage device and wait for activity from the client
59 .
60_verification:
61 Verify that udisks shows information about the USB device when it's plugged
62 in, and that it recognizes when the device is removed.
63 .
64 The information shown when the device is plugged in will be everything
65 udisks knows about the device and its partitions, expressed as DBus objects.
66 You will likely see something like:
67 .
68 21:37:20.621: Added /org/freedesktop/UDisks2/drives/Kingston_DataTraveler_2_2e0_0607151625382
69 org.freedesktop.UDisks2.Drive:
70 CanPowerOff: true
71 Configuration: {}
72 ConnectionBus: usb
73 Ejectable: true
74 Id: Kingston-DataTraveler-2.0-0607151625382
75 Media: thumb
76 MediaAvailable: true
77 MediaChangeDetected: true
78 .
79 and so on for the drive itself, each block device, and each filesystem
80
81id: udisks2/service/can_mount_unmount
82category_id: udisks2
83plugin: manual
84depends: udisks2/client/can_communicate_with_udisksd
85_summary: udisks can mount and unmount a storage device
86_purpose:
87 Makes sure that udisks can mount and unmount a storage device, and that
88 the mount points are visible outside of the snap
89_steps:
90 1. Plug in a USB storage device
91 2. Run the following:
92 .
93 $ sudo udisks2.udisksctl status
94 .
95 and note the name under the "DEVICE" column for the drive you just
96 plugged in. It should be "sdb" or similar
97 3. Mount the filesystem on the device:
98 .
99 $ sudo udisks2.udisksctl mount -b /dev/<device>1
100 .
101 Where <device> is the value from the status call above. This should mount
102 the first filesystem on the device and return the path
103 4. Examine the mount point:
104 .
105 $ sudo -s # You must be root to see the mount point
106 $ ls /mountpoint/path/returned/above
107 .
108 Make sure the contents of that directory match what you expect to see on
109 the drive.
110 .
111 Leave the root shell
112 $ exit
113 5. Unmount the filesystem:
114 .
115 $ sudo udisks2.udisksctl unmount -b /dev/<device>1
116_verification:
117 Verify that mounts and unmounts are performed successfully, and that the
118 contents of the drive can be seen from your shell (i.e. outside of the snap's
119 own namespace)
120
121id: udisks2/automount/default_off
122category_id: udisks2
123plugin: manual
124depends: udisks2/setup/ensure_interface_connection_setup
125_summary: The automount option defaults to off
126_purpose:
127 Makes sure that the automount option defaults to disabled
128_steps:
129 1. Install the udisks2 snap fresh (i.e. not an upgrade)
130 2. Run the following:
131 .
132 $ sudo snap get udisks2 automount.enable
133_verification:
134 Verify that the returned value is "false"
135
136id: udisks2/automount/disabling_works
137category_id: udisks2
138plugin: manual
139depends: udisks2/automount/default_off
140_summary: Disabling automount actually disables automount
141_purpose:
142 Makes sure that when automount.enabled is set to "false" automounts are
143 prevented
144_steps:
145 1. Make sure that automount is disabled. Run:
146 .
147 $ sudo snap set udisks2 automount.enable=false
148 2. Plug in a USB storage device
149_verification:
150 Verify that no filesystem on the USB device was mounted. /media/root/ should
151 be empty
152
153id: udisks2/automount/automount_works
154category_id: udisks2
155plugin: manual
156depends: udisks2/automount/default_off
157_summary: When enabled, automounting works
158_purpose:
159 Makes sure that automounting a USB drive works when the option is enabled
160_steps:
161 1. Make sure that automount is enabled. Run:
162 .
163 $ sudo snap set udisks2 automount.enable=true
164 2. Plug in a USB storage drive
165_verification:
166 Verify that the filesystem on the USB device was mounted to
167 /media/root/<filesystem label>, for example /media/root/7C69-3855
168
169id: udisks2/automount/automount_does_not_mount_system_devices
170category_id: udisks2
171plugin: manual
172depends: udisks2/automount/automount_works
173_summary: When enabled, automount does not mount system devices
174_purpose:
175 Makes sure that automount is only restricted to pluggable devices, not system
176_steps:
177 1. Make sure that automount is enabled. Run:
178 .
179 $ sudo snap set udisks2 automount.enable=true
180 3. Reboot the device and wait for it to power on again
181 4. Run the following:
182 .
183 $ sudo udisks2.udisksctl status
184 .
185 and check the names under the "DEVICE" column. It should just contain
186 thumb drives [if any is plugged-in] and the filesystem. For example,
187 in case of RaspberryPi it would contain just: mmcblk0.
188 Make sure that system devices, such as an ESP partition, are
189 not displayed here.
190 5. Examine the mount point:
191 .
192 $ sudo -s # You must be root to see the mount point
193 $ ls /media/root
194 .
195 Make sure the contents of that directory match what you expect to see on
196 the drive.
197 .
198 Leave the root shell
199 $ exit
200_verification:
201 Verify that none of the system devices were mounted and their filesystems
202 are not available /media/root/<filesystem label>
203
204id: udisks2/automount/automount_does_mount_usb_devices_plugged_before_boot
205category_id: udisks2
206plugin: manual
207depends: udisks2/automount/automount_works
208_summary: When enabled, automount mount USB devices plugged before boot
209_purpose:
210 Makes sure that system can mount devices plugged in before boot
211_steps:
212 1. Make sure that automount is enabled. Run:
213 .
214 $ sudo snap set udisks2 automount.enable=true
215 2. Switch the device off, insert USB flash drive and boot the device again
216 3. Run the following:
217 .
218 $ sudo udisks2.udisksctl status
219 .
220 and check the names under the "DEVICE" column. It should contain the device
221 that has been plugged before booting.
222 4. Examine the mount point:
223 .
224 $ sudo -s # You must be root to see the mount point
225 $ ls /media/root
226 .
227 Make sure the contents of that directory match what you expect to see on
228 the drive.
229 .
230 Leave the root shell
231 $ exit
232_verification:
233 Verify that the cold-plugged device is mounted on boot
234
235id: udisks2/automount/automount_works_after_manual_unmount
236category_id: udisks2
237plugin: manual
238depends: udisks2/automount/automount_works
239_summary: When enabled, automount still works after unmount has been called manually
240_purpose:
241 Make sure that the system can auto mount devices after the unmount has been called
242_steps:
243 1. Make sure that automount is enabled. Run:
244 .
245 $ sudo snap set udisks2 automount.enable=true
246 2. Insert USB flash drive
247 3. Examine the mount point:
248 .
249 $ sudo ls /media/root
250 .
251 Make sure the USB drive that just have been plugged in is listed there.
252 4. Run the following:
253 .
254 $ sudo udisks2.udisksctl status
255 .
256 and check the names under the "DEVICE" column. It should contain the device
257 that has been plugged before booting.
258 5. Unmount manually:
259 .
260 $ sudo udisks2.udisksctl unmount -b /dev/<device>1
261 6. Insert another USB flash drive
262 7. Examine the mount point:
263 .
264 $ sudo ls /media/root
265 .
266 Make sure the USB drive that just have been plugged in is listed there.
267_verification:
268 Verify that the USB drive that has been plugged in after the first one has
269 been unmounted manually is mounted.
270
271id: udisks2/debug/default_off
272category_id: udisks2
273plugin: manual
274depends: udisks2/setup/ensure_interface_connection_setup
275_summary: The debug option defaults to off
276_purpose:
277 Makes sure that the debug option defaults to disabled
278_steps:
279 1. Install the udisks2 snap fresh (i.e. not an upgrade)
280 2. Run the following:
281 .
282 $ sudo snap get udisks2 debug.enable
283_verification:
284 Verify that the returned value is "false"
285
286id: udisks2/debug/disabling_works
287category_id: udisks2
288plugin: manual
289depends: udisks2/debug/default_off
290_summary: Disabling debug actually disables debug logging
291_purpose:
292 Makes sure that when debug.enabled is set to "false" no loggs are written
293_steps:
294 1. Make sure that debug is disabled. Run:
295 .
296 $ sudo snap set udisks2 debug.enable=false
297 2. Plug in a USB storage device
298_verification:
299 Verify that no debug information coming from files such as jobs.go,
300 dispatcher.go, udisks2.go is written to the journal.
301
302id: udisks2/automount/debug_works
303category_id: udisks2
304plugin: manual
305depends: udisks2/debug/default_off
306_summary: When debug is enabled logs are available
307_purpose:
308 Makes sure that when debug option is enabled the logs are available
309_steps:
310 1. Make sure that debug is enabled. Run:
311 .
312 $ sudo snap set udisks2 debug.enable=true
313 2. Restart ciborium service for the above change to take effect
314 .
315 $ sudo systemctl restart snap.udisks2.ciborium.service
316 3. Plug in a USB storage drive
317_verification:
318 Verify that debug information coming from udisks2 snap (files such as
319 jobs.go, dispatcher.go, udisks2.go) is written to the journal.
diff --git a/com.canonical.se:engineering-tests/units/upower/00_general.pxu b/com.canonical.se:engineering-tests/units/upower/00_general.pxu
0new file mode 100644320new file mode 100644
index 0000000..3ab4d2d
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/upower/00_general.pxu
@@ -0,0 +1,52 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Simon Fels <simon.fels@canonical.com>
6
7unit: category
8id: upower
9_name: UPower
10
11id: upower/general/can_be_installed
12category_id: upower
13_description: Ensure that the upower snap can be installed
14plugin: manual
15_steps:
16 Install upower snap from the appropiate channel via the following command
17 .
18 $ snap install --candidate upower
19 .
20 NOTE: Depending on which channel you want to test you need to select the
21 right one. The command above uses the candidate channel as that is the
22 most used one.
23_verification:
24 The installation of the snap should finish with no errors.
25
26id: upower/general/service_started
27category_id: upower
28_description: Ensure that the upowerd service is up and running
29plugin: manual
30_steps:
31 Check service status with the following command
32 $ systemctl status snap.upower.service.service
33_verification:
34 Output should look like this:
35 snap.upower.upowerd.service - Service for snap application upower.upowerd
36 Loaded: loaded (/etc/systemd/system/snap.upower.upowerd.service; enabled; vendor preset: enabled)
37 Active: active (running) since Thu 2016-12-15 16:13:37 UTC; 54ms ago
38 Main PID: 4265 (upowerd)
39 CGroup: /system.slice/snap.upower.upowerd.service
40 4265 /snap/upower/x1/usr/libexec/upowerd
41
42id: upower/general/client_can_talk_to_service
43category_id: upower
44_description: Verify that the upower client can talk to the service
45plugin: manual
46_steps:
47 Enumerate available devices available on the system:
48 $ upower -e
49_verification:
50 Output should look like this and at least one device should be present:
51 /org/freedesktop/UPower/devices/battery_BAT0
52 /org/freedesktop/UPower/devices/DisplayDevice
diff --git a/com.canonical.se:engineering-tests/units/upower/10_battery_powered.pxu b/com.canonical.se:engineering-tests/units/upower/10_battery_powered.pxu
0new file mode 10064453new file mode 100644
index 0000000..d5fe6b1
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/upower/10_battery_powered.pxu
@@ -0,0 +1,129 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Simon Fels <simon.fels@canonical.com>
6
7id: upower/battery_powered/battery_detected
8category_id: upower
9_description: Verify upower is able to detect the battery of the system
10plugin: manual
11_steps:
12 Query upowerd for available power devices on the system
13 $ upower -d
14_verification:
15 The output should be similar to the following and should list the
16 battery of your system:
17 .
18 Device: /org/freedesktop/UPower/devices/line_power_AC
19 native-path: AC
20 power supply: yes
21 updated: Do 15 Dez 2016 15:06:55 CET (7163 seconds ago)
22 has history: no
23 has statistics: no
24 line-power
25 warning-level: none
26 online: yes
27 icon-name: 'ac-adapter-symbolic'
28 .
29 Device: /org/freedesktop/UPower/devices/battery_BAT0
30 native-path: BAT0
31 vendor: SANYO
32 model: 45N1779
33 serial: 115
34 power supply: yes
35 updated: Do 15 Dez 2016 17:05:18 CET (60 seconds ago)
36 has history: yes
37 has statistics: yes
38 battery
39 present: yes
40 rechargeable: yes
41 state: fully-charged
42 warning-level: none
43 energy: 97,05 Wh
44 energy-empty: 0 Wh
45 energy-full: 97,13 Wh
46 energy-full-design: 99,47 Wh
47 energy-rate: 7,983 W
48 voltage: 12,374 V
49 percentage: 99%
50 capacity: 97,6475%
51 technology: lithium-ion
52 icon-name: 'battery-full-charged-symbolic'
53 .
54 Device: /org/freedesktop/UPower/devices/DisplayDevice
55 power supply: yes
56 updated: Do 15 Dez 2016 16:41:14 CET (1504 seconds ago)
57 has history: no
58 has statistics: no
59 battery
60 present: yes
61 state: fully-charged
62 warning-level: none
63 energy: 97,05 Wh
64 energy-full: 97,13 Wh
65 energy-rate: 7,983 W
66 percentage: 99%
67 icon-name: 'battery-full-charged-symbolic'
68 .
69 Daemon:
70 daemon-version: 0.99.4
71 on-battery: no
72 lid-is-closed: no
73 lid-is-present: yes
74 critical-action: PowerOff
75
76id: upower/battery_powered/battery_discharging
77category_id: upower
78_description: Verify upower reports discharging battery
79plugin: manual
80_steps:
81 1. Unplug the power cable from your battery powered device
82 2. Listen to reported changes from upower
83 $ upower -m
84_verification:
85 Leave the monitor running for some time. It should report the device changes
86 for the battery device:
87 .
88 [17:22:55.277]device changed: /org/freedesktop/UPowerr/devices/battery_BAT0
89 [17:22:55.277]device changed: /org/freedesktop/UPower/devices/battery_BAT0
90 .
91 Verify with the following command that the actual battery capacity is changing
92 .
93 $ upower -i /org/freedesktop/UPower/devices/battery_BAT0 | capacity
94 capacity: 97,6475%
95 .
96 and that the state of the battery device is changed to 'discharging'
97 .
98 $ upower -i /org/freedesktop/UPower/devices/battery_BAT0 | grep state
99 state: discharging
100
101id: upower/battery_powered/low_battery_causes_poweroff
102category_id: upower
103_description: Verify upower reports discharging battery
104plugin: manual
105_steps:
106 A battery with a low capacity should cause the device to power off. This
107 involves as a first step to wait until the battery of the device under test
108 is close to being empty. The default trigger of the poweroff action is by
109 default at <= 2%
110 .
111 Depending on how full your device battery is you have to wait some time
112 until it is discharged enough.
113 .
114 Watch the system via
115 .
116 $ sudo journalctl --no-pager -f
117_verification:
118 Once the defined threshold of the battery capacity is reached the system
119 should power off itself. You can spot this by looking at the output
120 of the journalctl command which should print something like:
121 .
122 Dec 15 16:37:02 localhost.localdomain systemd-logind[1022]: System is powering down
123 [...]
124 Dec 15 16:37:02 localhost.localdomain systemd[1]: Starting Power-Off...
125 Dec 15 16:37:02 localhost.localdomain systemd[1]: Starting Update UTMP about System Runlevel Changes...
126 Dec 15 16:37:02 localhost.localdomain systemd[1]: Started Power-Off.
127 Dec 15 16:37:02 localhost.localdomain systemd[1]: Started Update UTMP about System Runlevel Changes.
128 Dec 15 16:37:02 localhost.localdomain systemd[1]: Reached target Power-Off.
129 Dec 15 16:37:02 localhost.localdomain systemd-logind[1022]: Operation 'shutdown' finished.
diff --git a/com.canonical.se:engineering-tests/units/upower/20_ac_powered.pxu b/com.canonical.se:engineering-tests/units/upower/20_ac_powered.pxu
0new file mode 100644130new file mode 100644
index 0000000..169f0e3
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/upower/20_ac_powered.pxu
@@ -0,0 +1,49 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Simon Fels <simon.fels@canonical.com>
6
7id: upower/ac_powered/source_detected
8category_id: upower
9_description: Verify upower is able to detect the AC power source
10plugin: manual
11_steps:
12 Query upowerd for available power devices on the system
13 .
14 $ upower -e
15_verification:
16 The AC power source should be listed in the result, e.g.:
17 .
18 /org/freedesktop/UPower/devices/line_power_AC
19
20id: upower/ac_powered/status_plugged
21category_id: upower
22_description: Verify upower report AC source as plugged
23plugin: manual
24_steps:
25 Plug in the AC adapter to your device and query upower with
26 .
27 $ upower -i /org/freedesktop/UPower/devices/line_power_AC | grep online
28 for the online status of the source.
29_verification:
30 The output should be
31 .
32 online: yes
33 .
34 If the source is not plugged upower will report a "no" instead of "yes".
35
36id: upower/ac_powered/status_unplugged
37category_id: upower
38_description: Verify upower report AC source as unplugged
39plugin: manual
40_steps:
41 Unplug the AC adapter from your device and query upower with
42 $ upower -i /org/freedesktop/UPower/devices/line_power_AC | grep online
43 for the online status of the source.
44_verification:
45 The output should be
46 .
47 online: no
48 .
49 If the source is plugged upower will report a "yes" instead of "no".
diff --git a/com.canonical.se:engineering-tests/units/wifi-ap/wifi-ap.pxu b/com.canonical.se:engineering-tests/units/wifi-ap/wifi-ap.pxu
0new file mode 10064450new file mode 100644
index 0000000..2f61369
--- /dev/null
+++ b/com.canonical.se:engineering-tests/units/wifi-ap/wifi-ap.pxu
@@ -0,0 +1,213 @@
1# Copyright 2016 Canonical Ltd.
2# All rights reserved.
3#
4# Written by:
5# Jim Hodapp <jim.hodapp@canonical.com>
6# Simon Fels <simon.fels@canonical.com>
7# Alfonso Sanchez-Beato <alfonso.sanchez-beato@canonical.com>
8
9unit: category
10id: wifi_ap
11_name: WiFi Access Point
12
13id: wifi_ap/setup/ensure_interface_connection_setup
14category_id: wifi_ap
15_description: Ensure that the wifi_ap interfaces are connected
16plugin: manual
17_steps:
18 0. If the wifi-ap snap is already installed remove it via
19 .
20 $ snap remove wifi-ap
21 .
22 1. Network Manager setup
23 .
24 For snaps on Classic:
25 NetworkManager is part of the base Ubuntu Desktop system
26 and does not need to be installed again.
27 .
28 For an Ubuntu Core system you have to install
29 the NetworkManager snap from the Ubuntu store.
30 $ snap install network-manager
31 .
32 2. Install now the wifi-ap snap via
33 .
34 $ snap install wifi-ap
35 .
36_verification:
37 Verify that you see the following when running snap interfaces:
38 :firewall-control wifi-ap
39 :network-control wifi-ap
40 :network wifi-ap
41 :network-bind wifi-ap
42 .
43 For snaps on Classic:
44 :network-manager wifi-ap
45 .
46 For full-snap based system:
47 network-manager:service wifi-ap:network-manager
48
49id: wifi_ap/configuration/get_all
50category_id: wifi_ap
51plugin: manual
52depends: wifi_ap/setup/ensure_interface_connection_setup
53_summary: All WiFi AP configuration items are listed
54_purpose:
55 Check if the configuration of the WiFi AP can be listed
56 with the expected command.
57_steps:
58 1. Execute the following command to retrieve all configuration items:
59 .
60 $ /snap/bin/wifi-ap.config get
61_verification:
62 All available configuration items are listed as output.
63
64id: wifi_ap/configuration/set_any
65category_id: wifi_ap
66plugin: manual
67depends: wifi_ap/setup/ensure_interface_connection_setup
68_summary: Setting any configuration key is possible
69_purpose:
70 Check if any configuration key can be set via the
71 configuration system of the snap.
72_steps:
73 1. Retrieve the current value of a specific configuration item
74 e.g. 'wifi.interface'.
75 .
76 $ /snap/bin/wifi-ap.config get wifi.interface
77 .
78 2. Set a new value for the same configuration item
79 .
80 $ sudo /snap/bin/wifi-ap.config set wifi.interface=wlan1
81 .
82 3. Retrieve the current value of the same configuration item
83 for comparison.
84 .
85 $ /snap/bin/wifi-ap.config get wifi.interface
86_verification:
87 Verify that the new value for the configuration is not the same
88 as before the 'set' operation and is now the the value specified
89 in step 2.
90
91id: wifi_ap/connect/enable_ap
92category_id: wifi_ap
93plugin: manual
94depends: wifi_ap/setup/ensure_interface_connection_setup
95_summary: Enabling the AP succeeds
96_purpose:
97 Check to make sure the AP can be enabled.
98_steps:
99 0. If the AP is already enabled as on some devices where the
100 automated wizard was able to find a good configuration
101 automatically you can skip the following steps. You can
102 verify if the AP is active by calling
103 .
104 $ /snap/bin/wifi-ap.status
105 ap.active: true
106 .
107 1. Look up and set the wifi interface to the proper name
108 e.g. from default wlan0 to enp0s3
109 .
110 $ ifconfig
111 $ /snap/bin/wifi-ap.config set wifi.interface=<interface_name>
112 .
113 2. Enable the AP with the default configuration for other key values
114 .