lp:~sylvain-pineau/plainbox-provider-snappy

Author: Sylvain Pineau
Author Date: 2020-03-26 14:53:27 UTC

Add docker-automated nested part to smoke test plans

Author: Sylvain Pineau
Author Date: 2019-11-21 17:11:51 UTC

units:test-plan: Remove the deprecated collect-manifest

Author: Sylvain Pineau
Author Date: 2019-05-20 08:10:05 UTC

test-plan.pxu: Add tpm2.0_3.0.4 to smoke and generic test plans

Author: Sylvain Pineau
Author Date: 2019-05-13 09:56:29 UTC

units:snappy: Add SNAPD_RETRY_SEC SNAPD_RETRY_INTERVAL to environ fields

Author: Sylvain Pineau
Author Date: 2019-03-30 10:19:27 UTC

watchdog: Add a new trigger-system-reset-auto job

The full test plan is still calling the user-interact-verify version.
watchdog-automated is meant for automated/headless runs.

Author: Sylvain Pineau
Author Date: 2019-03-14 09:41:52 UTC

info.pxu: Extend the parts_meta_info_attachment with content interface data

Author: Sylvain Pineau
Author Date: 2019-02-11 10:04:28 UTC

units:power: Fix warm/cold reboot tests to work from a systemd service

And avoid the 'Running in chroot, ignoring request.' errors.

See https://forum.snapcraft.io/t/running-systemd-commands-inside-a-snap/3591

Author: Sylvain Pineau
Author Date: 2018-12-21 13:21:41 UTC

units:usb:usb/storage-preinserted: Take into account usb2 and usb3 devices

Author: Sylvain Pineau
Author Date: 2018-12-17 08:09:46 UTC

bluetooth: blacklist bluez-internal-hci-tests_LE Generate DHKey

Author: Sylvain Pineau
Author Date: 2018-12-04 17:15:15 UTC

manage.py: Remove TestPlanReport extension

Author: Sylvain Pineau
Author Date: 2018-11-30 09:47:30 UTC

units:wwan:gsm-connection test: Add journalctl logs for both NM/MM on failures

Author: Sylvain Pineau
Author Date: 2018-11-29 22:07:12 UTC

units: Add system_info to all test plans

Author: Sylvain Pineau
Author Date: 2018-11-05 11:57:33 UTC

bin:wwan_tests: Fix a crash when mmcli is not available in resources mode

Author: Sylvain Pineau
Author Date: 2018-10-08 11:25:44 UTC

bluetooth: Remove obex-send test and prefer the beacon EddyStone scan instead

Author: Sylvain Pineau
Author Date: 2018-06-26 08:33:07 UTC

Group all wireless/wifi tests under the same category id

To avoid having them spread across two sections in the xlsx/html reports.

Author: Sylvain Pineau
Author Date: 2018-06-11 08:24:11 UTC

Add socketcan-full nested part (fully auto) to p-p-s test plans

Author: Sylvain Pineau
Author Date: 2018-06-06 07:15:19 UTC

Move ethernet job units to plainbox-provider-checkbox

Only keep snappy nested test plans here.

Author: Sylvain Pineau
Author Date: 2018-05-23 12:13:07 UTC

Add kernel config check test for 6lowpan

Author: Sylvain Pineau
Author Date: 2018-03-23 11:42:48 UTC

wwan: Update wwan/gsm-connection-interrupted-manual using template vars

And enable it again in test plans since it asks to place the SUT in a
Faraday bag instead of removing the antenna.

Author: Sylvain Pineau
Author Date: 2018-02-20 09:10:39 UTC

wwan: Use the new __checkbox_env__ parameter to template WWAN_NET_IF/WWAN_APN

Fixes lp:1748371

Author: Sylvain Pineau
Author Date: 2018-01-26 16:26:14 UTC

Update requirements to depend on checkbox-ng

Author: Sylvain Pineau
Author Date: 2018-01-24 12:24:57 UTC

power: Remove power-automated from power-full test plan

On core devices, running those automated tests in a manual test plan results
in unexpected behaviors as upon reboot a systemd unit automatically restart
checkbox as a background process.
If the tester decides to also resume the session manually it can lead to
several tests being skipped as:

- There is no UI to prompt the user for the job outcome
- Jobs can be left with the special outcome IJobResult.OUTCOME_UNDECIDED.

Thus power-automated nested part should be used in fully automated test plans
ONLY.

Fixes lp:1742889

Author: Sylvain Pineau
Author Date: 2018-01-19 09:19:54 UTC

stress: Add lsusb checks upon warm/cold boot

Using a new short mode option (-s), lsusb.py can deliver a mode that we can
diff across boot cycles.

Author: Sylvain Pineau
Author Date: 2017-12-11 13:37:18 UTC

wwan: Extend requirements to take into account running on classic

Author: Jonathan Cave
Author Date: 2017-10-13 13:03:56 UTC

Remove wireless jobs moved to p-p-checkbox

Author: Sylvain Pineau
Author Date: 2017-09-29 09:59:58 UTC

Add info/buildstamp to snappy-generic test plans

Author: Sylvain Pineau
Author Date: 2017-09-27 08:13:08 UTC

wifi-ap: only test b/g protocols with the nested test plan

802.11a and ad protocols will have to be included explicitly to be tested.

Author: Sylvain Pineau
Author Date: 2017-09-14 16:11:38 UTC

rename tpm test plans (info becomes tpm-automated)

and tpm-smoke-tests -> tpm-manual

Author: Sylvain Pineau
Author Date: 2017-08-21 08:25:53 UTC

audio: Add two manifest units for audio playback/capture capabilities

Author: Maciej Kisielewski
Author Date: 2017-07-04 13:08:33 UTC

remove 2013. prefix from all IDs

Signed-off-by: Maciej Kisielewski <maciej.kisielewski@canonical.com>

Author: Sylvain Pineau
Author Date: 2017-07-05 10:42:45 UTC

import bt_helper from checkbox-support

Author: Sylvain Pineau
Author Date: 2017-06-23 07:49:22 UTC

bluetooth: blacklist all bluez management tests and two new hci tests

- bluetooth/bluez-internal-hci-tests_LE Read Local PK
- bluetooth/bluez-internal-hci-tests_Read Local Supported Codecs

Author: Sylvain Pineau
Author Date: 2017-06-15 09:32:09 UTC

Use checkbox-support-usb_read_write

Author: Sylvain Pineau
Author Date: 2017-06-12 18:56:10 UTC

Check snap revision using int() to do the comparison in resource expressions

See http://checkbox.readthedocs.io/projects/plainbox/en/latest/dev/resources.html?highlight=resource#everything-is-a-string

Author: Sylvain Pineau
Author Date: 2017-05-19 16:35:36 UTC

wireless: Add a resource job to report STA capabilities (802.11n/ac)

The NM connection jobs (open and wpa) now require the corresponding protocol.

Author: Sylvain Pineau
Author Date: 2017-05-12 07:41:34 UTC

Remove kernel-security build artifacts

Author: Sylvain Pineau
Author Date: 2017-05-11 19:36:09 UTC

Update container tests to pull the generic resource provider

Author: Sylvain Pineau
Author Date: 2017-05-03 11:48:48 UTC

p-p-s: Remove obsolete suspend jobs (copy of classic checkbox provider)

The following categories all have after suspend jobs trough also-after-suspend
flags:

Audio, USB, Wireless, Monitor, BT (keyboard), Ethernet, Serial.

The "suspend" folder is (and will stay) a placeholder for suspend-auto and
hibernate-auto only.

Author: Sylvain Pineau
Author Date: 2017-04-26 10:10:14 UTC

p-p-s: Monitor mediacard insertion/removal using the journalctl command

Fixes: LP:1674960

Author: Sylvain Pineau
Author Date: 2017-04-19 11:08:50 UTC

p-p-s: Move attachments jobs from the resource provider to p-p-s

In order to use the generic resource provider (and to keep it a resource
only provider), some jobs have to be transferred to p-p-s.

Author: Sylvain Pineau
Author Date: 2017-04-19 08:56:14 UTC

p-p-s: Add the nm connections requirements to the wireless_scanning job

Author: Sylvain Pineau
Author Date: 2017-04-18 13:27:43 UTC

p-p-s: Use 2013.com.canonical.plainbox::monitor for monitor jobs

'display' is a resource job id (conflicting)

Author: Sylvain Pineau
Author Date: 2017-04-07 13:26:49 UTC

p-p-s: Update locationd.monitor test to properly exit on first data update

Author: Sylvain Pineau
Author Date: 2017-03-31 16:01:16 UTC

p-p-s: Run wireless connection tests as non-root user

Author: Sylvain Pineau
Author Date: 2017-03-13 17:09:38 UTC

p-p-s: Fix mmcli call in wwan/detect-manual

modem-manager.mmcli -l ==> modem-manager.mmcli -L

Author: Sylvain Pineau
Author Date: 2017-03-09 23:11:04 UTC

p-p-s:wifi-ap: Use the key=val format

Author: Sylvain Pineau
Author Date: 2017-03-09 20:34:37 UTC

p-p-s: Remove the manual bluetooth/hid test (obsolete)

replaced by bluetooth/keyboard based on bt_helper python module

Author: Sylvain Pineau
Author Date: 2017-03-03 10:59:47 UTC

p-p-s: Add WoLAN and WoWLAN from S4 tests

Author: Sylvain Pineau
Author Date: 2017-03-02 17:15:10 UTC

p-p-s: Add after-suspend-wwan-manual test plan

And stop running all auto/semi-auto tests relying on MM dbus API.

Author: Sylvain Pineau
Author Date: 2017-02-27 13:52:00 UTC

p-p-s:fwts_test: Flush the progress messages to follow them on stdout

Author: Sylvain Pineau
Author Date: 2017-02-23 23:48:18 UTC

p-p-stress: New version of S3 stress tests based on template jobs

One job per Suspend/resume iteration. This version will allow to quickly
detect which S3 loop failed.

Author: Sylvain Pineau
Author Date: 2017-02-22 12:26:55 UTC

p-p-s: Add WoWLAN tests using both wakeonlan and etherwake methods

Note: wakeonlan is a program that uses UDP packets not TCP,
and hence does not need to be run as root.

Author: Sylvain Pineau
Author Date: 2017-02-21 18:32:16 UTC

p-p-s: Remove after suspend flag from hotplug tests

Just keep the ping tests

Author: Sylvain Pineau
Author Date: 2017-02-21 14:04:42 UTC

p-p-s: Fix network_device_info to only check/report Ethernet devices

Since this script is only used by ethernet/detect job

Author: Sylvain Pineau
Author Date: 2017-02-20 17:20:18 UTC

p-p-s: Add a new category to group all BlueZ tests after suspend

Author: Sylvain Pineau
Author Date: 2017-02-20 12:16:42 UTC

p-p-s: Fix audio nested part names

Author: Maciej Kisielewski
Author Date: 2017-02-09 18:57:08 UTC

make location tests run again after suspend

Signed-off-by: Maciej Kisielewski <maciej.kisielewski@canonical.com>

Author: Sylvain Pineau
Author Date: 2017-02-10 11:14:14 UTC

p-p-stress: Replace depends by after in S3/S4 tests

To ensure both check and attachments jobs are executed

Author: Sylvain Pineau
Author Date: 2017-02-01 10:46:23 UTC

p-p-s: Fix broken i18n setup

Author: Sylvain Pineau
Author Date: 2017-02-01 09:16:21 UTC

p-p-s: Move the disk_stress_ng based test to the stress provider

Add a dedicated nested test plan for stress-ng tests:
2013.com.canonical.certification::stress-ng-automated

Author: Sylvain Pineau
Author Date: 2017-01-16 16:08:15 UTC

Update the pmr hook to upload to edge and beta

from their respective recipes

Author: Sylvain Pineau
Author Date: 2017-01-15 09:41:59 UTC

Update wifi-ap tests steps

Fixes https://bugs.launchpad.net/plano/+bug/1654496

Author: Sylvain Pineau
Author Date: 2017-01-04 11:32:32 UTC

p-p-s: Add stress-automated nested part to the main automated test plan

Author: Sylvain Pineau
Author Date: 2016-12-12 13:02:48 UTC

p-p-s: Add location service tests

Author: Sylvain Pineau
Author Date: 2016-11-24 20:03:24 UTC

p-p-s: Add WOL S5 tests (Power off)

Author: Sylvain Pineau
Author Date: 2016-11-24 16:34:52 UTC

p-p-s: Ensure that connections tests runs without any already active connections

As the test using "iw dev {interface} link | wc -l" passes but we were not
testing the right protocol.

Author: Sylvain Pineau
Author Date: 2016-11-24 13:18:14 UTC

p-p-r:udev_resource: Add mac addresses to the list of attributes

Author: Sylvain Pineau
Author Date: 2016-11-21 12:33:27 UTC

p-p-s: Remove automated master mode tests

Author: Sylvain Pineau
Author Date: 2016-11-08 14:00:27 UTC

UC16 tests for snap/os refresh/revert and failed boot (core)

The failed boot test is using the snap_mode grub env to simulate the failed boot.

Author: Sylvain Pineau
Author Date: 2016-11-02 09:54:45 UTC

Stop running stress-ng tests as root

The only stressor that won't run is apparmor, but it's non fatal and skipped.
Running as root in the other hand fails the tests on some platforms,
cpu and scheduler classes:

stress-ng: info: [30363] stress-ng-exec: running as root, won't run test.
stress-ng: error: [7373] process 30363 (stress-ng-exec) terminated with an error, exit status=1

Author: Sylvain Pineau
Author Date: 2016-10-31 13:02:34 UTC

Remove job summaries failing validation

Author: Sylvain Pineau
Author Date: 2016-10-28 13:29:28 UTC

Add missing tpm categories

Author: Sylvain Pineau
Author Date: 2016-10-28 12:38:03 UTC

Update watchdog tests now managed entirely by systemd

Fixes lp:1633086

Author: Sylvain Pineau
Author Date: 2016-10-27 13:35:43 UTC

tpm: use the dedicated snap commands

Author: Sylvain Pineau
Author Date: 2016-10-27 13:35:43 UTC

tpm: use the dedicated snap commands

Author: Sylvain Pineau
Author Date: 2016-10-25 12:12:07 UTC

Fix ethernet hotplug test ordering

The fix uses the new jinja2 template support to add an after field only for
jobs with a device index greater than 1.
This way the check-disconnected for the second device won't start until the
first device cycle is over (check/connect/disconnect).

Author: Sylvain Pineau
Author Date: 2016-10-24 07:56:52 UTC

Change ethernet hotplug ids to run them in - interface - order

eth0 check
eth0 connect
eth0 disconnect
eth1 check
eth1 connect
eth1 disconnect
eth2 ...

Author: Sylvain Pineau
Author Date: 2016-09-27 15:49:48 UTC

develop ipdt and docker providers for validation

Author: Sylvain Pineau
Author Date: 2016-09-23 09:22:59 UTC

p-p-s: Use the ipdt nested part now that it does not pull GPU tests

Author: Sylvain Pineau
Author Date: 2016-09-02 16:16:56 UTC

p-p-s: Add a server test plan combining snappy-generic and 16.04-server-soc-cert

Author: Sylvain Pineau
Author Date: 2016-09-01 15:55:33 UTC

Add docker an Intel ipdt tests to generic test plans

Author: Sylvain Pineau
Author Date: 2016-08-29 15:13:57 UTC

Version in the Makefile needs $$ to work right, and use awk to modify the version string so that we get the correct directory name

Author: Sylvain Pineau
Author Date: 2016-08-29 08:09:25 UTC

p-p-s-r: Add rtc and sleep resources

Author: Sylvain Pineau
Author Date: 2016-08-19 12:51:51 UTC

p-p-s-r: Run lspci job with -i $SNAP/usr/share/misc/pci.ids

Author: Sylvain Pineau
Author Date: 2016-08-18 08:19:59 UTC

p-p-s: Add new disk stress tests using stress-ng

Author: Sylvain Pineau
Author Date: 2016-07-19 13:01:04 UTC

p-p-c: Replace lsbutils lsusb binary by the lsusb.py python3 script

In the lsusb_attachment job.

Author: Sylvain Pineau
Author Date: 2016-04-14 16:31:09 UTC

p-p-s-r: Update the udev_resource script to benefit of the lsblk data