lp:~jocave/plainbox-provider-checkbox

Author: Jonathan Cave
Author Date: 2022-08-16 10:02:50 UTC

Fix: shellcheck errors on jammy

Author: Jonathan Cave
Author Date: 2022-08-09 13:09:55 UTC

Change: detect modem primary port using mmcli

The control interface needed to be specified in the checkbox.conf
to execute the gsm connection test. This is a minimal change to
now pull that information from mmcli

Author: Jonathan Cave
Author Date: 2022-05-04 11:45:45 UTC

Fix: snap list test failing on UC22

Include core22 in the list of valid core snaps to search for in
the snap list test

Author: Jonathan Cave
Author Date: 2022-04-20 14:41:55 UTC

Change: set TEST_SNAP when using checkbox22

Default the test snap to test-snapd-tools-core22 to avoid
need to download an extra base snap

Author: Jonathan Cave
Author Date: 2022-03-17 16:59:29 UTC

run dmesg after test

Author: Jonathan Cave
Author Date: 2022-02-21 17:52:48 UTC

Change: run graphics/opengl_support as root

This is a hack only to work around that checkbox does not currenlty
set up the environment correctly when running in remote

Author: Jonathan Cave
Author Date: 2022-02-15 18:12:07 UTC

Fix: UdevadmParser in touchpad_driver_info.py

Use new API defined in checkbox-support

Author: Jonathan Cave
Author Date: 2022-02-01 17:57:57 UTC

Change: gl_support job works from snaps

Prepend the path to the script used for testing with CHECKBOX_RUNTIME
so when checkbox is packaged in a snap it will be found in the checkbox
core snap

Author: Jonathan Cave
Author Date: 2022-01-28 17:30:07 UTC

Change: gather buildstamp for stock installs

The stock installer stores media info in /var/log, this is the data
collected by apport. Use this as the "buildstamp" when available.

Author: Jonathan Cave
Author Date: 2021-11-10 19:49:33 UTC

Change: run netplan test in try/finally

Author: Jonathan Cave
Author Date: 2021-11-15 12:09:02 UTC

Remove: kernel-snap/booted-dtbs from nested plan

Author: Jonathan Cave
Author Date: 2021-11-09 10:11:41 UTC

Fix: invalid manifest check on bluetooth/detect

Field requires a string comparison to work correctly

Author: Jonathan Cave
Author Date: 2021-10-26 15:36:54 UTC

Change: debsums & check_prelease desktop blockers

Author: Jonathan Cave
Author Date: 2021-10-13 16:59:01 UTC

Change: interlaced modes don't generate error

In the graphics/{}_cycle_resolution_{} test do not produce error output
when encountering interlaced modes which do not prevent cycling through
modes of the same resolution.

lp: #1946469

Author: Jonathan Cave
Author Date: 2021-08-23 17:15:17 UTC

Add: kernel-snap/booted-dtbs-match-current test

New test to check that DTB files distributed in kernel snaps
match the versions in the bootloader location for new snapd
versions that support updating

Author: Jonathan Cave
Author Date: 2021-10-12 16:57:40 UTC

Change: depend on debsums executable over package

Allow use of the debsums job from checkbox snaps for classic images
by switching to executable dependency (and adding the debsums exe to
the checkbox-core-snap).

Author: Jonathan Cave
Author Date: 2021-08-25 10:52:18 UTC

Change: make wwan/gsm-connection more verbose

Add some command echoing to help tracing job actions when attempting
to setup a GSM data connection

Author: Jonathan Cave
Author Date: 2021-08-17 13:37:21 UTC

Change: bluetooth/detect checks device resource

When the manifest indicates that a bluetooth device should be
present check that it is both listed by rfkill and identified in
the udev db as per the device resource

Author: Jonathan Cave
Author Date: 2021-07-02 15:14:29 UTC

Add: led test for specific AAEON devices

Author: Jonathan Cave
Author Date: 2021-06-29 11:00:03 UTC

Change: wireless-cert-manual don't nest automated

Author: Jonathan Cave
Author Date: 2021-06-28 16:26:38 UTC

Change: full/manual/automated for power-management

Standard layout for nested plans

Author: Jonathan Cave
Author Date: 2021-06-28 15:30:31 UTC

Change: ethernet/networking plans follow category

Create ethernet-cert-* test plans to have consistent assignement of
jobs to nested plans based on category

Author: Jonathan Cave
Author Date: 2021-06-28 14:53:19 UTC

Change: mobilebroadband don't next automated

Don't nest the automated plan in mobilebraodband-cert-manual

Author: Jonathan Cave
Author Date: 2021-06-28 13:29:17 UTC

Add: a fingerprint-automated nested test plan

Added for easier maintenance of test plans

Author: Jonathan Cave
Author Date: 2021-06-28 12:47:20 UTC

Change: misc-cert-manual doesn't nest -automated

Author: Jonathan Cave
Author Date: 2021-06-28 10:45:22 UTC

Change: cpu-cert-manual doesn't nest -automated

Remove the nesting of the automated plan in the manual specific
plan cpu-cert-manual

Author: Jonathan Cave
Author Date: 2021-06-25 14:45:54 UTC

Change: resource manage the key temp dir

Means this data is removed from the session share when the test
is finished running.

Author: Jonathan Cave
Author Date: 2021-06-18 09:40:58 UTC

Remove: firmware-uc20-fde checks

Compatibility with uc20 FDE functionality will now be tested using
FWTS

Author: Jonathan Cave
Author Date: 2021-06-01 13:23:44 UTC

Change: fde/encryption/detect passes without type

In rare cases the version of cryptsetup in the checkbox snap will
not match that used to creation the encrypted partition. In such
cases the type will appear as `n/a` and the regex will not match.
Relax the test to handle this situation.

Author: Jonathan Cave
Author Date: 2021-05-11 17:46:45 UTC

Add: nested test plans for virtualization tests

Author: Jonathan Cave
Author Date: 2021-04-28 09:37:18 UTC

Fix: make parts_meta_info_attachment work for all

This job would not work in both remote and local mode due to
another problem with the environ command. The only way to get the
CHECKBOX_RUNTIME variable available in both situations is to use
both root user and specifiy the variable

Author: Jonathan Cave
Author Date: 2021-04-27 13:40:38 UTC

Fix: lspci_attachment fails when no pci bus

If there is no PCI bus on the SUT exit with pass and don't attempt
to call lspci

Author: Jonathan Cave
Author Date: 2021-04-27 09:38:19 UTC

Fix: dont attempt to attach cstates_results on arm

On arm devices the cstates job cannot be run. However, the attachment
job will still attempt to find the log file and fail if it is not
present.

Author: Jonathan Cave
Author Date: 2021-04-22 14:56:47 UTC

Add: jobs to dump acpi tables and attach

Two new jobs that use fwts (which should be available on all systems
targetted by checkbox) to dump the acpi tables and that attach the
output. The attachment is gzipped to save reduce size of the submission.

LP: #1913714

Author: Jonathan Cave
Author Date: 2021-04-19 16:55:10 UTC

Fix: support nmcli AP settings names in focal

The names of certain wifi settings used in Access Point mode have
changed in network-manager version used in focal and later.

Author: Jonathan Cave
Author Date: 2021-04-09 16:47:30 UTC

Fix: ensure all plans have -manual variants

Ensure all plans have full/automated/manual variants. Also tidy
up some wording and consistency of captilisation.

Author: Jonathan Cave
Author Date: 2021-03-30 09:59:09 UTC

Change: net_if_mgmt selects master mode tests

Update the master mode nested plans to pull in the NetworkManager
based tests when the net_if_mgmt resoure indicates they should
be used

Author: Jonathan Cave
Author Date: 2021-03-30 13:25:25 UTC

Add: check that ubuntu-drivers can be run

Check this command is runnable and if a package list is produced
use an attachment job to include it.

Author: Jonathan Cave
Author Date: 2021-03-24 16:04:27 UTC

Change: modify some manifest names for clarity

Modify just the name field (what is presented to users in the UI)
for a number of manifest units to hopefully improve the clarity
of the question.

Author: Jonathan Cave
Author Date: 2021-02-16 10:59:39 UTC

snapd/*: ensure env vars available for local run

Some environment variables necessary for correct operations when
run use local checkbox session where not declared for passing
through the differential execution mechanism.

LP: #1910524

Author: Jonathan Cave
Author Date: 2021-02-12 15:32:52 UTC

units/submission: dkms_info_json empty json on UC

On Ubuntu Core the dkms_info_json job will output a traceback
to there being on dpkg status. This will just prevent that by
outputting empty json.

Author: Jonathan Cave
Author Date: 2021-01-27 11:19:04 UTC

wifi_nmcli_test: kill process group on timeout

Author: Jonathan Cave
Author Date: 2020-12-17 10:37:54 UTC

networking: dont run predicatable_names on pi

Pi devices intentionally use "stable" legacy interface names to
allow moving SD cards between devices.

Author: Jonathan Cave
Author Date: 2020-12-16 17:43:13 UTC

watchdog/*: add detect job to complement manifest

Create watchdog/detect job to check for a suitable device in the
udev db when the manifest indicates SUT should have one.

Author: Jonathan Cave
Author Date: 2020-12-09 18:25:47 UTC

firmware/*: jobs to check UC20 FDE compatibility

Use the tcglog-parser tools to test whether the device firmware
has the necessary features to enable UC20 FDE.

Author: Jonathan Cave
Author Date: 2020-12-07 14:34:33 UTC

socketcan/*: fix EFF ID flag on 32 bit arch

A misuse of the socket.CAN_EFF_FLAG constant was causing the
CAN ID to be out of range on 32bit architectures.

Author: Jonathan Cave
Author Date: 2020-12-08 14:36:48 UTC

kernel-snap: require kernel path for booted-kernel

Only attempt to run the booted-kernel-matches-current-.* test on
devices where we can identify the location of the booted kernel.
This will preclude devices using uboot bootloader currently.

Author: Jonathan Cave
Author Date: 2020-12-04 11:07:32 UTC

watchdog/*: move watchdog tests behind a manifest

To allow the watchdog tests to be included in generic test plans
they need to be placed behind a manifest entry. Lack of the manifest
results in fails on devices without a builtin watchdog (e.g. rpi).

Author: Jonathan Cave
Author Date: 2020-11-25 18:42:54 UTC

snap_tests.py: fix exit code, add core20

Ensure that the return value of each subcommand actually impacts
the test result. Add core20 to the list of core snaps.

Author: Jonathan Cave
Author Date: 2020-11-24 15:54:52 UTC

wireless/*: make auto jobs depend on manifest

Modify all the automated NM and networkd jobs such that they
depend on a detect job and the presence of a manifest entry.
This should protect aginst mis-reporting of device capabilities
by iw.

Author: Jonathan Cave
Author Date: 2020-11-13 14:54:35 UTC

roundtrip_qr.py: detect the active tty

UC20 seems to deviate from the norm of tty1 being the active tty
after install. Update the script to detect which is active.

Author: PMR
Author Date: 2020-10-28 12:10:44 UTC

Merge #392898 from ~jocave/plainbox-provider-checkbox:wwan-remove-core-req

Author: Jonathan Cave
Author Date: 2020-10-27 18:31:36 UTC

wwan/jobs: remove requirement for core snap

There should no longer be a requirement for the core snap to be
present when running the wwan tests, for exmaple on UC18 or UC20
images

Author: Jonathan Cave
Author Date: 2020-10-05 09:48:30 UTC

wireless/*: depend on net_if_mgmt resource again

Re-apply the requires field on wireless job to use the
net_if_managment resource to select which wireless jobs to run on
a device. This should now correctly select either network-manager
or networkd and allow for a single set of test plans.

Author: Jonathan Cave
Author Date: 2020-10-01 11:23:26 UTC

bin/booted_kernel_tests: kernel name for uc20

In UC20 images the kernel image is packed and distributed in
a different format and location. Handle this for the booted
kernel check test.

Author: Jonathan Cave
Author Date: 2020-09-10 15:53:21 UTC

fde_tests.py: update FDE detect for UC20

Update the script used by the disk/encryption/detect job to work
on UC20. The writable partition is now know as ubuntu-data in
this release.

Author: Jonathan Cave
Author Date: 2020-08-07 10:19:37 UTC

pulse_active_port_change.py: classic snap support

When this script is used to test a desktop system from a classic
checkbox snap and there is a switch from the root user down to
a non-priviledged user there is a need to modify the
PULSE_RUNTIME_PATH variable to allow comms between pactl and the
service.

Author: Jonathan Cave
Author Date: 2020-08-05 22:30:28 UTC

units/*: simplify requires fields

Improve performance of checkbox by reducing the complexity of
requires fields.

Author: Jonathan Cave
Author Date: 2020-08-05 14:54:34 UTC

wireless: NM auto jobs require root for ping

The addition of a ping test that binds to interface to be tested
requires root permisions.

Author: Jonathan Cave
Author Date: 2020-07-28 17:37:03 UTC

units/*: remove use of zenity dialogs

Modify any jobs still using zenity dialogs to only display the
information on the command line

Author: Jonathan Cave
Author Date: 2020-07-22 10:01:39 UTC

net_driver_info: handle empty argument

Handle the case where the NET_DRIVER_INFO config variable is
empty.

Author: Jonathan Cave
Author Date: 2020-07-21 16:17:10 UTC

fde_tests: fix bad search with line end

Fix use of '$' to match end of line in multiline string

Author: Jonathan Cave
Author Date: 2020-07-21 10:57:45 UTC

wifi: fixes for xenial, fix ping args

Need to support nmcli arguments back to xenial. Also the ping
test appears to have failed due to impossible to reach
requirement of 5 pings in 4 second timeout

Author: Jonathan Cave
Author Date: 2020-06-29 16:37:37 UTC

wifi_nmcli_test.py: add ping test, disable ipv6

Make sure connection status not influenced by ipv6. Make sure
and IP address is assigned and the gateway address can be pinged

Author: Jonathan Cave
Author Date: 2020-07-13 11:07:01 UTC

disk/encryption: support new LUKS in FDE test

LUKS2 is used in UC18 projects and newer, update the regex to
match the type string.

Author: Jonathan Cave
Author Date: 2020-07-01 13:57:35 UTC

units/image: tests for ODM IoT Certification

Tests intended for use checking the validity of the image used
when submitting a report for certification of an Ubuntu Core
system

Author: Jonathan Cave
Author Date: 2020-07-02 08:27:57 UTC

requirements: install shellcheck for testing

Make sure shellcheck is available during pmr merge testing

Author: Jonathan Cave
Author Date: 2020-06-29 17:01:32 UTC

gateway_ping_test: fix crash when using networkctl

When the information is available networkctl will include the
manufacturer in the gateway status field in brackets. This
handles that situation and problems splitting the line better.

Author: Jonathan Cave
Author Date: 2020-06-22 13:16:44 UTC

i2c/manifest: more specific naming

Author: Jonathan Cave
Author Date: 2020-06-03 13:03:27 UTC

ethernet: static config job networkd/nm support

Convert the old check-{if}-static script to python. Add support
for doing the same type of test when the interface is managed
by networkd. Allow check the address that was assigned through
a config variable.

Author: Jonathan Cave
Author Date: 2020-05-21 14:42:18 UTC

networking: limit predictable name test by release

On Ubuntu Core run the predictable names tests on UC20 or later
and on other systems test on 18.04 or later.

Author: Jonathan Cave
Author Date: 2020-04-23 17:38:42 UTC

ethernet: fix SO_BINDTODEVICE permissions errors

In some circumstances a test run will switch to normal_user that
does not have permission to bind a socket to a specific interface
which causes errors when running ping -I{interrface}. Force the
test to run as root.

Author: Jonathan Cave
Author Date: 2020-04-16 18:00:05 UTC

power-management: add failed service checks

After running tests that cause the device to reboot, check for
any failed services. Could be pertinent to the watchdog test
where the shutdown is not manageed.

Author: Jonathan Cave
Author Date: 2020-04-15 16:33:41 UTC

booted_kernel_tests: fix for base core18 snaps

Need to use snapd managed path to read hostfs

Author: Jonathan Cave
Author Date: 2020-02-18 10:08:50 UTC

camera: add packaging units for roundtrip-qr job

Author: Jonathan Cave
Author Date: 2020-02-21 18:44:19 UTC

wwan: remove missed context cruft from gsm test

Cleanup Some last traces of guacamole.

Author: Jonathan Cave
Author Date: 2020-02-21 17:17:24 UTC

wwan: use dbus on resource as with other jobs

Author: Jonathan Cave
Author Date: 2020-02-13 11:35:25 UTC

snap_tests.py: add verbose output on install tests

The snapd class now supports print task information during async
operations. Enable this for just install tests.

Author: Jonathan Cave
Author Date: 2020-02-11 13:41:12 UTC

camera: only run rpi test on armhf

The VideoCore userland currently only works on 32bit arch.

Author: Jonathan Cave
Author Date: 2020-02-07 17:10:54 UTC

info/packaging: remove python3-guacamole from deps

There are no longer any scripts using this module so removing the
packaging unit

Author: Jonathan Cave
Author Date: 2020-02-07 16:51:52 UTC

wifi_master_mode: remove use of guacamole module

Author: Jonathan Cave
Author Date: 2020-02-07 16:30:48 UTC

dkms_info: remove use of guacamole module

Author: Jonathan Cave
Author Date: 2020-02-07 14:43:17 UTC

dmi-sysfs-resource: script rename

Author: Jonathan Cave
Author Date: 2020-02-07 14:31:23 UTC

requirements: run `manage.py test` during merge

Author: Jonathan Cave
Author Date: 2020-02-07 12:50:22 UTC

recovery_info: remove guacamole, split out tests

This commit encapsulates changes to: remove guacamole dependency,
move out the unit tests to a dedicated script that can be run
with the `manage.py tests` framework, and give the script a .py
extension to allow importing

Author: Jonathan Cave
Author Date: 2020-02-07 10:22:43 UTC

tpm-sysfs-resource: rename script

Author: Jonathan Cave
Author Date: 2020-02-07 10:10:04 UTC

module_loaded_test: script rename

Author: Jonathan Cave
Author Date: 2020-02-04 12:28:56 UTC

wifi-ap: remove dependency on NM wireless job

It should be not necessary to keep this dependency for these
jobs given that it may not be included in the test plan if
netplan tests are used.

Author: Jonathan Cave
Author Date: 2020-02-03 17:47:52 UTC

wwan_tests: rename to wwan_tests.py

Author: Jonathan Cave
Author Date: 2020-01-27 16:50:55 UTC

snap_tests: remove use of guacamole module

Author: Jonathan Cave
Author Date: 2020-01-24 16:26:23 UTC

i2c: rename test script and use argparse

Author: Jonathan Cave
Author Date: 2020-01-24 14:35:15 UTC

info/stress/usb: use CHECKBOX_RUNTIME variable

Make calls to checkbox-support-lsusb use the CHECKBOX_RUNTIME
variable to find the usb.ids file.

Author: Jonathan Cave
Author Date: 2020-01-24 11:13:22 UTC

info: fix parts_meta_info_attach for classic snaps

Use new CHECKBOX_RUNTIME variable to make sure parts_meta_info
is collected even in classic snaps

Author: Jonathan Cave
Author Date: 2020-01-17 11:12:16 UTC

camera: tests specific to RPi camera

Author: Jonathan Cave
Author Date: 2019-12-19 16:16:53 UTC

monitor: add EDID stress test using chameleon

Imports a set of EDID example data from chromiomos project and
loads these one at a time in to the Chamelon board.

Author: Jonathan Cave
Author Date: 2019-12-12 08:52:13 UTC

monitor: rework hdmi hotlpug for chamelon board

Switch from use of the MuxPi to the chameleon board/service to
carry out an automated HDMI hotplug test.

Author: Jonathan Cave
Author Date: 2019-12-04 11:32:20 UTC

usb: add detect job for preinserted storage

Adds the detect job, a manifest entry and the test plan modifications

Author: Jonathan Cave
Author Date: 2019-12-03 11:33:55 UTC

usb: disk_detect requires udisks2 to be available

Author: Jonathan Cave
Author Date: 2019-11-07 17:25:08 UTC

wifi_nmcli_test: always clean up connections

A recent change broke the expected behaviour that connections
would be cleaned up and the end of the test. I don't think is much
need to catch any exceptions any more so removing the try block

Author: Jonathan Cave
Author Date: 2019-10-25 11:38:56 UTC

wifi_nmcli_backup: always create the backup dir

This ensures that the restore job will pass in the case when there
are no connections prior to running the save job