Disk tests ignored on VMs

Bug #1559189 reported by Jeff Lane 
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Checkbox Provider - Base
Fix Released
Critical
Jeff Lane 

Bug Description

Testing the latest RC2 for checkbox and I noticed the following messages in the checkbox-ng output when trying for the first time:

There are 0 incomplete sessions that might be resumed
Preparing...
Ignoring invalid instantiated unit Disk statistics for /dev/vda: Problem with field _description: wrong
Ignoring invalid instantiated unit disk/smart_vda: Problem with field _description: wrong
Ignoring invalid instantiated unit disk/read_performance_vda: Problem with field description: wrong
Ignoring invalid instantiated unit disk/storage_device_vda: Problem with field _description: wrong

canonical-certification-server 0.23~c1~ppa~ubuntu15.10.1
canonical-certification-submit 0.23~c1~ppa~ubuntu15.10.1
checkbox-ng 0.24~c2~ppa~ubuntu15.10.1
plainbox-provider-checkbox 0.26~c3~ppa~ubuntu15.10.1
python3-checkbox-ng 0.24~c2~ppa~ubuntu15.10.1
python3-checkbox-support 0.24~c2~ppa~ubuntu15.10.1

And sure enough, in the Test List, they are not present:
[X] + Benchmarks tests │
│ [X] + CPU tests │
│ [X] - Disk tests │
│ [X] disk/detect │
│ [X] disk/stats_vda <---- this appeared AFTER I changed {product} to {name} │
│ [X] + Ethernet Device tests │
│ [X] + Informational tests │
│ [X] + Memory tests │
│ [X] + Miscellaneous tests │
│ [X] + Optical Drive tests │
│ [X] + Power Management tests │
│ [X] + USB tests │
│ [X] + Uncategorised │
│ [X] + Virtualization tests

This was seen in a Digital Ocean droplet running amd64 15.10.

The problem seems to be the format of the description. As it currently sits, the description for Stats, for example, is this:
_description:
 This test checks disk stats, generates some activity and rechecks stats to
 verify they've changed. It also verifies that disks appear in the various
 files they're supposed to.
 .
 This test will inspect the following disk:
 .
     product name: {product}
     sysfs path: {path}
     device node path: /dev/{name}

Replacing instances of {product} with {name} in disk.txt.in resolved the issue for all of it in the Digital Ocean droplet.
I saw similar on a zKVM instance but in that case, changing {product} to {name} did not fix it and I only saw disk/detect as a disk test. Also, on Xenial, I never saw any of hte "Ignoring ..." messages.

We do a lot of testing on KVM and VM type scenarios (PowerVM, PowerKVM, sometimes standalone KVM devices from partners and also eventually z/KVM and z/VM)

Related branches

Jeff Lane  (bladernr)
description: updated
Jeff Lane  (bladernr)
tags: added: hwcert-server
summary: - Errors when running latest RC2
+ Disk tests ignored on VMs
Revision history for this message
Sylvain Pineau (sylvain-pineau) wrote :

Could you please attach the output of udevadm info --export-db for both VMs, it will help understanding why those devices don't get a valid product value. thanks

Changed in plainbox-provider-checkbox:
status: New → Incomplete
assignee: nobody → Jeff Lane (bladernr)
Revision history for this message
Jeff Lane  (bladernr) wrote :

udevadm info dumps from both

Changed in plainbox-provider-checkbox:
status: Incomplete → Confirmed
Revision history for this message
Jeff Lane  (bladernr) wrote :

https://pastebin.canonical.com/152867/

There's a trace from the most recent version of everything (Installed from the daily PPA).

Because of this, disk tests are not running on VMs and anything where {product} is invalid. That seems to be mostly the issue (with the exception of s390)

Changed in plainbox-provider-checkbox:
importance: Undecided → High
milestone: none → 0.27
Changed in plainbox-provider-checkbox:
assignee: Jeff Lane (bladernr) → Sylvain Pineau (sylvain-pineau)
status: Confirmed → In Progress
Changed in plainbox-provider-checkbox:
status: In Progress → Fix Committed
Pierre Equoy (pieq)
Changed in plainbox-provider-checkbox:
status: Fix Committed → Fix Released
Revision history for this message
Jeff Lane  (bladernr) wrote :

Still not working on z/VM and LPARs.

z/VM: https://certification.canonical.com/hardware/201604-21884/submission/110319/test-results/
LPAR: https://certification.canonical.com/hardware/201604-21859/submission/110321/test-results/

That's with the 0.27 version of the provider
plainbox-provider-checkbox 0.27+bzr4303+pkg36~ubuntu16.04.1

I can get you access to both z/VM and LPAR environments or help in debugging in any way I can. I have to set this to critical as it's now impacting live certifications on for z systems.

Changed in plainbox-provider-checkbox:
status: Fix Released → Confirmed
importance: High → Critical
Jeff Lane  (bladernr)
Changed in plainbox-provider-checkbox:
assignee: Sylvain Pineau (sylvain-pineau) → Jeff Lane (bladernr)
status: Confirmed → In Progress
Revision history for this message
Jeff Lane  (bladernr) wrote :

Per request, here's a udev dump from the lpar

Pierre Equoy (pieq)
Changed in plainbox-provider-checkbox:
status: In Progress → Fix Committed
Pierre Equoy (pieq)
Changed in plainbox-provider-checkbox:
milestone: 0.27 → 0.28
Pierre Equoy (pieq)
Changed in plainbox-provider-checkbox:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.