Dependency is not considered in re-run tests

Bug #1449837 reported by Kristin Chuang
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Next Generation Checkbox (CLI)
Confirmed
Wishlist
Unassigned

Bug Description

With Plainbox 0.21, we now have the possibility to re-run failed tests at the end of a session (before submitting it).

However, these re-run tests currently do not consider test case dependencies, which causes some problems.

For example:
A tester failed the following cases and is prompted to re-run these cases before submitting report:
- keys/wireless
- suspend/keys_after_suspend/wireless (this case depends on suspend/suspend_advanced)

Expected re-run scenario:
(1) [If possible] Plainbox triggers system to auto reboot to get clean "before s3" state
(2) keys/wireless is run
(3) suspend/suspend_advanced is run (so that cases which depend on it (i.e. after-suspend cases) do run in an after-suspend environment)
(4) suspend/keys_after_suspend/wireless is run

Actual re-run scenario:
(1) keys/wireless is run
(2) suspend/suspend_advanced is *not* run because it previously passed
(3) suspend/keys_after_suspend/wireless is run directly after keys/wireless

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

I think we will need to consider if we need to have a clean state for re-run job
e.g. reboot to get a real "before suspend" state.

As bug 1364861 was marked as Won't fix, I will leave this bug here.

Changed in checkbox-ng:
status: New → Confirmed
description: updated
description: updated
Changed in checkbox-ng:
importance: Undecided → Wishlist
milestone: none → future
Revision history for this message
Pierre Equoy (pieq) wrote :

This issue is probably connected to lp:1368001, however I don't think it's a duplicate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.