Code review comment for lp:~andrew-haigh-b/cdts/bug1218846

I tested this by merging this branch on checkbox-ihv-ng rev 2278.

I had to change the interface name to com.canonical.certification.PlainBox1 on line 35 of PBNames.h.

Then I observed some weird behavior (against plainbox from checkbox trunk rev 2335).

I started plainbox service, then the gui client. I selected only the default whitelist.

I deselected all the tests, then selected all the camera tests. This contains at least two manual tests (camera/still and camera/display). Then I started the test run.

camera/display runs first and I get a live video window but the manual test window that pops up is for camera/still. Once the live video window closes, I get another capture window but for a static image (meaning, that's the command for camera/still). An outcome of "No" is selected once the static image closes. Then I click on "continue" and the client crashes with a message (following).

So first, it seems to be running the command for a manual job as soon as it hits the manual job (not waiting for me to press "test"). Then capturing the test outcome seems to be wrong. Finally, it shouldn't crash at the end :(

uiEngine::PrepareJobs()

Time for summary: 29
Start Testing
Running Job "package"
Real outcome "pass"
Running Job "device"
Real outcome "pass"
Running Job "camera/detect"
Real outcome "pass"
Running Job "camera/display"
GuiEngine::CatchallAskForOutcomeSignalsHandler
Real outcome ""
GuiEngine::CatchallAskForOutcomeSignalsHandler - Done
Real outcome "none"
Running Job "camera/still"
GuiEngine::CatchallAskForOutcomeSignalsHandler
Real outcome ""
updateManualInteractionDialog
GuiEngine::CatchallAskForOutcomeSignalsHandler - Done
Real outcome "none"
onJobsCompleted
Continue
GuiEngine::ResumeFromManualInteraction()
GuiEngine::SetOutcome
GuiEngine::SetOutcome - Done
GuiEngine::ResumeFromManualInteraction()
Real outcome "pass"
ASSERT failure in QList<T>::at: "index out of range", file /usr/include/qt5/QtCore/qlist.h, line 454
The program has unexpectedly finished.
/media/bulk/checkboxes/checkbox-ihv-ng/trunk/build-plainbox-gui-Desktop-Debug/driver-testing/driver-testing exited with code 0

I was able to reproduce this with stubbox leaving all three whitelists selected, it just blasts through all the manual test windows and then crashes when I click on "continue" on the manual test window, which shows stub/dependency/good (it shouldn't as that's an automated job).