[UIFe] Interface asks about skipping tests that are automatically marked Yes

I'm going to mark this confirmed and set it to critical. I've noticed this myself in my own testing.

Can you confirm something for me, however. Checkbox 0.13 for Precise has a new Qt based interface. Can you confirm that this is the interface you are using? I do not believe the issue you mention is present in the older gtk interface. I have seen this behaviour myself with other tests using the Qt branch.

So let us know which interface you're running. The Qt one should be obvious (look for the grey bar at the top of the window, tabs int he middle of the window, colored circles highlighting each test step, etc).

I haven't seen the old GTK interface, and I'm away from the system where my report originated, but I believe I got the QT interface. The system was running a fresh 12.04 beta 2 installation, not an upgrade. Checkbox looked like the screenshot in bug 967445

Yeah, you're seeing the Qt interface. Ok, I've also added this to the upstream checkbox project and set it critical there as well. I'll have to do a run through again using the latest PPA version to double check, but I have seen exactly this issue before.

Here is a screenshot showing this...

This screenshot shows the MMC test with Yes highlighted. I haven't actually clicked "Test" the "Yes" button is highlighted after successfully completing a previous test and the button is NOT cleared out after moving on.

Here's the next one. After clicking "Next" because the UI confusingly informed me that I'd already tested this (The "Yes" was highlighted as though it had successfully passed, even though I never clicked "Test" to initiate the MMC Insert test) I got the popup asking me if I want to skip the test.

The buttons are not being reset from selected to unselected between tests. This can lead to the confusion that the OP mentioned and I have also observed.

To recreate this, just run checkbox-qt, leave all the tests selected and keep clicking yes. Click the "Test" button and you should see a test run then the Yes highlighted. Then click Next and you'll get the popup. Clicking the "Yes" button moves you on to the next test, but leaves Yes highlighted, which is creating the confusion.

Well, the problem here is that the 'Yes' button is merely focused and never actually clicked. Clicking 'Next' always implies you want to skip a test. Perhaps the description is the culprit here? Unfortunately we can't write a description that makes sense for both UI's though. The best workaround I can come up with is to intercept the click event on 'Next' and check if 'Yes' or 'No' is focused and then pass on the click event to the relevant button.

As per a discussion we had, the easiest workaround seems to be removing the "click on Next to proceed" step in test instructions. This may require a UIFe since we are changing some strings (though just removing that portion of the text).

I'll propose a merge request with those changes in a moment.

Presumably I'm to click the "Yes" or "No" button, which ever one the automated test selects? If so, you'll have to tell me to do that; otherwise, I'll just sit and stare, wondering whether the test is finished yet. I'm not necessarily going to notice that one of the buttons has been selected, let alone grasp the significance.

I also was confused by the phrase "the automatically selected result". I thought that the "results" are what I'm shown when I click the "results" button, and I couldn't see any way to change it ... If the last bullet point had the word "button" in it, I would have understood.

If the new message is needed to fix the functionality in a critical bug, then +1 from the translations team.

Please send a notification e-mail to the ubuntu-translators and ubuntu-docs team as per the UIFe procedure.

Thanks!

We notified ubuntu-translators, here's the thread:

https://lists.ubuntu.com/archives/ubuntu-translators/2012-April/005349.html

ubuntu-docs doesn't need to be notified as Checkbox has no documentation :(

Thanks!

Subscribed ubuntu-release and, as outlined in the Freeze Exception procedure, here's a description of the changes:

- Three test descriptions (optical/read, optical/cdrom-write and optical/dvd-write) are changed, by removing the 4th step in the test procedure. This reads 'When the test completes, proceed by clicking "Next"'. This means that the strings to be translated change, which impacts the translation team (see +1 from David Planella on comment #10). No new text is added, so the work to be done involves removing the translation for that step.

Rationale for the change:
- Having a test procedure with erroneous steps is confusing and leads to users skipping the test, which is considered core for Ubuntu Friendly submissions. Removing the erroneous instruction, in addition to the fact that "yes" or "no" are autoselected depending on the outcome of these tests, will hopefully result in less skipped tests and more valid submissions.

I consider that a bug fix. Please go ahead, approved.

This bug was fixed in the package checkbox - 0.13.7

---------------
checkbox (0.13.7) precise; urgency=low

  * New upstream release (LP: #980063)

  * Automated translation file updates from Launchpad

  [Jeff Lane]
  * qt/checkbox-qt.ui, qt/frontend/qtfront.cpp: Converted
    submissionWarningLabel and text to submissionUbuntuFriendlyLabel wtih
    instructional text for submitting results. This is a workaround for the
    bug causing the warning to be displayed at all times rather than only when
    testing is incomplete. (LP: #967457)
  * jobs/networking.txt.in: Removed dhclient call from networking/multi_nic
    tests because of a bug in dhclient that can cause it to hang when run on
    eth0. New test requirement will be that the tester must configure and
    bring up all ethernet devices prior to running checkbox. Also added a
    check to make sure we're not trying to run the test on a device that's not
    active. (LP: #926229)

  [Tiago Salem Herrmann]
  * checkbox_qt/qt_interface.py, qt/frontend/qtfront.cpp,
    qt/frontend/qtfront.h: Do async calls to some ui methods and avoid
    unexpected dbus timeouts (LP: #962333)

  [Sylvain Pineau]
  * qt/frontend/qtfront.cpp: Submit/View results buttons are disabled until
    every selected test has been run (LP: #937715)

  [Daniel Manrique]
  * jobs/optical.txt.in: Change test descriptions to avoid confusing
    instruction to press the "Next" button (which is incorrect). (LP: #971181)
  * jobs/local.txt.in: Fixed touchpad local job which was using suspend.txt
    as the job source) (LP: #979344)
  * jobs/mediacards.txt.in: Added usb and scsi devices to
    removable_storage_test commands (LP: #979356)
 -- Daniel Manrique <email address hidden> Wed, 11 Apr 2012 17:20:37 -0400