test_capture.TestCapture.test_shoot_button_disable autopilot test failure

Bug #1227373 reported by Paul Larson
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
camera-app
Fix Released
High
Günter Schwann
camera-app (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

Image: touch_ro build 56 (and others)

We sometimes see the camera-app tests fail on test_capture.TestCapture.test_shoot_button_disable
Here's a traceback from a recent failure:
21:25:04.845 INFO globals:49 - ************************************************************
21:25:04.846 INFO globals:50 - Starting test camera_app.tests.test_capture.TestCapture.test_shoot_button_disable (with touch)
21:25:04.847 WARNING testcase:118 - No tracing available - install the python-autopilot-trace package!
21:25:04.916 WARNING testcase:155 - Process manager backend unavailable, application snapshot support disabled.
21:25:04.934 INFO __init__:157 - Launching process: ['/usr/bin/camera-app', '-testability', '--fullscreen', '--desktop_file_hint=/usr/share/applications/camera-app.desktop']
21:25:05.311 INFO __init__:404 - dbus.DBusException while attempting to get PID for org.freedesktop.DBus: DBusException("Could not get PID of name 'org.freedesktop.DBus': no such name",)
21:25:06.590 DEBUG dbus:430 - Selecting objects of type QQuickView with attributes: {}
21:25:06.643 DEBUG dbus:430 - Selecting objects of type ShootButton with attributes: {}
21:25:06.679 INFO testcase:553 - waiting for process to exit.
21:25:06.679 INFO testcase:555 - Killing process 2164
}}}

Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/camera_app/tests/test_capture.py", line 145, in test_shoot_button_disable
self.assertEquals(exposure_button.enabled, True)
File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 322, in assertEqual
self.assertThat(observed, matcher, message)
File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 417, in assertThat
raise MismatchError(matchee, matcher, mismatch, verbose)
MismatchError: Boolean(False) != True

Ran 1 test in 2.433s
FAILED (failures=1)

More detailed logs can be found at: https://jenkins.qa.ubuntu.com/job/saucy-touch_ro-maguro-smoke-camera-app-autopilot/79/console

Tags: autopilot

Related branches

Revision history for this message
Sergio Schvezov (sergiusens) wrote :

It's a test that has a high race condition (http://bazaar.launchpad.net/~phablet-team/camera-app/trunk/view/head:/tests/autopilot/camera_app/tests/test_capture.py#L141)

Look at the final 3 lines there:
        self.pointing_device.click()

        self.assertThat(exposure_button.enabled, Eventually(Equals(False)))
        self.assertThat(exposure_button.enabled, Eventually(Equals(True)))

The point of the test seems to:
1- take a photo
2- see that the shutter is disabled (since camera is not ready)
3- reenable shutter (since camera is ready for new snaps)

Observations:
- The time between 2 and 3 can be anything.
- Nothing triggers the final assert, hence it's testing a transition.
- The 'False' assert could already have been done by the time it's 'looked' at.

Revision history for this message
Paul Larson (pwlars) wrote :

Added thomir as asac said that he thought there was a safe way to do this. Thomi - is there a sensible way to do something like this, or is it just too fast of a transition?

Revision history for this message
Olivier Tilloy (osomon) wrote :

The failure observed in the description of this bug happens even before hitting the last three lines of the test. So there are several problems in the test. I think the first one should be rather easy to address, by replacing self.assertEquals(…, True) with self.assertThat(…, Eventually(Equals(True))).

Changed in camera-app:
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Günter Schwann (schwann)
Revision history for this message
Olivier Tilloy (osomon) wrote :

On testing the transition, I can’t think of a safe way to do that.

I’d suggest we remove the test for the shutter disabled, and keep only the one for the shutter enabled after taking a picture (that would make for a more reliable test, however it could also very well be that when the assertion is tested, the shutter hasn’t had the time to be disabled yet, so we’d need to add an arbitrary sleep() in between to increase the meaningfulness of the test, and even then it wouldn’t guarantee much).

Suggestions welcome.

Revision history for this message
Günter Schwann (schwann) wrote :

Not testing for the button to be disabled, would make the whole test obsolete.

So we could just remove or disable that test, and add self.assertThat(exposure_button.enabled, Eventually(Equals(True))) at the end of the take_picture() test

Revision history for this message
Günter Schwann (schwann) wrote :

This is the MR to disable that test, and do the exposure_button.enabled check in the other test
https://code.launchpad.net/~schwann/camera-app/camera-disable-unstable-test/+merge/186753

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

Fix committed into lp:camera-app at revision 196, scheduled for release in camera-app, milestone ubuntu-13.05

Changed in camera-app:
status: Confirmed → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package camera-app - 2.9.1+13.10.20130923-0ubuntu1

---------------
camera-app (2.9.1+13.10.20130923-0ubuntu1) saucy; urgency=low

  [ Guenter Schwann ]
  * Disable test_shoot_button_disable(9 as it is unstable because of a
    race condition. (LP: #1227373)

  [ Ubuntu daily release ]
  * Automatic snapshot from revision 198
 -- Ubuntu daily release <email address hidden> Mon, 23 Sep 2013 07:49:28 +0000

Changed in camera-app (Ubuntu):
status: New → Fix Released
Changed in camera-app:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.