Ubuntu
webbrowser-app package

contextual actions tests in QmlTests::UbuntuWebView02 are flaky

Bug #1599630 reported by Olivier Tilloy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
webbrowser-app (Ubuntu)
Fix Released
High
Olivier Tilloy

Bug Description

I’m seeing intermittent failures in CI builds, e.g.:

https://jenkins.canonical.com/system-apps/job/build-2-binpkg/arch=amd64,release=vivid+overlay/778/console

https://jenkins.canonical.com/system-apps/job/build-2-binpkg/766/arch=amd64,release=vivid+overlay/console

https://jenkins.canonical.com/system-apps/job/build-2-binpkg/766/arch=amd64,release=xenial+overlay/console

All failures, when they happen, look like this:

FAIL! : QmlTests::UbuntuWebView02::test_contextual_actions() property loading
   Actual (): true
   Expected (): false
   Loc: [/«BUILDDIR»/…/tests/unittests/qml/tst_UbuntuWebView02.qml(98)]

IIRC I’ve seen similar failures on amd64 and i386, but not on armhf.

Related branches

lp:~osomon/webbrowser-app/increase-test-timeout
Rejected for merging into lp:webbrowser-app
Olivier Tilloy: Disapprove
system-apps-ci-bot: Needs Fixing (continuous-integration)
Diff: 30 lines (+3/-3)
1 file modified
tests/unittests/qml/tst_UbuntuWebView02.qml (+3/-3)
lp:~osomon/webbrowser-app/fix-1599630-increase-timeout
Olivier Tilloy: Approve
system-apps-ci-bot: Approve (continuous-integration)
Diff: 124 lines (+35/-17)
2 files modified
tests/unittests/qml/WebbrowserTestCase.qml (+13/-1)
tests/unittests/qml/tst_UbuntuWebView02.qml (+22/-16)
Olivier Tilloy (osomon)
Changed in webbrowser-app (Ubuntu):
status: New → In Progress
assignee: nobody → Olivier Tilloy (osomon)
importance: Undecided → High
Revision history for this message
Olivier Tilloy (osomon) wrote :

I haven’t been able to reproduce the failures locally in an amd64 vivid+overlay schroot like the one used by the system-apps jenkins instance.

Revision history for this message
Olivier Tilloy (osomon) wrote :

Done a few tests, and it *seems* that increasing the timeout does the trick. Two successful builds in a row with a timeout of 120 seconds, I’m now going to test again with 60 seconds.

Revision history for this message
Olivier Tilloy (osomon) wrote :

Got one failure (out of 9 binary builds) with a timeout of 60 seconds, bumping to 120 seconds again.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package webbrowser-app - 0.23+16.10.20160711-0ubuntu1

---------------
webbrowser-app (0.23+16.10.20160711-0ubuntu1) yakkety; urgency=medium

  [ Florian Boucault ]
  * Debian packaging crossbuild fixes:

  [ Olivier Tilloy ]
  * Create Downloads folder if it doesn’t exist. (LP: #1599436)
  * Increase timeout a lot to cope with flaky tests in CI. (LP:
    #1599630)

 -- Olivier Tilloy <email address hidden> Mon, 11 Jul 2016 15:20:04 +0000

Changed in webbrowser-app (Ubuntu):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.