Merge lp:~pitti/messaging-app/phonesim-tests into lp:messaging-app

Proposed by Martin Pitt on 2013-10-15
Status: Merged
Approved by: Tiago Salem Herrmann on 2013-11-04
Approved revision: 67
Merged at revision: 55
Proposed branch: lp:~pitti/messaging-app/phonesim-tests
Merge into: lp:messaging-app
Diff against target: 334 lines (+248/-9)
5 files modified
debian/control (+9/-0)
tests/autopilot/messaging_app/tests/__init__.py (+16/-0)
tests/autopilot/messaging_app/tests/test_messaging.py (+162/-9)
tests/autopilot/messaging_app/tests/test_ui.py (+49/-0)
tests/autopilot/messaging_app_connected/tests/__init__.py (+12/-0)
To merge this branch: bzr merge lp:~pitti/messaging-app/phonesim-tests
Reviewer Review Type Date Requested Status
Tiago Salem Herrmann (community) 2013-10-15 Approve on 2013-11-04
PS Jenkins bot continuous-integration Approve on 2013-11-04
Review via email: mp+191121@code.launchpad.net

Commit message

test_messaging.py: Cover SMS send/receive through ofono-phonesim.

Description of the change

This extends test_messaging.py to actually test a complete SMS send and
receive, using ofono-phonesim-autostart. The tests will be skipped (not fail)
if ofono-phonesim is not running. However, to make CI testing as simple as
possible I added a dependency to ofono-phonesim-autostart to the -autopilot
package, and a conflict to the -connected-autostart package.

To post a comment you must log in.
Martin Pitt (pitti) wrote :

This currently contains two helper methods:

 * go_back() for pressing the toolbar's "Back" button. This isn't i18n safe, and should eventually move into the Ubuntu toolkit emulator. I filed bug 1239751 about that yesterday, and Leo proposed a fix for that. Once that lands, we can drop this.

 * close_osk() for closing the on-screen keyboard. This is quite ugly, and I filed bug 1239753 for a better and generic solution instead of replicating something like that in each tests. This is currently being investigated.

PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Martin Pitt (pitti) wrote :

otto test failed with:

/var/local/autopilot/setup.log: The following packages have unmet dependencies:
/var/local/autopilot/setup.log: messaging-app-autopilot : Depends: ofono-phonesim-autostart but it is not going to be installed

That's because the otto test tries to install both messaging-app-autopilot and messaging-app-connected-autopilot, and the Conflicts: prevents that. -connected-autopilot isn't going to be useful on otto as that needs configuration and hardware.

The generic-mediumtests failures just seem to be a parent test of otto, there's no other failure in its logs.

I can't see an error message in the mako/maguro logs, but presumably they have the same problem at some point.

Any ideas here? Should I just drop the Conflicts:? Are we actually running the -connected-autopilot tests anywhere in CI with some real configuration/SIM hardware? If so, could we perhaps split the tests so that only one of these is installed?

Martin Pitt (pitti) wrote :

Discussed with Francis on IRC. I dropped the conflicts:, as our current CI machinery always installs all binaries, and does not have any configuration for the connected tests. Please note that the connected tests immediately crash with

  File "/home/phablet/messaging-app/tests/autopilot/messaging_app_connected/tests/__init__.py", line 43, in <module>
    class MessagingAppConnectedTestCase(MessagingAppTestCase):
  File "/home/phablet/messaging-app/tests/autopilot/messaging_app_connected/tests/__init__.py", line 52, in MessagingAppConnectedTestCase
    PHONE_NUMBER = config.get('connected_variables', 'dial_number')
  File "/usr/lib/python2.7/ConfigParser.py", line 607, in get
    raise NoSectionError(section)
NoSectionError: No section: 'connected_variables'

if there is no actual configuration file. This should be cleaned up, but the test is currently written in a way which runs a lot of static code which doesn't get skipped by testtools.skipIf() properly.

PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Martin Pitt (pitti) wrote :

It seems the tests already failed on the same issue (toolbar doesn't come up) before, like in
https://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2470/?

Not sure why it doesn't work. Is there any way to reproduce the CI test environment on the phone to reproduce this?

Gustavo Pichorim Boiko (boiko) wrote :

> It seems the tests already failed on the same issue (toolbar doesn't come up)
> before, like in
> https://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2470/?
>
> Not sure why it doesn't work. Is there any way to reproduce the CI test
> environment on the phone to reproduce this?

The autopilot tests are failing indeed, that's due to this bug:
https://bugs.launchpad.net/unity-mir/+bug/1238062

PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Martin Pitt (pitti) wrote :

Note to self: needs ofono-setup fix like http://bazaar.launchpad.net/~pitti/dialer-app/enable-phonesim-tests/revision/72 for amd64/otto/desktop environment.

PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:61
http://jenkins.qa.ubuntu.com/job/messaging-app-ci/83/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-saucy/5083/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-touch/3028/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-amd64-ci/83
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-armhf-ci/83
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-armhf-ci/83/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-i386-ci/83
    FAILURE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-saucy/1251/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-amd64/958
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-amd64/958/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-armhf/3030
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-armhf/3030/artifact/work/output/*zip*/output.zip
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2536
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2584/console
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/96
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/95

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/83/rebuild

review: Needs Fixing (continuous-integration)
PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:64
http://jenkins.qa.ubuntu.com/job/messaging-app-ci/84/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-saucy/5084/console
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-touch/3029
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-amd64-ci/84
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-armhf-ci/84
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-armhf-ci/84/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-i386-ci/84
    FAILURE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-saucy/1252/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-amd64/959
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-amd64/959/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-armhf/3031
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-armhf/3031/artifact/work/output/*zip*/output.zip
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2537
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2588
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/100
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/101

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/84/rebuild

review: Needs Fixing (continuous-integration)
PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:67
http://jenkins.qa.ubuntu.com/job/messaging-app-ci/85/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-saucy/5093/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-touch/3038/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-amd64-ci/85
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-armhf-ci/85
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-armhf-ci/85/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-saucy-i386-ci/85
    FAILURE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-saucy/1259/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-amd64/968
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-amd64/968/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-armhf/3040
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-saucy-armhf/3040/artifact/work/output/*zip*/output.zip
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2545/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2596/console
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/118
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/116

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/85/rebuild

review: Needs Fixing (continuous-integration)
PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:67
http://jenkins.qa.ubuntu.com/job/messaging-app-ci/86/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty/52/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty-touch/52
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-amd64-ci/1
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/1
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/1/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-i386-ci/1
    FAILURE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/46/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/52
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/52/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/52
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/52/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2683
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2734
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/405
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/406

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/86/rebuild

review: Needs Fixing (continuous-integration)
Martin Pitt (pitti) wrote :

I think I fixed ofono-phonesim-autostart hard enough now, at least http://10.97.0.26:8080/job/autopilot-testrunner-otto-trusty/105/console finally succeeded \o/. So trying once more..

PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:67
http://jenkins.qa.ubuntu.com/job/messaging-app-ci/87/
Executed test runs:
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty/120
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty-touch/114/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-amd64-ci/2
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/2
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/2/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-i386-ci/2
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/113
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/120
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/120/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/114
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/114/artifact/work/output/*zip*/output.zip
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2740/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2791
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/520
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/519

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/87/rebuild

review: Needs Fixing (continuous-integration)
PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:67
http://jenkins.qa.ubuntu.com/job/messaging-app-ci/88/
Executed test runs:
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty/123
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty-touch/117/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-amd64-ci/3
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/3
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/3/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-i386-ci/3
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/115
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/123
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/123/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/117
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/117/artifact/work/output/*zip*/output.zip
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2745/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2796
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/530
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/529

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/88/rebuild

review: Needs Fixing (continuous-integration)
Martin Pitt (pitti) wrote :

So maguro reproducibly fails because unity dies. To at least get tests running on mako, I'll skip the tests on maguro. That's better than blocking the landing of tests forever on all platforms.

PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:67
http://jenkins.qa.ubuntu.com/job/messaging-app-ci/89/
Executed test runs:
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty/150
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty-touch/144/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-amd64-ci/4
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/4
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/4/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-i386-ci/4
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/141
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/150
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/150/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/144
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/144/artifact/work/output/*zip*/output.zip
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2769/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2821
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/578
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/580

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/89/rebuild

review: Needs Fixing (continuous-integration)
Martin Pitt (pitti) wrote :

So the otto failure of making the SMS text entry field active also happens in the new dead-simple TestUI test, which doesn't involve any phonesim magic. I ran this in saucy live desktop environment in kvm which I upgraded to trusty, so I figure I somehow need to recreate the CI otto environment to investigate this.

PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:67
http://jenkins.qa.ubuntu.com/job/messaging-app-ci/90/
Executed test runs:
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty/152
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty-touch/146
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-amd64-ci/5
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/5
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/5/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-i386-ci/5
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/143
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/152
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/152/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/146
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/146/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2771
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2822
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/581
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/582

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/90/rebuild

review: Needs Fixing (continuous-integration)
Martin Pitt (pitti) wrote :

Yay, I finally reproduced it locally! It's the unity-daily PPA, when I dist-upgrade to that and run the test, I get exactly the same error. So something in unity-next breaks messaging-app on desktop, and the tests now work on phone. Can we land this, please?

Martin Pitt (pitti) wrote :

Proof: https://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/148/? ran on this exact branch, the only difference is that this ran without the unity-daily PPA.

Tiago Salem Herrmann (tiagosh) wrote :

I can confirm the sdk in ubuntu-daily ppa is broken.
All the views start with the toolbar open, and that makes the autopilot test fail.
It works fine on saucy though.

review: Approve
PS Jenkins bot (ps-jenkins) wrote :

FAILED: Autolanding.
More details in the following jenkins job:
http://jenkins.qa.ubuntu.com/job/messaging-app-autolanding/42/
Executed test runs:
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty/166
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty-touch/160
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-amd64-autolanding/1
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-autolanding/1
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-autolanding/1/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-i386-autolanding/1
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/157
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/166
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/166/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/160
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/160/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2784
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2835
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/608
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/607

review: Needs Fixing (continuous-integration)
Martin Pitt (pitti) wrote :

As that is a real failure and not a problem with this test or messaging-app itself, could this be landed despite the failure, so that the daily-migration has something to block on? Or do you want to wait until the toolkit regresssion is fixed?

PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:67
http://jenkins.qa.ubuntu.com/job/messaging-app-ci/91/
Executed test runs:
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty/320
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty-touch/308
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-amd64-ci/6
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/6
        deb: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-armhf-ci/6/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/messaging-app-trusty-i386-ci/6
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/301
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/320
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/320/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/308
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/308/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2925
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2976
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/897
    SUCCESS: http://10.97.0.26:8080/job/touch-flash-device/896

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/91/rebuild

review: Needs Fixing (continuous-integration)
Martin Pitt (pitti) wrote :

It seems the changed toolbar behaviour is actually intentional: bug 1207369 and also bug 1182734.

I'll update the test instead.

67. By Martin Pitt on 2013-11-04

get along with new toolbar behaviour to always open by default

PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Tiago Salem Herrmann (tiagosh) wrote :

Thanks for taking time to fix the tests.
It looks good to me now.
+1

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'debian/control'
2--- debian/control 2013-09-05 05:05:05 +0000
3+++ debian/control 2013-11-04 13:13:28 +0000
4@@ -41,8 +41,12 @@
5 messaging-app (>= ${binary:Version}),
6 python-autopilot,
7 ubuntu-ui-toolkit-autopilot,
8+ ofono-phonesim-autostart,
9 Description: autopilot tests for messaging-app
10 This package contains the autopilot tests for messaging-app
11+ .
12+ These tests run against the ofono phone simulator (pulled in through
13+ ofono-phonesim-autostart).
14
15 Package: messaging-app-connected-autopilot
16 Architecture: all
17@@ -54,3 +58,8 @@
18 ubuntu-ui-toolkit-autopilot,
19 Description: connected autopilot tests for messaging-app
20 This package contains the connected autopilot tests for messaging-app
21+ .
22+ These tests run against real SIM cards/hardware and need to be configured
23+ manually before they work. Do not install ubuntu-ui-toolkit-autopilot with
24+ these!
25+
26
27=== modified file 'tests/autopilot/messaging_app/tests/__init__.py'
28--- tests/autopilot/messaging_app/tests/__init__.py 2013-08-21 22:26:13 +0000
29+++ tests/autopilot/messaging_app/tests/__init__.py 2013-11-04 13:13:28 +0000
30@@ -19,11 +19,27 @@
31 from messaging_app import emulators
32
33 import os
34+import sys
35 from time import sleep
36 import logging
37+import subprocess
38
39 logger = logging.getLogger(__name__)
40
41+# ensure we have an ofono account; we assume that we have these tools,
42+# otherwise we consider this a test failure (missing dependencies)
43+def tp_has_ofono():
44+ mc_tool = subprocess.Popen(['mc-tool', 'list'], stdout=subprocess.PIPE,
45+ universal_newlines=True)
46+ mc_accounts = mc_tool.communicate()[0]
47+ return 'ofono/ofono/account' in mc_accounts
48+
49+if not tp_has_ofono():
50+ subprocess.check_call(['ofono-setup'])
51+ if not tp_has_ofono():
52+ sys.stderr.write('ofono-setup failed to create ofono account!\n')
53+ sys.exit(1)
54+
55
56 class MessagingAppTestCase(AutopilotTestCase):
57 """A common test case class that provides several useful methods for
58
59=== modified file 'tests/autopilot/messaging_app/tests/test_messaging.py'
60--- tests/autopilot/messaging_app/tests/test_messaging.py 2013-09-24 18:22:36 +0000
61+++ tests/autopilot/messaging_app/tests/test_messaging.py 2013-11-04 13:13:28 +0000
62@@ -7,33 +7,186 @@
63 # under the terms of the GNU General Public License version 3, as published
64 # by the Free Software Foundation.
65
66-"""Tests for the Messaging App"""
67+"""Tests for the Messaging App using ofono-phonesim"""
68
69 from __future__ import absolute_import
70
71+import subprocess
72+import os
73+import tempfile
74+import shutil
75+import time
76+
77+import dbus
78+
79 from autopilot.matchers import Eventually
80 from testtools.matchers import Equals
81+from testtools import skipIf, skipUnless
82
83 from messaging_app.tests import MessagingAppTestCase
84
85-
86+# determine whether we are running with phonesim
87+try:
88+ out = subprocess.check_output(["/usr/share/ofono/scripts/list-modems"],
89+ stderr=subprocess.PIPE)
90+ have_phonesim = out.startswith("[ /phonesim ]")
91+except subprocess.CalledProcessError:
92+ have_phonesim = False
93+
94+
95+@skipUnless(have_phonesim,
96+ "this test needs to run under with-ofono-phonesim")
97+@skipIf(os.uname()[2].endswith("maguro"),
98+ "tests cause Unity crashes on maguro")
99 class TestMessaging(MessagingAppTestCase):
100 """Tests for the communication panel."""
101
102 def setUp(self):
103+ # provide clean history
104+ self.history = os.path.expanduser(
105+ "~/.local/share/history-service/history.sqlite")
106+ if os.path.exists(self.history):
107+ os.rename(self.history, self.history + ".orig")
108+ subprocess.call(["pkill", "history-daemon"])
109+ subprocess.call(["pkill", "-f", "telephony-service-handler"])
110+
111 super(TestMessaging, self).setUp()
112
113- def test_click_new_message_button(self):
114- self.main_view.open_toolbar()
115- toolbar = self.main_view.get_toolbar()
116- toolbar.click_button("newMessageButton")
117+ # no initial messages
118+ self.thread_list = self.app.select_single(objectName="threadList")
119+ self.assertThat(self.thread_list.visible, Equals(True))
120+ self.assertThat(self.thread_list.count, Equals(0))
121+
122+ def tearDown(self):
123+ super(TestMessaging, self).tearDown()
124+
125+ # restore history
126+ try:
127+ os.unlink(self.history)
128+ except OSError:
129+ pass
130+ if os.path.exists(self.history + ".orig"):
131+ os.rename(self.history + ".orig", self.history)
132+ subprocess.call(["pkill", "history-daemon"])
133+ subprocess.call(["pkill", "-f", "telephony-service-handler"])
134+
135+ # on desktop, notify-osd may generate persistent popups (like for "SMS
136+ # received"), don't make that stay around for the tests
137+ subprocess.call(["pkill", "-f", "notify-osd"])
138
139 def test_write_new_message(self):
140- self.test_click_new_message_button()
141- self.assertThat(self.main_view.get_pagestack().depth, Eventually(Equals(2)))
142- self.assertThat(self.main_view.get_messages_page().visible, Eventually(Equals(True)))
143+ self.click_new_message_button()
144+
145+ # type address number
146 text_entry = self.main_view.get_newmessage_textfield()
147 text_entry.activeFocus.wait_for(True)
148 self.keyboard.type("123")
149 self.assertThat(text_entry.text, Eventually(Equals("123")))
150
151+ # type message
152+ text_entry = self.main_view.get_newmessage_textarea()
153+ self.pointing_device.click_object(text_entry)
154+ # need to click twice, first time to hide the toolbar (LP#1207369)
155+ self.pointing_device.click_object(text_entry)
156+ text_entry.activeFocus.wait_for(True)
157+ message = "hello from Ubuntu"
158+ self.keyboard.type(message)
159+ self.assertThat(text_entry.text, Eventually(Equals(message)))
160+
161+ # send
162+ button = self.main_view.get_send_button()
163+ self.assertThat(button.enabled, Eventually(Equals(True)))
164+ self.pointing_device.click_object(button)
165+ self.assertThat(button.enabled, Eventually(Equals(False)))
166+
167+ # verify that we get a bubble with our message
168+ messages_page = self.main_view.select_single(objectName="messagesPage")
169+ list_view = messages_page.select_single("MultipleSelectionListView")
170+ self.assertThat(list_view.count, Eventually(Equals(1)))
171+ l = list_view.select_single("Label", text="hello from Ubuntu")
172+ self.assertNotEqual(l, None)
173+
174+ # switch back to main page with thread list
175+ self.close_osk()
176+ self.go_back()
177+ self.assertThat(self.thread_list.visible, Eventually(Equals(True)))
178+
179+ # should show our message in the thread list
180+ self.assertThat(self.thread_list.count, Equals(1))
181+ # should show address number
182+ l = self.thread_list.select_single("Label", text="123")
183+ self.assertNotEqual(l, None)
184+ # should show text
185+ l = self.thread_list.select_single("Label", text="hello from Ubuntu")
186+ self.assertNotEqual(l, None)
187+
188+ def test_receive_message(self):
189+ self.receive_sms("0815", "hello to Ubuntu")
190+
191+ self.assertThat(self.thread_list.count, Eventually(Equals(1)))
192+ l = self.thread_list.select_single("Label", text="0815")
193+ self.assertNotEqual(l, None)
194+ time.sleep(1) # make it visible to human users for a sec
195+ # should show text
196+ l = self.thread_list.select_single("Label", text="hello to Ubuntu")
197+ self.assertNotEqual(l, None)
198+
199+ #
200+ # Helper methods
201+ #
202+
203+ def click_new_message_button(self):
204+ """Click "New message" menu button and wait for "New message" page"""
205+
206+ self.main_view.open_toolbar()
207+ toolbar = self.main_view.get_toolbar()
208+ toolbar.click_button("newMessageButton")
209+ self.assertThat(self.main_view.get_pagestack().depth,
210+ Eventually(Equals(2)))
211+ self.assertThat(self.main_view.get_messages_page().visible,
212+ Eventually(Equals(True)))
213+ self.assertThat(self.thread_list.visible, Equals(False))
214+
215+ def close_osk(self):
216+ """Swipe down to close on-screen keyboard"""
217+
218+ # TODO: hack! this belongs into the Ubuntu UI toolkit emulator,
219+ # LP#1239753
220+ x1, y1, x2, y2 = self.main_view.globalRect
221+ mid_x = (x2 - x1) // 2
222+ mid_y = (y2 - y1) * 7 // 10
223+ self.pointing_device.drag(mid_x, mid_y, mid_x, y2)
224+ time.sleep(1)
225+
226+ def go_back(self):
227+ """Click back button from toolbar"""
228+
229+ # will fail with i18n; this belongs into the Ubuntu UI toolkit
230+ # emulator, LP#1239751
231+ self.main_view.open_toolbar()
232+ toolbar = self.main_view.get_toolbar()
233+ back_button = toolbar.select_single("ActionItem", text=u"Back")
234+ self.assertNotEqual(back_button, None)
235+ self.pointing_device.click_object(back_button)
236+
237+ def receive_sms(self, sender, text):
238+ """Receive an SMS"""
239+
240+ # prepare and send a Qt GUI script to phonesim, over its private D-BUS
241+ # set up by ofono-phonesim-autostart
242+ script_dir = tempfile.mkdtemp(prefix="phonesim_script")
243+ os.chmod(script_dir, 0o755)
244+ self.addCleanup(shutil.rmtree, script_dir)
245+ with open(os.path.join(script_dir, "sms.js"), "w") as f:
246+ f.write("""tabSMS.gbMessage1.leMessageSender.text = "%s";
247+tabSMS.gbMessage1.leSMSClass.text = "1";
248+tabSMS.gbMessage1.teSMSText.setPlainText("%s");
249+tabSMS.gbMessage1.pbSendSMSMessage.click();
250+""" % (sender, text))
251+
252+ with open("/run/lock/ofono-phonesim-dbus.address") as f:
253+ phonesim_bus = f.read().strip()
254+ bus = dbus.bus.BusConnection(phonesim_bus)
255+ script_proxy = bus.get_object("org.ofono.phonesim", "/")
256+ script_proxy.SetPath(script_dir)
257+ script_proxy.Run("sms.js")
258
259=== added file 'tests/autopilot/messaging_app/tests/test_ui.py'
260--- tests/autopilot/messaging_app/tests/test_ui.py 1970-01-01 00:00:00 +0000
261+++ tests/autopilot/messaging_app/tests/test_ui.py 2013-11-04 13:13:28 +0000
262@@ -0,0 +1,49 @@
263+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
264+# Copyright 2013 Canonical
265+#
266+# This file is part of messaging-app.
267+#
268+# messaging-app is free software: you can redistribute it and/or modify it
269+# under the terms of the GNU General Public License version 3, as published
270+# by the Free Software Foundation.
271+
272+"""Pure UI tests for Messaging App"""
273+
274+from __future__ import absolute_import
275+
276+from autopilot.matchers import Eventually
277+from testtools.matchers import Equals
278+
279+from messaging_app.tests import MessagingAppTestCase
280+
281+
282+class TestUI(MessagingAppTestCase):
283+ def test_new_message_panel(self):
284+ """New message panel comes up with the toolbar button"""
285+
286+ # Click "New message" menu button
287+ self.main_view.open_toolbar()
288+ toolbar = self.main_view.get_toolbar()
289+ toolbar.click_button("newMessageButton")
290+
291+ # wait for "New message" page
292+ self.assertThat(self.main_view.get_pagestack().depth,
293+ Eventually(Equals(2)))
294+ self.assertThat(self.main_view.get_messages_page().visible,
295+ Eventually(Equals(True)))
296+
297+ # type address number
298+ text_entry = self.main_view.get_newmessage_textfield()
299+ text_entry.activeFocus.wait_for(True)
300+ self.keyboard.type("123")
301+ self.assertThat(text_entry.text, Eventually(Equals("123")))
302+
303+ # type message
304+ text_entry = self.main_view.get_newmessage_textarea()
305+ self.pointing_device.click_object(text_entry)
306+ # need to click twice, first time to hide the toolbar (LP#1207369)
307+ self.pointing_device.click_object(text_entry)
308+ text_entry.activeFocus.wait_for(True)
309+ message = "hello from Ubuntu"
310+ self.keyboard.type(message)
311+ self.assertThat(text_entry.text, Eventually(Equals(message)))
312
313=== modified file 'tests/autopilot/messaging_app_connected/tests/__init__.py'
314--- tests/autopilot/messaging_app_connected/tests/__init__.py 2013-09-25 20:24:30 +0000
315+++ tests/autopilot/messaging_app_connected/tests/__init__.py 2013-11-04 13:13:28 +0000
316@@ -19,8 +19,20 @@
317
318 import os
319 import shutil
320+import subprocess
321 import ConfigParser
322
323+# determine whether we are running with phonesim
324+try:
325+ out = subprocess.check_output(["/usr/share/ofono/scripts/list-modems"],
326+ stderr=subprocess.PIPE)
327+ if out.startswith("[ /phonesim ]"):
328+ print("ofono-phonesim is currently running, but this test is meant "
329+ "to run against real hardware. Please uninstall "
330+ "ofono-phonesim-autostart or run 'sudo stop ofono-phonesim'.")
331+except subprocess.CalledProcessError:
332+ pass
333+
334 config_file = os.path.expanduser('~/.testnumbers.cfg')
335
336

Subscribers

People subscribed via source and target branches