Merge into trunk : phonesim-tests : Code : messaging-app

Status:	Merged
Approved by:	Tiago Salem Herrmann on 2013-11-04
Approved revision:	67
Merged at revision:	55
Proposed branch:	lp:~pitti/messaging-app/phonesim-tests
Merge into:	lp:messaging-app
Diff against target:	334 lines (+248/-9) 5 files modified debian/control (+9/-0) tests/autopilot/messaging_app/tests/__init__.py (+16/-0) tests/autopilot/messaging_app/tests/test_messaging.py (+162/-9) tests/autopilot/messaging_app/tests/test_ui.py (+49/-0) tests/autopilot/messaging_app_connected/tests/__init__.py (+12/-0)
To merge this branch:	bzr merge lp:~pitti/messaging-app/phonesim-tests
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
Tiago Salem Herrmann (community)		2013-10-15	Approve on 2013-11-04
PS Jenkins bot	continuous-integration		Approve on 2013-11-04
Review via email: mp+191121@code.launchpad.net

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-15:

#

This currently contains two helper methods:

* go_back() for pressing the toolbar's "Back" button. This isn't i18n safe, and should eventually move into the Ubuntu toolkit emulator. I filed bug 1239751 about that yesterday, and Leo proposed a fix for that. Once that lands, we can drop this.

* close_osk() for closing the on-screen keyboard. This is quite ugly, and I filed bug 1239753 for a better and generic solution instead of replicating something like that in each tests. This is currently being investigated.

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-15:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/79/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-15:

#

otto test failed with:

/var/local/autopilot/setup.log: The following packages have unmet dependencies:
/var/local/autopilot/setup.log: messaging-app-autopilot : Depends: ofono-phonesim-autostart but it is not going to be installed

That's because the otto test tries to install both messaging-app-autopilot and messaging-app-connected-autopilot, and the Conflicts: prevents that. -connected-autopilot isn't going to be useful on otto as that needs configuration and hardware.

The generic-mediumtests failures just seem to be a parent test of otto, there's no other failure in its logs.

I can't see an error message in the mako/maguro logs, but presumably they have the same problem at some point.

Any ideas here? Should I just drop the Conflicts:? Are we actually running the -connected-autopilot tests anywhere in CI with some real configuration/SIM hardware? If so, could we perhaps split the tests so that only one of these is installed?

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-15:

#

Discussed with Francis on IRC. I dropped the conflicts:, as our current CI machinery always installs all binaries, and does not have any configuration for the connected tests. Please note that the connected tests immediately crash with

  File "/home/phablet/messaging-app/tests/autopilot/messaging_app_connected/tests/__init__.py", line 43, in <module>
    class MessagingAppConnectedTestCase(MessagingAppTestCase):
  File "/home/phablet/messaging-app/tests/autopilot/messaging_app_connected/tests/__init__.py", line 52, in MessagingAppConnectedTestCase
    PHONE_NUMBER = config.get('connected_variables', 'dial_number')
  File "/usr/lib/python2.7/ConfigParser.py", line 607, in get
    raise NoSectionError(section)
NoSectionError: No section: 'connected_variables'

if there is no actual configuration file. This should be cleaned up, but the test is currently written in a way which runs a lot of static code which doesn't get skipped by testtools.skipIf() properly.

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-15:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/80/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-15:

#

It seems the tests already failed on the same issue (toolbar doesn't come up) before, like in
https://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2470/?

Not sure why it doesn't work. Is there any way to reproduce the CI test environment on the phone to reproduce this?

Revision history for this message

Gustavo Pichorim Boiko (boiko) wrote on 2013-10-15:

#

> It seems the tests already failed on the same issue (toolbar doesn't come up)
> before, like in
> https://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/2470/?
>
> Not sure why it doesn't work. Is there any way to reproduce the CI test
> environment on the phone to reproduce this?

The autopilot tests are failing indeed, that's due to this bug:
https://bugs.launchpad.net/unity-mir/+bug/1238062

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-15:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/81/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-15:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/82/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-16:

#

Note to self: needs ofono-setup fix like http://bazaar.launchpad.net/~pitti/dialer-app/enable-phonesim-tests/revision/72 for amd64/otto/desktop environment.

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-17:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/83/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-17:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/84/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-17:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/85/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-24:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/86/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-26:

#

I think I fixed ofono-phonesim-autostart hard enough now, at least http://10.97.0.26:8080/job/autopilot-testrunner-otto-trusty/105/console finally succeeded \o/. So trying once more..

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-26:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/87/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-28:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/88/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-29:

#

So maguro reproducibly fails because unity dies. To at least get tests running on mako, I'll skip the tests on maguro. That's better than blocking the landing of tests forever on all platforms.

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-29:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/89/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-29:

#

So the otto failure of making the SMS text entry field active also happens in the new dead-simple TestUI test, which doesn't involve any phonesim magic. I ran this in saucy live desktop environment in kvm which I upgraded to trusty, so I figure I somehow need to recreate the CI otto environment to investigate this.

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-29:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/90/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-29:

#

Yay, I finally reproduced it locally! It's the unity-daily PPA, when I dist-upgrade to that and run the test, I get exactly the same error. So something in unity-next breaks messaging-app on desktop, and the tests now work on phone. Can we land this, please?

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-29:

#

Proof: https://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/148/? ran on this exact branch, the only difference is that this ran without the unity-daily PPA.

Revision history for this message

Tiago Salem Herrmann (tiagosh) wrote on 2013-10-29:

#

I can confirm the sdk in ubuntu-daily ppa is broken.
All the views start with the toolbar open, and that makes the autopilot test fail.
It works fine on saucy though.

review: Approve

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-10-29:

#

review: Needs Fixing (continuous-integration)

Revision history for this message

Martin Pitt (pitti) wrote on 2013-10-30:

#

As that is a real failure and not a problem with this test or messaging-app itself, could this be landed despite the failure, so that the daily-migration has something to block on? Or do you want to wait until the toolkit regresssion is fixed?

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-11-03:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/91/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Martin Pitt (pitti) wrote on 2013-11-04:

#

It seems the changed toolbar behaviour is actually intentional: bug 1207369 and also bug 1182734.

I'll update the test instead.

lp:~pitti/messaging-app/phonesim-tests updated on 2013-11-04

67. By Martin Pitt on 2013-11-04: get along with new toolbar behaviour to always open by default

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-11-04:

#

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/messaging-app-ci/92/rebuild

review: Approve (continuous-integration)

Revision history for this message

Tiago Salem Herrmann (tiagosh) wrote on 2013-11-04:

#

Thanks for taking time to fix the tests.
It looks good to me now.
+1

review: Approve

messaging-app

Merge lp:~pitti/messaging-app/phonesim-tests into lp:messaging-app

Commit message

Description of the change

Preview Diff

Subscribers

 === modified file 'debian/control'
 --- debian/control	2013-09-05 05:05:05 +0000
 +++ debian/control	2013-11-04 13:13:28 +0000
@@ -41,8 +41,12 @@
           messaging-app (>= ${binary:Version}),
           python-autopilot,
           ubuntu-ui-toolkit-autopilot,
++         ofono-phonesim-autostart,
  Description: autopilot tests for messaging-app
   This package contains the autopilot tests for messaging-app
++ .
++ These tests run against the ofono phone simulator (pulled in through
++ ofono-phonesim-autostart).
  Package: messaging-app-connected-autopilot
  Architecture: all
@@ -54,3 +58,8 @@
           ubuntu-ui-toolkit-autopilot,
  Description: connected autopilot tests for messaging-app
   This package contains the connected autopilot tests for messaging-app
++ .
++ These tests run against real SIM cards/hardware and need to be configured
++ manually before they work. Do not install ubuntu-ui-toolkit-autopilot with
++ these!
++
 === modified file 'tests/autopilot/messaging_app/tests/__init__.py'
 --- tests/autopilot/messaging_app/tests/__init__.py	2013-08-21 22:26:13 +0000
 +++ tests/autopilot/messaging_app/tests/__init__.py	2013-11-04 13:13:28 +0000
@@ -19,11 +19,27 @@
  from messaging_app import emulators
  import os
++import sys
  from time import sleep
  import logging
++import subprocess
  logger = logging.getLogger(__name__)
++# ensure we have an ofono account; we assume that we have these tools,
++# otherwise we consider this a test failure (missing dependencies)
++def tp_has_ofono():
++    mc_tool = subprocess.Popen(['mc-tool', 'list'], stdout=subprocess.PIPE,
++                               universal_newlines=True)
++    mc_accounts = mc_tool.communicate()[0]
++    return 'ofono/ofono/account' in mc_accounts
++
++if not tp_has_ofono():
++    subprocess.check_call(['ofono-setup'])
++    if not tp_has_ofono():
++        sys.stderr.write('ofono-setup failed to create ofono account!\n')
++        sys.exit(1)
++
  class MessagingAppTestCase(AutopilotTestCase):
      """A common test case class that provides several useful methods for
 === modified file 'tests/autopilot/messaging_app/tests/test_messaging.py'
 --- tests/autopilot/messaging_app/tests/test_messaging.py	2013-09-24 18:22:36 +0000
 +++ tests/autopilot/messaging_app/tests/test_messaging.py	2013-11-04 13:13:28 +0000
@@ -7,33 +7,186 @@
  # under the terms of the GNU General Public License version 3, as published
  # by the Free Software Foundation.
--"""Tests for the Messaging App"""
++"""Tests for the Messaging App using ofono-phonesim"""
  from __future__ import absolute_import
++import subprocess
++import os
++import tempfile
++import shutil
++import time
++
++import dbus
++
  from autopilot.matchers import Eventually
  from testtools.matchers import Equals
++from testtools import skipIf, skipUnless
  from messaging_app.tests import MessagingAppTestCase
--
++# determine whether we are running with phonesim
++try:
++    out = subprocess.check_output(["/usr/share/ofono/scripts/list-modems"],
++                                  stderr=subprocess.PIPE)
++    have_phonesim = out.startswith("[ /phonesim ]")
++except subprocess.CalledProcessError:
++    have_phonesim = False
++
++
++@skipUnless(have_phonesim,
++            "this test needs to run under with-ofono-phonesim")
++@skipIf(os.uname()[2].endswith("maguro"),
++        "tests cause Unity crashes on maguro")
  class TestMessaging(MessagingAppTestCase):
      """Tests for the communication panel."""
      def setUp(self):
++        # provide clean history
++        self.history = os.path.expanduser(
++            "~/.local/share/history-service/history.sqlite")
++        if os.path.exists(self.history):
++            os.rename(self.history, self.history + ".orig")
++        subprocess.call(["pkill", "history-daemon"])
++        subprocess.call(["pkill", "-f", "telephony-service-handler"])
++
          super(TestMessaging, self).setUp()
--    def test_click_new_message_button(self):
--        self.main_view.open_toolbar()
--        toolbar = self.main_view.get_toolbar()
--        toolbar.click_button("newMessageButton")
++        # no initial messages
++        self.thread_list = self.app.select_single(objectName="threadList")
++        self.assertThat(self.thread_list.visible, Equals(True))
++        self.assertThat(self.thread_list.count, Equals(0))
++
++    def tearDown(self):
++        super(TestMessaging, self).tearDown()
++
++        # restore history
++        try:
++            os.unlink(self.history)
++        except OSError:
++            pass
++        if os.path.exists(self.history + ".orig"):
++            os.rename(self.history + ".orig", self.history)
++        subprocess.call(["pkill", "history-daemon"])
++        subprocess.call(["pkill", "-f", "telephony-service-handler"])
++
++        # on desktop, notify-osd may generate persistent popups (like for "SMS
++        # received"), don't make that stay around for the tests
++        subprocess.call(["pkill", "-f", "notify-osd"])
      def test_write_new_message(self):
--        self.test_click_new_message_button()
--        self.assertThat(self.main_view.get_pagestack().depth, Eventually(Equals(2)))
--        self.assertThat(self.main_view.get_messages_page().visible, Eventually(Equals(True)))
++        self.click_new_message_button()
++
++        # type address number
          text_entry = self.main_view.get_newmessage_textfield()
          text_entry.activeFocus.wait_for(True)
          self.keyboard.type("123")
          self.assertThat(text_entry.text, Eventually(Equals("123")))
++        # type message
++        text_entry = self.main_view.get_newmessage_textarea()
++        self.pointing_device.click_object(text_entry)
++        # need to click twice, first time to hide the toolbar (LP#1207369)
++        self.pointing_device.click_object(text_entry)
++        text_entry.activeFocus.wait_for(True)
++        message = "hello from Ubuntu"
++        self.keyboard.type(message)
++        self.assertThat(text_entry.text, Eventually(Equals(message)))
++
++        # send
++        button = self.main_view.get_send_button()
++        self.assertThat(button.enabled, Eventually(Equals(True)))
++        self.pointing_device.click_object(button)
++        self.assertThat(button.enabled, Eventually(Equals(False)))
++
++        # verify that we get a bubble with our message
++        messages_page = self.main_view.select_single(objectName="messagesPage")
++        list_view = messages_page.select_single("MultipleSelectionListView")
++        self.assertThat(list_view.count, Eventually(Equals(1)))
++        l = list_view.select_single("Label", text="hello from Ubuntu")
++        self.assertNotEqual(l, None)
++
++        # switch back to main page with thread list
++        self.close_osk()
++        self.go_back()
++        self.assertThat(self.thread_list.visible, Eventually(Equals(True)))
++
++        # should show our message in the thread list
++        self.assertThat(self.thread_list.count, Equals(1))
++        # should show address number
++        l = self.thread_list.select_single("Label", text="123")
++        self.assertNotEqual(l, None)
++        # should show text
++        l = self.thread_list.select_single("Label", text="hello from Ubuntu")
++        self.assertNotEqual(l, None)
++
++    def test_receive_message(self):
++        self.receive_sms("0815", "hello to Ubuntu")
++
++        self.assertThat(self.thread_list.count, Eventually(Equals(1)))
++        l = self.thread_list.select_single("Label", text="0815")
++        self.assertNotEqual(l, None)
++        time.sleep(1)  # make it visible to human users for a sec
++        # should show text
++        l = self.thread_list.select_single("Label", text="hello to Ubuntu")
++        self.assertNotEqual(l, None)
++
++    #
++    # Helper methods
++    #
++
++    def click_new_message_button(self):
++        """Click "New message" menu button and wait for "New message" page"""
++
++        self.main_view.open_toolbar()
++        toolbar = self.main_view.get_toolbar()
++        toolbar.click_button("newMessageButton")
++        self.assertThat(self.main_view.get_pagestack().depth,
++                        Eventually(Equals(2)))
++        self.assertThat(self.main_view.get_messages_page().visible,
++                        Eventually(Equals(True)))
++        self.assertThat(self.thread_list.visible, Equals(False))
++
++    def close_osk(self):
++        """Swipe down to close on-screen keyboard"""
++
++        # TODO: hack! this belongs into the Ubuntu UI toolkit emulator,
++        # LP#1239753
++        x1, y1, x2, y2 = self.main_view.globalRect
++        mid_x = (x2 - x1) // 2
++        mid_y = (y2 - y1) * 7 // 10
++        self.pointing_device.drag(mid_x, mid_y, mid_x, y2)
++        time.sleep(1)
++
++    def go_back(self):
++        """Click back button from toolbar"""
++
++        # will fail with i18n; this belongs into the Ubuntu UI toolkit
++        # emulator, LP#1239751
++        self.main_view.open_toolbar()
++        toolbar = self.main_view.get_toolbar()
++        back_button = toolbar.select_single("ActionItem", text=u"Back")
++        self.assertNotEqual(back_button, None)
++        self.pointing_device.click_object(back_button)
++
++    def receive_sms(self, sender, text):
++        """Receive an SMS"""
++
++        # prepare and send a Qt GUI script to phonesim, over its private D-BUS
++        # set up by ofono-phonesim-autostart
++        script_dir = tempfile.mkdtemp(prefix="phonesim_script")
++        os.chmod(script_dir, 0o755)
++        self.addCleanup(shutil.rmtree, script_dir)
++        with open(os.path.join(script_dir, "sms.js"), "w") as f:
++            f.write("""tabSMS.gbMessage1.leMessageSender.text = "%s";
++tabSMS.gbMessage1.leSMSClass.text = "1";
++tabSMS.gbMessage1.teSMSText.setPlainText("%s");
++tabSMS.gbMessage1.pbSendSMSMessage.click();
++""" % (sender, text))
++
++        with open("/run/lock/ofono-phonesim-dbus.address") as f:
++            phonesim_bus = f.read().strip()
++        bus = dbus.bus.BusConnection(phonesim_bus)
++        script_proxy = bus.get_object("org.ofono.phonesim", "/")
++        script_proxy.SetPath(script_dir)
++        script_proxy.Run("sms.js")
 === added file 'tests/autopilot/messaging_app/tests/test_ui.py'
 --- tests/autopilot/messaging_app/tests/test_ui.py	1970-01-01 00:00:00 +0000
 +++ tests/autopilot/messaging_app/tests/test_ui.py	2013-11-04 13:13:28 +0000
@@ -0,0 +1,49 @@
++# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
++# Copyright 2013 Canonical
++#
++# This file is part of messaging-app.
++#
++# messaging-app is free software: you can redistribute it and/or modify it
++# under the terms of the GNU General Public License version 3, as published
++# by the Free Software Foundation.
++
++"""Pure UI tests for Messaging App"""
++
++from __future__ import absolute_import
++
++from autopilot.matchers import Eventually
++from testtools.matchers import Equals
++
++from messaging_app.tests import MessagingAppTestCase
++
++
++class TestUI(MessagingAppTestCase):
++    def test_new_message_panel(self):
++        """New message panel comes up with the toolbar button"""
++
++        # Click "New message" menu button
++        self.main_view.open_toolbar()
++        toolbar = self.main_view.get_toolbar()
++        toolbar.click_button("newMessageButton")
++
++        # wait for "New message" page
++        self.assertThat(self.main_view.get_pagestack().depth,
++                        Eventually(Equals(2)))
++        self.assertThat(self.main_view.get_messages_page().visible,
++                        Eventually(Equals(True)))
++
++        # type address number
++        text_entry = self.main_view.get_newmessage_textfield()
++        text_entry.activeFocus.wait_for(True)
++        self.keyboard.type("123")
++        self.assertThat(text_entry.text, Eventually(Equals("123")))
++
++        # type message
++        text_entry = self.main_view.get_newmessage_textarea()
++        self.pointing_device.click_object(text_entry)
++        # need to click twice, first time to hide the toolbar (LP#1207369)
++        self.pointing_device.click_object(text_entry)
++        text_entry.activeFocus.wait_for(True)
++        message = "hello from Ubuntu"
++        self.keyboard.type(message)
++        self.assertThat(text_entry.text, Eventually(Equals(message)))
 === modified file 'tests/autopilot/messaging_app_connected/tests/__init__.py'
 --- tests/autopilot/messaging_app_connected/tests/__init__.py	2013-09-25 20:24:30 +0000
 +++ tests/autopilot/messaging_app_connected/tests/__init__.py	2013-11-04 13:13:28 +0000
@@ -19,8 +19,20 @@
  import os
  import shutil
++import subprocess
  import ConfigParser
++# determine whether we are running with phonesim
++try:
++    out = subprocess.check_output(["/usr/share/ofono/scripts/list-modems"],
++                                  stderr=subprocess.PIPE)
++    if out.startswith("[ /phonesim ]"):
++        print("ofono-phonesim is currently running, but this test is meant "
++              "to run against real hardware. Please uninstall "
++              "ofono-phonesim-autostart or run 'sudo stop ofono-phonesim'.")
++except subprocess.CalledProcessError:
++    pass
++
  config_file = os.path.expanduser('~/.testnumbers.cfg')