Merge lp:~sil2100/unity/autopilot_hud_more_fixes into lp:unity

[Łukasz Zemczak]

Modify all the hud_query_check lambdas to be more smart and error-checking. Same for the selected_hud_button method in the HUD emulator.
Also, get rid of the troublesome test_gedit_undo and change it into test_gedit_save, which is easier to test and anyway tests what we want.
Besides that we also now try to handle the StateNotFoundError exception in the HUD emulator, another cause of some AP failures.

- Problem:

Some HUD tests are still failing, which is blocking the QA stack. The problem for that are differences on how HUD is returning the results, it's not instant.

- Fix:

Most of the fixes are related to changing the hud_query_check() lambda to a more 'smart' function. We first check if there is any HUD button we can fetch the label for. We do that every time hud_query_check was used.
We're adding one additional check to selected_hud_button() method in the hud emulator to make it safer to use. This fixes some of the errors I have been encountering.

Also, we get rid of the test_gedit_undo and change it into test_gedit_save. This test is used only to check if activating the menu entries in an application works and performs the actions (besides quitting). So, instead of performing a hard-to-test task like undo, I think 'save' is enough to test what we need.

- Tests:

N/A

[Andrea Azzarone (azzar1)]

LGTM.

review: Approve

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:3339
http://jenkins.qa.ubuntu.com/job/unity-ci/92/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-raring-amd64-ci/96
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-raring-armhf-ci/92/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-raring-i386-ci/92

Click here to trigger a rebuild:
http://s-jenkins:8080/job/unity-ci/92/rebuild

review: Needs Fixing (continuous-integration)

[Francis Ginther (fginther)]

The tests look better, but I get a StateNotFoundError on about every other test. My theory is that the contents of the hud have not yet 'settled' and the menu buttons are still changing. It might be worth merging this as is to make the testing better and addressing the StateNotFoundError later if it proves to be a problem for the integration testing.

Here's a stacktrace:

Traceback (most recent call last):
  File "/home/fginther/work/unity/autopilot_hud_more_fixes/tests/autopilot/unity/tests/test_hud.py", line 334, in test_app_activate_on_enter
    Eventually(Equals(u'Quit\u2002(File)'), timeout=30))
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 401, in assertThat
    mismatch = matcher.match(matchee)
  File "/usr/lib/python2.7/dist-packages/autopilot/matchers/__init__.py", line 103, in match
    wait_fun(self.matcher, self.timeout)
  File "/usr/lib/python2.7/dist-packages/autopilot/matchers/__init__.py", line 119, in _callable_wait_for
    new_value = refresh_fn()
  File "/home/fginther/work/unity/autopilot_hud_more_fixes/tests/autopilot/unity/tests/test_hud.py", line 328, in hud_query_check
    button = self.unity.hud.selected_hud_button
  File "/home/fginther/work/unity/autopilot_hud_more_fixes/tests/autopilot/unity/emulators/hud.py", line 123, in selected_hud_button
    [button] = filter(lambda x: x.focused, self.hud_buttons)
  File "/home/fginther/work/unity/autopilot_hud_more_fixes/tests/autopilot/unity/emulators/hud.py", line 123, in <lambda>
    [button] = filter(lambda x: x.focused, self.hud_buttons)
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/dbus.py", line 410, in __getattr__
    self.refresh_state()
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/dbus.py", line 363, in refresh_state
    _, new_state = self.get_new_state()
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/dbus.py", line 444, in get_new_state
    raise StateNotFoundError(self.__class__.__name__, self.id)
StateNotFoundError: State not found for class with name 'HudButton' and id '8793'.

review: Needs Fixing

[Francis Ginther (fginther)]

Ughh! I'm still seeing a StateNotFoundError, but this is when attempting to read the hud button label:

Traceback (most recent call last):
  File "/home/fginther/work/unity/lukasz/tests/autopilot/unity/tests/test_hud.py", line 334, in test_app_activate_on_enter
    Eventually(Equals(u'Quit\u2002(File)'), timeout=30))
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 401, in assertThat
    mismatch = matcher.match(matchee)
  File "/usr/lib/python2.7/dist-packages/autopilot/matchers/__init__.py", line 103, in match
    wait_fun(self.matcher, self.timeout)
  File "/usr/lib/python2.7/dist-packages/autopilot/matchers/__init__.py", line 119, in _callable_wait_for
    new_value = refresh_fn()
  File "/home/fginther/work/unity/lukasz/tests/autopilot/unity/tests/test_hud.py", line 331, in hud_query_check
    return button.label_no_formatting
  File "/home/fginther/work/unity/lukasz/tests/autopilot/unity/emulators/hud.py", line 168, in label_no_formatting
    return htmlparser.unescape(re.sub("<[^>]*>", "", self.label))
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/dbus.py", line 410, in __getattr__
    self.refresh_state()
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/dbus.py", line 363, in refresh_state
    _, new_state = self.get_new_state()
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/dbus.py", line 444, in get_new_state
    raise StateNotFoundError(self.__class__.__name__, self.id)
StateNotFoundError: State not found for class with name 'HudButton' and id '10497'.

I used your approach of handling the exception and added it to hud_query_check (along with moving it to the base class in test_hud): lp:~fginther/unity/autopilot_hud_button_label_fixes

review: Needs Fixing

[Francis Ginther (fginther)]

Per our IRC discussion, this is already an improvement, so I'll approve. We can then consider the additional fix separately.

review: Approve

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:3340
http://jenkins.qa.ubuntu.com/job/unity-ci/93/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-raring-amd64-ci/97
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-raring-armhf-ci/93
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-raring-i386-ci/93

Click here to trigger a rebuild:
http://s-jenkins:8080/job/unity-ci/93/rebuild

review: Approve (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Autolanding.
More details in the following jenkins job:
http://jenkins.qa.ubuntu.com/job/unity-autolanding/53/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-raring-amd64-autolanding/53/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-raring-armhf-autolanding/53/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-raring-i386-autolanding/53/console

review: Needs Fixing (continuous-integration)

[Francis Ginther (fginther)]

Jenkins barfed on itself, re-approving.

[PS Jenkins bot (ps-jenkins)]

FAILED: Autolanding.
More details in the following jenkins job:
http://jenkins.qa.ubuntu.com/job/unity-autolanding/55/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-raring-amd64-autolanding/55/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-raring-armhf-autolanding/55
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-raring-i386-autolanding/55

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

review: Approve (continuous-integration)

=== modified file 'tests/autopilot/unity/emulators/hud.py'
--- tests/autopilot/unity/emulators/hud.py	2013-05-10 05:16:07 +0000
+++ tests/autopilot/unity/emulators/hud.py	2013-05-21 16:21:28 +0000
@@ -9,7 +9,10 @@
 
 from __future__ import absolute_import
 
+from logging import getLogger
+
 from autopilot.input import Keyboard
+from autopilot.introspection.dbus import StateNotFoundError
 from autopilot.keybindings import KeybindingsHelper
 from HTMLParser import HTMLParser
 import re
@@ -18,6 +21,7 @@
 from unity.emulators.dash import SearchBar
 from unity.emulators.icons import HudEmbeddedIcon, HudLauncherIcon
 
+log = getLogger(__name__)
 
 class HudController(UnityIntrospectionObject, KeybindingsHelper):
     """Proxy object for the Unity Hud Controller."""
@@ -118,10 +122,15 @@
     @property
     def selected_hud_button(self):
         try:
+            if len(self.hud_buttons) is 0:
+                return 0
             [button] = filter(lambda x: x.focused, self.hud_buttons)
             return button
         except IndexError:
             raise RuntimeError("No HUD buttons found, is hud active?")
+        except StateNotFoundError:
+            log.warning("StateNotFoundError has been raised by HudController")
+            return 0
 
     @property
     def num_buttons(self):

=== modified file 'tests/autopilot/unity/tests/test_hud.py'
--- tests/autopilot/unity/tests/test_hud.py	2013-05-20 17:42:42 +0000
+++ tests/autopilot/unity/tests/test_hud.py	2013-05-21 16:21:28 +0000
@@ -195,8 +195,8 @@
 
         self.assertEqual(calc.is_active, True)
 
-    def test_gedit_undo(self):
-        """Test that the 'undo' action in the Hud works with GEdit."""
+    def test_gedit_save(self):
+        """Test that the 'save' action in the Hud works with GEdit."""
 
         file_path = mktemp()
         self.addCleanup(remove, file_path)
@@ -204,29 +204,27 @@
         self.addCleanup(self.process_manager.close_all_app, 'Text Editor')
         self.assertProperty(gedit_win, is_focused=True)
 
-        self.keyboard.type("0")
-        self.keyboard.type(" ")
-        self.keyboard.type("1")
+        self.keyboard.type("Test")
 
         self.unity.hud.ensure_visible()
 
-        self.keyboard.type("undo")
-        hud_query_check = lambda: self.unity.hud.selected_hud_button.label_no_formatting
+        self.keyboard.type("save")
+
+        def hud_query_check():
+            button = self.unity.hud.selected_hud_button
+            if not button:
+                return
+            return button.label_no_formatting
+
         self.assertThat(hud_query_check,
-                        Eventually(Equals(u'Undo\u2002(Edit)')))
+                        Eventually(Equals(u'Save\u2002(File)')))
         self.keyboard.press_and_release('Return')
+        self.addCleanup(self.keyboard.press_and_release, "Ctrl+s")
         self.assertThat(self.unity.hud.visible, Eventually(Equals(False), timeout=30))
 
         self.assertProperty(gedit_win, is_focused=True)
-        # XXX: Because of LP: #1180903, we need to give HUD some time after activating
-        #   any entry, because it takes some time
-        sleep(1.5)
-        self.keyboard.press_and_release("Ctrl+s")
         self.assertThat(lambda: exists(file_path), Eventually(Equals(True), timeout=30))
 
-        contents = open(file_path).read().strip('\n')
-        self.assertEqual("0 ", contents)
-
     def test_hud_to_dash_has_key_focus(self):
         """When switching from the hud to the dash you don't lose key focus."""
         self.unity.hud.ensure_visible()
@@ -325,7 +323,13 @@
 
         self.keyboard.type("Quit")
         self.assertThat(self.unity.hud.search_string, Eventually(Equals("Quit")))
-        hud_query_check = lambda: self.unity.hud.selected_hud_button.label_no_formatting
+
+        def hud_query_check():
+            button = self.unity.hud.selected_hud_button
+            if not button:
+                return
+            return button.label_no_formatting
+
         self.assertThat(hud_query_check,
                         Eventually(Equals(u'Quit\u2002(File)'), timeout=30))
 

=== modified file 'tests/autopilot/unity/tests/test_search.py'
--- tests/autopilot/unity/tests/test_search.py	2013-05-17 16:19:20 +0000
+++ tests/autopilot/unity/tests/test_search.py	2013-05-21 16:21:28 +0000
@@ -146,7 +146,12 @@
     def input_and_check_result(self, string, expected):
         self.keyboard.type(string)
         self.assertThat(self.unity.hud.search_string, Eventually(Equals(string), timeout=30))
-        hud_query_check = lambda: self.unity.hud.selected_hud_button.label_no_formatting
+        def hud_query_check():
+            button = self.unity.hud.selected_hud_button
+            if not button:
+                return
+            return button.label_no_formatting
+
         self.assertThat(hud_query_check, Eventually(Equals(expected), timeout=30))