Merge lp:~veebers/autopilot/add_OSK_keyboard_backend into lp:autopilot

[Christopher Lee]

Adds the Ubuntu Keyboard OSK as an input backend

Adds the Ubuntu Keyboard OSK as an input backend

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:322
http://jenkins.qa.ubuntu.com/job/autopilot-ci/202/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-amd64-ci/130
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-armhf-ci/130

Click here to trigger a rebuild:
http://s-jenkins:8080/job/autopilot-ci/202/rebuild

review: Approve (continuous-integration)

[lp:~veebers/autopilot/add_OSK_keyboard_backend]

323. By Christopher Lee on 2013-08-22

Documentation updates.

324. By Christopher Lee on 2013-08-23

Fix exception raised and docs

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:324
http://jenkins.qa.ubuntu.com/job/autopilot-ci/203/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-amd64-ci/131
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-armhf-ci/131

Click here to trigger a rebuild:
http://s-jenkins:8080/job/autopilot-ci/203/rebuild

review: Approve (continuous-integration)

[lp:~veebers/autopilot/add_OSK_keyboard_backend]

325. By Christopher Lee on 2013-08-23

Further docs improvements

326. By Christopher Lee on 2013-08-23

Minor doc change

[Thomi Richards (thomir-deactivatedaccount)]

LGTM

review: Approve

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:325
http://jenkins.qa.ubuntu.com/job/autopilot-ci/204/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-amd64-ci/132
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-armhf-ci/132

Click here to trigger a rebuild:
http://s-jenkins:8080/job/autopilot-ci/204/rebuild

review: Approve (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:326
http://jenkins.qa.ubuntu.com/job/autopilot-ci/205/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-amd64-ci/133
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-armhf-ci/133

Click here to trigger a rebuild:
http://s-jenkins:8080/job/autopilot-ci/205/rebuild

review: Approve (continuous-integration)

[lp:~veebers/autopilot/add_OSK_keyboard_backend]

327. By Christopher Lee on 2013-08-23

Re-order OSK Keyboard backend priority

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:327
http://jenkins.qa.ubuntu.com/job/autopilot-ci/206/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-amd64-ci/134
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-saucy-armhf-ci/134

Click here to trigger a rebuild:
http://s-jenkins:8080/job/autopilot-ci/206/rebuild

review: Approve (continuous-integration)

[Thomi Richards (thomir-deactivatedaccount)]

Still LGTM

review: Approve

=== modified file 'autopilot/input/__init__.py'
--- autopilot/input/__init__.py	2013-07-23 04:04:43 +0000
+++ autopilot/input/__init__.py	2013-08-23 05:01:40 +0000
@@ -56,6 +56,7 @@
 """
 
 from collections import OrderedDict
+from contextlib import contextmanager
 from autopilot.utilities import _pick_backend, CleanupRegistered
 from autopilot.input._common import get_center_point
 
@@ -81,12 +82,21 @@
         For more infomration on picking specific backends, see
         :ref:`tut-picking-backends`
 
+        For details regarding backend limitations please see:
+        :ref:`Keyboard backend limitations<keyboard_backend_limitations>`
+
+        .. warning:: The **OSK** (On Screen Keyboard) backend option does not
+         implement either :py:meth:`press` or :py:meth:`release` methods due to
+         technical implementation details and will raise a NotImplementedError
+         exception if used.
+
         :param preferred_backend: A string containing a hint as to which
             backend you would like. Possible backends are:
 
             * ``X11`` - Generate keyboard events using the X11 client
                 libraries.
             * ``UInput`` - Use UInput kernel-level device driver.
+            * ``OSK`` - Use the graphical On Screen Keyboard as a backend.
 
         :raises: RuntimeError if autopilot cannot instantate any of the
             possible backends.
@@ -104,17 +114,67 @@
             from autopilot.input._uinput import Keyboard
             return Keyboard()
 
+        def get_osk_kb():
+            try:
+                from autopilot.input._osk import Keyboard
+                return Keyboard()
+            except ImportError as e:
+                e.args += ("Unable to import the OSK backend",)
+                raise
+
         backends = OrderedDict()
         backends['X11'] = get_x11_kb
         backends['UInput'] = get_uinput_kb
+        backends['OSK'] = get_osk_kb
         return _pick_backend(backends, preferred_backend)
 
+    @contextmanager
+    def focused_type(self, input_target,  pointer=None):
+        """Type into an input widget.
+
+        This context manager takes care of making sure a particular
+        *input_target* UI control is selected before any text is entered.
+
+        Some backends extend this method to perform cleanup actions at the end
+        of the context manager block. For example, the OSK backend dismisses
+        the keyboard.
+
+        If the *pointer* argument is None (default) then either a Mouse or
+        Touch pointer will be created based on the current platform.
+
+        An example of using the context manager (with an OSK backend)::
+
+            from autopilot.input import Keyboard
+
+            text_area = self._launch_test_input_area()
+            keyboard = Keyboard.create('OSK')
+            with keyboard.focused_type(text_area) as kb:
+                kb.type("Hello World.")
+                self.assertThat(text_area.text, Equals("Hello World"))
+            # Upon leaving the context managers scope the keyboard is dismissed
+            # with a swipe
+
+        """
+        if pointer is None:
+            from autopilot.platform import model
+            if model() == 'Desktop':
+                pointer = Pointer(Mouse.create())
+            else:
+                pointer = Pointer(Touch.create())
+
+        pointer.click_object(input_target)
+        yield self
+
     def press(self, keys, delay=0.2):
         """Send key press events only.
 
         :param keys: Keys you want pressed.
         :param delay: The delay (in Seconds) after pressing the keys before
             returning control to the caller.
+        :raises: NotImplementedError If called when using the OSK Backend.
+
+        .. warning:: The **OSK** backend does not implement the press method
+          and will raise a NotImplementedError if called.
 
         Example:
 
@@ -131,6 +191,10 @@
         :param keys: Keys you want released.
         :param delay: The delay (in Seconds) after releasing the keys before
             returning control to the caller.
+        :raises: NotImplementedError If called when using the OSK Backend.
+
+        .. warning:: The **OSK** backend does not implement the press method
+         and will raise a NotImplementedError if called.
 
         Example:
 
@@ -219,14 +283,14 @@
         from autopilot.platform import model
         if model() != 'Desktop':
             logger.info(
-                "You cannot create a Mouse on the phablet devices. "
-                "consider using a Touch or Pointer device. "
+                "You cannot create a Mouse on the devices where X11 is not "
+                "available. consider using a Touch or Pointer device. "
                 "For more information, see: "
                 "http://unity.ubuntu.com/autopilot/api/input.html"
                 "#autopilot-unified-input-system"
             )
             raise RuntimeError(
-                "Cannot create a Mouse on the phablet devices."
+                "Cannot create a Mouse on devices where X11 is not available."
             )
 
         backends = OrderedDict()
 
=== added file 'autopilot/input/_osk.py'
--- autopilot/input/_osk.py	1970-01-01 00:00:00 +0000
+++ autopilot/input/_osk.py	2013-08-23 05:01:40 +0000
@@ -0,0 +1,121 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+#
+# Autopilot Functional Test Tool
+# Copyright (C) 2013 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import logging
+from time import sleep
+from contextlib import contextmanager
+
+from ubuntu_keyboard.emulators.keyboard import (
+    Keyboard as KeyboardDriver,
+    UnsupportedKey,
+)
+
+from autopilot.input import Keyboard as KeyboardBase
+
+
+logger = logging.getLogger(__name__)
+
+
+class Keyboard(KeyboardBase):
+
+    _keyboard = KeyboardDriver()
+
+    @contextmanager
+    def focused_type(self, input_target, pointer=None):
+        """Ensures that the keyboard is up and ready for input as well as
+        dismisses the keyboard afterward.
+
+        """
+        with super(Keyboard, self).focused_type(input_target, pointer):
+            try:
+                self._keyboard.wait_for_keyboard_ready()
+                yield self
+            finally:
+                self._keyboard.dismiss()
+
+    def press(self, keys, delay=0.2):
+        raise NotImplementedError(
+            "OSK Backend does not support the press method"
+        )
+
+    def release(self, keys, delay=0.2):
+        raise NotImplementedError(
+            "OSK Backend does not support the release method"
+        )
+
+    def press_and_release(self, key, delay=0.2):
+        """Press and release the key *key*.
+
+        The 'key' argument must be a string of the single key you want pressed
+        and released.
+
+        For example::
+
+            press_and_release('A')
+
+        presses then releases the 'A' key.
+
+        :raises: *UnsupportedKey* if the provided key is not supported by the
+         OSK Backend (or the current OSK langauge layout.)
+        :raises: *ValueError* if there is more than a single key supplied in
+         the *key* argument.
+
+        """
+
+        if len(self._sanitise_keys(key)) != 1:
+            raise ValueError("Only a single key can be passed in.")
+
+        try:
+            self._keyboard.press_key(key)
+            sleep(delay)
+        except UnsupportedKey as e:
+            e.args += ("OSK Backend is unable to type the key '%s" % key,)
+            raise
+
+    def type(self, string, delay=0.1):
+        """Simulate a user typing a string of text.
+
+        Only 'normal' keys can be typed with this method. There is no such
+        thing as Alt or Ctrl on the Onscreen Keyboard.
+
+        The OSK class back end will take care of ensuring that capitalized
+        keys are in fact capitalized.
+
+        :raises: *UnsupportedKey* if there is a key within the string that is
+        not supported by the OSK Backend (or the current OSK langauge layout.)
+
+        """
+        if not isinstance(string, basestring):
+            raise TypeError("'string' argument must be a string.")
+        logger.debug("Typing text: %s", string)
+        self._keyboard.type(string, delay)
+
+    @classmethod
+    def on_test_end(cls, test_instance):
+        """Dismiss (swipe hide) the keyboard.
+
+        """
+        logger.debug("Dismissing the OSK with a swipe.")
+        cls._keyboard.dismiss()
+
+    def _sanitise_keys(self, keys):
+        if keys == '+':
+            return [keys]
+        else:
+            return keys.split('+')
 
=== modified file 'autopilot/tests/functional/test_input_stack.py'
--- autopilot/tests/functional/test_input_stack.py	2013-07-24 08:26:10 +0000
+++ autopilot/tests/functional/test_input_stack.py	2013-08-23 05:01:40 +0000
@@ -21,11 +21,13 @@
 import json
 import os
 from tempfile import mktemp
-from testtools import TestCase
+from testtools import TestCase, skipIf
 from testtools.matchers import IsInstance, Equals, raises
+from textwrap import dedent
 from unittest import SkipTest
 from mock import patch
 
+from autopilot import platform
 from autopilot.testcase import AutopilotTestCase, multiply_scenarios
 from autopilot.input import Keyboard, Mouse, Pointer, Touch
 from autopilot.input._common import get_center_point
@@ -106,6 +108,24 @@
                         "app shows: " + text_edit.plainText
                         )
 
+    def test_typing_with_contextmanager(self):
+        """Typing text must produce the correct characters in the target
+        app.
+
+        """
+        app_proxy = self.start_mock_app()
+        text_edit = app_proxy.select_single('QTextEdit')
+
+        keyboard = Keyboard.create(self.backend)
+        with keyboard.focused_type(text_edit) as kb:
+            kb.type(self.input, 0.01)
+
+            self.assertThat(
+                text_edit.plainText,
+                Eventually(Equals(self.input)),
+                "app shows: " + text_edit.plainText
+            )
+
     def test_keyboard_keys_are_released(self):
         """Typing characters must not leave keys pressed."""
         app_proxy = self.start_mock_app()
@@ -137,6 +157,115 @@
                       )
 
 
+@skipIf(platform.model() == 'Desktop', "Only on device")
+class OSKBackendTests(AutopilotTestCase):
+    """Testing the Onscreen Keyboard (Ubuntu Keyboard) backend specifically.
+
+    There are limitations (i.e. on device only, window-mocker doesn't work on
+    the device, can't type all the characters that X11/UInput can.) that
+    necessitate this split into it's own test class.
+
+    """
+
+    scenarios = [
+        ('lower_alpha', dict(input='abcdefghijklmnopqrstuvwxyz')),
+        ('upper_alpha', dict(input='ABCDEFGHIJKLMNOPQRSTUVWXYZ')),
+        ('numeric', dict(input='0123456789')),
+        ('punctuation', dict(input='`~!@#$%^&*()_-+={}[]|\\:;"\'<>,.?/')),
+    ]
+
+    def launch_test_input_area(self):
+        self.app = self._launch_simple_input()
+        text_area = self.app.select_single("QQuickTextInput")
+
+        return text_area
+
+    def _start_qml_script(self, script_contents):
+        """Launch a qml script."""
+        qml_path = mktemp(suffix='.qml')
+        open(qml_path, 'w').write(script_contents)
+        self.addCleanup(os.remove, qml_path)
+
+        return self.launch_test_application(
+            "qmlscene",
+            qml_path,
+            app_type='qt',
+        )
+
+    def _launch_simple_input(self):
+        simple_script = dedent("""
+        import QtQuick 2.0
+        import Ubuntu.Components 0.1
+
+        Rectangle {
+            id: window
+            objectName: "windowRectangle"
+            color: "lightgrey"
+
+            Text {
+                id: inputLabel
+                text: "OSK Tests"
+                font.pixelSize: units.gu(3)
+                anchors {
+                    left: input.left
+                    top: parent.top
+                    topMargin: 25
+                    bottomMargin: 25
+                }
+            }
+
+            TextField {
+                id: input;
+                objectName: "input"
+                anchors {
+                    top: inputLabel.bottom
+                    horizontalCenter: parent.horizontalCenter
+                    topMargin: 10
+                }
+                inputMethodHints: Qt.ImhNoPredictiveText
+            }
+        }
+
+        """)
+
+        return self._start_qml_script(simple_script)
+
+    def test_can_type_string(self):
+        """Typing text must produce the expected characters in the input
+        field.
+
+        """
+
+        text_area = self.launch_test_input_area()
+        keyboard = Keyboard.create('OSK')
+        pointer = Pointer(Touch.create())
+        pointer.click_object(text_area)
+        keyboard._keyboard.wait_for_keyboard_ready()
+
+        keyboard.type(self.input)
+
+        self.assertThat(text_area.text, Eventually(Equals(self.input)))
+
+    def test_focused_typing_contextmanager(self):
+        """Typing text using the 'focused_typing' context manager must not only
+        produce the expected characters in the input field but also cleanup the
+        OSK afterwards too.
+
+        """
+        text_area = self.launch_test_input_area()
+        keyboard = Keyboard.create('OSK')
+        with keyboard.focused_type(text_area) as kb:
+            kb.type(self.input)
+            self.assertThat(
+                text_area.text,
+                Eventually(Equals(self.input))
+            )
+        self.assertThat(
+            keyboard._keyboard.is_available,
+            Eventually(Equals(False))
+        )
+
+
 class MouseTestCase(AutopilotTestCase):
 
     def test_move_to_nonint_point(self):
 
=== modified file 'docs/faq/faq.rst'
--- docs/faq/faq.rst	2013-07-24 08:38:59 +0000
+++ docs/faq/faq.rst	2013-08-23 05:01:40 +0000
@@ -20,7 +20,7 @@
 
 **I am running the latest development image!**
 
-In that case you can install autopilot directly - either by installing the ``autopilot-desktop`` or ``autopilot-touch`` packages, depending on whether you are installing to a desktop or phablet device.
+In that case you can install autopilot directly - either by installing the ``autopilot-desktop`` or ``autopilot-touch`` packages, depending on whether you are installing to a desktop or an Ubuntu Touch device.
 
 **I am running the Ubuntu release previous to the development release!**
 
@@ -108,8 +108,8 @@
 
 In general, autopilot tests are more relaxed about the 'one assertion per test' rule. However, care should still be taken to produce tests that are as small and understandable as possible.
 
-How do I write a test that uses either a Mouse or a Touch device interchangeably?
-==============================================================================================
+Q. How do I write a test that uses either a Mouse or a Touch device interchangeably?
+====================================================================================
 
 The :class:`autopilot.input.Pointer` class is a simple wrapper that unifies some of the differences between the :class:`~autopilot.input.Touch` and :class:`~autopilot.input.Mouse` classes. To use it, pass in the device you want to use under the hood, like so::
 
@@ -136,6 +136,33 @@
 
 If you only want to use the mouse on certain platforms, use the :mod:`autopilot.platform` module to determine the current platform at runtime.
 
+Q. How do I use the Onscreen Keyboard (OSK) to input text in my test?
+=====================================================================
+
+The OSK is an backend option for the :meth:`autopilot.input.Keyboard.create`
+method (see this :ref:`Advanced Autopilot<adv_picking_backend>` section for
+details regarding backend selection.)
+
+Unlike the other backends (X11, UInput) the OSK has a GUI presence and thus can
+be displayed on the screen.
+
+The :class:`autopilot.input.Keyboard` class provides a context manager that
+handles any cleanup required when dealing with the input backends.
+
+For example in the instance when the backend is the OSK, when leaving the scope
+of the context manager the OSK will be dismissed with a swipe::
+
+  from autopilot.input import Keyboard
+
+  text_area = self._launch_test_input_area()
+  keyboard = Keyboard.create('OSK')
+  with keyboard.focused_type(text_area) as kb:
+      kb.type("Hello World.")
+      self.assertThat(text_area.text, Equals("Hello World"))
+  # At this point now the OSK has been swiped away.
+  self.assertThat()
+
+
 Autopilot Qt & Gtk Support
 ++++++++++++++++++++++++++
 
 
=== modified file 'docs/tutorial/advanced_autopilot.rst'
--- docs/tutorial/advanced_autopilot.rst	2013-07-24 08:38:59 +0000
+++ docs/tutorial/advanced_autopilot.rst	2013-08-23 05:01:40 +0000
@@ -199,10 +199,12 @@
 
 The code snippet above will create an instance of the Keyboard class that uses X11 on Desktop systems, and UInput on other systems. On the rare occaison when test authors need to construct these objects themselves, we expect that the default creation pattern to be used.
 
+.. _adv_picking_backend:
+
 Picking a Backend
 +++++++++++++++++
 
-Test authors may sometimes want to pick a specific backend. The possible backends are documented in the API documentation for each class. For example, the documentation for the :meth:`autopilot.input.Keyboard.create` method says there are two backends available: the ``X11`` backend, and the ``UInput`` backend. These backends can be specified in the create method. For example, to specify that you want a Keyboard that uses X11 to generate it's input events::
+Test authors may sometimes want to pick a specific backend. The possible backends are documented in the API documentation for each class. For example, the documentation for the :meth:`autopilot.input.Keyboard.create` method says there are three backends available: the ``X11`` backend, the ``UInput`` backend, and the ``OSK`` backend. These backends can be specified in the create method. For example, to specify that you want a Keyboard that uses X11 to generate it's input events::
 
     >>> from autopilot.input import Keyboard
     >>> kbd = Keyboard.create("X11")
@@ -212,8 +214,15 @@
     >>> from autopilot.input import Keyboard
     >>> kbd = Keyboard.create("UInput")
 
+Finally, for the Onscreen Keyboard::
+
+    >>> from autopilot.input import Keyboard
+    >>> kbd = Keyboard.create("OSK")
+
 .. warning:: Care must be taken when specifying specific backends. There is no guarantee that the backend you ask for is going to be available across all platforms. For that reason, using the default creation method is encouraged.
 
+.. warning:: The **OSK** backend has some known implementation limitations, please see :meth:`autopilot.input.Keyboard.create` method documenation for further details.
+
 Possible Errors when Creating Backends
 ++++++++++++++++++++++++++++++++++++++
 
@@ -249,6 +258,92 @@
     'UInputError(\'"/dev/uinput" cannot be opened for writing\',)'
     'BackendException(\'Error while initialising backend. Original exception was: "/dev/uinput" cannot be opened for writing\',)'
 
+Keyboard Backends
+=================
+
+A quick introduction to the Keyboard backends
++++++++++++++++++++++++++++++++++++++++++++++
+
+Each backend has a different method of operating behind the scenes to provide
+the Keyboard interface.
+
+Here is a quick overview of how each backend works.
+
+.. list-table::
+   :widths: 15, 85
+   :header-rows: 1
+
+   * - Backend
+     - Description
+   * - X11
+     - The X11 backend generates X11 events using a mock input device which it
+       then syncs with X to actually action the input.
+   * - Uinput
+     - The UInput backend injects events directly in to the kernel using the
+       UInput device driver to produce input.
+   * - OSK
+     - The Onscreen Keyboard backend uses the GUI pop-up keyboard to enter
+       input. Using a pointer object it taps on the required keys to get the
+       expected output.
+
+.. _keyboard_backend_limitations:
+
+Limitations of the different Keyboard backends
+++++++++++++++++++++++++++++++++++++++++++++++
+
+While every effort has been made so that the Keyboard devices act the same
+regardless of which backend or platform is in use, the simple fact is that
+there can be some technical limitations for some backends.
+
+Some of these limitations are hidden when using the "create" method and won't
+cause any concern (i.e. X11 backend on desktop, UInput on an Ubuntu Touch device.)
+while others will raise exceptions (that are fully documented in the API docs).
+
+Here is a list of known limitations:
+
+**X11**
+
+* Only available on desktop platforms
+
+  - X11 isn't available on Ubuntu Touch devices
+
+**UInput**
+
+* Requires correct device access permissions
+
+  - The user (or group) that are running the autopilot tests need read/write
+    access to the UInput device (usually /dev/uinput).
+
+* Specific kernel support is required
+
+  - The kernel on the system running the tests must be running a kernel that
+    includes UInput support (as well as have the module loaded.
+
+**OSK**
+
+* Currently only available on Ubuntu Touch devices
+
+  - At the time of writing this the OSK/Ubuntu Keyboard is only
+    supported/available on the Ubuntu Touch devices. It is possible that it
+    will be available on the desktop in the near future.
+
+* Unable to type 'special' keys i.e. Alt
+
+  - This shouldn't be an issue as applications running on Ubuntu Touch devices
+    will be using the expected patterns of use on these platforms.
+
+* The following methods have limitations or are not implemented:
+
+  - :meth:`autopilot.input.Keyboard.press`: Raises NotImplementedError if
+    called.
+
+  - :meth:`autopilot.input.Keyboard.release`: Raises NotImplementedError if
+    called.
+
+  - :meth:`autopilot.input.Keyboard.press_and_release`: can can only handle
+    single keys/characters. Raises either ValueError if passed more than a
+    single character key or UnsupportedKey if passed a key that is not
+    supported by the OSK backend (or the current language layout).
 
 
 Process Control