Merge lp:~canonical-platform-qa/webapps-core/x-test into lp:webapps-core

[Leo Arias]

Added an initial selenium + autopilot test for a webapp. It runs locally, on qemu and on the phone.

[Federico Gimenez (fgimenez)]

Some inline comments, thanks!

[David Barth (dbarth)]

Would you have a rebuild handy for Trusty? Like app developers should, I
try to stick to Ubuntu LTS for development tasks.

On Mon, Feb 16, 2015 at 10:36 AM, Federico Gimenez <email address hidden>
wrote:

> Some inline comments, thanks!
>
>
> Diff comments:
>
> > === modified file 'README'
> > --- README 2013-09-13 08:20:52 +0000
> > +++ README 2015-02-14 05:29:25 +0000
> > @@ -1,9 +1,106 @@
> > -This package provides a set of default web application versions for
> popular web services.
> > +Core Webapps
> > +============
> > +
> > +This package provides a set of default web application versions for
> popular
> > +web services.
> >
> > The debian package itself serves 2 purposes:
> > -- generate click packages in a clean environment, that can be
> integrated with the rest of our toolchain
> > -- package a set of icons for the webapps, while click doesn't support
> icons directly (temporary)
> > -
> > -To generate the click packages directly, ie without the debian package:
> > -
> > -./build.sh
> > +- generate click packages in a clean environment, that can be
> integrated with
> > +the rest of our toolchain
> > +- package a set of icons for the webapps, while click doesn't support
> icons
> > +directly (temporary)
> > +
> > +Building the click packages
> > +===========================
> > +
> > +To generate the click packages directly, ie without the debian package::
> > +
> > + $ ./build.sh
> > +
> > +Running the user acceptance tests
> > +=================================
> > +
> > +Run tests locally
> > +-----------------
> > +
> > +The tests need an updated selenium package and the oxide webdriver::
> > +
> > + $ sudo add-apt-repository ppa:canonical-platform-qa/selenium
> > + $ sudo apt-get update
> > + $ sudo apt-get install python3-selenium oxideqt-chromedriver
> > +
> > +To run the user acceptance tests from the source branch::
> > +
> > + $ cd webapp-amazon/tests/autopilot
> > + $ autopilot3 run --config from_source amazon_webapp
> > +
> > +Run tests in the phone
> > +----------------------
> > +
> > +First flash the phone with developer mode enabled::
> > +
> > + $ ubuntu-device-flash --developer-mode --password 0000 \
> > + --channel="ubuntu-touch/devel-proposed" --wipe
> > +
> > +When the phone is flashed, complete the welcome wizard.
> > +
> > +To run the user accceptance tests into an Ubuntu phone with a read-only
> image::
> > +
> > + $ adt-run --click-source webapp-amazon/ \
> > + --click
> build/com.ubuntu.developer.webapps.webapp-amazon_*_all.click \
> > + -o /tmp/adt-webapps-tests \
> > + --setup-commands \
> > + 'mount -o remount,rw /' \
> > + --setup-commands \
> > + 'apt-add-repository -y ppa:canonical-platform-qa/selenium' \
> > + --setup-commands \
> > + 'apt-get --no-list-cleanup update -o
> Dir::Etc::SourceList=/dev/null' \
> > + --setup-commands \
> > + 'sync; sleep 2; mount -o remount,ro /' \
> > + --- ssh -s adb -- -p 0000
> > +
> > +Run tests in a virtual machine
> > +------------------------------
> > +
> > +The user acceptance tests also can be run against a built and installed
> click
> > +package, under qemu. To do so,...

[Leo Arias (elopio)]

David, I'm not sure what build are you asking for. Doesn't ./build.sh work on trusty?
I only have vivid here.

[lp:~canonical-platform-qa/webapps-core/x-test]

120. By Leo Arias on 2015-02-17

Extracted a method to get dependencies path.

[Leo Arias (elopio)]

Replied and pushed to federico's comments.

[Federico Gimenez (fgimenez)]

Thanks Leo, some inline comments, cheers!

[Federico Gimenez (fgimenez)]

One inline comment, all working now :)

[Alexandre Abreu (abreu-alexandre)]

Could we make the webapp agnostic bits (chromedriver related, PageObject, etc.) in an emulator?

[Leo Arias (elopio)]

Do you mean, to move the things that will be common to all the webapps to a module that all of them can share?

Yes we can, but we will need to package those common bits into a deb. It can be called something like webapp-test-helpers, and every test in here will import it.
I imagine that shared package will have a base test class, a base page object, and all the helpers to get chromedriver running.

It will get a little weird if they are in the same branch though, because adt-run will use the archive to install dependencies in the test bed, so if a test needs an update in the common helpers, the common helpers will have to be pushed to the archive first before being able to run the new tests.

Does that answer your question? Or did I miss the point?

[Alexandre Abreu (abreu-alexandre)]

review: Approve

[Leo Arias (elopio)]

woohoo \o/
Let us know if you need some help from us extracting the pieces that will be common to all the apps. Well, actually, let Julien know :)

=== modified file 'README'
--- README	2013-09-13 08:20:52 +0000
+++ README	2015-02-17 19:29:56 +0000
@@ -1,9 +1,106 @@
-This package provides a set of default web application versions for popular web services.
+Core Webapps
+============
+
+This package provides a set of default web application versions for popular
+web services.
 
 The debian package itself serves 2 purposes:
-- generate click packages in a clean environment, that can be integrated with the rest of our toolchain
-- package a set of icons for the webapps, while click doesn't support icons directly (temporary)
-
-To generate the click packages directly, ie without the debian package:
-
-./build.sh
+- generate click packages in a clean environment, that can be integrated with
+the rest of our toolchain
+- package a set of icons for the webapps, while click doesn't support icons
+directly (temporary)
+
+Building the click packages
+===========================
+
+To generate the click packages directly, ie without the debian package::
+
+  $ ./build.sh
+
+Running the user acceptance tests
+=================================
+
+Run tests locally
+-----------------
+
+The tests need an updated selenium package and the oxide webdriver::
+
+    $ sudo add-apt-repository ppa:canonical-platform-qa/selenium
+    $ sudo apt-get update
+    $ sudo apt-get install python3-selenium oxideqt-chromedriver
+
+To run the user acceptance tests from the source branch::
+
+    $ cd webapp-amazon/tests/autopilot
+    $ autopilot3 run --config from_source amazon_webapp
+
+Run tests in the phone
+----------------------
+
+First flash the phone with developer mode enabled::
+
+    $ ubuntu-device-flash --developer-mode --password 0000 \
+      --channel="ubuntu-touch/devel-proposed" --wipe
+
+When the phone is flashed, complete the welcome wizard.
+
+To run the user accceptance tests into an Ubuntu phone with a read-only image::
+
+    $ adt-run --click-source webapp-amazon/ \
+      --click build/com.ubuntu.developer.webapps.webapp-amazon_*_all.click \
+      -o /tmp/adt-webapps-tests \
+      --setup-commands \
+      'mount -o remount,rw /' \
+      --setup-commands \
+      'apt-add-repository -y ppa:canonical-platform-qa/selenium' \
+      --setup-commands \
+      'apt-get --no-list-cleanup update -o Dir::Etc::SourceList=/dev/null' \
+      --setup-commands \
+      'sync; sleep 2; mount -o remount,ro /' \
+      --- ssh -s adb -- -p 0000
+
+Run tests in a virtual machine
+------------------------------
+
+The user acceptance tests also can be run against a built and installed click
+package, under qemu. To do so, install autopkgtest and qemu::
+
+    $ sudo apt-get install autopkgtest qemu
+
+After installing autopkgtest and qemu, you need to build an image for qemu
+to use. We use vivid here, as building an image to closely resemble the actual
+stable phone images is quite difficult. When this is easier in the future, we
+will switch to using stable phone images for this. We output the image to ~/
+rather than the current directory, so it will be in a safer place to avoid
+rebuilding images every time. You can store it in any directory you wish.
+
+::
+
+    $ adt-buildvm-ubuntu-cloud -r vivid -a amd64 -o ~/
+
+Then the tests may be run using adt-run with the qemu virtualization host.
+The output directory option here can be wherever you like, and is where the
+test artifacts will be placed. The ordering of the arguments to adt-run is
+important so try to keep them in this order.
+
+::
+
+    $ adt-run --click-source webapp-amazon/ \
+      --click build/com.ubuntu.developer.webapps.webapp-amazon_*_all.click \
+      -o /tmp/adt-webapps-tests \
+      --setup-commands ubuntu-touch-session \
+      --setup-commands \
+      'add-apt-repository -y ppa:canonical-platform-qa/selenium' \
+      -U --apt-pocket proposed \
+      --- qemu ~/adt-vivid-amd64-cloud.img
+
+See the detailed results
+------------------------
+
+To examine the test results, which are in subunit format, additional tools are
+required::
+
+    $ sudo add-apt-repository ppa:thomir/trv
+    $ sudo apt-get update
+    $ sudo apt-get install trv
+    $ trv /tmp/adt-webapps-tests/artifacts/autopilot.subunit

=== modified file 'webapp-amazon/manifest.json'
--- webapp-amazon/manifest.json	2014-09-19 16:29:06 +0000
+++ webapp-amazon/manifest.json	2015-02-17 19:29:56 +0000
@@ -11,6 +11,19 @@
     "maintainer": "Webapps Team <[e-mail]>",
     "name": "com.ubuntu.developer.webapps.webapp-amazon",
     "title": "webapp-amazon",
-    "version": "1.0.10"
+    "version": "1.0.10",
+    "x-test": {
+        "autopilot": {
+            "autopilot_module": "amazon_webapp",
+            "depends": [
+                "click",
+                "dpkg-dev",
+                "qtdeclarative5-ubuntu-ui-toolkit-plugin",
+                "ubuntu-sdk-libs",
+                "oxideqt-chromedriver",
+                "python3-selenium",
+                "webapp-container"
+            ]
+        }
+    }
 }
-

=== added directory 'webapp-amazon/tests'
=== added directory 'webapp-amazon/tests/autopilot'
=== added directory 'webapp-amazon/tests/autopilot/amazon_webapp'
=== added file 'webapp-amazon/tests/autopilot/amazon_webapp/__init__.py'
--- webapp-amazon/tests/autopilot/amazon_webapp/__init__.py	1970-01-01 00:00:00 +0000
+++ webapp-amazon/tests/autopilot/amazon_webapp/__init__.py	2015-02-17 19:29:56 +0000
@@ -0,0 +1,138 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu Core Webapps
+# Copyright (C) 2015 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import contextlib
+import logging
+
+import autopilot.logging
+import ubuntuuitoolkit
+from autopilot import platform
+
+logger = logging.getLogger(__name__)
+
+
+class PageObject():
+
+    """Base page object."""
+
+    def __init__(self, proxy, driver):
+        """Initialize the page object.
+
+        :param proxy: The autopilot application proxy.
+        :param driver: The webdriver used to introspect and control the web
+            container.
+
+        """
+        super().__init__()
+        self.proxy = proxy
+        self.driver = driver
+
+    def get_container_proxy(self):
+        return self.proxy.select_single('WebappContainerWebview')
+
+    def find_element_by_css_in_desktop_or_mobile(
+            self, desktop_css_selector, mobile_css_selector):
+        return self.driver.find_element_by_css_selector(
+            '{}, {}'.format(desktop_css_selector, mobile_css_selector))
+
+    @contextlib.contextmanager
+    def focused_type(self, input_web_element):
+        """Type into an input web element.
+
+        This context manager takes care of making sure a particular
+        *input_target* UI control is selected before any text is entered.
+
+        """
+
+        pointer = ubuntuuitoolkit.get_pointing_device()
+        pointer.move(*self.get_center_point(input_web_element))
+        pointer.click()
+
+        yield ubuntuuitoolkit.get_keyboard()
+
+    def get_center_point(self, web_element):
+        web_element_center_x = web_element.location.get('x') + (
+            web_element.size.get('width') // 2)
+        web_element_center_y = web_element.location.get('y') + (
+            web_element.size.get('height') // 2)
+
+        container_x, container_y, _, _ = self.get_container_proxy().globalRect
+
+        return (
+            container_x + web_element_center_x,
+            container_y + web_element_center_y
+        )
+
+
+class AmazonMainPage(PageObject):
+
+    """Page object for the main page of the Amazon website."""
+
+    @autopilot.logging.log_action(logger.info)
+    def search(self, query):
+        search_bar = self._get_search_bar()
+        self._enter_search_query(search_bar, query)
+        go_button = self._get_go_button(search_bar)
+        go_button.click()
+        return AmazonResultsPage(self.proxy, self.driver)
+
+    def _get_search_bar(self):
+        desktop_css_selector = '#nav-searchbar'
+        mobile_css_selector = '.nav-searchbar'
+        return self.find_element_by_css_in_desktop_or_mobile(
+            desktop_css_selector, mobile_css_selector)
+
+    def _enter_search_query(self, search_bar, query):
+        desktop_css_selector = '#twotabsearchtextbox'
+        mobile_css_selector = 'input.nav-input[type="text"]'
+        search_text_field = self.find_element_by_css_in_desktop_or_mobile(
+            desktop_css_selector, mobile_css_selector)
+        if platform.model() == 'Desktop':
+            with self.focused_type(search_text_field) as keyboard:
+                keyboard.type(query)
+        else:
+            # The location returned by the driver in the phone is not accurate.
+            # TODO report a bug.
+            # XXX We can't send keys because of bug http://pad.lv/1421423
+            # Remove the workaround once that bug is fixed.
+            # --elopio - 2015-02-12
+            # search_text_field.send_keys('Test')
+            self.driver.execute_script(
+                "document.getElementsByClassName("
+                "'nav-searchbar')[0].getElementsByTagName("
+                "'input')[1].setAttribute('value', '{}')".format(query)
+            )
+
+    def _get_go_button(self, search_bar):
+        desktop_css_selector = '.nav-submit-input'
+        mobile_css_selector = '.nav-input'
+        return self.find_element_by_css_in_desktop_or_mobile(
+            desktop_css_selector, mobile_css_selector)
+
+
+class AmazonResultsPage(PageObject):
+
+    """Page object for the main page of the Amazon Website."""
+
+    def get_search_results_count_label(self):
+        desktop_css_selector = '#s-result-count'
+        mobile_css_selector = '#results span'
+        return self.find_element_by_css_in_desktop_or_mobile(
+            desktop_css_selector, mobile_css_selector).text

=== added file 'webapp-amazon/tests/autopilot/amazon_webapp/test_amazon_webapp.py'
--- webapp-amazon/tests/autopilot/amazon_webapp/test_amazon_webapp.py	1970-01-01 00:00:00 +0000
+++ webapp-amazon/tests/autopilot/amazon_webapp/test_amazon_webapp.py	2015-02-17 19:29:56 +0000
@@ -0,0 +1,152 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu Core Webapps
+# Copyright (C) 2015 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import shutil
+import subprocess
+
+from autopilot import (
+    get_test_configuration,
+    testcase
+)
+from selenium import webdriver
+from selenium.webdriver.chrome import options
+from ubuntuuitoolkit import fixture_setup
+
+import amazon_webapp
+
+
+DESKTOP_FILE_NAME = 'webapp-amazon.desktop'
+
+DEFAULT_IMPLICIT_WAIT = 10  # seconds
+
+
+def get_source_desktop_file_path():
+    return os.path.join(
+        os.path.dirname(amazon_webapp.__file__), '..', '..', '..',
+        DESKTOP_FILE_NAME)
+
+
+def get_local_desktop_file_directory():
+    return os.path.join(
+        os.environ.get('HOME'), '.local', 'share', 'applications')
+
+
+def get_chrome_driver_path():
+    env = os.environ.copy()
+    readonly_dependencies_path = get_readonly_dependencies_path()
+    if readonly_dependencies_path:
+        # XXX When running with adt-run, this is needed if the image is
+        # read-only so we can find the chromium driver path.
+        env['PERL5LIB'] = os.path.join(
+            readonly_dependencies_path, 'usr', 'share', 'perl5')
+
+    architecture = subprocess.check_output(
+        ['dpkg-architecture', '-qDEB_HOST_MULTIARCH'],
+        env=env, universal_newlines=True).strip()
+
+    chrome_driver_path = '/usr/lib/{}/oxide-qt/chromedriver'.format(
+        architecture)
+
+    if readonly_dependencies_path and not os.path.exists(chrome_driver_path):
+        # Try the read-only install path.
+        chrome_driver_path = os.path.join(
+            readonly_dependencies_path, chrome_driver_path.lstrip('/'))
+
+    return chrome_driver_path
+
+
+def get_readonly_dependencies_path():
+    adt_artifacts_path = os.environ.get('ADT_ARTIFACTS', None)
+    if adt_artifacts_path:
+        return os.path.join(os.path.dirname(adt_artifacts_path), 'deps')
+    else:
+        return None
+
+
+class AmazonWebappTestCase(testcase.AutopilotTestCase):
+
+    DEFAULT_WEBVIEW_INSPECTOR_IP = '127.0.0.1'
+    DEFAULT_WEBVIEW_INSPECTOR_PORT = 9221
+
+    def setUp(self):
+        super().setUp()
+        self.setup_webdriver_environment()
+
+    def setup_webdriver_environment(self):
+        self.useFixture(fixture_setup.InitctlEnvironmentVariable(
+            global_=True,
+            UBUNTU_WEBVIEW_DEVTOOLS_HOST=self.DEFAULT_WEBVIEW_INSPECTOR_IP,
+            UBUNTU_WEBVIEW_DEVTOOLS_PORT=str(
+                self.DEFAULT_WEBVIEW_INSPECTOR_PORT)
+        ))
+
+    def launch_application(self):
+        if (get_test_configuration().get('from_source', None) and
+                os.path.exists(get_source_desktop_file_path())):
+            return self.launch_application_from_source()
+        else:
+            return self.launch_installed_application()
+
+    def launch_application_from_source(self):
+        test_desktop_file_path = os.path.join(
+            get_local_desktop_file_directory(), 'webapps-test.desktop')
+        shutil.copy(get_source_desktop_file_path(), test_desktop_file_path)
+        self.addCleanup(os.remove, test_desktop_file_path)
+
+        return self.launch_upstart_application('webapps-test')
+
+    def launch_installed_application(self):
+        return self.launch_click_package(
+            'com.ubuntu.developer.webapps.webapp-amazon')
+
+    def get_webdriver(self):
+        driver_options = options.Options()
+        driver_options.binary_location = ''
+        driver_options.debugger_address = '{}:{}'.format(
+            self.DEFAULT_WEBVIEW_INSPECTOR_IP,
+            self.DEFAULT_WEBVIEW_INSPECTOR_PORT)
+        driver = webdriver.Chrome(
+            executable_path=get_chrome_driver_path(),
+            chrome_options=driver_options
+        )
+
+        self.assertIsNot(driver, None)
+
+        self.addCleanup(self.close_webdriver, driver)
+
+        driver.implicitly_wait(DEFAULT_IMPLICIT_WAIT)
+        return driver
+
+    def close_webdriver(self, driver):
+        driver.close()
+        driver.quit()
+
+    def test_search_in_amazon_must_show_results(self):
+        proxy = self.launch_application()
+        driver = self.get_webdriver()
+
+        amazon = amazon_webapp.AmazonMainPage(proxy, driver)
+        results_page = amazon.search('Test')
+
+        results = results_page.get_search_results_count_label()
+        self.assertTrue(
+            results.endswith('results for "Test"') or
+            results.endswith('Results'))