Autopilot

Merge lp:~nskaggs/autopilot/page-object-docs into lp:autopilot

page-object-docs
Merge into trunk

Proposed by Nicholas Skaggs on 2015-01-14

Status:	Merged
Approved by:	Christopher Lee on 2015-01-22
Approved revision:	528
Merged at revision:	530
Proposed branch:	lp:~nskaggs/autopilot/page-object-docs
Merge into:	lp:autopilot
Diff against target:	257 lines (+231/-0) 3 files modified docs/contents.rst (+1/-0) docs/guides/page_object.rst (+226/-0) docs/tutorial/good_tests.rst (+4/-0)
To merge this branch:	bzr merge lp:~nskaggs/autopilot/page-object-docs
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
PS Jenkins bot	continuous-integration		Approve on 2015-01-22
Christopher Lee (community)			Approve on 2015-01-22
Richard Huddie (community)			Needs Fixing on 2015-01-16
Allan LeSage (community)		2015-01-14	Needs Fixing on 2015-01-15
Review via email: mp+246504@code.launchpad.net

Commit message

Convert https://developer.ubuntu.com/en/apps/platform/guides/acceptance-testing-using-the-page-object-model/ into the autopilot documentation

Description of the change

Convert https://developer.ubuntu.com/en/apps/platform/guides/acceptance-testing-using-the-page-object-model/ into the autopilot documentation.

Open questions:
I didn't convert all of the text as it was originally intended as a guide and didn't fit well within the theme and voice of the 'writing good tests' section of autopilot. I also considered creating it as a separate page. Opinions welcome!

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2015-01-14:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/947/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Allan LeSage (allanlesage) wrote on 2015-01-15:

A couple of improvements possible :) , wonder if we want to make an item for "study the Ubuntu SDK or UI Toolkit helpers".

review: Needs Fixing

Revision history for this message

Richard Huddie (rhuddie) wrote on 2015-01-16:

This looks good. I've made a couple of minor points below.

I was also thinking, should we mention about only driving the UI input from the page object helpers, and not directly from the test itself? This may be mentioned somewhere else, but I just thought about it when reading through this. It's probably covered under making tests less flaky as all the UI input is driven by the helpers, not repeated throughout different tests.

review: Needs Fixing

Revision history for this message

Christopher Lee (veebers) wrote on 2015-01-19:

Having a separate page is a good idea, perhaps a "case study" or a suggestion for structed testing for applications.

review: Needs Fixing

Revision history for this message

Nicholas Skaggs (nskaggs) wrote on 2015-01-21:

I addressed many of the comments, but left the text on the same page for now. I'll look at what the structure would look like for migrating it out to a separate page. My original plan was to leave the small paragraph on 'think about design' in the good tests, and link to the remainder.

lp:~nskaggs/autopilot/page-object-docs updated on 2015-01-21

522. By Nicholas Skaggs on 2015-01-21: address MP comments

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2015-01-21:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/986/rebuild

review: Approve (continuous-integration)

Revision history for this message

Nicholas Skaggs (nskaggs) wrote on 2015-01-21:

Adding the major content to it's own page in the new guides section.

lp:~nskaggs/autopilot/page-object-docs updated on 2015-01-22

523. By Nicholas Skaggs on 2015-01-22: move to guides
524. By Nicholas Skaggs on 2015-01-22: fix guides location

Revision history for this message

Nicholas Skaggs (nskaggs) wrote on 2015-01-22:

This is ready for review again :-)

lp:~nskaggs/autopilot/page-object-docs updated on 2015-01-22

525. By Nicholas Skaggs on 2015-01-22: fix trunk issue

Revision history for this message

Christopher Lee (veebers) wrote on 2015-01-22:

(comment about warning when building removed as it's fixed.)

Looking good, jut a couple of minor things pointed out.

review: Needs Fixing

lp:~nskaggs/autopilot/page-object-docs updated on 2015-01-22

526. By Nicholas Skaggs on 2015-01-22: rebase trunk
527. By Nicholas Skaggs on 2015-01-22: fixes for veebers
528. By Nicholas Skaggs on 2015-01-22: add ...

Revision history for this message

Christopher Lee (veebers) wrote on 2015-01-22:

Awesome, LGTM. Thanks for that.

review: Approve

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2015-01-22:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/993/rebuild

review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk

Subscribers

People subscribed via source and target branches

to all changes:

Autopilot Hackers

Chris Gagnon

Nicholas Skaggs

 === modified file 'docs/contents.rst'
 --- docs/contents.rst	2014-05-20 14:02:18 +0000
 +++ docs/contents.rst	2015-01-22 01:16:09 +0000
@@ -9,6 +9,7 @@
      tutorial/tutorial
      api/index
      porting/porting
++    guides/page_object
      faq/faq
      faq/contribute
      faq/troubleshooting
 === added directory 'docs/guides'
 === added file 'docs/guides/page_object.rst'
 --- docs/guides/page_object.rst	1970-01-01 00:00:00 +0000
 +++ docs/guides/page_object.rst	2015-01-22 01:16:09 +0000
@@ -0,0 +1,226 @@
++.. _page_object_guide:
++
++Page Object Pattern
++####################
++
++.. contents::
++
++Introducing the Page Object Pattern
++-----------------------------------
++Automated testing of an application through the Graphical User Interface (GUI) is inherently fragile.
++These tests require regular review and attention during the development cycle. This is known as Interface Sensitivity (`"even minor changes to the interface can cause tests to fail" <https://books.google.com/books?isbn=0132797461>`_).
++Utilizing the page-object pattern, alleviates some of the problems stemming from this fragility, allowing us to do automated user acceptance testing (UAT) in a sustainable manner.
++
++The Page Object Pattern comes from the `Selenium community <https://code.google.com/p/selenium/wiki/PageObjects>`_ and is the best way to turn a flaky and unmaintainable user acceptance test into a stable and useful
++part of your release process. A page is what's visible on the screen at a single moment.
++A user story consists of a user jumping from page to page until they achieve their goal.
++Thus pages are modeled as objects following these guidelines:
++
++#. The public methods represent the services that the page offers.
++#. Try not to expose the internals of the page.
++#. Methods return other PageObjects.
++#. Assertions should exist only in tests
++#. Objects need not represent the entire page.
++#. Actions which produce multiple results should have a test for each result
++
++Lets take the page objects of the `Ubuntu Clock App  <http://bazaar.launchpad.net/~ubuntu-clock-dev/ubuntu-clock-app/trunk/view/399/tests/autopilot/ubuntu_clock_app/emulators.py>`__ as an example, with some simplifications. This application is written in
++QML and Javascript using the `Ubuntu SDK <http://developer.ubuntu.com/apps/sdk/>`__.
++
++1. The public methods represent the services that the page offers.
++~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++
++This application has a stopwatch page that lets users measure elapsed
++time. It offers services to start, stop and reset the watch, so we start
++by defining the stop watch page object as follows:
++
++.. code-block:: python
++
++    class Stopwatch(object):
++
++        def start(self):
++            raise NotImplementedError()
++
++        def stop(self):
++            raise NotImplementedError()
++
++        def reset(self):
++            raise NotImplementedError()
++
++2. Try not to expose the internals of the page.
++~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++
++The internals of the page are more likely to change than the services it
++offers. A stopwatch will keep the same three services we defined above
++even if the whole design changes. In this case, we reset the stop watch
++by clicking a button on the bottom-left of the window, but we hide that
++as an implementation detail behind the public methods. In Python, we can
++indicate that a method is for internal use only by adding a single
++leading underscore to its name. So, lets implement the reset\_stopwatch
++method:
++
++.. code-block:: python
++
++    def reset(self):
++        self._click_reset_button()
++
++    def _click_reset_button(self):
++        reset_button = self.wait_select_single(
++            'ImageButton', objectName='resetButton')
++        self.pointing_device.click_object(reset_button)
++
++Now if the designers go crazy and decide that it's better to reset the
++stop watch in a different way, we will have to make the change only in
++one place to keep all the tests working. Remember that this type of
++tests has Interface Sensitivity, that's unavoidable; but we can reduce
++the impact of interface changes with proper encapsulation and turn these
++tests into a useful way to verify that a change in the GUI didn't
++introduce any regressions.
++
++.. _page_object_guide_guideline_3:
++
++3. Methods return other PageObjects
++~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++
++An UAT checks a user story. It will involve the journey of the user
++through the system, so he will move from one page to another. Lets take
++a look at how a journey to reset the stop watch will look like:
++
++.. code-block:: python
++
++    stopwatch = clock_page.open_stopwatch()
++    stopwatch.start()
++    stopwatch.reset()
++
++In our sample application, the first page that the user will encounter
++is the Clock. One of the things the user can do from this page is to
++open the stopwatch page, so we model that as a service that the Clock
++page provides. Then return the new page object that will be visible to
++the user after completing that step.
++
++.. code-block:: python
++
++    class Clock(object):
++
++        ...
++
++        def open_stopwatch(self):
++            self._switch_to_tab('StopwatchTab')
++            return self.wait_select_single(Stopwatch)
++
++Now the return value of open\_stopwatch will make available to the
++caller all the available services that the stopwatch exposes to the
++user. Thus it can be chained as a user journey from one page to the
++other.
++
++4. Assertions should exist only in tests
++~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++
++A well written UAT consists of a sequence of
++steps or user actions and ends with one single assertion that verifies
++that the user achieved its goal. The page objects are the helpers for
++the user actions part of the test, so it's better to leave the check for
++success out of them. With that in mind, a test for the reset of the
++stopwatch would look like this:
++
++.. code-block:: python
++
++    def test_restart_button_must_restart_stopwatch_time(self):
++        # Set up.
++        stopwatch = self.clock_page.open_stopwatch()
++
++        stopwatch.start()
++        stopwatch.reset_stopwatch()
++
++        # Check that the stopwatch has been reset.
++        self.assertThat(
++            stopwatch.get_time,
++            Eventually(Equals('00:00.0')))
++
++We have to add a new method to the stopwatch page object: get\_time. But
++it only returns the state of the GUI as the user sees it. We leave in
++the test method the assertion that checks it's the expected value.
++
++.. code-block:: python
++
++    class Stopwatch(object):
++
++        ...
++
++        def get_time(self):
++            return self.wait_select_single(
++                'Label', objectName='time').text
++
++5. Need not represent an entire page
++~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++
++The objects we are modeling here can just represent a part of the page.
++Then we build the entire page that the user is seeing by composition of
++page parts. This way we can reuse test code for parts of the GUI that
++are reused in the application or between different applications. As an
++example, take the \_switch\_to\_tab('StopwatchTab') method that we are
++using to open the stopwatch page. The Clock application is using the
++Header component provided by the Ubuntu SDK, as all the other Ubuntu
++applications are doing too. So, the Ubuntu SDK also provides helpers to
++make it easier the user acceptance testing of the applications, and you
++will find an object like this:
++
++.. code-block:: python
++
++    class Header(object):
++
++        def switch_to_tab(tab_object_name):
++            """Open a tab.
++
++            :parameter tab_object_name: The QML objectName property of the tab.
++            :return: The newly opened tab.
++            :raise ToolkitException: If there is no tab with that object
++                name.
++
++            """
++        ...
++
++This object just represents the header of the page, and inside the
++object we define the services that the header provides to the users. If
++you dig into the full implementation of the Clock test class you will
++find that in order to open the stopwatch page we end up calling Header
++methods.
++
++6. Actions which produce multiple results should have a test for each result
++~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++
++According to the guideline :ref:`page_object_guide_guideline_3`, we are returning page objects every time
++that a user action opens the option for new actions to execute.
++Sometimes the same action has different results depending on the context
++or the values used for the action. For example, the Clock app has an
++Alarm page. In this page you can add new alarms, but if you try to add
++an alarm for sometime in the past, it will result in an error. So, we
++will have two different tests that will look something like this:
++
++.. code-block:: python
++
++    def test_add_alarm_for_tomorrow_must_add_to_alarm_list(self):
++        tomorrow = ...
++        test_alarm_name = 'Test alarm for tomorrow'
++        alarm_page = self.alarm_page.add_alarm(
++            test_alarm_name, tomorrow)
++
++        saved_alarms = alarm_page.get_saved_alarms()
++        self.assertIn(
++            (test_alarm_name, tomorrow),
++            saved_alarms)
++
++    def test_add_alarm_for_earlier_today_must_display_error(self):
++        earlier_today = ...
++        test_alarm_name = 'Test alarm for earlier_today'
++        error_dialog = self.alarm_page.add_alarm_with_error(
++            test_alarm_name, earlier_today)
++
++        self.assertEqual(
++            error_dialog.text,
++            'Please select a time in the future.')
++
++Take a look at the methods add\_alarm and add\_alarm\_with\_error. The
++first one returns the Alarm page again, where the user can continue his
++journey or finish the test checking the result. The second one returns
++the error dialog that's expected when you try to add an alarm with the
++wrong values.
 === modified file 'docs/tutorial/good_tests.rst'
 --- docs/tutorial/good_tests.rst	2015-01-19 22:48:14 +0000
 +++ docs/tutorial/good_tests.rst	2015-01-22 01:16:09 +0000
@@ -98,6 +98,10 @@
 . Commit. Push. Superseed original merge proposal with your branch.
 . Celebrate!
++Think about design
++++++++++++++++++++
++Much in the same way you might choose a functional or objective-oriented paradigm for a piece of code, a testsuite can benefit from choosing a good design pattern. One such design pattern is the page object model. The page object model can reduce testcase complexity and allow the testcase to grow and easily adapt to changes within the underlying application. Check out :ref:`page_object_guide`.
++
  Test Length
  +++++++++++