Merge lp:~canonical-platform-qa/ubuntu-system-tests/pull_to_refresh_video_scope into lp:ubuntu-system-tests

[Omer Akram]

Stabilize local videos display test

manually refresh the video scope before trying to assess its elements.

@run_tests: ubuntu_system_tests.tests.scopes.test_video_scope.VideoScopeTest.test_display_local_video_content

[Omer Akram (om26er)]

My hunch is that it could be a timing issue between the files are copied vs the scope is opened. So I implemented a mechanism that refreshes the video scope when its opened. If we still see the test failure after this update, we can declare that to be a real bug.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:462
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/678/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3422
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3422/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3422
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3422/artifact/work/output/*zip*/output.zip
    UNSTABLE: https://platform-qa-jenkins.ubuntu.com/job/ust-ci-krillin/34
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/930/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/678/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:462
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/678/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3422
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3422/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3422
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3422/artifact/work/output/*zip*/output.zip
    UNSTABLE: https://platform-qa-jenkins.ubuntu.com/job/ust-ci-krillin/34
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/930
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/931/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/678/rebuild

review: Needs Fixing (continuous-integration)

[Santiago Baldassin (sbaldassin)]

Comments inline

review: Needs Fixing

[lp:~canonical-platform-qa/ubuntu-system-tests/pull_to_refresh_video_scope]

463. By Omer Akram on 2016-09-15

minor changes per review

[Omer Akram (om26er)]

Thanks for the review, updated the code.

[Santiago Baldassin (sbaldassin)]

Code looks good to me. Let's wait until Jenkins vote before emerge

review: Approve

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:463
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/682/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3427
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3427/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3427
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3427/artifact/work/output/*zip*/output.zip
    UNSTABLE: https://platform-qa-jenkins.ubuntu.com/job/ust-ci-krillin/38
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/938/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/682/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:463
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/682/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3427
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3427/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3427
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3427/artifact/work/output/*zip*/output.zip
    UNSTABLE: https://platform-qa-jenkins.ubuntu.com/job/ust-ci-krillin/38
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/938
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/939/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/682/rebuild

review: Needs Fixing (continuous-integration)

[Omer Akram (om26er)]

Interestingly the test is still failing and I don't seem to be able to reproduce it here. Will probably need to replicate a test run exactly like the lab.

[Richard Huddie (rhuddie)]

> Interestingly the test is still failing and I don't seem to be able to
> reproduce it here. Will probably need to replicate a test run exactly like the
> lab.

I would recommend not using a pull action to refresh. - We've had many problems with this in the past as it is not reliable. There is already a refresh() method in the base class which uses url dispatcher to navigate back to the current scope and it causes a refresh without any UI interaction. So I would try using that first.

[Richard Huddie (rhuddie)]

Setting as needs fixing for above.

review: Needs Fixing

[Omer Akram (om26er)]

It seems we don't really need a fix on this one. Its something to do with the lab as the scope is completely blank there.

I'll probably need remote access to the device in the lab.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:463
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/697/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3449
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3449/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3449
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3449/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/ust-ci-krillin/50
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/960/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/697/rebuild

review: Approve (continuous-integration)

[Santiago Baldassin (sbaldassin)]

> > Interestingly the test is still failing and I don't seem to be able to
> > reproduce it here. Will probably need to replicate a test run exactly like
> the
> > lab.
>
> I would recommend not using a pull action to refresh. - We've had many
> problems with this in the past as it is not reliable. There is already a
> refresh() method in the base class which uses url dispatcher to navigate back
> to the current scope and it causes a refresh without any UI interaction. So I
> would try using that first.

I'm sorry but I disagree with Richard on this one. I think that the use of the pull_to_refresh method is actually a good solution. That's what the end user would do if the local videos/photos are not displayed. Using the url dispatcher is not really a navigation option for the end users so I would avoid it. If the pull to refresh method is not reliable, me can work and making it more robust

[lp:~canonical-platform-qa/ubuntu-system-tests/pull_to_refresh_video_scope]

464. By Omer Akram on 2016-09-20

dummy commit for CI

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:464
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/718/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3473
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3473/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3473
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3473/artifact/work/output/*zip*/output.zip
    UNSTABLE: https://platform-qa-jenkins.ubuntu.com/job/ust-ci-krillin/59
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/983/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/718/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:464
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/718/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3473
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3473/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3473
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3473/artifact/work/output/*zip*/output.zip
    UNSTABLE: https://platform-qa-jenkins.ubuntu.com/job/ust-ci-krillin/59
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/983
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/985/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/718/rebuild

review: Needs Fixing (continuous-integration)

[Richard Huddie (rhuddie)]

We had a similar problem on this mp: https://code.launchpad.net/~canonical-platform-qa/ubuntu-system-tests/fix_sd_card/+merge/305711

The solution there was to use the mediascanner observer to wait for the new media files to be scanned before navigating to the scope. I think similar fix would likely work here also.

review: Needs Fixing

[Omer Akram (om26er)]

This is more of an issue with the Device setup than the test code. The scope being completely blank, all the times indicates of a situation with network.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:464
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/732/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3494
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3494/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3494
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3494/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/ust-ci-krillin/72
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1005/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/732/rebuild

review: Approve (continuous-integration)

[Allan LeSage (allanlesage)]

Rebuilding this a few times just for kicks.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:464
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/733/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3495
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3495/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3495
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3495/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/ust-ci-krillin/73
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1006/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/733/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:464
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1076/
Executed test runs:
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/308/console
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/306/console
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1697/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1076/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:464
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1093/
Executed test runs:
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/332/console
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/330/console
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1803/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1093/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:464
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1122/
Executed test runs:
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/378/console
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/376/console
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1859/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1122/rebuild

review: Needs Fixing (continuous-integration)

Unmerged revisions

464. By Omer Akram on 2016-09-20

dummy commit for CI

463. By Omer Akram on 2016-09-15

minor changes per review

462. By Omer Akram on 2016-09-15

Implement a pull_to_refresh for scopes

=== modified file 'ubuntu_system_tests/helpers/scopes/base.py'
--- ubuntu_system_tests/helpers/scopes/base.py	2016-09-15 14:00:15 +0000
+++ ubuntu_system_tests/helpers/scopes/base.py	2016-09-20 16:11:55 +0000
@@ -55,6 +55,13 @@
                                        objectName='categoryListView')
         list_view.swipe_to_top()
 
+    def pull_to_refresh(self):
+        """Refresh the scope by a pull down."""
+        width, height = self.globalRect.w, self.globalRect.h
+        self.pointing_device.drag(width / 2, height / 2, width / 2, height)
+        self.wait_for_processing_to_start(silent_fail=True)
+        self.wait_for_processing_to_complete()
+
     def _get_icon(self, category, title, subtitle=None):
         """Return an icon that has specified title and subtitle.
         :param category: The category to search.
@@ -315,7 +322,7 @@
             self.processing.wait_for(True, timeout=timeout)
         except AssertionError as e:
             if not silent_fail:
-                raise(e)
+                raise e
 
     def wait_for_processing_to_complete(self):
         """Wait for the processing indicator to disappear."""

=== modified file 'ubuntu_system_tests/helpers/scopes/video/_cpo.py'
--- ubuntu_system_tests/helpers/scopes/video/_cpo.py	2016-08-31 11:40:49 +0000
+++ ubuntu_system_tests/helpers/scopes/video/_cpo.py	2016-09-20 16:11:55 +0000
@@ -51,10 +51,9 @@
             return False
 
     def _get_my_videos_section(self):
-        self.wait_select_many(
-            ap_query_timeout=30, objectName='dashCategorylocal')
-        return self.select_single(
-            objectName='dashCategorylocal').select_single('QQuickGridView')
+        local_videos_category = self.wait_select_single(
+            ap_query_timeout=30, objectName='dashCategorylocal', visible=True)
+        return local_videos_category.select_single('QQuickGridView')
 
     def get_my_videos_thumbnails(self):
         """
@@ -71,10 +70,8 @@
         return labels, sources
 
     def get_my_videos(self):
-        """ Retrieve all the youtube videos """
-        self.wait_for_processing_to_complete()
-        buttons = self._get_videos(self._get_my_videos_section())
-        return buttons
+        """ Retrieve all the local videos """
+        return self._get_videos(self._get_my_videos_section())
 
     def is_my_videos_displayed(self):
         """ Indicate if the my videos section is being displayed """

=== modified file 'ubuntu_system_tests/tests/scopes/test_video_scope.py'
--- ubuntu_system_tests/tests/scopes/test_video_scope.py	2016-09-16 14:12:34 +0000
+++ ubuntu_system_tests/tests/scopes/test_video_scope.py	2016-09-20 16:11:55 +0000
@@ -79,8 +79,8 @@
         shutil.copy2(test_data.SRC_VIDEO_FILE_1, video2)
 
         scope = scopes.go_to_video_scope()
-
         self.dash.wait_for_dash_loaded()
+        scope.pull_to_refresh()
         video_content = scope.get_my_videos_thumbnails()
         video_names = [video1_name, video2_name]
         video_files = [THUMBNAILER_PREFIX + video1,