Ubuntu Shorts App

Merge lp:~carla-sella/ubuntu-rssreader-app/fixing-flaky-test into lp:~ubuntu-shorts-dev/ubuntu-rssreader-app/trunk

  1. fixing-flaky-test
  2. Merge into trunk
Proposed by Carla Sella on 2015-01-23
Status: Merged
Approved by: Nicholas Skaggs on 2015-01-30
Approved revision: 364
Merged at revision: 360
Proposed branch: lp:~carla-sella/ubuntu-rssreader-app/fixing-flaky-test
Merge into: lp:~ubuntu-shorts-dev/ubuntu-rssreader-app/trunk
Diff against target: 130 lines (+28/-32)
2 files modified
tests/autopilot/shorts_app/__init__.py (+5/-5)
tests/autopilot/shorts_app/tests/test_rssreader.py (+23/-27)
To merge this branch: bzr merge lp:~carla-sella/ubuntu-rssreader-app/fixing-flaky-test
Reviewer Review Type Date Requested Status
Ubuntu Phone Apps Jenkins Bot continuous-integration Approve on 2015-01-30
Nicholas Skaggs 2015-01-23 Approve on 2015-01-29
Review via email: mp+247482@code.launchpad.net

Commit Message

Autopilot test for rssreader app.

To post a comment you must log in.
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Approve (continuous-integration)
Roman Shchekin (mrqtros) wrote :

So simple solution :)
I'll try to "top approve" it, but I don't believe in success...

Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Needs Fixing (continuous-integration)
Carla Sella (carla-sella) wrote :

:) Roman I am working on it, wanted to see what Jenkins would do after my change. I will tell you when I think I have fixed the flaky tests so you can merge, meanwhile wait it's just work in progress, but I need Jenkins to run the changes I make.

Carla Sella (carla-sella) wrote :

This is just another try, let's see what Jenkins says about it :).

Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Needs Fixing (continuous-integration)
Carla Sella (carla-sella) wrote :

@ Nicholas: What is happening is quite weird, if you look at the video of the failed test: http://91.189.93.70:8080/job/generic-mediumtests-vivid/826/artifact/shorts_app.tests.test_rssreader.TestMainWindow.test_edit_topic.ogv

you can see that in Edit feeds page, when changing topic, the test does not click on the new topic "Ubuntu" but clicks on the valueselector thus non changing the topic, so the test fails.
Looks like it does not find the LabelVisual object "Ubuntu", but on desktop the test works fine, so I am a bit puzzled.
I have tried some workarounds thinking it was a timing issue, but looks like it's not so.
What do you think about this ?
Thanks.
Carla

Carla Sella (carla-sella) wrote :
Nicholas Skaggs (nskaggs) wrote :

This fails on the desktop for me still, when run in the sandbox. you need to set the screen size to 400x600. By default on your desktop the screen size is larger and the display is different. You could also just run it on the phone to see, but use autopilot3-sandbox, specifying the small screen size to see it.

Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Approve (continuous-integration)
Nicholas Skaggs (nskaggs) wrote :

Carla, I'm re-running it over and over to see if it's consistent in passing / failing.

Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Needs Fixing (continuous-integration)
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Approve (continuous-integration)
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Needs Fixing (continuous-integration)
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Needs Fixing (continuous-integration)
Nicholas Skaggs (nskaggs) wrote :

Are you able to figure out why this still occasionally fails?

Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Needs Fixing (continuous-integration)
Nicholas Skaggs (nskaggs) wrote :

I see it's consistent with:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/shorts_app/tests/test_rssreader.py", line 140, in test_edit_topic
    "FeedComponent", text=test_feed_title).text,
  File "/usr/lib/python3/dist-packages/autopilot/introspection/dbus.py", line 295, in wait_select_single
    return self.select_single(type_name, **kwargs)
  File "/usr/lib/python3/dist-packages/autopilot/introspection/dbus.py", line 244, in select_single
    raise StateNotFoundError(type_name_str, **kwargs)
autopilot.exceptions.StateNotFoundError: Object not found with name 'FeedComponent' and properties {'text': 'Ubuntu Insights'}.

Roman Shchekin (mrqtros) wrote :

Nick, we already solved it (in little bit hacky way), take a look:
https://code.launchpad.net/~mrqtros/ubuntu-rssreader-app/ubuntu-rssreader-app-non-dev/+merge/247777

Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Approve (continuous-integration)
Nicholas Skaggs (nskaggs) wrote :

Going to re-run this several more times.

Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Needs Fixing (continuous-integration)
Nicholas Skaggs (nskaggs) wrote :

LGTM! You've solved the timing issue with the waitfor.

review: Approve
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Approve (continuous-integration)
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Approve (continuous-integration)
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Approve (continuous-integration)
Nicholas Skaggs (nskaggs) wrote :

Last little bit is working now. I approve (again, hehe).

review: Approve
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Needs Fixing (continuous-integration)
Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Needs Fixing (continuous-integration)
Nicholas Skaggs (nskaggs) wrote :

Carla, bah flake8 failures. Can you fix and I or Roman will re-approve?

lp:~carla-sella/ubuntu-rssreader-app/fixing-flaky-test updated on 2015-01-30
364. By Carla Sella on 2015-01-30

Fixed pyflakes and pep8.

Carla Sella (carla-sella) wrote :

Nicholas, should be ok now.

Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot) wrote :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'tests/autopilot/shorts_app/__init__.py'
2--- tests/autopilot/shorts_app/__init__.py 2015-01-28 22:31:43 +0000
3+++ tests/autopilot/shorts_app/__init__.py 2015-01-30 19:40:55 +0000
4@@ -499,17 +499,17 @@
5 """ change feed's topic """
6 topicValueselector = self._get_editfeed_topic_valueselector()
7 self.pointing_device.click_object(topicValueselector)
8- sleep(3)
9- newTopic = self._get_editfeed_valueselector_value(
10- topicValueselector, name)
11+ topicValueselector.expanded.wait_for(True)
12+ newTopic = self._get_editfeed_valueselector_value(name)
13+ newTopic.visible.wait_for(True)
14 self.pointing_device.click_object(newTopic)
15
16 def _get_editfeed_topic_valueselector(self):
17 return self.wait_select_single(
18 "ValueSelector", objectName="valueselector")
19
20- def _get_editfeed_valueselector_value(self, topicValueselector, name):
21- return topicValueselector.wait_select_single("LabelVisual", text=name)
22+ def _get_editfeed_valueselector_value(self, name):
23+ return self.wait_select_single("LabelVisual", text=name)
24
25
26 class FeedComponent(toolkit_emulators.Empty):
27
28=== modified file 'tests/autopilot/shorts_app/tests/test_rssreader.py'
29--- tests/autopilot/shorts_app/tests/test_rssreader.py 2015-01-17 15:15:47 +0000
30+++ tests/autopilot/shorts_app/tests/test_rssreader.py 2015-01-30 19:40:55 +0000
31@@ -33,6 +33,10 @@
32
33 def setUp(self):
34 super(BaseShortsAppTestCase, self).setUp()
35+ self.test_feed_url = _EXTERNAL_FEED_URL
36+ self.test_feed_title = _EXTERNAL_FEED_TITLE
37+ self.test_topic = 'Test topic'
38+ self.test_change_topic = 'Ubuntu'
39
40 # wait for any updates to finish before beginning tests
41 self._wait_for_refresh()
42@@ -68,13 +72,11 @@
43
44 def test_add_feed_to_new_topic(self):
45 """Test that adding a feed to a new topic must show it in a new tab."""
46- test_feed_url = _EXTERNAL_FEED_URL
47- test_feed_title = _EXTERNAL_FEED_TITLE
48- test_topic = 'Test topic'
49- self.add_feed_to_new_topic(test_feed_url, test_feed_title, test_topic)
50+ self.add_feed_to_new_topic(
51+ self.test_feed_url, self.test_feed_title, self.test_topic)
52
53 selected_tab_title = self.app.main_view.get_selected_tab_title()
54- self.assertEqual('Test topic', selected_tab_title)
55+ self.assertEqual(self.test_topic, selected_tab_title)
56
57 def test_switch_to_list_view_mode(self):
58 """ test switching to list view mode"""
59@@ -101,55 +103,49 @@
60 def test_open_listmode_feed_item(self):
61 """"test to ensure list mode feed items can be opened"""
62
63- test_feed_url = _EXTERNAL_FEED_URL
64- test_feed_title = _EXTERNAL_FEED_TITLE
65- test_topic = 'Test topic'
66-
67- self.add_feed_to_new_topic(test_feed_url, test_feed_title, test_topic)
68- new_topic_tab = self.app.main_view.get_topic_tab(test_topic)
69+ self.add_feed_to_new_topic(
70+ self.test_feed_url, self.test_feed_title, self.test_topic)
71+ new_topic_tab = self.app.main_view.get_topic_tab(self.test_topic)
72 self._ensure_list_view_mode(new_topic_tab)
73
74- self.app.main_view.open_feed_item(test_topic, test_feed_title)
75+ self.app.main_view.open_feed_item(
76+ self.test_topic, self.test_feed_title)
77 self.assertEqual(
78 self.app.main_view.get_feed_in_feedlist_title(
79- test_topic, test_feed_title),
80+ self.test_topic, self.test_feed_title),
81 self.app.main_view.get_articleviewitem_title())
82
83 def test_edit_topic(self):
84 """test edit topic"""
85- test_feed_url = _EXTERNAL_FEED_URL
86- test_feed_title = _EXTERNAL_FEED_TITLE
87- test_topic = 'Test topic'
88
89- self.add_feed_to_new_topic(test_feed_url, test_feed_title, test_topic)
90+ self.add_feed_to_new_topic(
91+ self.test_feed_url, self.test_feed_title, self.test_topic)
92
93 self.app.main_view.click_edit_topics_in_header()
94 topicManagementPage = self.app.main_view.get_feed_management_page()
95- topicManagementPage.expand_topic(test_topic)
96+ topicManagementPage.expand_topic(self.test_topic)
97
98 # change feed's topic
99- topicManagementPage.goto_feed_in_topic(test_topic, test_feed_title)
100+ topicManagementPage.goto_feed_in_topic(
101+ self.test_topic, self.test_feed_title)
102 editFeedpage = self.app.main_view.get_edit_feed_page()
103- editFeedpage.change_feeds_topic("Ubuntu")
104+ editFeedpage.change_feeds_topic(self.test_change_topic)
105 self.app.main_view.click_header_done_button()
106
107 # verify topic has changed
108 newTopic = topicManagementPage.wait_select_single(
109- "TopicComponent", topicName="Ubuntu")
110+ "TopicComponent", topicName=self.test_change_topic)
111 self.assertThat(newTopic.wait_select_single(
112- "FeedComponent", text=test_feed_title).text,
113- Eventually(Equals(test_feed_title)))
114+ "FeedComponent", text=self.test_feed_title).text,
115+ Eventually(Equals(self.test_feed_title)))
116
117
118 class ShortsTestCaseWithTopicAndFeed(BaseShortsAppTestCase):
119
120 def setUp(self):
121 super(ShortsTestCaseWithTopicAndFeed, self).setUp()
122- test_feed_url = _EXTERNAL_FEED_URL
123- self.test_feed_title = _EXTERNAL_FEED_TITLE
124- self.test_topic = 'Test topic'
125 self.add_feed_to_new_topic(
126- test_feed_url, self.test_feed_title, self.test_topic)
127+ self.test_feed_url, self.test_feed_title, self.test_topic)
128
129 def test_remove_feed(self):
130 """Test the removal of a feed from the Edit Feeds page."""

Subscribers

People subscribed via source and target branches