Merge lp:~konrad11901/ubuntu-system-tests/add-gnome-software-tests into lp:ubuntu-system-tests

[Konrad Krawiec]

Add eight testcases:
1. Existing application packaged as deb (not installed)
2. Existing application packaged as snap (installed)
3. Existing application packaged as snap (not installed)
4. Mixed results (deb, snap, installed and not installed)
5. Number of characters in search string
6. Multi words search
7. Several searches in a row
8. Featured application - Verify that clicking on the image opens the detailed page of the application

It's important to note that testcase number 4 works only sometimes (chromium snap isn't always detected as installed, don't know why) and the testcase no. 8 doesn't work at all and requires fixing (I'd be really thankful for the help with this).

Also, you might notice that I added 0.1s sleep after clicking the search button in every search testcase. It's because in my testing environment sometimes the first letter of the string wasn't typed, so sometimes instead of i.e. 'gnome-software', 'nome-software' was typed. If it shouldn't be here, feel free to tell me and I'll remove that.

[Heber Parrucci (heber013)]

Looks good in general. Some corrections to make.

review: Needs Fixing

[Konrad Krawiec (konrad11901)]

I added the suggested changes.

[Heber Parrucci (heber013)]

Looks good. For those tests that fail randomly or do not work at all, please add @skip and the reason so we can fix them later. Apart from the ones you mentioned, this one is also failing:
2. Existing application packaged as snap

review: Needs Fixing

[Heber Parrucci (heber013)]

Looks good to me.

review: Approve

[platform-qa-bot (platform-qa-bot)]

FAILED: Autolanding.
No commit message was specified in the merge proposal. Hit 'Add commit message' on the merge proposal web page or follow the link below. You can approve the merge proposal yourself to rerun.
https://code.launchpad.net/~konrad11901/ubuntu-system-tests/add-gnome-software-tests/+merge/336169/+edit-commit-message

review: Needs Fixing (continuous-integration)

=== modified file 'ubuntu_system_tests/helpers/application.py'
--- ubuntu_system_tests/helpers/application.py	2017-09-14 14:06:34 +0000
+++ ubuntu_system_tests/helpers/application.py	2018-01-18 21:17:49 +0000
@@ -36,6 +36,7 @@
 )
 
 from ubuntu_system_tests.helpers.utils import is_discovery
+from ubuntu_system_tests.helpers.testbed import run_command_with_sudo
 
 logger = logging.getLogger(__name__)
 logger.setLevel(logging.INFO)
@@ -207,7 +208,15 @@
             return len(output) > 0 and 'ok installed' in output.decode()
         except subprocess.CalledProcessError:
             try:
-                output = subprocess.check_output(cmd + [self.package_name])
+                output = subprocess.check_output(cmd + [self.package_name or ''])
                 return len(output) > 0 and 'ok installed' in output.decode()
             except subprocess.CalledProcessError:
                 return False
+
+    def install(self):
+        cmd = ['apt', 'install', self.app_name, '-y']
+        run_command_with_sudo(cmd)
+
+    def remove(self):
+        cmd = ['apt', 'remove', self.app_name, '-y']
+        run_command_with_sudo(cmd)

=== added directory 'ubuntu_system_tests/helpers/chromium'
=== added file 'ubuntu_system_tests/helpers/chromium/app.py'
--- ubuntu_system_tests/helpers/chromium/app.py	1970-01-01 00:00:00 +0000
+++ ubuntu_system_tests/helpers/chromium/app.py	2018-01-18 21:17:49 +0000
@@ -0,0 +1,30 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2015-2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+from ubuntu_system_tests.helpers.application import Deb
+
+APP_NAME = 'chromium-browser'
+APP_ID = 'chromium-browser'
+
+
+class Chromium(Deb):
+
+    def __init__(self):
+        super().__init__(APP_NAME, APP_ID)

=== added directory 'ubuntu_system_tests/helpers/entangle'
=== added file 'ubuntu_system_tests/helpers/entangle/app.py'
--- ubuntu_system_tests/helpers/entangle/app.py	1970-01-01 00:00:00 +0000
+++ ubuntu_system_tests/helpers/entangle/app.py	2018-01-18 21:17:49 +0000
@@ -0,0 +1,30 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2015-2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+from ubuntu_system_tests.helpers.application import Deb
+
+APP_NAME = 'entangle'
+APP_ID = 'entangle'
+
+
+class Entangle(Deb):
+
+    def __init__(self):
+        super().__init__(APP_NAME, APP_ID)

=== modified file 'ubuntu_system_tests/helpers/gnome_sofware/cpo.py'
--- ubuntu_system_tests/helpers/gnome_sofware/cpo.py	2018-01-04 00:36:29 +0000
+++ ubuntu_system_tests/helpers/gnome_sofware/cpo.py	2018-01-18 21:17:49 +0000
@@ -60,6 +60,7 @@
         :param text: the string to write in search box
         :return: ResultsPage object
         """
+        time.sleep(0.1)
         self.get_entry_search(is_focus=True)
         input_manager.keyboard.type(text)
         return ResultsPage(self.proxy)
@@ -163,6 +164,19 @@
         except (DBusException, StateNotFoundError):
             return False
 
+    def open_featured_app(self):
+        """Open the featured app's description
+        :return The page with the featured app's description
+        """
+        item = self.proxy.wait_select_single(BuilderName='GsFeatureTile', visible=True)
+        box = item.wait_select_single(BuilderName='box')
+        # This line throws an exception:
+        input_manager.pointer.click_object(box)
+        self.proxy.wait_select_single(
+            BuilderName='box_details',
+            visible=True)
+        return ItemDetailsPage(self.proxy.select_single(BuilderName='details_page'))
+
 
 class ResultsPage(GnomeSoftwareCPO):
     def __init__(self, proxy):
@@ -172,12 +186,12 @@
         """Get the current results
         :return the list with the current results or empty
         """
+        attempts = 1
+        retries = kwargs.pop('retries', 10)
+        wait = kwargs.pop('wait', 5)
         search = self.proxy.wait_select_single(BuilderName='list_box_search',
                                                *args,
                                                **kwargs)
-        attempts = 1
-        retries = kwargs.pop('retries', 10)
-        wait = kwargs.pop('wait', 5)
         # Wait until search finishes and return results
         while attempts < retries:
             try:
@@ -205,6 +219,28 @@
         except (DBusException, StateNotFoundError):
             return False
 
+    @staticmethod
+    def check_app_present(items, app_to_search):
+        """Check if the given application is present in the search results
+        :param items: the list of results to check
+        :param app_to_search: the name (visible in GNOME Software)
+         of the application to find
+        :return True if the given application is present in the search results
+        """
+        for item in items:
+            try:
+                label = item.select_single(BuilderName="name_label", visible=True).label
+                if label == app_to_search:
+                    return True
+            except (DBusException, StateNotFoundError):
+                pass
+        return False
+
+    @staticmethod
+    def get_app_name(item):
+        """Gets the name of the application taken from the search results."""
+        return item.select_single(BuilderName="name_label", visible=True).label
+
     def open_item(self, item):
         """Click on the given item
         :param item: the item click on

=== modified file 'ubuntu_system_tests/tests/test_gnome_software.py'
--- ubuntu_system_tests/tests/test_gnome_software.py	2018-01-11 13:14:30 +0000
+++ ubuntu_system_tests/tests/test_gnome_software.py	2018-01-18 21:17:49 +0000
@@ -16,7 +16,7 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-from testtools import skipIf
+from testtools import skipIf, skip
 
 from ubuntu_system_tests.helpers.session.release import get_release, XENIAL
 
@@ -24,6 +24,9 @@
 from ubuntu_system_tests.helpers.testbed import run_command
 from ubuntu_system_tests.helpers.gnome_sofware.app import GnomeSoftware
 from ubuntu_system_tests.tests.base import BaseUbuntuSystemTestCase
+from ubuntu_system_tests.helpers.snapd.snapd import Snapd
+from ubuntu_system_tests.helpers.chromium.app import Chromium
+from ubuntu_system_tests.helpers.entangle.app import Entangle
 
 
 class TestGnomeSoftware(BaseUbuntuSystemTestCase):
@@ -47,7 +50,7 @@
                              'Search entry is not hidden')
 
     def test_check_installed_deb(self):
-        """ Check already installed deb """
+        """Check already installed deb"""
         # Perform the search
         self.app.click_search_button()
         results_page = self.app.write_search('gnome-software')
@@ -66,6 +69,60 @@
         item_details = results_page.open_item(item)
         self.assertTrue(item_details.check_description())
 
+    def test_number_of_characters_in_search_string(self):
+        """Check the 2, 3 and 5 chars long search strings"""
+        # Perform the search with 2 chars
+        self.app.click_search_button()
+        results_page = self.app.write_search('ge')
+        results = results_page.get_results(retries=2)
+        # Verify that no search was performed
+        self.assertEquals(0, len(results),
+                          'Expected 0 items. But found: %s' % len(results))
+        # Add the third character
+        results_page = self.app.write_search('d')
+        results = results_page.get_results()
+        # Check if the search results contain 'gedit'
+        self.assertTrue(results_page.check_app_present(results,
+                                                       'gedit'))
+        items_number = len(results)
+        # Add another two chars, so we have 'gedit' in the box
+        results_page = self.app.write_search('it')
+        results = results_page.get_results()
+        # Check if the search results contain 'gedit'
+        self.assertTrue(results_page.check_app_present(results,
+                                                       'gedit'))
+        # Check if there are fewer results than before
+        self.assertTrue(items_number > len(results))
+
+    def test_several_searches(self):
+        """Check if several searches in a row perform normally"""
+        # Perform the first search
+        self.app.click_search_button()
+        results_page = self.app.write_search('gimp')
+        # Verify that the search results contain 'GIMP'
+        results = results_page.get_results()
+        self.assertTrue(results_page.check_app_present(results,
+                                                       'GIMP'))
+        # Clean the search box by double clicking the button and
+        # perform the second search
+        self.app.click_search_button()
+        self.app.click_search_button()
+        results_page = self.app.write_search('editor')
+        # Check if the search results contain 'gedit'
+        results = results_page.get_results()
+        self.assertTrue(results_page.check_app_present(results,
+                                                       'gedit'))
+
+    def test_multi_words_search(self):
+        """Check if multi words search performs normally"""
+        # Perform the multi words search
+        self.app.click_search_button()
+        results_page = self.app.write_search('gnome software')
+        # Verify that the search results contain 'GNOME Software'
+        results = results_page.get_results()
+        self.assertTrue(results_page.check_app_present(results,
+                                                       'GNOME Software'))
+
     def test_all_page_sections(self):
         """Check categories after clicking on down arrow"""
         labels = self.app.get_all_page_sections()
@@ -111,3 +168,184 @@
         items = self.app.get_editors_picks()
         for item in items:
             self.assertTrue(self.app.check_item(item, 'image'))
+
+    @skip('This test never passes, open_featured_app function needs to be fixed')
+    def test_featured_application_description(self):
+        """Check there is a description after clicking the featured
+        application image"""
+        featured_details = self.app.open_featured_app()
+        self.assertTrue(featured_details.check_description())
+
+
+class TestGnomeSoftwareMixedResults(BaseUbuntuSystemTestCase):
+
+    def setUp(self):
+        super(TestGnomeSoftwareMixedResults, self).setUp()
+        self.snapd = Snapd()
+        run_command('killall gnome-software')
+        self.is_snap_installed = self.snapd.is_installed('chromium')
+        if not self.is_snap_installed:
+            self.snapd.install('chromium')
+        self.chromium = Chromium()
+        self.is_deb_installed = self.chromium._is_installed()
+        if self.is_deb_installed:
+            self.chromium.remove()
+        self.root = GnomeSoftware().launch(app_type='gtk')
+        self.app = GnomeSoftwareCPO(self.root)
+
+    @skip('This tests randomly fails, so it needs to be fixed')
+    def test_mixed_results(self):
+        """Check there are deb and snap versions"""
+        # Perform the search
+        self.app.click_search_button()
+        results_page = self.app.write_search('chromium')
+        # Verify that there is more than one result
+        results = results_page.get_results()
+        self.assertTrue(len(results) > 1)
+        # Check item attributes in results
+        self.assertTrue(results_page.check_item(results[0],
+                                                'image',
+                                                'description_label'))
+        self.assertTrue(results_page.check_item(results[1],
+                                                'image',
+                                                'description_label'))
+        # Determine which result is a snap and which is a deb
+        if (results_page.get_app_name(results[0]) == 'Chromium'):
+            i = 0
+            j = 1
+        else:
+            i = 1
+            j = 0
+        # Check if the snap is visible as installed
+        self.assertTrue(results_page.check_item(results[i],
+                                                'label_installed'))
+        # Check if there is rating of the deb
+        self.assertTrue(results_page.check_item(results[j],
+                                                'star'))
+        # Check if the deb is visible as not installed
+        self.assertFalse(results_page.check_item(results[j],
+                                                 'label_installed'))
+
+    def tearDown(self):
+        super(TestGnomeSoftwareMixedResults, self).tearDown()
+        if not self.is_snap_installed:
+            self.snapd.remove('chromium')
+        if self.is_deb_installed:
+            self.chromium.install()
+
+
+class TestGnomeSoftwareInstalledSnap(BaseUbuntuSystemTestCase):
+
+    def setUp(self):
+        super(TestGnomeSoftwareInstalledSnap, self).setUp()
+        self.snapd = Snapd()
+        run_command('killall gnome-software')
+        self.is_installed = self.snapd.is_installed('hello-unity')
+        if not self.is_installed:
+            self.snapd.install('hello-unity')
+        self.root = GnomeSoftware().launch(app_type='gtk')
+        self.app = GnomeSoftwareCPO(self.root)
+
+    @skip('In some cases this test freezes at setup, so it needs to be fixed')
+    def test_check_installed_snap(self):
+        """Check already installed snap"""
+        # Perform the search
+        self.app.click_search_button()
+        results_page = self.app.write_search('hello-unity')
+        # Verify that the result contains only 1 line with hello-unity.
+        results = results_page.get_results()
+        self.assertEquals(1, len(results),
+                          'Expected 1 item. But found: %s' % len(results))
+        item = results[0]
+        # Check item attributes in results
+        self.assertTrue(results_page.check_item(item,
+                                                'image',
+                                                'description_label',
+                                                'label_installed'))
+        # Open item and check details
+        item_details = results_page.open_item(item)
+        self.assertTrue(item_details.check_description())
+
+    def tearDown(self):
+        super(TestGnomeSoftwareInstalledSnap, self).tearDown()
+        if not self.is_installed:
+            self.snapd.remove('hello-unity')
+
+
+class TestGnomeSoftwareNotInstalledSnap(BaseUbuntuSystemTestCase):
+
+    def setUp(self):
+        super(TestGnomeSoftwareNotInstalledSnap, self).setUp()
+        self.snapd = Snapd()
+        run_command('killall gnome-software')
+        self.is_installed = self.snapd.is_installed('hugo')
+        if self.is_installed:
+            self.snapd.remove('hugo')
+        self.root = GnomeSoftware().launch(app_type='gtk')
+        self.app = GnomeSoftwareCPO(self.root)
+
+    def test_check_not_installed_snap(self):
+        """Check not installed snap"""
+        # Verify if the snap is not installed
+        self.assertFalse(self.snapd.is_installed('hugo'))
+        # Perform the search
+        self.app.click_search_button()
+        results_page = self.app.write_search('hugo')
+        # Verify that the result contains only 1 line with hugo.
+        results = results_page.get_results()
+        self.assertEquals(1, len(results),
+                          'Expected 1 item. But found: %s' % len(results))
+        item = results[0]
+        # Check item attributes in results
+        self.assertTrue(results_page.check_item(item,
+                                                'image',
+                                                'description_label'))
+        self.assertFalse(results_page.check_item(item,
+                                                 'label_installed'))
+        # Open item and check details
+        item_details = results_page.open_item(item)
+        self.assertTrue(item_details.check_description())
+
+    def tearDown(self):
+        super(TestGnomeSoftwareNotInstalledSnap, self).tearDown()
+        if self.is_installed:
+            self.snapd.install('hugo')
+
+
+class TestGnomeSoftwareNotInstalledDeb(BaseUbuntuSystemTestCase):
+
+    def setUp(self):
+        super(TestGnomeSoftwareNotInstalledDeb, self).setUp()
+        run_command('killall gnome-software')
+        self.entangle = Entangle()
+        self.is_installed = self.entangle._is_installed()
+        if self.is_installed:
+            self.entangle.remove()
+        self.root = GnomeSoftware().launch(app_type='gtk')
+        self.app = GnomeSoftwareCPO(self.root)
+
+    def test_check_not_installed_deb(self):
+        """Check not installed deb"""
+        # Perform the search
+        self.app.click_search_button()
+        results_page = self.app.write_search('entangle')
+        # Verify that the result contains only 1 line with entangle.
+        results = results_page.get_results()
+        self.assertEquals(1, len(results),
+                          'Expected 1 item. But found: %s' % len(results))
+        item = results[0]
+        # Check item attributes in results
+        self.assertTrue(results_page.check_item(item,
+                                                'image',
+                                                'star',
+                                                'description_label'))
+        self.assertFalse(results_page.check_item(item,
+                                                 'label_installed'))
+        # Open item and check details
+        item_details = results_page.open_item(item)
+        self.assertTrue(item_details.check_description())
+
+    def tearDown(self):
+        super(TestGnomeSoftwareNotInstalledDeb, self).tearDown()
+        if self.is_installed:
+            self.entangle.install()