Merge lp:~canonical-platform-qa/ubuntu-system-tests/app-launch-introspect into lp:ubuntu-system-tests

[Richard Huddie]

Use introspection in app launch tests to validate app has launched.

Changes:
- Change config value package_type to unity8_mode for clarity
- Add Snap and Deb application classes, derived from Application.
- Add some missing cpos used to validate app has launched
- Fix the is_discovery() method which was not working correctly when checking if an app is installed.
- Update target_setup.py script to install snap version of apps on a deb session.
- Remove all image analysis from test_launch_apps and replace with introspection.

Currently only the deb version works when launching app once on unity8 deb session. In this case there should be 2 pass and 8 failed. All other variants will fail due to problems with app launch tools.

To test:

python3 -m ubuntu_system_tests.run run ubuntu_system_tests.tests.test_launch_apps.LaunchAppOnceTestCase

python3 -m ubuntu_system_tests.run run ubuntu_system_tests.tests.test_launch_apps.LaunchAppTwiceTestCase

This should be landed in conjunction with following for qa-jenkins-jobs: https://code.launchpad.net/~canonical-platform-qa/qa-jenkins-jobs/ust-add-app-modes/+merge/319313

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:527
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1172/
Executed test runs:
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/543/console
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/540/console
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2069/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1172/rebuild

review: Needs Fixing (continuous-integration)

[Santiago Baldassin (sbaldassin)]

Thanks for the mp Richard. I'm afraid I disagree on the design followed. With minimal changes we would be able to run tests against an image which has both snaps and deb installed, all as part of the same job. Would you please take a look at this? https://pastebin.ubuntu.com/24140328/

I've just run ./system-tests run ubuntu_system_tests.tests.test_launch_apps.LaunchAppOnceTestCase with the design above and I was able to run the tests for calc, webbrowser and system settings as part of the same execution

review: Needs Fixing

[Richard Huddie (rhuddie)]

The reason for doing this way is to write the tests without knowledge of the type of app being used. - Hence using the config value.

Although the app-launch tests could be implemented with snap/deb specific knowledge, it wouldn't be good to have a suite of sanity or regression tests where we need to have specific test cases for specific app versions. - The way I see it is you would want to run a sanity suite on a deb image for deb apps and then run exactly the same set of tests on same image for snap apps. With the current way that would just mean changing the config value and re-running the same set of tests on same image. Similarly for a snap based image, you would want to run the suite using snap apps only.

The jenkins jobs I am creating will do this from a single job. See: https://code.launchpad.net/~canonical-platform-qa/qa-jenkins-jobs/ust-add-app-modes/+merge/319313

I'm not sure if I understood your concerns correctly or not, what do you think?

[Santiago Baldassin (sbaldassin)]

> The reason for doing this way is to write the tests without knowledge of the
> type of app being used. - Hence using the config value.
>
> Although the app-launch tests could be implemented with snap/deb specific
> knowledge, it wouldn't be good to have a suite of sanity or regression tests
> where we need to have specific test cases for specific app versions.

This is actually my point, with the approach I suggested, the tests do not know if they are dealing with a snap or a deb, they are just dealing with an application. So for example in the case of the app launch test cases, the test would call calculator.launch() or web_browser.launch(), and since they are both extending from different classes, the apps will be launched differently.

Right now, we specify a mode and we skip those tests that do not comply with that mode right? With the approach I suggest, we don't specify a mode at all. Every app (either deb or snap) knows how to launch itself

 - The way
> I see it is you would want to run a sanity suite on a deb image for deb apps
> and then run exactly the same set of tests on same image for snap apps.

There you go...so this is what we need to agree on. The way I see it, you have an image and you want to run the sanity suite against that image, no matter what that image has, either debs or snaps or both, as you said, the test do not care

If we follow the mode approach, we would have to start skipping test cases based on the mode right? just as tests are skipped here. And when a deb is moved to snap we'll need to do the other way around. With the approach that I proposed, we don't skip tests at all and when a deb app is snapped, then we just change the inheritance from Deb to Snap

With
> the current way that would just mean changing the config value and re-running
> the same set of tests on same image. Similarly for a snap based image, you
> would want to run the suite using snap apps only.
>

From my point of view the suite of tests is exactly the same either in a pure snap, pure deb or in a mix of debs and snaps like we have now. The tests do not change, what changes, is how you start ans introspect a app or a snap

> The jenkins jobs I am creating will do this from a single job. See:
> https://code.launchpad.net/~canonical-platform-qa/qa-jenkins-jobs/ust-add-app-
> modes/+merge/319313
>

I was not clear. What I don't want to do is to re run the tests and merge the results when we could run the tests once and avoid the post processing

> I'm not sure if I understood your concerns correctly or not, what do you
> think?

Let's go through this over the phone. It'll be much easier, and we can get everyone opinion on this

[Richard Huddie (rhuddie)]

> This is actually my point, with the approach I suggested, the tests do not
> know if they are dealing with a snap or a deb, they are just dealing with an
> application. So for example in the case of the app launch test cases, the test
> would call calculator.launch() or web_browser.launch(), and since they are
> both extending from different classes, the apps will be launched differently.

But this would only work for for one instance of the app, either deb or snap, right? So if you wanted to run your tests against both deb and snap versions of a single application (as is the case with the app launch tests) you would have to change the base class to do so.

If we were running these tests on an deb based image we would want to test both deb and snap versions of the app, but for a snap based image we would only want to test the snap version of the app. I don't see how the approach you mentioned could work with that scenario?

>
> If we follow the mode approach, we would have to start skipping test cases
> based on the mode right? just as tests are skipped here. And when a deb is
> moved to snap we'll need to do the other way around. With the approach that I
> proposed, we don't skip tests at all and when a deb app is snapped, then we
> just change the inheritance from Deb to Snap

I think this is the key point. Doing this would require a code change, and does not allow for testing both deb and snap versions of a single application at the same time.

>
> Let's go through this over the phone. It'll be much easier, and we can get
> everyone opinion on this

Sure, we can discuss more later.

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:528
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1177/
Executed test runs:
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/561/console
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/558/console
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2122/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1177/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:538
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1178/
Executed test runs:
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/567/console
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/564/console
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2130/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1178/rebuild

review: Needs Fixing (continuous-integration)

[lp:~canonical-platform-qa/ubuntu-system-tests/app-launch-introspect]

539. By Richard Huddie on 2017-03-14

Fix flake8.

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:539
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1179/
Executed test runs:
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/569/console
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/566/console
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2132/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1179/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:539
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1180/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2133/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1180/rebuild

review: Approve (continuous-integration)

[Santiago Baldassin (sbaldassin)]

Code looks good to me

review: Approve

=== modified file 'ubuntu_system_tests/common/config.py'
--- ubuntu_system_tests/common/config.py	2017-03-06 17:27:25 +0000
+++ ubuntu_system_tests/common/config.py	2017-03-14 11:03:55 +0000
@@ -211,10 +211,9 @@
         default='qemu',
         help_string='Type of target. Either qemu or adb.'),
     options.Option(
-        'package_type',
+        'unity8_mode',
         default='deb',
-        help_string='Which unity8 package type should be installed during '
-                    'setup. Either snap or deb.'),
+        help_string='Which unity8 session to use, either deb or snap.'),
 ]
 
 

=== modified file 'ubuntu_system_tests/helpers/addressbook/app.py'
--- ubuntu_system_tests/helpers/addressbook/app.py	2017-02-08 15:03:35 +0000
+++ ubuntu_system_tests/helpers/addressbook/app.py	2017-03-14 11:03:55 +0000
@@ -18,21 +18,20 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 from collections import namedtuple
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 
 APP_NAME = 'Contacts'
 APP_ID = 'address-book-app'
-APP_SNAP = 'address-book-app'
 
 
 Contact = namedtuple('Contact', ['name', 'number', 'expected_reply'])
 Contact.__new__.__defaults__ = (None, None, None)
 
 
-class AddressBook(Application):
+class AddressBook(Snap):
 
     def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
+        super().__init__(APP_NAME, APP_ID)
 
     def get_main_view(self):
         # TODO Move the address book helper upstream migrating the

=== modified file 'ubuntu_system_tests/helpers/application.py'
--- ubuntu_system_tests/helpers/application.py	2017-03-08 01:00:05 +0000
+++ ubuntu_system_tests/helpers/application.py	2017-03-14 11:03:55 +0000
@@ -29,76 +29,75 @@
 
 from ubuntu_system_tests.helpers import autopilot
 from ubuntu_system_tests.helpers import processes
-from ubuntu_system_tests.helpers.scopes.apps import (
-    launch_application_from_apps_scope)
+from ubuntu_system_tests.helpers.snap import (
+    get_snap_revision,
+    is_installed as is_snap_installed,
+)
 from ubuntu_system_tests.helpers.unity8.utils import (
     is_unity8_snap,
     launch_application_from_launcher,
 )
 from ubuntu_system_tests.helpers.utils import is_discovery
 
-UNITY8_DEB_CMD_PREFIX = ''
-UNITY8_SNAP_CMD_PREFIX = 'unity8-session.'
-APP_SNAP_URL = 'appid://{n}/{n}/current-user-version'
-APP_DEB_URL = 'application:///{n}.desktop'
-
 logger = logging.getLogger(__name__)
-
-
-class ApplicationManagerBase:
-    """Base class to launch and close unity8 apps."""
-
-    def __init__(self, name, cmd_prefix, launch_param):
-        self.name = name
-        self.cmd_prefix = cmd_prefix
-        self.launch_param = launch_param
-        self.rev_id = self._get_rev_id()
-
-    def _get_app_ids(self):
-        """Return list of installed app ids."""
-        return subprocess.check_output(
-            self._get_ids_command()).decode().splitlines()
-
-    def _get_rev_id(self):
+logger.setLevel(logging.INFO)
+
+
+class Application:
+    __metaclass__ = ABCMeta
+
+    def __init__(self, app_name, app_id):
+        """
+        :param app_name: Display name for application.
+        :param app_id: Application id.
+        """
+        self.app_name = app_name
+        self.app_id = app_id
+        self.proxy_object = None
+        self.cmd_prefix = None
+
+    def _get_cmd_prefix(self):
+        """Return prefix to use with all ubuntu-app-launch commands.
+        This will depend on whether unity8 is running from a snap
+        session or not.
+        """
+        if self.cmd_prefix is None:
+            self.cmd_prefix = 'unity8-session.' if is_unity8_snap() else ''
+        return self.cmd_prefix
+
+    @abstractmethod
+    def _get_app_rev_id(self, app_id):
         """Return the application revision ID."""
-        for app_id in self._get_app_ids():
-            if app_id.startswith(self.name):
-                return app_id
-        logger.warning(
-            'No id found for app: {}. '
-            'Proceeding with this id value.'.format(self.name))
-        return self.name
+        pass
 
     def _get_start_command(self):
         """Return command to start application."""
         return [
-            self.cmd_prefix + 'url-dispatcher',
-            self.launch_param.format(n=self.name)]
+            self._get_cmd_prefix() + 'url-dispatcher',
+            self._get_launch_params()]
 
     def _get_stop_command(self):
         """Return command to stop application."""
-        return [self.cmd_prefix + 'ubuntu-app-stop', self.rev_id]
+        id = self._get_app_rev_id()
+        return [self._get_cmd_prefix() + 'ubuntu-app-stop', id]
 
     def _get_pid_command(self):
         """Return command to get application process ID."""
-        return [self.cmd_prefix + 'ubuntu-app-pid', self.rev_id]
-
-    def _get_ids_command(self):
-        """Return command to get list of available application IDs."""
-        return [self.cmd_prefix + 'ubuntu-app-launch-appids']
-
-    def launch(self):
-        """Launch application."""
-        subprocess.check_call(self._get_start_command())
+        id = self._get_app_rev_id()
+        return [self._get_cmd_prefix() + 'ubuntu-app-pid', id]
 
     def stop(self):
         """Stop application."""
-        subprocess.check_call(self._get_stop_command())
+        cmd = self._get_stop_command()
+        logger.info('application stop command: {}'.format(' '.join(cmd)))
+        subprocess.check_call(cmd)
 
-    def pid(self):
+    def get_pid(self):
         """Return application process ID."""
+        cmd = self._get_pid_command()
+        logger.info('application pid command: {}'.format(' '.join(cmd)))
         try:
-            out = subprocess.check_output(self._get_pid_command()).decode()
+            out = subprocess.check_output(cmd).decode()
         except subprocess.CalledProcessError:
             pass
         else:
@@ -106,84 +105,24 @@
                 if line.isdigit():
                     return line
         raise RuntimeError(
-            'Process ID not found for app ID: {}'.format(self.rev_id))
+            'Process ID not found for app ID: {}'.format(self.app_id))
 
     def running(self):
         """Return boolean to indicate if application is running."""
         return subprocess.call(self._get_pid_command()) == 0
 
-    def installed(self):
-        """Return boolean to indicate if application is installed."""
-        app_ids = self._get_app_ids()
-        return any(app_id.startswith(self.rev_id) for app_id in app_ids)
-
-
-class DebApplicationManager(ApplicationManagerBase):
-    """Application manager for deb based apps on deb based unity8 session."""
-
-    def __init__(self, name):
-        super().__init__(name, UNITY8_DEB_CMD_PREFIX, APP_DEB_URL)
-
-
-class SnapApplicationManager(ApplicationManagerBase):
-    """Application manager for snap based apps on snap based unity8 session."""
-
-    def __init__(self, name):
-        super().__init__(name, UNITY8_SNAP_CMD_PREFIX, APP_SNAP_URL)
-
-
-class SnapOnDebApplicationManager(ApplicationManagerBase):
-    """Application manager for snap based apps on deb based unity8 session."""
-
-    def __init__(self, name):
-        super().__init__(name, UNITY8_DEB_CMD_PREFIX, APP_SNAP_URL)
-
-
-class Application:
-    __metaclass__ = ABCMeta
-
-    def __init__(self, app_name, app_id=None, app_snap=None, use_snap=False):
-        """
-        :param app_name: Display name for application.
-        :param app_id: Application id.
-        :param app_snap: Name of application snap.
-        :param use_snap: Use snap version of application even if running from
-                         a deb based unity8 session.
-        """
-        self.app_name = app_name
-        self.app_id = app_id
-        self.app_snap = app_snap
-        self.proxy_object = None
-        self.app_manager = None
-        self.use_snap = use_snap
-
-    def _get_application_manager(self):
-        """Get application manager for required application. The type of
-        application used will either be deb or snap based depending on which
-        type of unity8 session is being used. In the case where unity8 is deb
-        based but use_snap property is True, the snap version of app is used.
-        """
-        if self.app_manager is None:
-            if is_unity8_snap():
-                # This is a snap based unity8 session, so must use snap
-                # based application.
-                self.app_manager = SnapApplicationManager(self.app_snap)
-            elif self.use_snap:
-                # This is a deb based unity8 session, but we're requested to
-                # use a snap based application.
-                self.app_manager = SnapOnDebApplicationManager(self.app_snap)
-            else:
-                # This is a deb based unity8 session using deb based
-                # application.
-                self.app_manager = DebApplicationManager(self.app_id)
-
-    def launch_from_scope(self, return_proxy=True):
-        """
-        Launch app from apps scope
-        """
-        launch_application_from_apps_scope(self.app_name)
-        if return_proxy:
-            return self.get_proxy_object()
+    def launch(self):
+        """Launch application using command."""
+        cmd = self._get_start_command()
+        logger.info('application launch command: {}'.format(' '.join(cmd)))
+        subprocess.check_call(cmd)
+
+    def launch_from_app_drawer(self, return_proxy=True):
+        """
+        Launch app from app drawer.
+        """
+        # TODO: Implement with app drawer helpers.
+        pass
 
     def launch_from_launcher(self, return_proxy=True):
         """Drag out the launcher and tap the application icon to start it."""
@@ -191,33 +130,16 @@
         if return_proxy:
             return self.get_proxy_object()
 
-    def launch(self):
-        self._get_application_manager()
-        self.app_manager.launch()
-
-    def stop(self):
-        self._get_application_manager()
-        self.app_manager.stop()
-
-    def get_pid(self):
-        self._get_application_manager()
-        return self.app_manager.pid()
-
-    def is_running(self):
-        self._get_application_manager()
-        return self.app_manager.running()
-
-    @retry(stop_max_delay=30000,
-           wait_exponential_multiplier=1000,
-           wait_exponential_max=10000,
+    @retry(stop_max_delay=3000,
+           wait_fixed=1000,
            retry_on_exception=lambda exception: (
                isinstance(exception, ProcessSearchError) or
                isinstance(exception, processes.PidNotFoundError) or
+               isinstance(exception, RuntimeError) or
                isinstance(exception, NoSuchProcess)))
     def get_proxy_object(self):
         if self.proxy_object is None:
-            self.proxy_object = autopilot.get_proxy_object(
-                int(self.get_pid()))
+            self.proxy_object = autopilot.get_proxy_object(int(self.get_pid()))
         return self.proxy_object
 
     @abstractmethod
@@ -231,5 +153,56 @@
             # In this case always return True to ensure all tests can be
             # listed on the host machine.
             return True
-        self._get_application_manager()
-        return self.app_manager.installed()
+        return self._is_installed()
+
+    @abstractmethod
+    def _is_installed(self, app_id):
+        """Return boolean indicating if app is installed."""
+        pass
+
+
+class Snap(Application):
+
+    app_type = 'snap'
+
+    def __init__(self, app_name, app_id):
+        super().__init__(app_name, app_id)
+        self.app_rev_id = None
+
+    def _get_launch_params(self):
+        """Return required parameters for app launch command."""
+        return 'appid://{n}/{n}/current-user-version'.format(n=self.app_id)
+
+    def _get_app_rev_id(self):
+        """Return the application revision ID."""
+        if self.app_rev_id is None:
+            rev = get_snap_revision(self.app_id)
+            self.app_rev_id = '{n}_{n}_{r}'.format(n=self.app_id, r=rev)
+        return self.app_rev_id
+
+    def _is_installed(self):
+        """Return boolean indicating if app is installed."""
+        return is_snap_installed(self.app_id)
+
+
+class Deb(Application):
+
+    app_type = 'deb'
+
+    def __init__(self, app_name, app_id, package_id=None):
+        super().__init__(app_name, app_id)
+        self.package_id = package_id or app_id
+
+    def _get_launch_params(self):
+        """Return required parameters for app launch command."""
+        return 'application:///{n}.desktop'.format(n=self.app_id)
+
+    def _get_app_rev_id(self):
+        """Return the application revision ID."""
+        return self.app_id
+
+    def _is_installed(self):
+        """Return boolean indicating if app is installed."""
+        output = subprocess.check_output(
+            ['apt-cache', 'policy', self.package_id])
+        return len(output) > 0 and 'Installed: (none)' not in output.decode()

=== modified file 'ubuntu_system_tests/helpers/calculator/app.py'
--- ubuntu_system_tests/helpers/calculator/app.py	2017-02-09 15:14:56 +0000
+++ ubuntu_system_tests/helpers/calculator/app.py	2017-03-14 11:03:55 +0000
@@ -18,17 +18,16 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 
 APP_NAME = 'Calculator'
 APP_ID = 'ubuntu-calculator-app'
-APP_SNAP = 'ubuntu-calculator-app'
-
-
-class Calculator(Application):
+
+
+class Calculator(Snap):
 
     def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
+        super().__init__(APP_NAME, APP_ID)
 
     def get_main_view(self):
         from ubuntu_system_tests.helpers.calculator import cpo  # NOQA

=== modified file 'ubuntu_system_tests/helpers/calculator/config.py'
--- ubuntu_system_tests/helpers/calculator/config.py	2017-01-26 13:44:38 +0000
+++ ubuntu_system_tests/helpers/calculator/config.py	2017-03-14 11:03:55 +0000
@@ -19,9 +19,7 @@
 #
 import os
 from ubuntu_system_tests.helpers import file_system as fs
-from ubuntu_system_tests.helpers.calculator.app import (
-    APP_SNAP as SNAP_CALCULATOR,
-)
+from ubuntu_system_tests.helpers.calculator.app import APP_ID
 from ubuntu_system_tests.helpers.configuration import Config
 from ubuntu_system_tests.helpers.snap import (
     get_snap_revision,
@@ -35,9 +33,9 @@
 
     def __init__(self):
         if is_unity8_snap():
-            rev = get_snap_revision(SNAP_CALCULATOR)
+            rev = get_snap_revision(APP_ID)
             conf_dir = fs.get_home_snap_path(
-                SNAP_CALCULATOR, rev, fs.DIR_CONFIG, fs.DIR_CALCULATOR)
+                APP_ID, rev, fs.DIR_CONFIG, fs.DIR_CALCULATOR)
         else:
             conf_dir = os.path.join(fs.DIR_HOME_CONFIG, fs.DIR_CALCULATOR)
         self.conf_file_path = os.path.join(conf_dir, CALCULATOR_CONF_FILE)

=== modified file 'ubuntu_system_tests/helpers/calendar/app.py'
--- ubuntu_system_tests/helpers/calendar/app.py	2017-02-08 17:47:07 +0000
+++ ubuntu_system_tests/helpers/calendar/app.py	2017-03-14 11:03:55 +0000
@@ -18,17 +18,16 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 
 APP_NAME = 'Calendar'
 APP_ID = 'ubuntu-calendar-app'
-APP_SNAP = 'ubuntu-calendar-app'
-
-
-class Calendar(Application):
+
+
+class Calendar(Snap):
 
     def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
+        super().__init__(APP_NAME, APP_ID)
 
     def get_main_view(self):
         from ubuntu_system_tests.helpers.clock.cpo import MainView  # NOQA

=== modified file 'ubuntu_system_tests/helpers/camera/app.py'
--- ubuntu_system_tests/helpers/camera/app.py	2017-02-08 17:47:07 +0000
+++ ubuntu_system_tests/helpers/camera/app.py	2017-03-14 11:03:55 +0000
@@ -17,17 +17,16 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 
 APP_NAME = 'Camera'
 APP_ID = 'camera-app'
-APP_SNAP = 'camera-app'
-
-
-class Camera(Application):
+
+
+class Camera(Snap):
 
     def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
+        super().__init__(APP_NAME, APP_ID)
 
     def get_main_view(self):
         from ubuntu_system_tests.helpers.camera import cpo  # NOQA

=== modified file 'ubuntu_system_tests/helpers/clock/app.py'
--- ubuntu_system_tests/helpers/clock/app.py	2017-02-08 17:47:07 +0000
+++ ubuntu_system_tests/helpers/clock/app.py	2017-03-14 11:03:55 +0000
@@ -17,17 +17,16 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 
 APP_NAME = 'Clock'
 APP_ID = 'ubuntu-clock-app'
-APP_SNAP = 'ubuntu-clock-app'
-
-
-class Clock(Application):
+
+
+class Clock(Snap):
 
     def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
+        super().__init__(APP_NAME, APP_ID)
 
     def get_main_view(self):
         from ubuntu_system_tests.helpers.clock.cpo import MainView  # NOQA

=== modified file 'ubuntu_system_tests/helpers/dialer_app/app.py'
--- ubuntu_system_tests/helpers/dialer_app/app.py	2017-02-08 15:03:35 +0000
+++ ubuntu_system_tests/helpers/dialer_app/app.py	2017-03-14 11:03:55 +0000
@@ -17,13 +17,13 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 
 APP_NAME = 'Phone'
 APP_ID = 'com.ubuntu.dialer'
 
 
-class Dialer(Application):
+class Dialer(Snap):
 
     def __init__(self):
         super().__init__(APP_NAME, APP_ID)

=== modified file 'ubuntu_system_tests/helpers/filemanager/app.py'
--- ubuntu_system_tests/helpers/filemanager/app.py	2017-02-09 11:43:45 +0000
+++ ubuntu_system_tests/helpers/filemanager/app.py	2017-03-14 11:03:55 +0000
@@ -18,17 +18,16 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 
 APP_NAME = 'Files'
 APP_ID = 'ubuntu-filemanager-app'
-APP_SNAP = 'ubuntu-filemanager-app'
-
-
-class FileManager(Application):
+
+
+class FileManager(Snap):
 
     def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
+        super().__init__(APP_NAME, APP_ID)
 
     def get_main_view(self):
         pass

=== modified file 'ubuntu_system_tests/helpers/gallery/app.py'
--- ubuntu_system_tests/helpers/gallery/app.py	2017-02-08 17:47:07 +0000
+++ ubuntu_system_tests/helpers/gallery/app.py	2017-03-14 11:03:55 +0000
@@ -20,7 +20,7 @@
 
 from ubuntu_system_tests.helpers import autopilot
 from ubuntu_system_tests.helpers import processes
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 from ubuntu_system_tests.helpers.scopes.apps import (
     launch_application_from_apps_scope
 )
@@ -33,19 +33,19 @@
 APP_NAME = 'Gallery'
 APP_WIN_ID = 'com.ubuntu.gallery_gallery'
 APP_CLICK = 'com.ubuntu.gallery'
+APP_DOMAIN = 'com.ubuntu.gallery'
 APP_ICON_NAME = 'gallery-app.png'
 APP_ID = 'gallery-app'
-APP_SNAP = 'gallery-app'
-
-
-class Gallery(Application):
+
+
+class Gallery(Snap):
 
     def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
+        super().__init__(APP_NAME, APP_ID)
 
     def get_main_view(self):
         from ubuntu_system_tests.helpers.gallery import cpo  # NOQA
-        return self.get_proxy_object().events_view
+        return self.get_proxy_object().application
 
 
 def launch_gallery_app():

=== modified file 'ubuntu_system_tests/helpers/gallery/cpo.py'
--- ubuntu_system_tests/helpers/gallery/cpo.py	2016-12-19 18:50:17 +0000
+++ ubuntu_system_tests/helpers/gallery/cpo.py	2017-03-14 11:03:55 +0000
@@ -22,6 +22,7 @@
 from autopilot.introspection.utilities import sort_by_keys
 from retrying import retry
 
+from ubuntu_system_tests.helpers.gallery.app import APP_DOMAIN
 from ubuntu_system_tests.helpers.input_manager import input_manager
 from ubuntu_system_tests.helpers.ubuntuuitools.cpo.header import Header
 from ubuntu_system_tests.helpers.ubuntuuitools.cpo.common import CustomProxyObjectBase  # NOQA
@@ -34,6 +35,19 @@
 APP_PATH_ROOT = b'comubuntugallery'
 
 
+class comubuntugallery(CustomProxyObjectBase):
+
+    @classmethod
+    def validate_dbus_object(cls, path, state):
+        return validate_dbus_object(
+            path, state, APP_PATH_ROOT, APP_PATH_ROOT,
+            applicationName=APP_DOMAIN)
+
+    @property
+    def application(self):
+        return self.select_single(GalleryApplication, visible=True)
+
+
 class GalleryApplication(CustomProxyObjectBase):
 
     @classmethod

=== modified file 'ubuntu_system_tests/helpers/messaging/app.py'
--- ubuntu_system_tests/helpers/messaging/app.py	2017-02-08 15:03:35 +0000
+++ ubuntu_system_tests/helpers/messaging/app.py	2017-03-14 11:03:55 +0000
@@ -17,7 +17,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 from ubuntu_system_tests.helpers import autopilot
 from ubuntu_system_tests.helpers import processes
 from ubuntu_system_tests.helpers.scopes.apps import (
@@ -34,7 +34,7 @@
 APP_WIN_ID = 'messaging-app'
 
 
-class Messaging(Application):
+class Messaging(Snap):
 
     def __init__(self):
         super().__init__(APP_NAME, APP_ID)

=== modified file 'ubuntu_system_tests/helpers/snap.py'
--- ubuntu_system_tests/helpers/snap.py	2017-01-26 14:06:10 +0000
+++ ubuntu_system_tests/helpers/snap.py	2017-03-14 11:03:55 +0000
@@ -25,15 +25,29 @@
 SOCK_SNAPD = '/run/snapd.socket'
 
 
+def get_snap_data(snap_name):
+    """Return snap data dictionary or None if not found."""
+    session = requests_unixsocket.Session()
+    path = quote_plus(SOCK_SNAPD)
+    url = 'http+unix://{p}/v2/snaps/{s}'.format(p=path, s=snap_name)
+    response = session.get(url)
+    if not response.ok:
+        return None
+    return json.loads(response.content.decode())['result']
+
+
 def get_snap_revision(snap_name):
     """Return the revision of the specified snap.
     :param snap_name: Name of the required snap package.
     :return: Revision number as string.
     """
-    session = requests_unixsocket.Session()
-    path = quote_plus(SOCK_SNAPD)
-    url = 'http+unix://{p}/v2/snaps/{s}'.format(p=path, s=snap_name)
-    response = session.get(url)
-    if not response.ok:
-        raise ConnectionError('Could not query snapd socket: {}'.format(url))
-    return json.loads(response.content.decode())['result']['revision']
+    snap_data = get_snap_data(snap_name)
+    if snap_data is None:
+        raise RuntimeError(
+            'Snap {s} not found'.format(s=snap_name))
+    return snap_data['revision']
+
+
+def is_installed(snap_name):
+    """Return boolean indicating if snap is installed or not."""
+    return bool(get_snap_data(snap_name))

=== modified file 'ubuntu_system_tests/helpers/system_settings/app.py'
--- ubuntu_system_tests/helpers/system_settings/app.py	2017-03-11 09:35:38 +0000
+++ ubuntu_system_tests/helpers/system_settings/app.py	2017-03-14 11:03:55 +0000
@@ -19,27 +19,55 @@
 #
 from ubuntu_system_tests.helpers import autopilot
 from ubuntu_system_tests.helpers import processes
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import (
+    Deb,
+    Snap,
+)
 from ubuntu_system_tests.helpers.processes import get_process_id
 from ubuntu_system_tests.helpers.scopes.apps import (
-    launch_application_from_apps_scope)
+    launch_application_from_apps_scope
+)
 
 APP = 'system-settings'
 APP_NAME = 'System Settings'
 APP_WIN_ID = 'ubuntu-system-settings'
 APP_ID = 'ubuntu-system-settings'
-APP_SNAP = 'ubuntu-system-settings'
-
-
-class SystemSettings(Application):
-
-    def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
-
-    def get_main_view(self):
-        from ubuntu_system_tests.helpers.system_settings import cpo  # NOQA
-        pid = get_process_id(APP)
-        return autopilot.get_proxy_object(pid=pid).main_view
+
+
+class SystemSettingsDeb(Deb):
+
+    def __init__(self):
+        super().__init__(APP_NAME, APP_ID)
+
+    def get_main_view(self):
+        return get_main_view()
+
+
+class SystemSettingsSnap(Snap):
+
+    def __init__(self):
+        super().__init__(APP_NAME, APP_ID)
+
+    def get_main_view(self):
+        return get_main_view()
+
+
+def get_main_view():
+    from ubuntu_system_tests.helpers.system_settings import cpo  # NOQA
+    pid = get_process_id(APP)
+    return autopilot.get_proxy_object(pid=pid).main_view
+
+
+def get_system_settings_app(mode):
+    """Return system settings application class of required type.
+    :param mode: Either 'snap' or 'deb'.
+    :return: Either SystemSettingsDeb or SystemSettingsSnap class reference.
+    """
+    app_modes = {
+        'deb': SystemSettingsDeb,
+        'snap': SystemSettingsSnap
+    }
+    return app_modes[mode]
 
 
 def launch_system_settings(shell):

=== modified file 'ubuntu_system_tests/helpers/telegram/app.py'
--- ubuntu_system_tests/helpers/telegram/app.py	2017-02-08 15:03:35 +0000
+++ ubuntu_system_tests/helpers/telegram/app.py	2017-03-14 11:03:55 +0000
@@ -16,13 +16,13 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 
 APP_NAME = 'Telegram'
 APP_ID = 'com.ubuntu.telegram_telegram'
 
 
-class Telegram(Application):
+class Telegram(Snap):
 
     def __init__(self):
         super().__init__(APP_NAME, APP_ID)

=== modified file 'ubuntu_system_tests/helpers/terminal/app.py'
--- ubuntu_system_tests/helpers/terminal/app.py	2017-02-09 11:43:45 +0000
+++ ubuntu_system_tests/helpers/terminal/app.py	2017-03-14 11:03:55 +0000
@@ -18,17 +18,18 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Deb
 
 APP_NAME = 'Terminal'
-APP_ID = 'ubuntu-terminal-app'
-APP_SNAP = 'ubuntu-terminal-app'
-
-
-class Terminal(Application):
+APP_ID = 'com.ubuntu.terminal'
+APP_PACKAGE_ID = 'ubuntu-terminal-app'
+
+
+class Terminal(Deb):
 
     def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
+        super().__init__(APP_NAME, APP_ID, APP_PACKAGE_ID)
 
     def get_main_view(self):
-        pass
+        from ubuntu_system_tests.helpers.terminal import cpo  # NOQA
+        return self.get_proxy_object().main_view

=== added file 'ubuntu_system_tests/helpers/terminal/cpo.py'
--- ubuntu_system_tests/helpers/terminal/cpo.py	1970-01-01 00:00:00 +0000
+++ ubuntu_system_tests/helpers/terminal/cpo.py	2017-03-14 11:03:55 +0000
@@ -0,0 +1,46 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+# Ubuntu System Tests
+# Copyright (C) 2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from ubuntu_system_tests.helpers.autopilot import validate_dbus_object
+from ubuntu_system_tests.helpers.ubuntuuitools.cpo.common import (
+    CustomProxyObjectBase,
+)
+
+APP_PATH_ROOT = b'comubuntuterminal'
+
+
+class comubuntuterminal(CustomProxyObjectBase):
+    """Autopilot helper object for terminal app."""
+
+    @classmethod
+    def validate_dbus_object(cls, path, state):
+        return validate_dbus_object(
+            path, state, APP_PATH_ROOT, APP_PATH_ROOT,
+            applicationName='com.ubuntu.terminal')
+
+    @property
+    def main_view(self):
+        return self.wait_select_single(MainView)
+
+
+class MainView(CustomProxyObjectBase):
+
+    @classmethod
+    def validate_dbus_object(cls, path, state):
+        return validate_dbus_object(
+            path, state, APP_PATH_ROOT, b'MainView', objectName='terminal')

=== modified file 'ubuntu_system_tests/helpers/utils.py'
--- ubuntu_system_tests/helpers/utils.py	2017-02-28 10:57:31 +0000
+++ ubuntu_system_tests/helpers/utils.py	2017-03-14 11:03:55 +0000
@@ -30,7 +30,7 @@
 DUMMY_IMPORT_ERR = ('DummyImportObject being used because try_import failed. '
                     'Check all required dependencies are installed.')
 
-is_discovery = None
+discovery = None
 
 
 def try_import(name, package=None):
@@ -144,16 +144,16 @@
 
 def is_discovery():
     """Return True if being imported during test discovery."""
-    global is_discovery
-    if is_discovery is None:
+    global discovery
+    if discovery is None:
         stack_trace = stack()
         stack_trace.reverse()
         for frame in stack_trace:
-            if frame[3] == 'load_test_suite_from_name':
-                is_discovery = True
-                return is_discovery
-        is_discovery = False
-    return is_discovery
+            if frame[3] == 'list_tests':
+                discovery = True
+                return discovery
+        discovery = False
+    return discovery
 
 
 class AttrDict(dict):

=== modified file 'ubuntu_system_tests/helpers/webbrowser/app.py'
--- ubuntu_system_tests/helpers/webbrowser/app.py	2017-02-08 17:02:14 +0000
+++ ubuntu_system_tests/helpers/webbrowser/app.py	2017-03-14 11:03:55 +0000
@@ -18,17 +18,16 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from ubuntu_system_tests.helpers.application import Application
+from ubuntu_system_tests.helpers.application import Snap
 
 APP_NAME = 'Browser'
 APP_ID = 'webbrowser-app'
-APP_SNAP = 'webbrowser-app'
-
-
-class WebBrowser(Application):
+
+
+class WebBrowser(Snap):
 
     def __init__(self):
-        super().__init__(APP_NAME, APP_ID, APP_SNAP)
+        super().__init__(APP_NAME, APP_ID)
 
     def get_main_view(self):
         from ubuntu_system_tests.helpers.webbrowser import cpo  # NOQA

=== modified file 'ubuntu_system_tests/host/target_setup.py'
--- ubuntu_system_tests/host/target_setup.py	2017-02-20 19:49:08 +0000
+++ ubuntu_system_tests/host/target_setup.py	2017-03-14 11:03:55 +0000
@@ -178,6 +178,17 @@
                 ['apt-get', '-y', '--no-install-recommends', 'install'] +
                 packages)
 
+    def install_snaps(self, snaps):
+        if snaps:
+            self.mount_fs_rw()
+            for snap in snaps:
+                cmd = 'refresh' if self.is_snap_installed(snap) else 'install'
+                subprocess.check_call(
+                    ['snap', cmd, '--devmode', '--edge', snap])
+
+    def is_snap_installed(self, snap):
+        return os.path.isdir(os.path.join('/snap', snap))
+
     def are_packages_installed(self, packages):
         output = subprocess.check_output(['apt-cache', 'policy'] + packages)
         return len(output) > 0 and 'Installed: (none)' not in output.decode()
@@ -206,15 +217,31 @@
             # In this case we assume running vivid based ubuntu-touch which
             # already has unity8 running, so just return.
             return
-        if self.config.get('package_type') == 'snap':
-            # install and setup unity8-snap-session
+        self.setup_snapd_https_proxy()
+        if self.config.get('unity8_mode') == 'snap':
+            # install unity8-session snap and set as default session
             self.install_packages(['unity8-session-snap'])
-            self.setup_snapd_https_proxy()
             subprocess.check_call(['unity8-snap-install'])
             session_name = 'unity8-snap'
         else:
-            # install unity8-desktop-session
-            self.install_packages(['unity8-desktop-session'])
+            # install deb based unity8-desktop-session and set as default.
+            # also install ubuntu-terminal-app which is tested as deb
+            self.install_packages([
+                'unity8-desktop-session',
+                'ubuntu-terminal-app',
+            ])
+            # Also install applications as snaps, which can also be launched
+            # in a deb based session
+            self.install_snaps([
+                'address-book-app',
+                'camera-app',
+                'gallery-app',
+                'ubuntu-calculator-app',
+                'ubuntu-calendar-app',
+                'ubuntu-clock-app',
+                'ubuntu-filemanager-app',
+                'webbrowser-app',
+            ])
             session_name = 'unity8'
         self.set_unity8_default_session(session_name)
 

=== modified file 'ubuntu_system_tests/host/targets.py'
--- ubuntu_system_tests/host/targets.py	2017-03-07 11:20:22 +0000
+++ ubuntu_system_tests/host/targets.py	2017-03-14 11:03:55 +0000
@@ -183,7 +183,7 @@
                     'network_ready', 'update_apt', 'verbose']:
             config[key] = getattr(args, key, False)
         # Add options from config_stack
-        for key in ['wifi_ssid', 'wifi_password', 'package_type']:
+        for key in ['wifi_ssid', 'wifi_password', 'unity8_mode']:
             config[key] = config_stack.get(key)
         # Add list of required test dependencies
         config['test_dependencies'] = debian.get_dependencies(

=== modified file 'ubuntu_system_tests/tests/base.py'
--- ubuntu_system_tests/tests/base.py	2017-03-09 02:42:26 +0000
+++ ubuntu_system_tests/tests/base.py	2017-03-14 11:03:55 +0000
@@ -58,10 +58,6 @@
     disable_edges_intro,
     setup_unity
 )
-from ubuntu_system_tests.helpers.webapp import (
-    DEFAULT_WEBVIEW_INSPECTOR_IP,
-    DEFAULT_WEBVIEW_INSPECTOR_PORT,
-)
 
 from ubuntu_system_tests.tests.data.definitions import DESKTOP_PLATFORM_NAME
 
@@ -211,6 +207,10 @@
         """ Setup the environment for the webapps. This method has to be
         called before launch the first web app in the test.
         """
+        from ubuntu_system_tests.helpers.webapp import (
+            DEFAULT_WEBVIEW_INSPECTOR_IP,
+            DEFAULT_WEBVIEW_INSPECTOR_PORT,
+        )
         self.useFixture(JobEnvironmentVariable(
             global_=True,
             UBUNTU_WEBVIEW_DEVTOOLS_HOST=DEFAULT_WEBVIEW_INSPECTOR_IP,

=== modified file 'ubuntu_system_tests/tests/test_launch_apps.py'
--- ubuntu_system_tests/tests/test_launch_apps.py	2017-02-13 14:05:03 +0000
+++ ubuntu_system_tests/tests/test_launch_apps.py	2017-03-14 11:03:55 +0000
@@ -18,399 +18,124 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-import dbus
 import logging
-import os
-import pwd
-import subprocess
-import tempfile
-import time
-
-from autopilot.testcase import AutopilotTestCase
-from collections import namedtuple
-from PIL import Image
-from retrying import retry
-from testtools.content import (
-    attach_file,
-    ContentType,
-)
-
-
+
+from ubuntu_system_tests.common.config import get_device_config_stack
 from ubuntu_system_tests.helpers.addressbook.app import AddressBook
 from ubuntu_system_tests.helpers.calculator.app import Calculator
+from ubuntu_system_tests.helpers.calculator.fixture_setup import (
+    CalculatorConfigFixture,
+)
 from ubuntu_system_tests.helpers.calendar.app import Calendar
 from ubuntu_system_tests.helpers.camera.app import Camera
 from ubuntu_system_tests.helpers.clock.app import Clock
 from ubuntu_system_tests.helpers.filemanager.app import FileManager
 from ubuntu_system_tests.helpers.gallery.app import Gallery
 from ubuntu_system_tests.helpers.terminal.app import Terminal
-from ubuntu_system_tests.helpers.system_settings.app import SystemSettings
+from ubuntu_system_tests.helpers.system_settings.app import (
+    get_system_settings_app,
+)
 from ubuntu_system_tests.helpers.webbrowser.app import WebBrowser
-
-from ubuntu_system_tests.helpers.calculator.fixture_setup import (
-    CalculatorConfigFixture,
-)
-from ubuntu_system_tests.helpers.images import (
-    convert_rgba_image,
-    get_color_count,
-    get_color_percentage,
-    _rmsdiff as rmsdiff,
-)
-from ubuntu_system_tests.helpers.mir import get_rgba_screenshot
-
-from ubuntu_system_tests.helpers.unity8.utils import is_unity8_snap
-
-APP_LAUNCH_TIMEOUT = 30
-BLACK_THRESHOLD = 17
-PIXELS_X = '1024'
-PIXELS_Y = '768'
-RESOLUTION = '{x}x{y}'.format(x=PIXELS_X, y=PIXELS_Y)
-
-app_ids = None
+from ubuntu_system_tests.tests.base import BaseUbuntuSystemTestCase
 
 logger = logging.getLogger(__name__)
 logger.setLevel(logging.INFO)
 
-# TODO: Remove these AppInfo definitions when autopilot introspection is
-#       working on snap based session.
-AppInfo = namedtuple('AppInfo', ['app_class', 'color_stats_list'])
-APP_INFO_ADDRESSBOOK = AppInfo(AddressBook, [(8, 200, 400)])
-APP_INFO_CAMERA = AppInfo(Camera, [
-                              (16, 1700, 2000),  # Full screen
-                              (16, 28000, 31000)])  # Low space warning
-APP_INFO_GALLERY = AppInfo(Gallery, [(10, 200, 400)])
-APP_INFO_CALCULATOR = AppInfo(Calculator, [(8, 4400, 4800)])
-APP_INFO_CALENDAR = AppInfo(Calendar, [(8, 2300, 2700)])
-APP_INFO_CLOCK = AppInfo(Clock, [
-                              (8, 300, 600),  # Full screen
-                              (8, 4000, 4500)])  # Right hand side dock
-APP_INFO_FILEMANAGER = AppInfo(FileManager, [(8, 1900, 2200)])
-APP_INFO_TERMINAL = AppInfo(Terminal, [(85, 600, 900)])
-APP_INFO_WEBBROWSER = AppInfo(WebBrowser, [(8, 2000, 2300)])
-APP_INFO_SETTINGS = AppInfo(SystemSettings, [
-                              (5, 200, 600),  # Full screen
-                              (5, 2100, 2500),  # Left hand side
-                              (5, 2800, 3100)])  # Left hand side, compacted
-
-unity8_snap = None
-
-
-# TODO: Remove and replace with SetupUnity8 fixture when working in snap based
-#       session.
-def restart_unity8():
-    """Restart unity8 process."""
-    # Ensure unity8 is running in first place
-    wait_for_unity8_running()
-    # kill process and check that unity8 restarts by itself
-    subprocess.check_call(['pkill', 'unity8'])
-    # Allow screen to go black before waiting for it to run again
-    time.sleep(10)
-    wait_for_unity8_running()
-
-
-def wait_for_unity8_running():
-    """Wait until unity8 is displaying on screen."""
-    for c in range(60):
-        screen = get_screenshot()
-        black = get_color_percentage(screen, BLACK_THRESHOLD)
-        count = get_color_count(screen)
-        logger.info('#### BLACK :: {} :: COLORS :: {}'.format(black, count))
-        os.remove(screen)
-        if black < 50 and count > 100:
-            # Black screen is no longer displayed
-            return
-        time.sleep(1)
-    raise RuntimeError('Unity8 UI is not displayed!')
-
-
-def get_command(command):
-    """Return command based on whether unity8 is running from a snap or not.
-    If it is then prefix command with 'unity8-session', otherwise return
-    command with no prefix.
-    """
-    global unity8_snap
-    if unity8_snap is None:
-        unity8_snap = is_unity8_snap()
-    if unity8_snap:
-        return 'unity8-session.{}'.format(command)
-    return command
-
-
-# TODO: Remove all below screenshot analysis when autopilot introspection
-#       working on snap based session.
-def get_screenshot():
-    """
-    Take screen shot and return file path of png file
-    :return: Path of PNG screen image file
-    """
-    timestamp = int(time.time())
-    filename_rgba = 'screenshot-{}.rgba'.format(timestamp)
-    filename_png = 'screenshot-{}.png'.format(timestamp)
-    temp_dir = tempfile.gettempdir()
-    filepath_rgba = os.path.join(temp_dir, filename_rgba)
-    filepath_png = os.path.join(temp_dir, filename_png)
-    get_rgba_screenshot(filepath_rgba)
-    convert_rgba_image(filepath_rgba, filepath_png, RESOLUTION)
-    # finished with rgba file so delete it
-    os.remove(filepath_rgba)
-    return filepath_png
-
-
-def images_almost_equal(im1, im2):
-    """ Determine if two images are almost equal or not
-    :return: True if the images have almost same content, False otherwise
-    """
-    try:
-        difference = rmsdiff(im1, im2)
-    except ValueError:
-        logger.info('### Images have no similarity.')
-        return False
-    logger.info('### Image difference :: {}'.format(difference))
-    return difference < 4
-
-
-def image_files_almost_equal(im1_path, im2_path):
-    """ Determine if two image files are equal or not.
-    :param im1_path: File path to first image file.
-    :param im2_path: File path to second image file.
-    :return: True if the image files have same content, False otherwise.
-    """
-    return images_almost_equal(Image.open(im1_path), Image.open(im2_path))
-
-
-# TODO: Remove this when Tutorial and SettingsWizard fixtures working.
-def get_user_account():
-    for account in sorted(pwd.getpwall(), key=lambda x: x.pw_uid):
-        if account.pw_uid > 500 and account.pw_dir.startswith('/home/'):
-            return account
-
-
-# TODO: Remove this and replace with Tutorial fixture when working.
-def disable_edges_demo():
-    # Getting or setting the dbus property must be done from user account,
-    # so use dbus-send with sudo -u to set the user.
-    setting_changed = False
-    user_id = 'User{}'.format(get_user_account().pw_uid)
-    wait_for_user(user_id)
-    user_path = '/org/freedesktop/Accounts/{}'.format(user_id)
-    user_name = get_user_account().pw_name
-    if get_edge_demo_status(user_name, user_path):
-        subprocess.check_call([
-            'sudo', '-u', user_name,
-            'dbus-send', '--system', '--print-reply',
-            '--dest=org.freedesktop.Accounts', user_path,
-            'org.freedesktop.DBus.Properties.Set',
-            'string:com.canonical.unity.AccountsService',
-            'string:demo-edges', 'variant:boolean:false'],
-            stdout=subprocess.DEVNULL)
-        setting_changed = True
-    return setting_changed
-
-
-@retry(wait_fixed=1000,
-       stop_max_attempt_number=30,
-       retry_on_exception=lambda e: (
-            isinstance(e, subprocess.CalledProcessError)))
-def get_edge_demo_status(user_name, user_path):
-    """Return edge demo boolean status."""
-    output = subprocess.check_output([
-        'sudo', '-u', user_name,
-        'dbus-send', '--system', '--print-reply',
-        '--dest=org.freedesktop.Accounts', user_path,
-        'org.freedesktop.DBus.Properties.Get',
-        'string:com.canonical.unity.AccountsService',
-        'string:demo-edges']).decode().strip().split('\n')[-1]
-    return output.split(' ')[-1] == 'true'
-
-
-@retry(wait_fixed=1000,
-       stop_max_attempt_number=30,
-       retry_on_exception=lambda e: isinstance(e, RuntimeError))
-def wait_for_user(user_id):
-    """Wait until user_id object exists under Accounts object."""
-    bus = dbus.SystemBus()
-    obj = bus.get_object(
-        'org.freedesktop.Accounts', '/org/freedesktop/Accounts')
-    intf = dbus.Interface(obj, 'org.freedesktop.DBus.Introspectable')
-    if not 'name="{}"'.format(user_id) in intf.Introspect():
-        raise RuntimeError('User {} does not exist.'.format(user_id))
-
-
-# TODO: Remove this and replace with SettingsWizard fixture when working.
-def disable_settings_wizard():
-    """Disable settings wizard from running on first boot. Return bool
-    indicating if setting was changed."""
-    setting_changed = False
-    base_path = '/home/{}'.format(get_user_account().pw_name)
-    if is_unity8_snap():
-        base_path = os.path.join(base_path, 'snap/unity8-session/common')
-    wizard_file = os.path.join(
-        base_path, '.config/ubuntu-system-settings/wizard-has-run')
-    if not os.path.isfile(wizard_file):
-        wizard_dir = os.path.dirname(wizard_file)
-        if not os.path.isdir(wizard_dir):
-            os.makedirs(wizard_dir)
-        open(wizard_file, 'a').close()
-        setting_changed = True
-    return setting_changed
-
-
-# TODO: Change to BaseUbuntuSystemTestCase when it works on snap based session.
-class AppLaunchBase(AutopilotTestCase):
-
-    @classmethod
-    def setUpClass(cls):
-        wizard_changed = disable_settings_wizard()
-        edges_changed = disable_edges_demo()
-        if wizard_changed or edges_changed:
-            restart_unity8()
+
+class AppLaunchBase(BaseUbuntuSystemTestCase):
 
     def setUp(self):
         super().setUp()
         self.app = None
-        self.before = None
-        self.after = None
-        wait_for_unity8_running()
+        self.config_stack = get_device_config_stack()
 
     def tearDown(self):
         if self.app:
-            self.close_app()
-        if self.before and os.path.isfile(self.before):
-            os.remove(self.before)
-            self.before = None
-        if self.after and os.path.isfile(self.after):
-            os.remove(self.after)
-            self.after = None
+            self.app.stop()
         super().tearDown()
 
-    def close_app(self):
-        self.app.stop()
-        time.sleep(10)
-
-    def maximise_window(self):
-        self.keyboard.press_and_release('Ctrl+LeftMeta+Up')
-        time.sleep(10)
-
-    def check_app_launch(self, app_info, launch_count):
-        self.app = app_info.app_class()
-        logger.info('### TESTING APP NAME :: {}'.format(self.app.app_name))
-        self.maximise_window()
+    def check_app_launch(self, app_class, launch_count):
+        self.app = app_class()
+        if not self.app.is_installed():
+            reason = '{n} not installed as {t}'.format(
+                n=self.app.app_id, t=self.app.app_type)
+            self.app = None
+            self.skipTest(reason)
         for count in range(1, launch_count + 1):
-            logger.info('### APP LAUNCH :: {}'.format(count))
-            # TODO: Replace all screenshot analysis with introspection when
-            #       autopilot working on snap based session.
-            self.before = get_screenshot()
-            self.attach_screenshot(
-                self.before, 'before_launched_{}'.format(count))
             self.app.launch()
-            time.sleep(10)
-            self.maximise_window()
-            self.after = get_screenshot()
-            self.attach_screenshot(
-                self.after, 'after_launched_{}'.format(count))
+            self.assertIsNotNone(self.app.get_main_view())
+            logger.info('App launch {c} successful'.format(c=count))
             if count < launch_count:
                 # Only close the app here if this is not the last iteration.
                 # Otherwise it will be closed automatically in tearDown.
-                self.close_app()
-            black = get_color_percentage(self.after, BLACK_THRESHOLD)
-            count = get_color_count(self.after)
-            logger.info(
-                '### ID {} :: BLACK {} :: COLORS '
-                '{}'.format(self.app.app_id, black, count, self.before,
-                            self.after))
-            self.assertFalse(
-                image_files_almost_equal(self.before, self.after),
-                'No difference on screen detected after app launched.')
-            self.validate_color_stats(app_info, count, black)
-
-    def validate_color_stats(self, app_info, color_count, black_percent):
-        exceptions = []
-        for stats in app_info.color_stats_list:
-            black_max = stats[0]
-            colors_min = stats[1]
-            colors_max = stats[2]
-            try:
-                self.assertGreater(color_count, colors_min)
-                self.assertLess(color_count, colors_max)
-                self.assertLess(black_percent, black_max)
-            except AssertionError as e:
-                exceptions.append(e)
-            else:
-                break
-        if len(exceptions) == len(app_info.color_stats_list):
-            # An exception was raised for every stats item, which means
-            # that none of them matched. So raise the first exception here.
-            raise exceptions[0]
-
-    def attach_screenshot(self, file_path, name):
-        attach_file(
-            self, file_path, name=name,
-            content_type=ContentType('image', 'png'), buffer_now=True)
+                self.app.stop()
 
 
 class LaunchAppOnceTestCase(AppLaunchBase):
 
     def test_launch_addressbook_app(self):
-        self.check_app_launch(APP_INFO_ADDRESSBOOK, 1)
+        self.check_app_launch(AddressBook, 1)
 
     def test_launch_camera_app(self):
-        self.check_app_launch(APP_INFO_CAMERA, 1)
+        self.check_app_launch(Camera, 1)
 
     def test_launch_gallery_app(self):
-        self.check_app_launch(APP_INFO_GALLERY, 1)
+        self.check_app_launch(Gallery, 1)
 
     def test_launch_calculator_app(self):
         self.useFixture(CalculatorConfigFixture(first_run=False))
-        self.check_app_launch(APP_INFO_CALCULATOR, 1)
+        self.check_app_launch(Calculator, 1)
 
     def test_launch_calendar_app(self):
-        self.check_app_launch(APP_INFO_CALENDAR, 1)
+        self.check_app_launch(Calendar, 1)
 
     def test_launch_clock_app(self):
-        self.check_app_launch(APP_INFO_CLOCK, 1)
+        self.check_app_launch(Clock, 1)
 
     def test_launch_filemanager_app(self):
-        self.check_app_launch(APP_INFO_FILEMANAGER, 1)
+        self.check_app_launch(FileManager, 1)
 
     def test_launch_terminal_app(self):
-        self.check_app_launch(APP_INFO_TERMINAL, 1)
+        self.check_app_launch(Terminal, 1)
 
     def test_launch_webbrowser_app(self):
-        self.check_app_launch(APP_INFO_WEBBROWSER, 1)
+        self.check_app_launch(WebBrowser, 1)
 
     def test_launch_systemsettings_app(self):
-        self.check_app_launch(APP_INFO_SETTINGS, 1)
+        app = get_system_settings_app(self.config_stack.get('unity8_mode'))
+        self.check_app_launch(app, 1)
 
 
 class LaunchAppTwiceTestCase(AppLaunchBase):
 
     def test_launch_addressbook_app_twice(self):
-        self.check_app_launch(APP_INFO_ADDRESSBOOK, 2)
+        self.check_app_launch(AddressBook, 2)
 
     def test_launch_camera_app_twice(self):
-        self.check_app_launch(APP_INFO_CAMERA, 2)
+        self.check_app_launch(Camera, 2)
 
     def test_launch_gallery_app_twice(self):
-        self.check_app_launch(APP_INFO_GALLERY, 2)
+        self.check_app_launch(Gallery, 2)
 
     def test_launch_calculator_app_twice(self):
         self.useFixture(CalculatorConfigFixture(first_run=False))
-        self.check_app_launch(APP_INFO_CALCULATOR, 2)
+        self.check_app_launch(Calculator, 2)
 
     def test_launch_calendar_app_twice(self):
-        self.check_app_launch(APP_INFO_CALENDAR, 2)
+        self.check_app_launch(Calendar, 2)
 
     def test_launch_clock_app_twice(self):
-        self.check_app_launch(APP_INFO_CLOCK, 2)
+        self.check_app_launch(Clock, 2)
 
     def test_launch_filemanager_app_twice(self):
-        self.check_app_launch(APP_INFO_FILEMANAGER, 2)
+        self.check_app_launch(FileManager, 2)
 
     def test_launch_terminal_app_twice(self):
-        self.check_app_launch(APP_INFO_TERMINAL, 2)
+        self.check_app_launch(Terminal, 2)
 
     def test_launch_webbrowser_app_twice(self):
-        self.check_app_launch(APP_INFO_WEBBROWSER, 2)
+        self.check_app_launch(WebBrowser, 2)
 
     def test_launch_systemsettings_app_twice(self):
-        self.check_app_launch(APP_INFO_SETTINGS, 2)
+        app = get_system_settings_app(self.config_stack.get('unity8_mode'))
+        self.check_app_launch(app, 2)

=== modified file 'ubuntu_system_tests/tests/test_system_settings.py'
--- ubuntu_system_tests/tests/test_system_settings.py	2017-03-08 01:00:05 +0000
+++ ubuntu_system_tests/tests/test_system_settings.py	2017-03-14 11:03:55 +0000
@@ -20,6 +20,8 @@
 
 from autopilot.matchers import Eventually
 from testtools.matchers import Equals
+
+from ubuntu_system_tests.common.config import get_device_config_stack
 from ubuntu_system_tests.helpers.network.utils import turn_on_wifi
 
 from ubuntu_system_tests.helpers.indicators.network.cpo import (
@@ -28,7 +30,9 @@
 from ubuntu_system_tests.helpers import network
 from ubuntu_system_tests.helpers import system_settings
 from ubuntu_system_tests.helpers.indicators.indicators import IndicatorsFactory
-from ubuntu_system_tests.helpers.system_settings.app import SystemSettings
+from ubuntu_system_tests.helpers.system_settings.app import (
+    get_system_settings_app,
+)
 from ubuntu_system_tests.tests import base
 
 SOUND_SETTINGS_PAGE = 'soundPage'
@@ -76,7 +80,9 @@
 
     def setUp(self):
         super().setUp()
-        system_settings_app = SystemSettings()
+        self.config_stack = get_device_config_stack()
+        system_settings_app = get_system_settings_app(
+            self.config_stack.get('unity8_mode'))()
         system_settings_app.launch()
         self.system_settings = system_settings_app.get_main_view()