Merge lp:~rhuddie/ubuntu-push/push-autopilot-tests into lp:ubuntu-push/automatic

[John Lenton]

autopilot test framework and basic coverage of broadcast notifications

Work around for excessive list of reviewers on https://code.launchpad.net/~rhuddie/ubuntu-push/push-autopilot-tests/+merge/221359

[John Lenton (chipaca)]

Already approved by everybody and their mom in the previous one.

review: Approve

=== added directory 'tests'
=== added directory 'tests/autopilot'
=== added directory 'tests/autopilot/push_notifications'
=== added file 'tests/autopilot/push_notifications/README'
--- tests/autopilot/push_notifications/README	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/README	2014-07-10 10:50:28 +0000
@@ -0,0 +1,74 @@
+==================
+README
+==================
+
+To run ubuntu-push autopilot tests you need to have a push server available. This can be running locally using loopback (127.0.0.1) or remotely on the same network.
+
+----------------------------------
+To configure and build the server:
+----------------------------------
+
+1) export GOPATH=${PWD}/push
+2) mkdir -p push/src/launchpad.net
+3) cd push/src/launchpad.net
+4) bzr branch lp:ubuntu-push
+5) Edit ubuntu-push/sampleconfigs/dev.json with the correct server IP address and ports.
+   E.g. for a server IP address 192.168.1.2 use:
+    "addr": "192.168.1.2:9090",
+    "http_addr": "192.168.1.2:8080",
+6) cd ubuntu-push
+7) ensure all dependencies are installed: sudo apt-get build-dep ubuntu-push
+8) install additional cm tools: sudo apt-get install git mercurial
+9) make bootstrap
+10) make run-server-dev
+   Following output should be observed:
+     INFO listening for http on 192.168.1.2:8080
+     INFO listening for devices on 192.168.1.2:9090
+
+------------------------
+To configure the client:
+------------------------
+
+Install depenendencies:
+
+1) sudo apt-get install unity8-autopilot unity-click-scope
+   Note: unity-click-scope is required for unity8-autopilot tests and is currently required for running tests on emulator.
+2) bzr branch lp:ubuntu-push
+3) Edit ip address and ports to match environment: ubuntu-push/tests/autopilot/push_notifications/config/push.conf:
+    [config]
+    addr = 192.168.1.2
+    listener_port = 8080
+    device_port = 9090
+    cert_pem_file = testing.cert
+4) initctl stop unity8
+5) cd ubuntu-push/tests/autopilot
+6) autopilot3 list push_notifications
+7) autopilot3 run push_notifications
+8) To run a specific test case use the test case identifier from the list command:
+     - e.g. autopilot3 run push_notifications.tests.test_broadcast_notifications.TestPushClientBroadcast.test_broadcast_push_notification
+
+----------------
+Troubleshooting:
+----------------
+
+1) Ping from client to server to ensure connectivity is correct
+2) Delete ~/.local/share/ubuntu-push-client/levels.db if no notifications are being displayed:
+    rm ~/.local/share-ubuntu-push-client/levels.db
+3) Check client log file at ~/.cache/upstart/ubuntu-push-client.log:
+    tail -f --line=30 ~/.cache/upstart/ubuntu-push-client.log
+4) To send a notification manually:
+     echo '{"channel":"system", "data": {"ubuntu-touch/utopic-proposed/mako": [94, ""]}, "expire_on": "2015-12-19T16:39:57-08:00"}' | POST -c application/json http://192.168.1.2:8080/broadcast
+   Response should be:
+     {"ok":true}
+   Note that:
+     - The channel and device names must match the client.
+     - The build number must be greater than current installed version in order to trigger an update message.
+     - The expiration time must be in the future.
+5) Ensure unity8-autopilot is installed
+6) Ensure unity8 is not running before executing the tests:
+     - initctl stop unity8
+7) Unity8 has a 2 minute timeout period, so stopping and starting it can take up to this long.
+8) If device/emulator is unresponsive then reboot and stop unity8 before re-running the tests (initctl stop unity8).
+9) To get additional autopilot logging use -v option:
+     - autopilot3 run -v push_notifications
+

=== added file 'tests/autopilot/push_notifications/__init__.py'
--- tests/autopilot/push_notifications/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/__init__.py	2014-07-10 10:50:28 +0000
@@ -0,0 +1,20 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+#
+# Push Notifications Autopilot Test Suite
+# Copyright (C) 2014 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+"""push-notifications autopilot tests."""

=== added directory 'tests/autopilot/push_notifications/config'
=== added file 'tests/autopilot/push_notifications/config/__init__.py'
--- tests/autopilot/push_notifications/config/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/config/__init__.py	2014-07-10 10:50:28 +0000
@@ -0,0 +1,39 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+#
+# Push Notifications Autopilot Test Suite
+# Copyright (C) 2014 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+import os
+
+CONFIG_FILE = 'push.conf'
+
+
+def get_config_file():
+    """
+    Return the path for the config file
+    """
+    config_dir = os.path.dirname(__file__)
+    return os.path.join(config_dir, CONFIG_FILE)
+
+
+def get_cert_file(cert_file_name):
+    """
+    Return the path for the testing certificate file
+    """
+    config_dir = os.path.dirname(__file__)
+    return os.path.join(config_dir, cert_file_name)

=== added file 'tests/autopilot/push_notifications/config/push.conf'
--- tests/autopilot/push_notifications/config/push.conf	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/config/push.conf	2014-07-10 10:50:28 +0000
@@ -0,0 +1,5 @@
+[config]
+addr = 192.168.1.3
+listener_port = 8080
+device_port = 9090
+cert_pem_file = testing.cert

=== added file 'tests/autopilot/push_notifications/config/testing.cert'
--- tests/autopilot/push_notifications/config/testing.cert	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/config/testing.cert	2014-07-10 10:50:28 +0000
@@ -0,0 +1,10 @@
+-----BEGIN CERTIFICATE-----
+MIIBYzCCAQ+gAwIBAgIBADALBgkqhkiG9w0BAQUwEjEQMA4GA1UEChMHQWNtZSBD
+bzAeFw0xMzEyMTkyMDU1NDNaFw0yMzEyMTcyMDU1NDNaMBIxEDAOBgNVBAoTB0Fj
+bWUgQ28wWjALBgkqhkiG9w0BAQEDSwAwSAJBAPw+niki17X2qALE2A2AzE1q5dvK
+9CI4OduRtT9IgbFLC6psqAT21NA+QbY17nWSSpyP65zkMkwKXrbDzstwLPkCAwEA
+AaNUMFIwDgYDVR0PAQH/BAQDAgCkMBMGA1UdJQQMMAoGCCsGAQUFBwMBMA8GA1Ud
+EwEB/wQFMAMBAf8wGgYDVR0RBBMwEYIJbG9jYWxob3N0hwR/AAABMAsGCSqGSIb3
+DQEBBQNBAFqiVI+Km2XPSO+pxITaPvhmuzg+XG3l1+2di3gL+HlDobocjBqRctRU
+YySO32W07acjGJmCHUKpCJuq9X8hpmk=
+-----END CERTIFICATE-----

=== added file 'tests/autopilot/push_notifications/data.py'
--- tests/autopilot/push_notifications/data.py	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/data.py	2014-07-10 10:50:28 +0000
@@ -0,0 +1,98 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+#
+# Push Notifications Autopilot Test Suite
+# Copyright (C) 2014 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+"""Push-Notifications autopilot data structure classes"""
+
+
+class PushNotificationMessage:
+    """
+    Class to hold all the details required for a
+    push notification message
+    """
+
+    def __init__(self, channel='system', data=None, expire_after=None):
+        """
+        Constructor
+        :param channel: Name of channel
+        :param data: Data value
+        :param expire_after: expiration time
+        """
+        self.channel = channel
+        self.data = data
+        self.expire_after = expire_after
+
+    def to_json(self):
+        """
+        Return JSON representation of message
+        :return: JSON representation of message
+        """
+        json_str = '{{"channel":"{0}", "data":{{{1}}}, "expire_on":"{2}"}}'
+        return json_str.format(self.channel, self.data, self.expire_after)
+
+
+class DeviceNotificationData:
+    """
+    Class to represent device's data used for sending notification, including:
+    - Device software channel
+    - Device build number
+    - Device model
+    - Device last update
+    - Data for the notification
+    """
+
+    def __init__(self, channel=None, device=None, build_number=None,
+                 last_update=None, version=None, data=None):
+        """
+        Constructor
+        :param channel: Name of channel
+        :param device: Name of device
+        :param build_number: Build number
+        :param last_update: Last update time
+        :param version: Build version
+        :param data: Device specific data
+        """
+        self.channel = channel
+        self.build_number = build_number
+        self.device = device
+        self.last_update = last_update
+        self.version = version
+        self.data = data
+
+    def inc_build_number(self):
+        """
+        Increment build number
+        """
+        self.build_number = str(int(self.build_number) + 1)
+
+    def dec_build_number(self):
+        """
+        Decrement build number
+        """
+        self.build_number = str(int(self.build_number) - 1)
+
+    def to_json(self):
+        """
+        Return json representation of info based:
+        "IMAGE-CHANNEL/DEVICE-MODEL": [BUILD-NUMBER, CHANNEL-ALIAS]"
+        :return: JSON representation of device data
+        """
+        json_str = '"{0}/{1}": [{2}, "{3}"]'
+        return json_str.format(self.channel, self.device, self.build_number,
+                               self.data)

=== added directory 'tests/autopilot/push_notifications/helpers'
=== added file 'tests/autopilot/push_notifications/helpers/__init__.py'
--- tests/autopilot/push_notifications/helpers/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/helpers/__init__.py	2014-07-10 10:50:28 +0000
@@ -0,0 +1,18 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+#
+# Push Notifications Autopilot Test Suite
+# Copyright (C) 2014 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#

=== added file 'tests/autopilot/push_notifications/helpers/push_notifications_helper.py'
--- tests/autopilot/push_notifications/helpers/push_notifications_helper.py	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/helpers/push_notifications_helper.py	2014-07-10 10:50:28 +0000
@@ -0,0 +1,280 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+#
+# Push Notifications Autopilot Test Suite
+# Copyright (C) 2014 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import configparser
+import datetime
+import http.client as http
+import json
+import os
+import subprocess
+import systemimage.config as sys_info
+
+from push_notifications import config as push_config
+from push_notifications.data import (
+    PushNotificationMessage,
+    DeviceNotificationData
+)
+
+
+class PushClientConfig:
+    """
+    Container class to read and hold all required server config
+    - Server listener address
+    - Server device address
+    - Certificate PEM file path
+    """
+
+    @staticmethod
+    def read_config(config_file_path):
+        """
+        Return PushClientConfig object containing all test config parameters
+        which have been read from specified config file.
+        :param config_file_path: path to required config file
+        :return: PushClientConfig object containing all config parameters
+        """
+        KEY_ADDR = 'addr'
+        KEY_LISTENER_PORT = 'listener_port'
+        KEY_DEVICE_PORT = 'device_port'
+        KEY_CONFIG = 'config'
+        KEY_CERT_PEM_FILE = 'cert_pem_file'
+
+        config = PushClientConfig()
+        parser = configparser.ConfigParser()
+        parser.read(config_file_path)
+        server_addr = parser[KEY_CONFIG][KEY_ADDR]
+        device_port = parser[KEY_CONFIG][KEY_DEVICE_PORT]
+        listener_port = parser[KEY_CONFIG][KEY_LISTENER_PORT]
+        addr_fmt = '{0}:{1}'
+        config.server_listener_addr = addr_fmt.format(
+            server_addr, listener_port)
+        config.server_device_addr = addr_fmt.format(server_addr, device_port)
+        config.cert_pem_file = push_config.get_cert_file(
+            parser[KEY_CONFIG][KEY_CERT_PEM_FILE])
+        return config
+
+
+class PushClientController:
+    """
+    Class used to reconfigure and re-start ubuntu-push-client for testing
+    """
+
+    PUSH_CLIENT_DEFAULT_CONFIG_FILE = '/etc/xdg/ubuntu-push-client/config.json'
+    PUSH_CLIENT_CONFIG_FILE = '~/.config/ubuntu-push-client/config.json'
+
+    def restart_push_client_using_config(self, client_config=None):
+        """
+        Restart the push client using the config provided
+        If the config is none then revert to default client behaviour
+        :param client_config: PushClientConfig object containing
+                              required config
+        """
+        if client_config is None:
+            # just delete the local custom config file
+            # client will then just use the original config
+            abs_config_file = self.get_abs_local_config_file_path()
+            if os.path.exists(abs_config_file):
+                os.remove(abs_config_file)
+        else:
+            # write the config to local config file
+            self.write_client_test_config(client_config)
+
+        # Now re-start the client
+        self.restart_push_client()
+
+    def write_client_test_config(self, client_config):
+        """
+        Write the test server address and certificate path
+        to the client config file
+        :param client_config: PushClientConfig object containing
+                              required config
+        """
+        # read the original push client config file
+        with open(self.PUSH_CLIENT_DEFAULT_CONFIG_FILE) as config_file:
+            config = json.load(config_file)
+        # change server address
+        config['addr'] = client_config.server_device_addr
+        # add certificate file path
+        config['cert_pem_file'] = client_config.cert_pem_file
+        # write the config json out to the ~.local address
+        # creating the directory if it doesn't already exist
+        abs_config_file = self.get_abs_local_config_file_path()
+        config_dir = os.path.dirname(abs_config_file)
+        if not os.path.exists(config_dir):
+            os.makedirs(config_dir)
+        with open(abs_config_file, 'w+') as outfile:
+            json.dump(config, outfile, indent=4)
+            outfile.close()
+
+    def get_abs_local_config_file_path(self):
+        """
+        Return absolute path of ~.local config file
+        """
+        return os.path.expanduser(self.PUSH_CLIENT_CONFIG_FILE)
+
+    def control_client(self, command):
+        """
+        start/stop/restart the ubuntu-push-client using initctl
+        """
+        subprocess.call(
+            ['/sbin/initctl', command, 'ubuntu-push-client'],
+            stdout=subprocess.DEVNULL)
+
+    def stop_push_client(self):
+        """
+        Stop the push client
+        """
+        self.control_client('stop')
+
+    def start_push_client(self):
+        """
+        Start the push client
+        """
+        self.control_client('start')
+
+    def restart_push_client(self):
+        """
+        Restart the push client
+        """
+        self.stop_push_client()
+        self.start_push_client()
+
+
+class PushNotificationHelper:
+    """
+    Utility class to create and send push notification messages
+    """
+
+    DEFAULT_BROADCAST_URL = '/broadcast'
+
+    def get_device_info(self):
+        """
+        Discover the device's model and build info
+        - device name e.g. mako
+        - channel name e.g. ubuntu-touch/utopic-proposed
+        - build_number e.g. 101
+        :return: DeviceNotificationData object containing device info
+        """
+        # channel info needs to be read from file
+        parser = configparser.ConfigParser()
+        channel_config_file = '/etc/system-image/channel.ini'
+        parser.read(channel_config_file)
+        channel = parser['service']['channel']
+        return DeviceNotificationData(
+            device=sys_info.config.device,
+            channel=channel,
+            build_number=sys_info.config.build_number)
+
+    def send_push_broadcast_notification(self, msg_json, server_addr,
+                                         url=DEFAULT_BROADCAST_URL):
+        """
+        Send the specified push message to the server broadcast url
+        using an HTTP POST command
+        :param msg_json: JSON representation of message to send
+        :param url: destination server url to send message to
+        """
+        headers = {'Content-type': 'application/json'}
+        conn = http.HTTPConnection(server_addr)
+        conn.request(
+            'POST',
+            url,
+            headers=headers,
+            body=msg_json)
+        return conn.getresponse()
+
+    def create_push_message(self, channel='system', data=None,
+                            expire_after=None):
+        """
+        Return a new push message
+        If no expiry time is given, a future date will be assigned
+        :param channel: name of the channel
+        :param data: data value of the message
+        :param expire_after: expiry time for message
+        :return: PushNotificationMessage object containing specified parameters
+        """
+        if expire_after is None:
+            expire_after = self.get_future_iso_time()
+        return PushNotificationMessage(
+            channel=channel,
+            data=data,
+            expire_after=expire_after)
+
+    def get_past_iso_time(self):
+        """
+        Return time 1 year in past in ISO format
+        """
+        return self.get_iso_time(year_offset=-1)
+
+    def get_near_past_iso_time(self):
+        """
+        Return time 5 seconds in past in ISO format
+        """
+        return self.get_iso_time(sec_offset=-5)
+
+    def get_near_future_iso_time(self):
+        """
+        Return time 5 seconds in future in ISO format
+        """
+        return self.get_iso_time(sec_offset=5)
+
+    def get_future_iso_time(self):
+        """
+        Return time 1 year in future in ISO format
+        """
+        return self.get_iso_time(year_offset=1)
+
+    def get_current_iso_time(self):
+        """
+        Return current time in ISO format
+        """
+        return self.get_iso_time()
+
+    def get_iso_time(self, year_offset=0, month_offset=0, day_offset=0,
+                     hour_offset=0, min_offset=0, sec_offset=0,
+                     tz_hour_offset=0, tz_min_offset=0):
+        """
+        Return an ISO8601 format date-time string, including time-zone
+        offset: YYYY-MM-DDTHH:MM:SS-HH:MM
+        :param year_offset: number of years to offset
+        :param month_offset: number of months to offset
+        :param day_offset: number of days to offset
+        :param hour_offset: number of hours to offset
+        :param min_offset: number of minutes to offset
+        :param sec_offset: number of seconds to offset
+        :param tz_hour_offset: number of hours to offset time zone
+        :param tz_min_offset: number of minutes to offset time zone
+        :return: string representation of required time in ISO8601 format
+        """
+        # calulate target time based on current time and format it
+        now = datetime.datetime.now()
+        target_time = datetime.datetime(
+            year=now.year + year_offset,
+            month=now.month + month_offset,
+            day=now.day + day_offset,
+            hour=now.hour + hour_offset,
+            minute=now.minute + min_offset,
+            second=now.second + sec_offset)
+        target_time_fmt = target_time.strftime('%Y-%m-%dT%H:%M:%S')
+        # format time zone offset
+        tz = datetime.time(
+            hour=tz_hour_offset,
+            minute=tz_min_offset)
+        tz_fmt = tz.strftime('%H:%M')
+        # combine target time and time zone offset
+        iso_time = '{0}-{1}'.format(target_time_fmt, tz_fmt)
+        return iso_time

=== added directory 'tests/autopilot/push_notifications/tests'
=== added file 'tests/autopilot/push_notifications/tests/__init__.py'
--- tests/autopilot/push_notifications/tests/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/tests/__init__.py	2014-07-10 10:50:28 +0000
@@ -0,0 +1,245 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+#
+# Push Notifications Autopilot Test Suite
+# Copyright (C) 2014 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+"""push-notifications autopilot tests."""
+
+import copy
+
+import evdev
+
+from autopilot.introspection import dbus
+from autopilot.matchers import Eventually
+from push_notifications import config as push_config
+import push_notifications.helpers.push_notifications_helper as push_helper
+from testtools.matchers import Equals, NotEquals
+from unity8.shell.tests import UnityTestCase
+
+
+class PushNotificationTestBase(UnityTestCase):
+    """
+    Base class for push notification test cases
+    """
+
+    @classmethod
+    def setUpClass(cls):
+        """
+        Executed once before all the tests run
+        Restart the push client using the test config
+        """
+        test_config = push_helper.PushClientConfig.read_config(
+            push_config.get_config_file())
+        push_client_controller = push_helper.PushClientController()
+        push_client_controller.restart_push_client_using_config(test_config)
+
+    @classmethod
+    def tearDownClass(cls):
+        """
+        Executed once after all tests have completed
+        Reset the push client to use the device's original config
+        """
+        push_client_controller = push_helper.PushClientController()
+        push_client_controller.restart_push_client_using_config(None)
+
+    def setUp(self):
+        """
+        Setup phase executed before each test
+        """
+        # setup
+        super(PushNotificationTestBase, self).setUp()
+
+        # read and store the test config data
+        self.test_config = push_helper.PushClientConfig.read_config(
+            push_config.get_config_file())
+        # create a push helper object which will do all the message sending
+        self.push_helper = push_helper.PushNotificationHelper()
+        # create a push controller object
+        self.push_client_controller = push_helper.PushClientController()
+        # get and store device and build info
+        self.device_info = self.push_helper.get_device_info()
+        # start unity8
+        self._qml_mock_enabled = False
+        self._data_dirs_mock_enabled = False
+        self.unity = self.launch_unity()
+        # dismiss any outstanding dialog
+        self.dismiss_outstanding_dialog()
+
+    def create_device_info_copy(self):
+        """
+        Return a copy of the device's model and build info
+        :return: DeviceNotificationData object containging device's model
+                 and build info
+        """
+        return copy.deepcopy(self.device_info)
+
+    def press_power_button(self):
+        """
+        Simulate a power key press event
+        """
+        uinput = evdev.UInput(name='push-autopilot-power-button',
+                              devnode='/dev/autopilot-uinput')
+        # One press and release to turn screen off (locking unity)
+        uinput.write(evdev.ecodes.EV_KEY, evdev.ecodes.KEY_POWER, 1)
+        uinput.write(evdev.ecodes.EV_KEY, evdev.ecodes.KEY_POWER, 0)
+        uinput.syn()
+
+    def unlock_greeter(self):
+        """
+        Unlock the greeter to display home screen
+        """
+        self.main_window.get_greeter().swipe()
+
+    def validate_response(self, response, expected_status_code=200):
+        """
+        Validate the received response status code against expected code
+        :param response: response to validate
+        :param expected_status_code: value of expected http status code
+        """
+        self.assertThat(response.status, Equals(expected_status_code))
+
+    def assert_notification_dialog(self, notification, summary=None,
+                                   body=None, icon=True, secondary_icon=False,
+                                   opacity=None):
+        """
+        Assert that the properties of the notification are as
+        expected
+        :param notification: notification object to validate
+        :param summary: expected notification summary value
+        :param body: expected notification body value
+        :param icon: expected icon status
+        :param secondary_icon: expected secondary icon status
+        :param opacity: expected opacity value
+        """
+        if summary is not None:
+            self.assertThat(notification.summary, Eventually(Equals(summary)))
+        if body is not None:
+            self.assertThat(notification.body, Eventually(Equals(body)))
+        if opacity is not None:
+            self.assertThat(notification.opacity, Eventually(Equals(opacity)))
+
+        if icon:
+            self.assertThat(
+                notification.iconSource, Eventually(NotEquals('')))
+        else:
+            self.assertThat(
+                notification.iconSource, Eventually(Equals('')))
+
+        if secondary_icon:
+            self.assertThat(
+                notification.secondaryIconSource, Eventually(NotEquals('')))
+        else:
+            self.assertThat(
+                notification.secondaryIconSource, Eventually(Equals('')))
+
+    def validate_notification_not_displayed(self, wait=True):
+        """
+        Validate that the notification is not displayed
+        If wait is True then wait for default timeout period
+        If wait is False then do not wait at all
+        :param wait: wait status
+        """
+        found = True
+        try:
+            if wait is True:
+                self.main_window.wait_select_single(
+                    'Notification', objectName='notification1')
+            else:
+                self.main_window.select_single(
+                    'Notification', objectName='notification1')
+        except dbus.StateNotFoundError:
+            found = False
+        self.assertFalse(found)
+
+    def send_push_broadcast_message(self):
+        """
+        Send a push broadcast message which should trigger a notification
+        to be displayed on the client
+        """
+        # create a copy of the device's build info
+        device_info = self.create_device_info_copy()
+        # increment the build number to trigger an update
+        device_info.inc_build_number()
+        # create push message based on the device data
+        push_msg = self.push_helper.create_push_message(
+            data=device_info.to_json())
+        # send the notification message to the server and check response
+        response = self.push_helper.send_push_broadcast_notification(
+            push_msg.to_json(), self.test_config.server_listener_addr)
+        self.validate_response(response)
+
+    def get_notification_dialog(self, wait=True):
+        """
+        Get the notification dialog being displaye on screen
+        If wait is True then wait for default timeout period
+        If wait is False then do not wait at all
+        :param wait: wait status
+        :return: dialog introspection object
+        """
+        if wait is True:
+            dialog = self.main_window.wait_select_single(
+                'Notification', objectName='notification1')
+        else:
+            dialog = self.main_window.select_single(
+                'Notification', objectName='notification1')
+        return dialog
+
+    def validate_and_dismiss_notification_dialog(self, message):
+        """
+        Validate a notification dialog is displayed and dismiss it
+        :param message: expected message displayed in summary
+        """
+        # get the dialog
+        dialog = self.get_notification_dialog()
+        # validate dialog
+        self.assert_notification_dialog(
+            dialog, summary=message)
+        # wait for dialog to dismiss automatically
+        self.wait_until_dialog_dismissed(dialog)
+        # check the dialog is no longer displayed
+        self.validate_notification_not_displayed(wait=False)
+
+    def wait_until_dialog_dismissed(self, dialog):
+        """Wait for the dialog to dismiss automatically"""
+        dialog_disappeared = False
+        try:
+            # waiting for this property to change will cause a not found
+            # exception once the dialog disappears
+            dialog.visible.wait_for(False)
+        except dbus.StateNotFoundError:
+            dialog_disappeared = True
+        # check that the dialog did disappear
+        self.assertTrue(dialog_disappeared)
+
+    def press_notification_dialog(self, dialog):
+        """
+        Press the dialog to dismiss it
+        """
+        self.touch.tap_object(dialog)
+
+    def dismiss_outstanding_dialog(self):
+        """
+        Dismiss outstanding notification dialog that may be displayed
+        from an aborted previous test
+        """
+        try:
+            dialog = self.main_window.select_single(
+                'Notification', objectName='notification1')
+        except dbus.StateNotFoundError:
+            dialog = None
+        if dialog:
+            self.wait_until_dialog_dismissed(dialog)

=== added file 'tests/autopilot/push_notifications/tests/test_broadcast_notifications.py'
--- tests/autopilot/push_notifications/tests/test_broadcast_notifications.py	1970-01-01 00:00:00 +0000
+++ tests/autopilot/push_notifications/tests/test_broadcast_notifications.py	2014-07-10 10:50:28 +0000
@@ -0,0 +1,141 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+#
+# Push Notifications Autopilot Test Suite
+# Copyright (C) 2014 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+"""Tests broadcast push notifications sent to the client"""
+
+import time
+
+from push_notifications.tests import PushNotificationTestBase
+
+
+class TestPushClientBroadcast(PushNotificationTestBase):
+    """
+    Test cases for broadcast push notifications
+    """
+
+    DEFAULT_DISPLAY_MESSAGE = 'There\'s an updated system image.'
+
+    def test_broadcast_push_notification_screen_off(self):
+        """
+        Send a push message whilst the device's screen is turned off
+        Notification should still be displayed when it is turned on
+        """
+        # Assumes greeter starts in locked state
+        # Turn display off
+        self.press_power_button()
+        # send message
+        self.send_push_broadcast_message()
+        # wait before turning screen on
+        time.sleep(2)
+        # Turn display on
+        self.press_power_button()
+        self.validate_and_dismiss_notification_dialog(
+            self.DEFAULT_DISPLAY_MESSAGE)
+
+    def test_broadcast_push_notification_locked_greeter(self):
+        """
+        Positive test case to send a valid broadcast push notification
+        to the client and validate that a notification message is displayed
+        whist the greeter screen is displayed
+        """
+        # Assumes greeter starts in locked state
+        self.send_push_broadcast_message()
+        self.validate_and_dismiss_notification_dialog(
+            self.DEFAULT_DISPLAY_MESSAGE)
+
+    def test_broadcast_push_notification(self):
+        """
+        Positive test case to send a valid broadcast push notification
+        to the client and validate that a notification message is displayed
+        """
+        # Assumes greeter starts in locked state
+        self.unlock_greeter()
+        self.send_push_broadcast_message()
+        self.validate_and_dismiss_notification_dialog(
+            self.DEFAULT_DISPLAY_MESSAGE)
+
+    def test_broadcast_push_notification_on_connect(self):
+        """
+        Send a broadcast notification whilst the push client is disconnected
+        from the server. Then reconnect and ensure message is displayed
+        """
+
+        # Assumes greeter starts in locked state
+        self.unlock_greeter()
+        self.push_client_controller.stop_push_client()
+        self.send_push_broadcast_message()
+        self.push_client_controller.start_push_client()
+        self.validate_and_dismiss_notification_dialog(
+            self.DEFAULT_DISPLAY_MESSAGE)
+
+    def test_expired_broadcast_push_notification(self):
+        """
+        Send an expired broadcast notification message to server
+        """
+        # Assumes greeter starts in locked state
+        self.unlock_greeter()
+        # create notification message using past expiry time
+        device_info = self.create_device_info_copy()
+        device_info.inc_build_number()
+        push_msg = self.push_helper.create_push_message(
+            data=device_info.to_json(),
+            expire_after=self.push_helper.get_past_iso_time())
+        # send message
+        response = self.push_helper.send_push_broadcast_notification(
+            push_msg.to_json(),
+            self.test_config.server_listener_addr)
+        # 400 status is received for an expired message
+        self.validate_response(response, expected_status_code=400)
+        # validate no notification is displayed
+        self.validate_notification_not_displayed()
+
+    def test_older_version_broadcast_push_notification(self):
+        """
+        Send an old version broadcast notification message to server
+        """
+        # Assumes greeter starts in locked state
+        self.unlock_greeter()
+        # create notification message using previous build number
+        device_info = self.create_device_info_copy()
+        device_info.dec_build_number()
+        push_msg = self.push_helper.create_push_message(
+            data=device_info.to_json())
+        response = self.push_helper.send_push_broadcast_notification(
+            push_msg.to_json(),
+            self.test_config.server_listener_addr)
+        self.validate_response(response)
+        # validate no notification is displayed
+        self.validate_notification_not_displayed()
+
+    def test_equal_version_broadcast_push_notification(self):
+        """
+        Send an equal version broadcast notification message to server
+        """
+        # Assumes greeter starts in locked state
+        self.unlock_greeter()
+        # create notification message using equal build number
+        device_info = self.create_device_info_copy()
+        push_msg = self.push_helper.create_push_message(
+            data=device_info.to_json())
+        response = self.push_helper.send_push_broadcast_notification(
+            push_msg.to_json(),
+            self.test_config.server_listener_addr)
+        self.validate_response(response)
+        # validate no notification is displayed
+        self.validate_notification_not_displayed()