Merge lp:~nskaggs/ubuntu-weather-app/manual-tests into lp:ubuntu-weather-app

[Nicholas Skaggs]

Add manual tests

Adding the checkbox tests that shipped long ago with Pilot. This includes the provider and is a nice drop-in solution. I took the comments from the first merge as well.

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

PASSED: Continuous integration, rev:129
http://91.189.93.70:8080/job/ubuntu-weather-app-reboot-ci/255/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/ubuntu-weather-app-reboot-utopic-amd64-ci/225
    SUCCESS: http://91.189.93.70:8080/job/ubuntu-weather-app-reboot-vivid-amd64-ci/255

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/ubuntu-weather-app-reboot-ci/255/rebuild

review: Approve (continuous-integration)

[Nicholas Skaggs (nskaggs)]

These are fine, but will need a provider and folder layout to make them plug and play. I'll bundle them and try them before providing a diff.

[Nicholas Skaggs (nskaggs)]

These work. Some comments

Avoid using dashes for the file name
s/first-run/firstrun/

Prefix the tests file names with 'weather-'

You are missing a colon
s/_description/_description:/g

review: Needs Fixing

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

PASSED: Continuous integration, rev:130
http://91.189.93.70:8080/job/ubuntu-weather-app-reboot-ci/258/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/ubuntu-weather-app-reboot-utopic-amd64-ci/228
    SUCCESS: http://91.189.93.70:8080/job/ubuntu-weather-app-reboot-vivid-amd64-ci/258

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/ubuntu-weather-app-reboot-ci/258/rebuild

review: Approve (continuous-integration)

[Jenkins Bot (ubuntu-core-apps-jenkins-bot)]

FAILED: Continuous integration, rev:130
https://core-apps-jenkins.ubuntu.com/job/weather-app-ci/5/
Executed test runs:
    None: https://core-apps-jenkins.ubuntu.com/job/generic-update-mp/59/console

Click here to trigger a rebuild:
https://core-apps-jenkins.ubuntu.com/job/weather-app-ci/5/rebuild

review: Needs Fixing (continuous-integration)

[Jenkins Bot (ubuntu-core-apps-jenkins-bot)]

PASSED: Continuous integration, rev:130
https://core-apps-jenkins.ubuntu.com/job/weather-app-ci/8/
Executed test runs:
    None: https://core-apps-jenkins.ubuntu.com/job/generic-update-mp/63/console

Click here to trigger a rebuild:
https://core-apps-jenkins.ubuntu.com/job/weather-app-ci/8/rebuild

review: Approve (continuous-integration)

[Victor Thompson (vthompson)]

See 1 inline comment.

Also:
1. Please add an entry to the changelog, we're trying to maintain this file as we make fixes/updates.
2. There seem to be a lot of indentation issues. Is that problematic at all? Will it show up in an odd fashion to those running through the tests?

review: Needs Fixing

[Nicholas Skaggs (nskaggs)]

Sorry for overlaying your old MP popey; it was very helpful.

[Jenkins Bot (ubuntu-core-apps-jenkins-bot)]

PASSED: Continuous integration, rev:188
https://core-apps-jenkins.ubuntu.com/job/weather-app-ci/29/
Executed test runs:
    None: https://core-apps-jenkins.ubuntu.com/job/generic-update-mp/259/console

Click here to trigger a rebuild:
https://core-apps-jenkins.ubuntu.com/job/weather-app-ci/29/rebuild

review: Approve (continuous-integration)

[Jenkins Bot (ubuntu-core-apps-jenkins-bot)]

PASSED: Continuous integration, rev:189
https://core-apps-jenkins.ubuntu.com/job/weather-app-ci/30/
Executed test runs:
    None: https://core-apps-jenkins.ubuntu.com/job/generic-update-mp/260/console

Click here to trigger a rebuild:
https://core-apps-jenkins.ubuntu.com/job/weather-app-ci/30/rebuild

review: Approve (continuous-integration)

[Victor Thompson (vthompson)]

lgtm! Thanks balloons! :)

review: Approve

[Andrew Hayzen (ahayzen)]

LGTM! lets land it :-)

review: Approve

[Jenkins Bot (ubuntu-core-apps-jenkins-bot)]

review: Approve (continuous-integration)

=== added directory 'tests/manual'
=== added directory 'tests/manual/2015.com.ubuntu.weather:weather-tests'
=== added directory 'tests/manual/2015.com.ubuntu.weather:weather-tests/jobs'
=== added file 'tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-firstrun.pxu'
--- tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-firstrun.pxu	1970-01-01 00:00:00 +0000
+++ tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-firstrun.pxu	2015-12-16 21:52:36 +0000
@@ -0,0 +1,65 @@
+plugin: resource
+id: weather-firstrun/system-image
+_summary: [Automated] gather system information
+command: system-image-cli -i
+_description:
+ Gather system info
+
+id: weather-firstrun/check-icon
+plugin: manual
+_summary: Test to check correct icon is used
+estimated_duration: 600
+_description:
+    Test to check correct icon is used
+    1. Install weather reboot
+        Weather icon with a cloud and 12-spiked sun should appear in the dash
+
+id: weather-firstrun/location-check
+plugin: manual
+_summary: Test to confirm location is requested
+estimated_duration: 600
+_description:
+    Test to confirm location is requested
+    1. Launch weather reboot
+        Expect to see 'location acceptance screen'
+
+id: weather-firstrun/location-deny
+plugin: manual
+_summary: Test to confirm effect of denying location access
+estimated_duration: 600
+_description:
+    Test to confirm effect of denying location access
+    1. Launch weather reboot
+        Expect to see 'location acceptance screen'
+    2. Choose 'deny'
+        Expect to see the "Cannot determine your location", "Manually add" screen
+
+id: weather-firstrun/location-allow
+plugin: manual
+_summary: Test to confirm effect of allowing location access
+estimated_duration: 600
+_description:
+    Test to confirm effect of allowing location access
+    1. Close weather app (if already open)
+    2. Launch System Settings
+    3. Choose "Security & Privacy"
+    4. Choose "Location Access"
+    5. Switch location access for weather app on
+        This reverses the location denial from the previous test
+    6. Launch weather reboot
+        Expect to _not_ see 'location acceptance screen'
+    7. Wait for a short while
+        Expect to see weather for your current location
+
+id: weather-firstrun/pull-refresh
+plugin: manual
+_summary: Test to confirm pull to refresh works
+estimated_duration: 600
+_description:
+    Test to confirm pull to refresh works
+    1. Launch weather reboot
+        Expect to see weather details for current location
+    2. Pull down the screen to refresh
+        Expect to see 'Pull down to refresh' then 'Release to refresh'
+    3. Let go of the screen
+        Expect view to refresh - although you may not see any difference if the data recently refreshed
 
=== added file 'tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-forecasts.pxu'
--- tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-forecasts.pxu	1970-01-01 00:00:00 +0000
+++ tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-forecasts.pxu	2015-12-16 21:52:36 +0000
@@ -0,0 +1,62 @@
+plugin: resource
+id: weather-forecasts/system-image
+_summary: [Automated] gather system information
+command: system-image-cli -i
+_description:
+ Gather system info
+
+id: weather-forecasts/test-daily-forecast
+plugin: manual
+_summary: Test to future day forecast is shown
+estimated_duration: 600
+_description:
+    Test to future day forecast is shown
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Scroll up
+        Expect to see weather for at least the following 4 days
+
+id: weather-forecasts/test-hourly-forecast-detail
+plugin: manual
+_summary: Test to ensure hourly forecast detail is shown
+estimated_duration: 600
+_description:
+    Test to ensure hourly forecast detail is shown
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Tap large current weather icon (cloud/sun/rain) in the top half of the display
+        Expect to see weather for the following hours
+    3. Swipe right and left
+        Expect to see further details about the weather on coming hours
+    4. Tap in that area again
+        Expect the previously shown current weather icon to be displayed
+
+id: weather-forecasts/test-daily-forecast-detail
+plugin: manual
+_summary: Test to ensure daily forecast detail is shown
+estimated_duration: 600
+_description:
+    Test to ensure daily forecast detail is shown
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Scroll up
+        Expect to see weather for at least the following 4 days
+    3. Tap any day in the future
+        Expect to see further details about the weather on that day
+    4. Tap any other day
+        Expect the previously tapped day to shrink, and the newly tapped day to expand
+    5. Tap the same day again
+        Expect the tapped day detail to shrink
+
+id: weather-forecasts/test-final-day
+plugin: manual
+_summary: Test to ensure UI expands properly
+estimated_duration: 600
+_description:
+    Test to ensure UI expands properly
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Scroll up to the very bottom
+        Expect to see weather for at least the following 4 days
+    3. Tap the final day in the list
+        Expect it to expand, and still be on screen (i.e. not expanded off-screen)
 
=== added file 'tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-locations.pxu'
--- tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-locations.pxu	1970-01-01 00:00:00 +0000
+++ tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-locations.pxu	2015-12-16 21:52:36 +0000
@@ -0,0 +1,79 @@
+plugin: resource
+id: weather-locations/system-image
+_summary: [Automated] gather system information
+command: system-image-cli -i
+_description:
+ Gather system info
+
+id: weather-locations/show-list-of-locations
+plugin: manual
+_summary: Test to see a list of locations
+estimated_duration: 600
+_description:
+    Test to see a list of locations
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Swipe up from the bottom of the screen
+        Expect to see a list of locations (only one location on first run, or no locations if gps access denied)
+
+id: weather-locations/add-pre-defined-location
+plugin: manual
+_summary: Test to add a location from a pre-defined list
+estimated_duration: 600
+_description:
+    Test to add a location from a pre-defined list
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Swipe up from the bottom of the screen
+        Expect to see a list of locations (only one location on first run, or no locations if gps access denied)
+    3. Press "+" icon
+        Expect to see a pre-defined list of city names
+    4. Tap any city on the list
+        Expect to see it added to the list of locations
+    5. Continue to add more cities
+        There is a hard wired limit of 7 locations currently
+
+id: weather-locations/switch-location-by-swipe
+plugin: manual
+_summary: Test to switch locations by swiping
+estimated_duration: 600
+_description:
+    Test to switch locations by swiping
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Swipe left
+        Expect to switch location
+    3. Swipe right
+        Expect to switch back to previous location
+
+id: weather-locations/switch-location-by-edge
+plugin: manual
+_summary: Test to switch locations using bottom edge
+estimated_duration: 600
+_description:
+    Test to switch locations using bottom edge
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Swipe up from the bottom of the screen
+        Expect to see a list of locations
+    3. Tap a location
+        Expect to be returned to the main screen, showing weather for selected location
+
+id: weather-locations/delete-location
+plugin: manual
+_summary: Test to remove locations
+estimated_duration: 600
+_description:
+    Test to remove locations
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Swipe up from the bottom of the screen
+        Expect to see a list of locations
+    3. Swipe a location to the right
+        Expect to see a delete option
+    4. Swipe location left again
+        Expect location NOT to be deleted
+    5. Swipe a location to the right
+        Expect to see a delete option
+    6. Tap delete (trash) button
+        Expect location to be deleted
 
=== added file 'tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-settings.pxu'
--- tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-settings.pxu	1970-01-01 00:00:00 +0000
+++ tests/manual/2015.com.ubuntu.weather:weather-tests/jobs/weather-settings.pxu	2015-12-16 21:52:36 +0000
@@ -0,0 +1,40 @@
+plugin: resource
+id: weather-settings/system-image
+_summary: [Automated] gather system information
+command: system-image-cli -i
+_description:
+ Gather system info
+
+id: weather-settings/change-units-temp
+plugin: manual
+_summary: Test to change temperature units
+estimated_duration: 600
+_description:
+    Test to change temperature units
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Tap settings cog in top right corner
+        Expect to see a list of settings
+    3. Tap "Units"
+        Expect to see 'Units' screen
+    4. Tap "Temperature"
+        Expect to see options unfolded
+    5. Switch to different temperature units and go back to main screen
+        Expect units shown to be different
+
+id: weather-settings/change-units-speed
+plugin: manual
+_summary: Test to change wind speed units
+estimated_duration: 600
+_description:
+    Test to change wind speed units
+    1. Launch weather app
+        Expect to see weather for your current location
+    2. Tap settings cog in top right corner
+        Expect to see a list of settings
+    3. Tap "Units"
+        Expect to see 'Units' screen
+    4. Tap "Wind Speed"
+        Expect to see options unfolded
+    5. Switch to different wind speed units and go back to main screen
+        Expect wind speed units shown to be different
 
=== added file 'tests/manual/2015.com.ubuntu.weather:weather-tests/manage.py'
--- tests/manual/2015.com.ubuntu.weather:weather-tests/manage.py	1970-01-01 00:00:00 +0000
+++ tests/manual/2015.com.ubuntu.weather:weather-tests/manage.py	2015-12-16 21:52:36 +0000
@@ -0,0 +1,21 @@
+#!/usr/bin/env python3
+from plainbox.provider_manager import setup, N_
+
+# You can inject other stuff here but please don't go overboard.
+#
+# In particular, if you need comprehensive compilation support to get
+# your bin/ populated then please try to discuss that with us in the
+# upstream project IRC channel #checkbox on irc.freenode.net.
+
+# NOTE: one thing that you could do here, that makes a lot of sense,
+# is to compute version somehow. This may vary depending on the
+# context of your provider. Future version of PlainBox will offer git,
+# bzr and mercurial integration using the versiontools library
+# (optional)
+
+setup(
+    name='2015.com.ubuntu.weather:weather-tests',
+    version="1.0",
+    description=N_("The 2015.com.ubuntu.weather:weather-tests provider"),
+    gettext_domain="2015_com_ubuntu_weather_weather-tests",
+)
 
=== added directory 'tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists'
=== added file 'tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-firstrun.whitelist'
--- tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-firstrun.whitelist	1970-01-01 00:00:00 +0000
+++ tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-firstrun.whitelist	2015-12-16 21:52:36 +0000
@@ -0,0 +1,1 @@
+weather-firstrun/.*
 \ No newline at end of file
 
=== added file 'tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-forecasts.whitelist'
--- tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-forecasts.whitelist	1970-01-01 00:00:00 +0000
+++ tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-forecasts.whitelist	2015-12-16 21:52:36 +0000
@@ -0,0 +1,1 @@
+weather-forecasts/.*
 \ No newline at end of file
 
=== added file 'tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-locations.whitelist'
--- tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-locations.whitelist	1970-01-01 00:00:00 +0000
+++ tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-locations.whitelist	2015-12-16 21:52:36 +0000
@@ -0,0 +1,1 @@
+weather-locations/.*
 \ No newline at end of file
 
=== added file 'tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-settings.whitelist'
--- tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-settings.whitelist	1970-01-01 00:00:00 +0000
+++ tests/manual/2015.com.ubuntu.weather:weather-tests/whitelists/weather-settings.whitelist	2015-12-16 21:52:36 +0000
@@ -0,0 +1,1 @@
+weather-settings/.*
 \ No newline at end of file