Merge lp:~mrazik/unity/tests-feedback-from-nice into lp:unity

[Martin Mrazik]

Improving manual test-cases based on the feedback from nice.fi.

Our test-contractor (nice.fi) had few difficulties with the current test-cases. This MP is incorporating their feedback.

[Martin Mrazik (mrazik)]

8 -#. Open the Dash - there shouldn't be any results displayed, since this user
9 - account didn't interact with anything yet
10 +#. Open the Dash

I think this needs some more info -- I've removed the "there shouldn't be any results displayed" part as it is not true. Even in guest session you have a section with recent apps and you can see stuff like thunderbird there.This confused our testers enough to make them think something is wrong.

[MC Return (mc-return)]

RhytmBox -> Rhythmbox

[lp:~mrazik/unity/tests-feedback-from-nice]

2522. By Martin Mrazik on 2012-07-25

correct spelling of Rhythmbox

[jenkins (martin-mrazik+qa)]

FAILED: Continuous integration, rev:2521
http://jenkins.qa.ubuntu.com/job/unity-ci/119/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-ci/./distribution=quantal,flavor=amd64/119/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-ci/./distribution=quantal,flavor=i386/119/console

review: Needs Fixing (continuous-integration)

[Martin Mrazik (mrazik)]

The failure above is clearly not related to this MP.

[jenkins (martin-mrazik+qa)]

FAILED: Continuous integration, rev:2522
http://jenkins.qa.ubuntu.com/job/unity-ci/120/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-ci/./distribution=quantal,flavor=amd64/120/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-ci/./distribution=quantal,flavor=i386/120/console

review: Needs Fixing (continuous-integration)

[Thomi Richards (thomir-deactivatedaccount)]

Hi Martin,

IMO these tests could still use some fixing:

18 +#. Make sure you already opened a music file using your Unity desktop (e.g. by
19 + playing it with Rhythmbox). Downloading a file, e.g. using a Firefox will not
20 + show the music in the Music Lens (see the steps below).
21 +

I think this needs to be more explicit. We ship a bunch of sound files in /usr/share/sounds, so how about something like:

 #. Open Rhythmbox
 #. Inside Rhythmbox, open the file /usr/share/sounds/some_sound_file.ogg
 #. Click Play.

39 + * Assert: The numbers [0-9] should be displayed overlaid on the icons
40 + (0 is the last number as it is the 10th icon). If there are more than 10
41 + application icons they don't have any overlaid numbers.

I don't think we should be using the word "Assert" in the test text. Also, instead of using the word "should" we should say "must". Finally, I think multiple-sentence outcome statements like this one should be broken down. I suggest replacing the above with the following:

 * The numbers 0-9 must be overlaid on the icons starting with the first icon after the BFB.
 * After the 10th icon, icons must have no overlay text displayed.
 * The workspace switcher must have an overlay text of "s"
 * The trash can icon must have an overlay text of "t"

review: Needs Fixing (quality)

[Martin Mrazik (mrazik)]

Some more feedback from nice.

On 07/27/2012 11:44 AM, Slade Martin wrote:
> Hi,
>
> Thank you for the updates. I just have one query for the ' Dash no-results-hint' test case.
> With the changes is this test testing anything?
> As the purpose of the test was:
> "This tests that the text displaying "Sorry nothing matches your search" is hidden when results are present (even in hard-to-automatically-test scenarios)."
>

You are right. I will find out what changed and will update (or delete?) the test-case accordingly.

Unmerged revisions

2522. By Martin Mrazik on 2012-07-25

correct spelling of Rhythmbox

2521. By Martin Mrazik on 2012-07-25

improved some test-cases based on feedback from nice.fi

=== modified file 'manual-tests/Dash.txt'
--- manual-tests/Dash.txt	2012-07-12 12:03:05 +0000
+++ manual-tests/Dash.txt	2012-07-25 14:34:22 +0000
@@ -88,8 +88,7 @@
 
 Actions:
 #. Start guest session
-#. Open the Dash - there shouldn't be any results displayed, since this user
-   account didn't interact with anything yet
+#. Open the Dash
 #. Run an application not present in the launcher (for example gedit)
 #. Close previously opened application
 
@@ -201,10 +200,13 @@
 -----------------------------------
 
 Setup:
+#. Make sure you already opened a music file using your Unity desktop (e.g. by
+   playing it with Rhythmbox). Downloading a file, e.g. using a Firefox will not
+   show the music in the Music Lens (see the steps below).
+
+Actions:
 #. Open the Dash (Super).
 #. Go to the Music Lens.
-
-Actions:
 #. Drag any music file into the Firefox icon on the launcher.
 
 Expected Result:

=== modified file 'manual-tests/Launcher.txt'
--- manual-tests/Launcher.txt	2012-07-20 02:24:47 +0000
+++ manual-tests/Launcher.txt	2012-07-25 14:34:22 +0000
@@ -502,11 +502,15 @@
 
 Actions:
   * Hold down Super until the "Keyboard Shortcuts" screen appears
-  * Assert: The numbers [1-9] should be displayed overlaid on the icons.
+  * Assert: The numbers [0-9] should be displayed overlaid on the icons 
+    (0 is the last number as it is the 10th icon). If there are more than 10 
+    application icons they don't have any overlaid numbers.
     "Keyboard Shortcut" menu should appear as well.
   * Mouse over an icon in the launcher
   * Hold down Super until the "Keyboard Shortcuts" screen appears
-  * Assert: The numbers [1-9] should be displayed overlaid on the icons.
+  * Assert: The numbers [0-9] should be displayed overlaid on the icons
+    (0 is the last number as it is the 10th icon). If there are more than 10 
+    application icons they don't have any overlaid numbers.
     "Keyboard Shortcut" menu should appear as well.
 
 Expected Result:
@@ -612,9 +616,11 @@
 
 Actions:
  * Open Nautilus
- * Mount an internal partition using the sidebar
+ * There is a "Devices" section in the upper left corner of nautilus
+ * Mount one of the devices there by clicking on it  
  * Make sure the launcher has the launcher icon for the partition
- * Unmount that partition using Nautilus
+ * Unmount that partition using Nautilus by clicking on the "eject" button in 
+   the "Devices" section
 
 Expected Result:
  * The device launcher icon has been removed.

=== modified file 'manual-tests/Panel.txt'
--- manual-tests/Panel.txt	2012-07-20 02:24:47 +0000
+++ manual-tests/Panel.txt	2012-07-25 14:34:22 +0000
@@ -83,6 +83,9 @@
 
 Disabled Top-level Menus
 ------------------------
+This test-case is not valid anymore. See also 
+https://bugs.launchpad.net/unity/+bug/1028836
+
 Setup:
 #. Install the 'gnome-dictionary' package.
 

=== modified file 'manual-tests/WindowButtons.txt'
--- manual-tests/WindowButtons.txt	2012-07-12 12:03:05 +0000
+++ manual-tests/WindowButtons.txt	2012-07-25 14:34:22 +0000
@@ -1,5 +1,8 @@
 Window Close Button should be Disabled if not available
 -------------------------------------------------------
+This test-case is not valid anymore. See also 
+https://bugs.launchpad.net/unity/+bug/1028922
+
 Setup:
 Actions:  
  * Start the Update Manager