Merge lp:~ahayzen/music-app/qml-unit-test-filenames into lp:music-app

[Andrew Hayzen]

* Add qml unit tests for different filenames with media-hub and mediascanner

* Add qml unit tests for different filenames with media-hub and mediascanner

This add a set of qml unit tests, which can be run manually to test that all the possible filenames in the ascii table can be played/found. (Plan is to add this to the manual tests list once that is in the repo).

[lp:~ahayzen/music-app/qml-unit-test-filenames]

892. By Andrew Hayzen on 2015-09-09

* Merge of trunk

893. By Andrew Hayzen on 2015-09-09

* Update code comments

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

PASSED: Continuous integration, rev:892
http://91.189.93.70:8080/job/music-app-ci/1379/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/music-app-vivid-amd64-ci/231

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/music-app-ci/1379/rebuild

review: Approve (continuous-integration)

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

PASSED: Continuous integration, rev:893
http://91.189.93.70:8080/job/music-app-ci/1380/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/music-app-vivid-amd64-ci/232

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/music-app-ci/1380/rebuild

review: Approve (continuous-integration)

[Victor Thompson (vthompson)]

Just briefly looking this over. I think we should also add a README.unittests with instructions on running the tests. Also, if we have a script to run on the device, would it also make sense to have a script that runs all the tests locally?

review: Needs Fixing

[lp:~ahayzen/music-app/qml-unit-test-filenames]

894. By Andrew Hayzen on 2015-09-25

* Add README.unittests and setup_run_locally.sh files

[Andrew Hayzen (ahayzen)]

* Add README.unittests and setup_run_locally.sh files

Please retest :-)

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

PASSED: Continuous integration, rev:894
http://91.189.93.70:8080/job/music-app-ci/1384/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/music-app-vivid-amd64-ci/236

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/music-app-ci/1384/rebuild

review: Approve (continuous-integration)

[Victor Thompson (vthompson)]

Things to fix:

1. Please update to UC 1.3.
2. I also experienced your FIXME when running on the device. I think it's because the password isn't echoed back through adb--my password shows up in the terminal. There should be a way to echo it back.

Things to consider:

1. Oh dear that tone for the test files is kind of annoying. :)
2. Will it still make sense to have this run locally to test mh3 bg playlists? I think we might need to update the installation requirements before running the tests to install media hub. The unit tests fail locally on Xenial for me.
3. Is there a way to have the Ubuntu Application Test close after the unit tests run?

review: Needs Fixing

[Jenkins Bot (ubuntu-core-apps-jenkins-bot)]

PASSED: Continuous integration, rev:894
https://core-apps-jenkins.ubuntu.com/job/music-app-ci/10/
Executed test runs:
    None: https://core-apps-jenkins.ubuntu.com/job/generic-update-mp/143/console

Click here to trigger a rebuild:
https://core-apps-jenkins.ubuntu.com/job/music-app-ci/10/rebuild

review: Approve (continuous-integration)

Unmerged revisions

894. By Andrew Hayzen on 2015-09-25

* Add README.unittests and setup_run_locally.sh files

893. By Andrew Hayzen on 2015-09-09

* Update code comments

892. By Andrew Hayzen on 2015-09-09

* Merge of trunk

891. By Andrew Hayzen on 2015-09-08

* Tweaks to test files

890. By Andrew Hayzen on 2015-09-08

* Merge of trunk

889. By Andrew Hayzen on 2015-08-09

* Skip invalid file paths
* Test metadata from mediascanner is correct
* Improve init and how data is exposed

888. By Andrew Hayzen on 2015-07-26

* First round of changes for unit tests

=== added file 'README.unittests'
--- README.unittests	1970-01-01 00:00:00 +0000
+++ README.unittests	2015-09-25 11:15:25 +0000
@@ -0,0 +1,47 @@
+QML Unit Tests
+==============
+
+QML Unit Tests help with testing the internal working of components while
+autopilot tests help with testing the UI workflow as experience by the user.
+
+If you are submitting your bugfix/patch to the music app, please follow the
+following steps below to check whether that all tests pass before proposing a
+merge request.
+
+Note that these tests use the folder ~/Music/QMLTest and any content inside
+will be removed.
+
+Also the application currently crashes on exit, so the test pass or fail lines
+are a few lines back up the console.
+
+Running QML Unit Tests on device
+================================
+
+Use the tests/unit/setup_run_on_device.sh note that some commands may have to
+be run manually.
+
+Running QML Unit Tests on desktop
+=================================
+
+Note that you need the mediascanner2 service to be running.
+
+Run the setup_run_locally.sh
+
+$ cd tests/unit/
+$ ./setup_run_locally.sh
+
+Running individual test cases
+=============================
+
+To run individual tests firstly created the mocked environment
+$ ./generate_files.sh
+$ cp -r files/generated ~/Music/QMLTest
+
+Then ensure that all requirements are installed
+$ sudo apt-get install ubuntu-app-test qtdeclarative5-dev-tools qml-module-qttest -y
+
+Navigate to the unit tests directory
+$ cd tests/unit
+
+Run the test by providing its filename
+$ qmltestrunner -input tst_filenames.qml

=== modified file 'app/components/Player.qml'
--- app/components/Player.qml	2015-06-28 03:06:49 +0000
+++ app/components/Player.qml	2015-09-25 11:15:25 +0000
@@ -40,6 +40,7 @@
     property int currentIndex: -1
     property int duration: 1
     readonly property bool isPlaying: player.playbackState === MediaPlayer.PlayingState
+    readonly property bool loaded: mediaPlayerLoader.status == Loader.Ready
     readonly property var playbackState: mediaPlayerLoader.status == Loader.Ready ? mediaPlayerLoader.item.playbackState : MediaPlayer.StoppedState
     property int position: 0
     property alias repeat: settings.repeat

=== modified file 'debian/changelog'
--- debian/changelog	2015-09-25 08:11:22 +0000
+++ debian/changelog	2015-09-25 11:15:25 +0000
@@ -1,5 +1,8 @@
 music-app (2.2ubuntu2) UNRELEASED; urgency=medium
 
+  [ Andrew Hayzen ]
+  * Add qml unit tests for different filenames with media-hub and mediascanner
+
   [ Bartosz Kosiorek ]
   *  Reduce size of images (with tinypng.com) to decrease click size and improve performance
 

=== modified file 'tests/CMakeLists.txt'
--- tests/CMakeLists.txt	2014-01-31 20:43:59 +0000
+++ tests/CMakeLists.txt	2015-09-25 11:15:25 +0000
@@ -1,1 +1,2 @@
 add_subdirectory(autopilot)
+add_subdirectory(unit)

=== added directory 'tests/unit'
=== added file 'tests/unit/CMakeLists.txt'
--- tests/unit/CMakeLists.txt	1970-01-01 00:00:00 +0000
+++ tests/unit/CMakeLists.txt	2015-09-25 11:15:25 +0000
@@ -0,0 +1,6 @@
+# make the emulator files visible on qtcreator
+file(GLOB QML_UNIT_TEST_FILES
+     RELATIVE ${CMAKE_CURRENT_SOURCE_DIR}
+     *.qml *.sh)
+
+add_custom_target(com_ubuntu_music_QMLUNITTESTFiles ALL SOURCES ${QML_UNIT_TEST_FILES})

=== added directory 'tests/unit/files'
=== added directory 'tests/unit/files/original'
=== added file 'tests/unit/files/original/1.ogg'
Binary files tests/unit/files/original/1.ogg	1970-01-01 00:00:00 +0000 and tests/unit/files/original/1.ogg	2015-09-25 11:15:25 +0000 differ
=== added file 'tests/unit/generate_files.sh'
--- tests/unit/generate_files.sh	1970-01-01 00:00:00 +0000
+++ tests/unit/generate_files.sh	2015-09-25 11:15:25 +0000
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+mkdir files/generated
+
+for i in {32..126}  # from 'Space' to '~'
+do
+    if [ $i -eq 46 ] || [ $i -eq 47 ]  # skip '.' and '/'
+    then
+        continue
+    else
+        chr=$(printf \\$(printf '%03o' $i))
+
+        echo "Creating file $chr.ogg from 1.ogg"
+        cp "files/original/1.ogg" "files/generated/$chr.ogg"
+    fi
+done

=== added file 'tests/unit/setup_run_locally.sh'
--- tests/unit/setup_run_locally.sh	1970-01-01 00:00:00 +0000
+++ tests/unit/setup_run_locally.sh	2015-09-25 11:15:25 +0000
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+# Generate files
+./generate_files.sh
+
+# Copy files to device
+cp -r files/generated ~/Music/QMLTest
+
+# Install requirements
+sudo apt-get install ubuntu-app-test qtdeclarative5-dev-tools qml-module-qttest -y
+
+# Run qmltestrunner
+ubuntu-app-test qmltestrunner -silent -eventdelay 500
+
+# Clean up
+rm -rf ~/Music/QMLTest
+rm -rf files/generated

=== added file 'tests/unit/setup_run_on_device.sh'
--- tests/unit/setup_run_on_device.sh	1970-01-01 00:00:00 +0000
+++ tests/unit/setup_run_on_device.sh	2015-09-25 11:15:25 +0000
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+# Copy music app to device
+adb push ../../. /tmp/music-app-unit-test
+
+# Generate files
+./generate_files.sh
+
+# Copy files to device
+adb push files/generated /home/phablet/Music/QMLTest
+
+# TODO: make writeable ?
+
+# Install requirements
+# FIXME: I have to adb into the device and run manually?
+adb shell "sudo apt-get install ubuntu-app-test qtdeclarative5-dev-tools qml-module-qttest -y"
+
+# Run qmltestrunner
+adb shell "cd /tmp/music-app-unit-test/tests/unit && ubuntu-app-test qmltestrunner -silent -eventdelay 500"
+
+# Clean up
+adb shell "rm -rf /home/phablet/Music/QMLTest"
+adb shell "rm -rf /tmp/music-app-unit-test"
+rm -rf files/generated

=== added file 'tests/unit/tst_filenames.qml'
--- tests/unit/tst_filenames.qml	1970-01-01 00:00:00 +0000
+++ tests/unit/tst_filenames.qml	2015-09-25 11:15:25 +0000
@@ -0,0 +1,116 @@
+
+import QtTest 1.0
+import QtQuick 2.4
+import Ubuntu.Components 1.2
+import Ubuntu.MediaScanner 0.1
+import Ubuntu.Test 1.0
+
+import "../../app/components"
+
+MainView {
+    // Wrapper function around decodeURIComponent() to prevent exceptions
+    // from bubbling up to the app.
+    function decodeFileURI(filename)
+    {
+        var newFilename = "";
+        try {
+            newFilename = decodeURIComponent(filename);
+        } catch (e) {
+            newFilename = filename;
+            console.log("Unicode decoding error:", filename, e.message)
+        }
+
+        return newFilename;
+    }
+
+    Item {
+        id: trackQueue
+
+        property ListModel model: ListModel {
+
+        }
+    }
+
+    /* Mediahub */
+    Player {
+        id: player
+    }
+
+    /* Mediascanner */
+    SongsModel {
+        id: songsModel
+        store: MediaStore {
+            id: musicStore
+        }
+    }
+
+    /* UnitTests */
+    UbuntuTestCase {
+        property var filepaths: [
+
+        ]
+
+        function init() {
+            // Wait for the async MediaPlayer to load
+            tryCompare(player, "loaded", true, 250, "Player has not loaded");
+        }
+
+        function init_data() {
+            return filepaths;
+        }
+
+        function initTestCase() {
+            // Build up filenames to test
+            var blacklist = [46, 47];  // '.', '/'
+
+            for (var i=32; i < 127; i++) {
+                if (blacklist.indexOf(i) > -1) {
+                    continue;
+                } else {
+                    var chr = String.fromCharCode(i);
+                    filepaths.push({ filepath: "/home/phablet/Music/QMLTest/" + chr + ".ogg", tag: chr });
+                }
+            }
+        }
+
+        function test_mediascanner(data) {
+            var filepath = decodeFileURI(data.filepath);
+            var msResult = musicStore.lookup(filepath);
+
+            // Check lookup from ms2 returned something
+            verify(msResult !== null, "Mediascanner2 could not find track - " + filepath);
+
+            compare(msResult.title, "Test Sound", "Mediascanner title is not the same - " + filepath);
+            compare(msResult.album, "Test Album", "Mediascanner album is not the same - " + filepath);
+            compare(msResult.author, "Test Artist", "Mediascanner author is not the same - " + filepath);
+            compare(msResult.art, "image://albumart/artist=Test%20Artist&album=Test%20Album", "Mediascanner art is not the same - " + filepath);
+        }
+
+        function test_media_hub(data) {
+            if (data.tag === "#") {  // manually skip otherwise media-hub crashes
+                // FIXME: MediaHub won't play '#' due to http://pad.lv/1449790
+                // expectFail("#", "MediaHub won't play '#' due to http://pad.lv/1449790")
+
+                /*
+                  // FIXME: Not sure why this doesn't work, it should only skip this data row but it skips the test
+                if (data.tag === "#") {
+                    skip("MediaHub won't play '#' due to http://pad.lv/1449790")
+                }
+                */
+            } else {
+                var filepath = data.filepath;
+
+                player.setSource(filepath);
+                player.play();
+
+                wait(1100);  // Wait for the track to reach > 1s
+
+                // Check that track is playing and the position has changed
+                compare(player.isPlaying, true, "MediaHub is not playing the track - " + filepath);
+                verify(player.position > 0, "MediaHub position has not changed - " + filepath + " - " + player.position);
+
+                player.pause();
+            }
+        }
+    }
+}