Merge lp:~canonical-ci-engineering/ubuntu-test-cases/testing-docs into lp:ubuntu-test-cases/touch

[Francis Ginther]

Add README documentation for developer centric testing of a modified debian or click application.

Add README documentation for developer centric testing of a modified debian or click application.

[Francis Ginther (fginther)]

I've also been playing with a modified version oh phablet-click-test-setup (from phablet-tools) which takes care of installation of the test sources and would allow removing the following:

33 +The above phablet-click-test-setup command will install the standard test
34 +dependencies and the click application's test sources as specified in the
35 +manifest. This is usually the application's trunk branch. To override the test
36 +sources with local changes, replace the test sources that were copied to the
37 +device. This example assumes the application code is checked out under the
38 +'dropping-letters' directory with the test sources under 'tests/autopilot'::
39 +
40 + adb shell rm -rf /home/phablet/autopilot/dropping_letters_app
41 + adb push dropping-letters/tests/autopilot \
42 + /home/phablet/autopilot

[Paul Larson (pwlars)]

is pkcon preferred over click install?

[Francis Ginther (fginther)]

> is pkcon preferred over click install?

I've been unable to find info in any of the wiki's but the 'click' command provides this:
"
phablet@ubuntu-phablet:~$ click install --help
Usage: click install [options] PACKAGE-FILE

This is a low-level tool; to install a package as an ordinary user
you should generally use "pkcon install-local PACKAGE-FILE"
instead.
"

Will keep looking.

[Colin Watson (cjwatson)]

The click documentation should be considered authoritative on this.

[lp:~canonical-ci-engineering/ubuntu-test-cases/testing-docs]

377. By Francis Ginther on 2015-03-02

A couple of minor corrections and updates.

[Thomi Richards (thomir-deactivatedaccount)]

Found a typo

review: Needs Fixing

[Francis Ginther (fginther)]

> Found a typo

plars fixed this in rev 377.

[Thomi Richards (thomir-deactivatedaccount)]

That looks better then.

review: Approve

[lp:~canonical-ci-engineering/ubuntu-test-cases/testing-docs]

378. By Paul Larson on 2015-03-03

Add a section on running tests for a modified debian package

[Paul Larson (pwlars)]

This now has the debian package testing instructions merged, and I think that's all we were intending to add. So there was something else, I'm +1 to go ahead and merge this in.

review: Approve

[Francis Ginther (fginther)]

Also approve.

review: Approve

=== modified file 'README-cli.rst'
--- README-cli.rst	2014-11-10 19:28:56 +0000
+++ README-cli.rst	2015-03-03 13:54:49 +0000
@@ -6,10 +6,11 @@
 things you need are:
 
  * This bzr branch
- * The phablet-tools_ package
+ * The phablet-tools_ and ubuntu-device-flash_ packages
  * An Ubuntu Touch supported_ device
 
 .. _phablet-tools: http://launchpad.net/phablet-tools
+.. _ubuntu-device-flash: http://launchpad.net/goget-ubuntu-touch
 .. _supported: http://wiki.ubuntu.com/Touch/Devices
 
 There are two pieces to touch testing, provisioning and test execution. These
@@ -27,10 +28,20 @@
 
 will list supported options.
 
+Provisioning using this script requires that you start off with the
+device booted and accessible via ADB. The device will be rebooted
+automatically and completely reinstalled - ALL DATA WILL BE LOST.
+
 NOTE: provision.sh requires a path to a network-manager wifi connection that
 can be copied to the target device. By default this is set to
 ${HOME}/.ubuntu-ci/wifi.conf. This can be overridden with the -n parameter.
 
+By default, the latest devel-proposed image will be installed. If you
+wish to install the latest ubuntu-rtm image instead, use::
+
+  export IMAGE_OPT="--bootstrap --developer-mode --channel=ubuntu-touch/ubuntu-rtm/14.09-proposed"
+  ./scripts/provision.sh -w
+
 Executing Tests
 ---------------
 
@@ -52,6 +63,10 @@
 These sub-directories include a xUnit XML formatted file, *test_results.xml*,
 as well as several log files from the device to help with debugging failures.
 
+NOTE: run-autopilot-tests.sh will call a script that installs 
+unity8-autopilot if it is not already installed, to allow the device to
+be unlocked automatically.
+
 An example testing two applications::
 
  ./scripts/run-autopilot-tests.sh -a dropping_letters_app -a music_app
@@ -96,3 +111,71 @@
 Then execute the following script::
 
   ./scripts/run-mp.sh
+
+Running Tests for a Modified Click Application
+----------------------------------------------
+
+First provision the device with the desired image using the instructions
+in the "Provisioning" section of this README.
+
+Once the image has been provisioned, install the click app to test.
+The dropping-letters application is used in this example::
+
+  adb push com.ubuntu.dropping-letters_0.1.2.2.67_all.click /tmp
+  adb shell pkcon --allow-untrusted install-local \
+      /tmp/com.ubuntu.dropping-letters_0.1.2.2.67_all.click
+
+Now install the test sources ('--wipe' will remove any previously installed
+test sources)::
+
+  phablet-click-test-setup --wipe --click com.ubuntu.dropping-letters
+
+The above phablet-click-test-setup command will install the standard test
+dependencies and the click application's test sources as specified in the
+manifest. This is usually the application's trunk branch. To override the test
+sources with local changes, replace the test sources that were copied to the
+device. This example assumes the application code is checked out under the
+'dropping-letters' directory with the test sources under 'tests/autopilot'::
+
+  adb shell rm -rf /home/phablet/autopilot/dropping_letters_app
+  adb push dropping-letters/tests/autopilot \
+      /home/phablet/autopilot
+
+Finally, run the application tests::
+
+  ./scripts/run-autopilot-tests.sh -a dropping_letters_app
+
+The test results are available under::
+
+  clientlogs/dropping_letters_app/test_results.xml
+
+Running Tests for a Modified Debian Package
+-------------------------------------------
+
+First provision the device with the desired image using the instructions
+in the "Provisioning" section of this README.
+
+If the device is provisioned, and you have built the debian package
+you wish to test with locally, install it on the device. For instance,
+if you are building and installing dialer-app::
+
+  phablet-config writable-image -r 0000 --package-dir /path/to/packages -p dialer-app
+
+Alternatively, if you have built the packages in a ppa, you could use::
+
+  phablet-config writable-image -r 0000 --ppa ppa:ci-train-ppa-service/landing-004 -p dialer-app
+
+NOTE: If you have updates to the dependencies or tests in debian
+packages, make sure to also install packages for those if required for
+the change you are making. Some tests need a few extra dependencies 
+installed for the tests to function correctly.  To see a list of them, 
+look at jenkins/testconfig.py.
+
+Finally, run the application tests::
+
+  ./scripts/run-autopilot-tests.sh -a dialer_app
+
+The test results are available under::
+
+  clientlogs/dialer_app/test_results.xml
+