Merge lp:~elopio/snapcraft/coverage into lp:~snappy-dev/snapcraft/core

[Leo Arias]

Add test coverage report.

[Michael Terry (mterry)]

Nice! :)

Is there any way we could include the plainbox runs in the coverage report? The unit tests are only a part of the story, and so the coverage report would be misleading without the integration tests too.

review: Needs Information

[lp:~elopio/snapcraft/coverage]

91. By Michael Vogt on 2015-07-21

Fix setting of snapcraft.common.plugindir (thanks to cjwatson). by mvo approved by mterry

[Leo Arias (elopio)]

I haven't been able to run the plainbox tests on vivid yet. I get:

usage: manage.py [--help] [--version] [options]
manage.py: error: unrecognized arguments: -d /tmp/tmp.FrgNhrCr9x

I have the ppa mentioned on the README. I'll ask on IRC.

I'm not sure if we want a unified report of unit test + functional test coverage, or we want to reports. We have the same question for go, so we'll be talking about that. Anyway, we can add that in a following branch.

[Leo Arias (elopio)]

I found I was missing an update. But it seems python3-coverage can run only python files. We'll have to see how to make it work with plainbox.

[Michael Terry (mterry)]

OK, this is fine/harmless. I just don't want us to get caught up in chasing those numbers, since they aren't super useful without incorporating the integration tests.

review: Approve

[lp:~elopio/snapcraft/coverage]

92. By Leo Arias on 2015-07-21

Add test coverage report. by elopio approved by mterry

=== modified file 'README.md'
--- README.md	2015-07-13 21:48:10 +0000
+++ README.md	2015-07-21 02:05:48 +0000
@@ -10,6 +10,11 @@
 
     ./runtests.sh
 
+- If you want to get a test coverage report, install python3-coverage before running the tests:
+
+    sudo apt-get install python3-coverage
+
+
 - If you don't want to run the plainbox integration tests, you can skip them by setting SNAPCRAFT_TESTS_SKIP_PLAINBOX=1 in your environment.
 
 - If you are on 15.04 or earlier, you will need to run:

=== modified file 'runtests.sh'
--- runtests.sh	2015-07-15 16:16:45 +0000
+++ runtests.sh	2015-07-21 02:05:48 +0000
@@ -27,7 +27,13 @@
 
 pyflakes3 $SRC_PATHS
 
-python3 -m unittest
+if which python3-coverage >/dev/null 2>&1; then
+    python3-coverage erase
+    python3-coverage run --branch --source snapcraft -m unittest    
+    python3-coverage report
+else
+    python3 -m unittest    
+fi
 
 if [ -z "$SNAPCRAFT_TESTS_SKIP_PLAINBOX" ]; then
 (