Merge lp:~fginther/nux/add-code-coverage into lp:nux

[Francis Ginther]

Add code coverage reporting with coverage-html and coverage-xml targets.

Coverage reporting can be enabled with --enable-gcov.

Add code coverage reporting with coverage-html and coverage-xml targets.

Coverage reporting can be enabled with --enable-gcov.

Testing:
 - Built coverage targets and verified results.
 - Built in pbuilder chroot with hooks to enable gcovr report.

[Allan LeSage (allanlesage)]

* Teensy correction restores coverage for the NuxGraphics dir:

=== modified file 'NuxGraphics/Makefile.am'
--- NuxGraphics/Makefile.am 2012-12-04 19:36:00 +0000
+++ NuxGraphics/Makefile.am 2012-12-05 15:32:28 +0000
@@ -18,7 +18,7 @@
   $(NUX_GRAPHICS_CFLAGS) \
   $(MAINTAINER_CFLAGS) \
   $(GEIS_CFLAGS) \
- $(COVERAGE_FLAGS)
+ $(COVERAGE_CFLAGS)

 libnux_graphics_@NUX_API_VERSION@_la_LIBADD = \
   $(top_builddir)/NuxCore/libnux-core-@NUX_API_VERSION@.la \

* I'll attach a diff of what was necessary to get tests running FYI.

* We could consider splitting these tests into "small" and "large" but can take that up in another MP :) .

* The choice to make "make coverage-html" et.al. not depend on "make check" is evidently a considered one to which I don't object.

Approval contingent on the small correction above.

review: Approve

[Brandon Schaefer (brandontschaefer)]

It looks good to me, but possibly making it easier to know you need to do a make check or make check-headless before running make coverage-html would be nice. Or possibly to check if no *.gcda files are found through an error message saying to run those commands.

Otherwise looks good to me :)

[Francis Ginther (fginther)]

> It looks good to me, but possibly making it easier to know you need to do a
> make check or make check-headless before running make coverage-html would be
> nice. Or possibly to check if no *.gcda files are found through an error
> message saying to run those commands.
>
> Otherwise looks good to me :)

Brandon, I added make check to the two main coverage targets to make it more intuitive for developers. This still allows our jenkins automation to run make check-headless via hook scripts instead of make check and collect the results with gcovr.

[Brandon Schaefer (brandontschaefer)]

Awesome, im happy with that :)

review: Approve

[PS Jenkins bot (ps-jenkins)]

FAILED: Autolanding.
More details in the following jenkins job:
http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/68/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=panda-pbuilder,distribution=quantal,flavor=armhf/68/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=pbuilder,distribution=quantal,flavor=amd64/68/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=pbuilder,distribution=quantal,flavor=i386/68/console

review: Needs Fixing (continuous-integration)

[Martin Mrazik (mrazik)]

Sorry, Jenkins failure which should be fixed now -> reapproving.

[PS Jenkins bot (ps-jenkins)]

FAILED: Autolanding.
More details in the following jenkins job:
http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/70/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=panda-pbuilder,distribution=quantal,flavor=armhf/70/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=pbuilder,distribution=quantal,flavor=amd64/70/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=pbuilder,distribution=quantal,flavor=i386/70/console

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Autolanding.
More details in the following jenkins job:
http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/73/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=panda-pbuilder,distribution=quantal,flavor=armhf/73/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=pbuilder,distribution=quantal,flavor=amd64/73/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=pbuilder,distribution=quantal,flavor=i386/73/console

review: Needs Fixing (continuous-integration)

[Martin Mrazik (mrazik)]

It looks like Brandon did something weird in his branch and hence the conflict. I see some coverage related commits in bzr log --include-merged lp:nux ....

[Brandon Schaefer (brandontschaefer)]

What the heck...all I did was a bzr merge...check it out, then a bzr revert...then did my own thing and pushed the other branch. I don't see how this should carry any info with it...

=== modified file 'Makefile.am'
--- Makefile.am	2012-05-25 20:36:09 +0000
+++ Makefile.am	2012-12-06 01:28:22 +0000
@@ -2,6 +2,7 @@
 
 CXXFLAGS += -fno-permissive
 
+include $(top_srcdir)/Makefile.am.coverage
 include doxygen-include.am
 
 if BUILD_DOCUMENTATION
 
=== added file 'Makefile.am.coverage'
--- Makefile.am.coverage	1970-01-01 00:00:00 +0000
+++ Makefile.am.coverage	2012-12-06 01:28:22 +0000
@@ -0,0 +1,49 @@
+
+# Coverage targets
+
+.PHONY: clean-gcno clean-gcda \
+	coverage-html generate-coverage-html clean-coverage-html \
+	coverage-gcovr generate-coverage-gcovr clean-coverage-gcovr
+
+clean-local: clean-gcno clean-coverage-html clean-coverage-gcovr
+
+if HAVE_GCOV
+
+clean-gcno:
+	@echo Removing old coverage instrumentation
+	-find -name '*.gcno' -print | xargs -r rm
+
+clean-gcda:
+	@echo Removing old coverage results
+	-find -name '*.gcda' -print | xargs -r rm
+  
+coverage-html:
+	-$(MAKE) $(AM_MAKEFLAGS) -k check
+	$(MAKE) $(AM_MAKEFLAGS) generate-coverage-html
+  
+generate-coverage-html:
+	@echo Collecting coverage data
+	$(LCOV) --directory $(top_builddir) --capture --output-file coverage.info --no-checksum --compat-libtool
+	$(LCOV) --extract coverage.info "$(abs_top_srcdir)/*" -o coverage.info
+	LANG=C $(GENHTML) --prefix $(top_builddir) --output-directory coveragereport --title "Code Coverage" --legend --show-details coverage.info
+  
+clean-coverage-html: clean-gcda
+	-$(LCOV) --directory $(top_builddir) -z
+	-rm -rf coverage.info coveragereport
+
+if HAVE_GCOVR
+
+coverage-gcovr:
+	-$(MAKE) $(AM_MAKEFLAGS) -k check
+	$(MAKE) $(AM_MAKEFLAGS) generate-coverage-gcovr
+	
+generate-coverage-gcovr:
+	@echo Generating coverage GCOVR report
+	$(GCOVR) --xml -r $(top_builddir) -o $(top_builddir)/coverage.xml
+	
+clean-coverage-gcovr: clean-gcda
+	-rm -rf $(top_builddir)/coverage.xml
+
+endif # HAVE_GCOVR
+
+endif # HAVE_GCOV
 
=== modified file 'Nux/Makefile.am'
--- Nux/Makefile.am	2012-11-20 22:13:38 +0000
+++ Nux/Makefile.am	2012-12-06 01:28:22 +0000
@@ -17,7 +17,8 @@
   $(NUX_CFLAGS) \
   $(IBUS_CFLAGS) \
   $(GEIS_CFLAGS) \
-  $(MAINTAINER_CFLAGS)
+  $(MAINTAINER_CFLAGS) \
+  $(COVERAGE_CFLAGS)
 
 libnux_@NUX_API_VERSION@_la_LIBADD = \
   $(top_builddir)/NuxCore/libnux-core-@NUX_API_VERSION@.la \
@@ -27,7 +28,8 @@
   $(GEIS_LIBS)
 
 libnux_@NUX_API_VERSION@_la_LDFLAGS = \
-  $(NUX_LT_LDFLAGS)
+  $(NUX_LT_LDFLAGS) \
+  $(COVERAGE_LDFLAGS)
 
 source_cpp = \
   $(srcdir)/AbstractButton.cpp \
 
=== modified file 'NuxCore/Makefile.am'
--- NuxCore/Makefile.am	2012-09-12 00:52:05 +0000
+++ NuxCore/Makefile.am	2012-12-06 01:28:22 +0000
@@ -16,13 +16,15 @@
 libnux_core_@NUX_API_VERSION@_la_CPPFLAGS= \
   $(GCC_FLAGS) \
   $(NUX_CORE_CFLAGS) \
-  $(MAINTAINER_CFLAGS)
+  $(MAINTAINER_CFLAGS) \
+  $(COVERAGE_CFLAGS)
 
 libnux_core_@NUX_API_VERSION@_la_LIBADD = \
   $(NUX_CORE_LIBS) -lpthread
 
 libnux_core_@NUX_API_VERSION@_la_LDFLAGS = \
-  $(NUX_LT_LDFLAGS)
+  $(NUX_LT_LDFLAGS) \
+  $(COVERAGE_LDFLAGS)
 
 source_cpp = \
   $(srcdir)/Animation.cpp \
 
=== modified file 'NuxGraphics/Makefile.am'
--- NuxGraphics/Makefile.am	2012-11-05 17:47:31 +0000
+++ NuxGraphics/Makefile.am	2012-12-06 01:28:22 +0000
@@ -17,7 +17,8 @@
   $(GCC_FLAGS) \
   $(NUX_GRAPHICS_CFLAGS) \
   $(MAINTAINER_CFLAGS) \
-  $(GEIS_CFLAGS)
+  $(GEIS_CFLAGS) \
+  $(COVERAGE_CFLAGS)
 
 libnux_graphics_@NUX_API_VERSION@_la_LIBADD = \
   $(top_builddir)/NuxCore/libnux-core-@NUX_API_VERSION@.la \
@@ -25,7 +26,8 @@
   $(GEIS_LIBS)
 
 libnux_graphics_@NUX_API_VERSION@_la_LDFLAGS = \
-  $(NUX_LT_LDFLAGS)
+  $(NUX_LT_LDFLAGS) \
+  $(COVERAGE_LDFLAGS)
 
 source_h = \
   $(srcdir)/BitmapFormats.h \
 
=== modified file 'configure.ac'
--- configure.ac	2012-11-27 15:12:37 +0000
+++ configure.ac	2012-12-06 01:28:22 +0000
@@ -342,6 +342,17 @@
 
 AM_CONDITIONAL(BUILD_TESTS, [test "x$enable_tests" = "xyes"])
 
+###########################
+# gcov coverage reporting
+###########################
+m4_include([m4/gcov.m4])
+AC_TDD_GCOV
+AM_CONDITIONAL([HAVE_GCOV], [test "x$ac_cv_check_gcov" = xyes])
+AM_CONDITIONAL([HAVE_LCOV], [test "x$ac_cv_check_lcov" = xyes])
+AM_CONDITIONAL([HAVE_GCOVR], [test "x$ac_cv_check_gcovr" = xyes])
+AC_SUBST(COVERAGE_CFLAGS)
+AC_SUBST(COVERAGE_LDFLAGS)
+
 dnl ===========================================================================
 
 # sets up doxygen
@@ -485,6 +496,7 @@
 echo -e "        Build Examples     : ${BOLD_WHITE}${enable_examples}${RESET}"
 echo -e "        Build Gpu Tests    : ${BOLD_WHITE}${enable_gputests}${RESET}"
 echo -e "        Build Nux Tests    : ${BOLD_WHITE}${enable_tests}${RESET}"
+echo -e "        Coverage Reporting : ${BOLD_WHITE}${use_gcov}${RESET}"
 echo -e "        Gestures support   : ${BOLD_WHITE}${have_geis}${RESET}"
 echo -e "        X11 build support  : ${BOLD_WHITE}${enable_x_support}${RESET}"
 echo -e "        Minimal build      : ${BOLD_WHITE}${enable_minimal_build}${RESET}"
 
=== added file 'm4/gcov.m4'
--- m4/gcov.m4	1970-01-01 00:00:00 +0000
+++ m4/gcov.m4	2012-12-06 01:28:22 +0000
@@ -0,0 +1,89 @@
+# Checks for existence of coverage tools:
+#  * gcov
+#  * lcov
+#  * genhtml
+#  * gcovr
+# 
+# Sets ac_cv_check_gcov to yes if tooling is present
+# and reports the executables to the variables LCOV, GCOVR and GENHTML.
+AC_DEFUN([AC_TDD_GCOV],
+[
+  AC_ARG_ENABLE(gcov,
+  AS_HELP_STRING([--enable-gcov],
+		 [enable coverage testing with gcov]),
+  [use_gcov=$enableval], [use_gcov=no])
+
+  AM_CONDITIONAL(HAVE_GCOV, test "x$use_gcov" = "xyes")
+
+  if test "x$use_gcov" = "xyes"; then
+  # we need gcc:
+  if test "$GCC" != "yes"; then
+    AC_MSG_ERROR([GCC is required for --enable-gcov])
+  fi
+
+  # Check if ccache is being used
+  AC_CHECK_PROG(SHTOOL, shtool, shtool)
+  if test "$SHTOOL"; then
+    AS_CASE([`$SHTOOL path $CC`],
+                [*ccache*], [gcc_ccache=yes],
+                [gcc_ccache=no])
+  fi
+
+  if test "$gcc_ccache" = "yes" && (test -z "$CCACHE_DISABLE" || test "$CCACHE_DISABLE" != "1"); then
+    AC_MSG_ERROR([ccache must be disabled when --enable-gcov option is used. You can disable ccache by setting environment variable CCACHE_DISABLE=1.])
+  fi
+
+  lcov_version_list="1.6 1.7 1.8 1.9"
+  AC_CHECK_PROG(LCOV, lcov, lcov)
+  AC_CHECK_PROG(GENHTML, genhtml, genhtml)
+
+  if test "$LCOV"; then
+    AC_CACHE_CHECK([for lcov version], glib_cv_lcov_version, [
+      glib_cv_lcov_version=invalid
+      lcov_version=`$LCOV -v 2>/dev/null | $SED -e 's/^.* //'`
+      for lcov_check_version in $lcov_version_list; do
+        if test "$lcov_version" = "$lcov_check_version"; then
+          glib_cv_lcov_version="$lcov_check_version (ok)"
+        fi
+      done
+    ])
+  else
+    lcov_msg="To enable code coverage reporting you must have one of the following lcov versions installed: $lcov_version_list"
+    AC_MSG_ERROR([$lcov_msg])
+  fi
+
+  case $glib_cv_lcov_version in
+    ""|invalid[)]
+      lcov_msg="You must have one of the following versions of lcov: $lcov_version_list (found: $lcov_version)."
+      AC_MSG_ERROR([$lcov_msg])
+      LCOV="exit 0;"
+      ;;
+  esac
+
+  if test -z "$GENHTML"; then
+    AC_MSG_ERROR([Could not find genhtml from the lcov package])
+  fi
+
+  ac_cv_check_gcov=yes
+  ac_cv_check_lcov=yes
+
+  # Remove all optimization flags from CFLAGS
+  changequote({,})
+  CFLAGS=`echo "$CFLAGS" | $SED -e 's/-O[0-9]*//g'`
+  changequote([,])
+
+  # Add the special gcc flags
+  COVERAGE_CFLAGS="-O0 --coverage"
+  COVERAGE_CXXFLAGS="-O0 --coverage"
+  COVERAGE_LDFLAGS="-lgcov"
+
+  # Check availability of gcovr
+  AC_CHECK_PROG(GCOVR, gcovr, gcovr)
+  if test -z "$GCOVR"; then
+    ac_cv_check_gcovr=no
+  else
+    ac_cv_check_gcovr=yes
+  fi
+fi
+]) # AC_TDD_GCOV
+