Merge lp:~deeptik/lava-test/gmpbench into lp:lava-test/0.0

[Deepti B. Kalakeri]

This patch automates the installation, execution, and parsing of GMPbench test suite. This patch has been tested on Manderick for installation/execution and parsing of results with the latest abrek revision 35.

[lp:~deeptik/lava-test/gmpbench]

37. By Deepti B. Kalakeri on 2010-09-28

Fix typo in the comment

[Paul Larson (pwlars)]

+RUNSTEPS = ['cd gmpbench-0.2 && PATH=$PATH:. ./runbench ']
> +PATTERN = "\s*(?P<test_case_id>GMPbench\.*\w*\.*\w*):?\s*(?P<result>\d+.\d+)"
> +
> +class GMPParser(abrek.testdef.AbrekTestParser):
> + def parse(self):
> + filename = "testoutput.log"
> + pat = re.compile(self.pattern)
> + with open(filename) as fd:
> + for line in fd:
> + match = pat.match(line)
> + if match:
> + results = match.groupdict()
> + self.results['test_results'].append(results)
> Since this is a performance benchmark, I would expect that you would have a measurement field, and I don't see that here. Also, I don't see where you needed to extend AbrekTestParser yet for this test. It seems that the base class would be sufficient here. Take a look at stream for an example of a performance test.

review: Needs Fixing

[Deepti B. Kalakeri (deeptik)]

> +RUNSTEPS = ['cd gmpbench-0.2 && PATH=$PATH:. ./runbench ']
> > +PATTERN =
> "\s*(?P<test_case_id>GMPbench\.*\w*\.*\w*):?\s*(?P<result>\d+.\d+)"
> > +
> > +class GMPParser(abrek.testdef.AbrekTestParser):
> > + def parse(self):
> > + filename = "testoutput.log"
> > + pat = re.compile(self.pattern)
> > + with open(filename) as fd:
> > + for line in fd:
> > + match = pat.match(line)
> > + if match:
> > + results = match.groupdict()
> > + self.results['test_results'].append(results)
> > Since this is a performance benchmark, I would expect that you would have a
> measurement field, and I don't see that here. Also, I don't see where you
> needed to extend AbrekTestParser yet for this test. It seems that the base
> class would be sufficient here. Take a look at stream for an example of a
> performance test.

Yeah!! the AbrekTestParser would do the same work for parsing instead of rewriting my own parse function. Somehow it slipped from my mind. I will use the default parser available. The result field contains the measurement field. I will change the name of the results to measurement so that it makes more sense.

[lp:~deeptik/lava-test/gmpbench]

38. By Deepti B. Kalakeri on 2010-09-28

Addressing review comments

[Deepti B. Kalakeri (deeptik)]

This patch addresses the review comments and includes the following changes:

1) Removed the parse() function in the file and used the default available in abrek framework.
2) Renamed the parse field from results to measurement to make it more meaningful.

review: Needs Resubmitting

[Paul Larson (pwlars)]

Please add units and result via appendall as you see in stream. Discussed this with you over email earlier, but wanted to mention it here as well.

[lp:~deeptik/lava-test/gmpbench]

39. By Deepti B. Kalakeri on 2010-09-29

Adding the results and unit field to the test_results

[Deepti B. Kalakeri (deeptik)]

Added the results and units field to the gmpbench test_results

[Paul Larson (pwlars)]

28 +import re
unused

Other than that and a few minor whitespace cleanups, it all looks good. I will go ahead and merge. Thanks!

review: Approve

=== added file 'abrek/test_definitions/gmpbench.py'
--- abrek/test_definitions/gmpbench.py	1970-01-01 00:00:00 +0000
+++ abrek/test_definitions/gmpbench.py	2010-09-29 11:21:18 +0000
@@ -0,0 +1,49 @@
+# Copyright (c) 2010 Linaro
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+"""
+   This script automates the automate installation, execution, and
+   results parsing for the GMPbench test suite.
+   The GMPbench test Suite is an open source test suite with the goal of
+   comparing different processors performance against eachother.
+
+"""
+
+import re
+
+import abrek.testdef
+
+VERSION='0.2'
+URL="ftp://ftp.gmplib.org/pub/misc/gmpbench-%s.tar.bz2" %(VERSION)
+URL_gexpr="http://www.gmplib.org/gexpr.c"
+DEPS = ['libgmp3-dev', 'wget']
+
+INSTALLSTEPS = ['tar -xjf  gmpbench-0.2.tar.bz2',
+                'wget -c %s' %(URL_gexpr),
+                'mv gexpr.c gmpbench-0.2', 
+                'cd gmpbench-0.2 && gcc -o gexpr gexpr.c  -static -lm']
+                       
+RUNSTEPS = ['cd  gmpbench-0.2 && PATH=$PATH:. ./runbench ']
+PATTERN = "\s*(?P<test_case_id>GMPbench\.*\w*\.*\w*):?\s*"\
+          "(?P<measurement>\d+.\d+)"
+
+gmpbenchinst = abrek.testdef.AbrekTestInstaller(INSTALLSTEPS, deps=DEPS, 
+                                                url=URL)
+gmpbenchrun = abrek.testdef.AbrekTestRunner(RUNSTEPS)
+gmpbenchparser = abrek.testdef.AbrekTestParser(PATTERN, 
+                                               appendall={'units':'operations/s',
+                                               'result':'pass'})
+testobj = abrek.testdef.AbrekTest(testname="gmpbench", installer=gmpbenchinst, 
+                                  runner=gmpbenchrun, parser=gmpbenchparser)