Merge lp:~le-chi-thu/lava-test/override-test-options into lp:lava-test/0.0

[Le Chi Thu]

Implemented the blueprint https://blueprints.launchpad.net/lava-test/+spec/linaro-platforms-o-override-test-options

See the updated test definition peacekeeper.py how to use this feature.

from the peacekeeper.py :

RUNSTEPS = ['python peacekeeper_runner.py $(OPTIONS)']

The $(OPTIONS) in the run steps will be replaced with 'firefox' when issuing the command "lava-test run peacekeeper firefox"

[Paul Larson (pwlars)]

This will require modifications to all of those tests of course, not just peacekeeper... so I'm assuming this is just an example and not meant to be ready yet right? Also, there will of course need to be a dispatcher change to make use of this.

32 + parser.add_argument("test_options",
33 + help="Override the default test options by this test options. \n"
34 + "Example : lava-test run peacekeeper firefox")
35 +
I think we should specify a -/-- option name for this rather than taking all the extra args off the command line. It makes it more clear with passing other options to the lava-test run command.

Also, we could use either that option with no arguments if it makes sense, or add another option for displaying the default commandline use with that test.

review: Needs Fixing

[lp:~le-chi-thu/lava-test/override-test-options]

102. By Le Chi Thu <email address hidden> <email address hidden> on 2011-10-18

Make testoptions as option parameter in run command.

[Le Chi Thu (le-chi-thu)]

I updated the lava-test override-test-options. It is still the same syntax as before but the test options is an optional parameter.

lava-test run [-h] [-q] [-Q] [-S] [-H] [--trusted-time] [-o FILE] [-A] test_id [test_options]

review: Needs Resubmitting

[Spring Zhang (qzhang)]

182 +if result_url.find("http") == -1:
is it started with "http"?

+ "Example of multiple options : lava-test run foo_test 'arg1 arg2")
lost a single-quote

[Paul Larson (pwlars)]

> I updated the lava-test override-test-options. It is still the same syntax as
> before but the test options is an optional parameter.
>
> lava-test run [-h] [-q] [-Q] [-S] [-H] [--trusted-time] [-o FILE] [-A] test_id
> [test_options]

Indeed, it should be optional, however please see my previous comments. I think if we used something like --test-options "-a foo -b bar" instead, it would be better. Rational is explained in previous comment.

[lp:~le-chi-thu/lava-test/override-test-options]

103. By Le Chi Thu <email address hidden> <email address hidden> on 2011-10-19

make the test_options an option instead of argument.

104. By Le Chi Thu <email address hidden> <email address hidden> on 2011-10-19

made the test_options as a option to the run command

105. By Le Chi Thu <email address hidden> <email address hidden> on 2011-10-19

Renamed --test_options to --test-options

[Le Chi Thu (le-chi-thu)]

Now it is an option instead of a argument.

 lava-test run [-h] [-q] [-Q] [-S] [-H] [--trusted-time] [-o FILE] [-A]
                     [-t TEST_OPTIONS]
                     test_id

positional arguments:
  test_id Test or test suite identifier

optional arguments:
  -h, --help show this help message and exit
  -q, --quiet Be less verbose about undertaken actions
  -Q, --quiet-subcommands
                        Hide the output of all sub-commands (including tests)
  -t TEST_OPTIONS, --test-options TEST_OPTIONS
                        Override the default test options. The $(OPTIONS) in
                        the run steps will be replaced by the options.See
                        peacekeeper.py as example how to use this feature.To
                        provide multiple options, use quote character.Example
                        : lava-test run peacekeeper -t firefox. Example of
                        multiple options : lava-test run foo_test -t 'arg1
                        arg2'

review: Needs Resubmitting

[Paul Larson (pwlars)]

The change still needs to be made to allow all tests to allow options sent to them, not just peacekeeper.

[lp:~le-chi-thu/lava-test/override-test-options]

106. By Le Chi Thu <email address hidden> <email address hidden> on 2011-10-20

Updated all test definitions with

[Paul Larson (pwlars)]

I haven't tested all of them, but I spot checked a few and they seemed to run ok. I also fixed the unit tests. The change in number of args to ...
67 - def _run_lava_test_steps(self, artifacts, observer):
68 + def _run_lava_test_steps(self, artifacts, observer, test_options):
...broke things a bit.

Also, it seems that there's still no way to see information on the test from the command line about what the default args will be. I'd really like to have that and it should be pretty simple. I'd like to move this along though, so I'll just add a bug for it.

review: Approve

=== modified file 'lava_test/api/delegates.py'
--- lava_test/api/delegates.py	2011-09-27 20:07:52 +0000
+++ lava_test/api/delegates.py	2011-10-20 12:16:24 +0000
@@ -65,7 +65,7 @@
     """
 
     @abstractmethod
-    def run(self, artifacts, observer):
+    def run(self, artifacts, observer, test_options):
         """
         Run the test and create artifacts (typically log files).
 
@@ -78,7 +78,9 @@
         :param observer:
             Observer object that makes it possible to monitor the actions
             performed by the test runner.
-        :type observer: :class:`~lava_test.api.observers.ITestRunnerObserver` 
+        :type observer: :class:`~lava_test.api.observers.ITestRunnerObserver`
+        :param test_options:
+           A string with space separated options to pass to the test.
 
         :return true if any test step return none-zero return code
         .. versionadded:: 0.2

=== modified file 'lava_test/commands.py'
--- lava_test/commands.py	2011-10-05 21:57:34 +0000
+++ lava_test/commands.py	2011-10-20 12:16:24 +0000
@@ -221,11 +221,20 @@
                                   " is not affected as it never stores any"
                                   " attachments."))
 
+        parser.add_argument("-t", "--test-options",
+                            default=None,
+                            help="Override the default test options. "
+                                 "The $(OPTIONS) in the run steps will be replaced by the options."
+                                 "See peacekeeper.py as example how to use this feature."
+                                 "To provide multiple options, use quote character."
+                                 "Example : lava-test run peacekeeper -t firefox.  "
+                                 "Example of multiple options : lava-test run foo_test -t 'arg1 arg2'")
+
     def invoke_with_test(self, test):
         if not test.is_installed:
             raise LavaCommandError("The specified test is not installed")
         try:
-            artifacts, run_fail = test.run(self)
+            artifacts, run_fail = test.run(self, test_options=self.args.test_options)
         except subprocess.CalledProcessError as ex:
             if ex.returncode is None:
                 raise LavaCommandError("Command %r was aborted" % ex.cmd)

=== modified file 'lava_test/core/runners.py'
--- lava_test/core/runners.py	2011-10-05 21:57:34 +0000
+++ lava_test/core/runners.py	2011-10-20 12:16:24 +0000
@@ -32,14 +32,14 @@
     :ivar steps:
         list of shell commands to execute
     """
-    def __init__(self, steps=None):
+    def __init__(self, steps=None, default_options=None):
         self.steps = steps or []
-        self.testoutput = []  # XXX: is this still used?
+        self.default_options = default_options
 
     def __repr__(self):
         return "<%s steps=%r>" % (self.__class__.__name__, self.steps)
 
-    def _run_lava_test_steps(self, artifacts, observer):
+    def _run_lava_test_steps(self, artifacts, observer, test_options):
         stdout = open(artifacts.stdout_pathname, 'at')
         stderr = open(artifacts.stderr_pathname, 'at')
         delegate = DisplayDelegate(stdout, stderr, observer)
@@ -47,6 +47,15 @@
         run_failed = False
         try:
             for cmd in self.steps:
+                # should override the test options in the step ?
+                if cmd.find("$(OPTIONS)") > 0:
+                    # check if test options is provided or use default options
+                    if not test_options and self.default_options == None:
+                        raise RuntimeError("Test options is missing. No default value is provided.")
+                    if not test_options:
+                        test_options = self.default_options
+                    cmd = cmd.replace("$(OPTIONS)",test_options)
+
                 if observer: observer.about_to_run_shell_command(cmd)
                 returncode = extcmd.call(cmd, shell=True)
                 if observer: observer.did_run_shell_command(cmd, returncode)
@@ -56,7 +65,7 @@
             stderr.close()
         return run_failed
 
-    def run(self, artifacts, observer=None):
+    def run(self, artifacts, observer=None, test_options = None):
         """
         Run the test program by executing steps in sequence.
 
@@ -65,6 +74,6 @@
             :meth:`~lava_test.api.delegates.TestRunner.run`
         """
         self.starttime = datetime.datetime.utcnow()
-        run_failed = self._run_lava_test_steps(artifacts, observer)
+        run_failed = self._run_lava_test_steps(artifacts, observer, test_options)
         self.endtime = datetime.datetime.utcnow()
         return run_failed

=== modified file 'lava_test/core/tests.py'
--- lava_test/core/tests.py	2011-10-05 21:57:34 +0000
+++ lava_test/core/tests.py	2011-10-20 12:16:24 +0000
@@ -52,13 +52,15 @@
     """
 
     def __init__(self, test_id, test_version=None,
-                 installer=None, runner=None, parser=None):
+                 installer=None, runner=None, parser=None, default_options=None):
         self._test_id = test_id
         self._test_version = test_version
         # Delegate objects
         self.installer = installer
         self.runner = runner
         self.parser = parser
+        self.default_options = default_options
+        
         # Config instance
         self._config = get_config()
 
@@ -117,7 +119,7 @@
         if os.path.exists(self.install_dir):
             shutil.rmtree(self.install_dir)
 
-    def run(self, observer=None):
+    def run(self, observer=None, test_options=None):
         if not self.runner:
             raise RuntimeError(
                 "no test runner defined for '%s'" % self.test_id)
@@ -126,7 +128,7 @@
             logging.debug(
                 "Invoking %r.run_and_store_artifacts(...)",
                 self.runner, observer)
-            run_fail = self.runner.run(artifacts, observer)
+            run_fail = self.runner.run(artifacts, observer, test_options)
 
         return artifacts, run_fail
 

=== modified file 'lava_test/test_definitions/bootchart.py'
--- lava_test/test_definitions/bootchart.py	2011-09-23 08:10:25 +0000
+++ lava_test/test_definitions/bootchart.py	2011-10-20 12:16:24 +0000
@@ -18,13 +18,14 @@
 from lava_test.core.runners import TestRunner
 from lava_test.core.tests import Test
 
+DEFAULT_OPTIONS = ""
 INSTALLSTEPS = ['bzr branch lp:~linaro-foundations/lava-test/bootchartscript']
 DEPS = ['bootchart', 'pybootchartgui', 'bzr']
-RUNSTEPS = ['./bootchartscript/bootchartscript.sh']
+RUNSTEPS = ['./bootchartscript/bootchartscript.sh $(OPTIONS)']
 PATTERN = "^(?P<test_case_id>\w+):\W+(?P<measurement>\d+\.\d+)"
 
 bootchartinst = TestInstaller(INSTALLSTEPS, deps=DEPS)
-bootchartrun = TestRunner(RUNSTEPS)
+bootchartrun = TestRunner(RUNSTEPS, default_options=DEFAULT_OPTIONS)
 bootchartparser = TestParser(PATTERN,
                    appendall={'units':'sec', 'result':'pass'})
 testobj = Test(test_id="bootchart", installer=bootchartinst,

=== modified file 'lava_test/test_definitions/firefox.py'
--- lava_test/test_definitions/firefox.py	2011-09-12 09:19:10 +0000
+++ lava_test/test_definitions/firefox.py	2011-10-20 12:16:24 +0000
@@ -19,14 +19,14 @@
 from lava_test.core.runners import TestRunner
 from lava_test.core.tests import Test
 
-
+DEFAULT_OPTIONS = ""
 INSTALLSTEPS = ['git clone git://github.com/janimo/firefox-startup-timing.git']
 DEPS = ['firefox', 'git-core', 'gcalctool']
-RUNSTEPS = ['cd firefox-startup-timing; ./firefox_startup_timing.sh']
+RUNSTEPS = ['cd firefox-startup-timing; ./firefox_startup_timing.sh $(OPTIONS)']
 PATTERN = "^(?P<test_case_id>\w+):(?P<measurement>\d+)"
 
 firefoxinst = TestInstaller(INSTALLSTEPS, deps=DEPS)
-firefoxrun = TestRunner(RUNSTEPS)
+firefoxrun = TestRunner(RUNSTEPS, default_options=DEFAULT_OPTIONS)
 firefoxparser = TestParser(PATTERN,
                appendall={'units':'ms', 'result':'pass'})
 testobj = Test(test_id="firefox", installer=firefoxinst,

=== modified file 'lava_test/test_definitions/glmemperf.py'
--- lava_test/test_definitions/glmemperf.py	2011-09-12 09:19:10 +0000
+++ lava_test/test_definitions/glmemperf.py	2011-10-20 12:16:24 +0000
@@ -19,12 +19,12 @@
 from lava_test.core.runners import TestRunner
 from lava_test.core.tests import Test
 
-
-RUNSTEPS = ["glmemperf -e shmimage"]
+DEFAULT_OPTIONS = "-e shmimage"
+RUNSTEPS = ["glmemperf $(OPTIONS)"]
 PATTERN = "^(?P<test_case_id>\w+):\W+(?P<measurement>\d+) fps"
 
 inst = TestInstaller(deps=["glmemperf"])
-run = TestRunner(RUNSTEPS)
+run = TestRunner(RUNSTEPS, default_options=DEFAULT_OPTIONS)
 parse = TestParser(PATTERN,
                                       appendall={'units':'fps',
                                                  'result':'pass'})

=== modified file 'lava_test/test_definitions/gmpbench.py'
--- lava_test/test_definitions/gmpbench.py	2011-09-12 09:19:10 +0000
+++ lava_test/test_definitions/gmpbench.py	2011-10-20 12:16:24 +0000
@@ -33,17 +33,18 @@
 URL_gexpr="http://www.gmplib.org/gexpr.c"
 DEPS = ['gcc', 'libgmp3-dev', 'wget', 'bzip2']
 
+DEFAULT_OPTIONS = ""
 INSTALLSTEPS = ['tar -xjf  gmpbench-0.2.tar.bz2',
                 'wget -c %s' %(URL_gexpr),
                 'mv gexpr.c gmpbench-0.2',
                 'cd gmpbench-0.2 && gcc -o gexpr gexpr.c  -static -lm']
-RUNSTEPS = ['cd  gmpbench-0.2 && PATH=$PATH:. ./runbench ']
+RUNSTEPS = ['cd  gmpbench-0.2 && PATH=$PATH:. ./runbench $(OPTIONS)']
 PATTERN = "\s*(?P<test_case_id>GMPbench\.*\w*\.*\w*):?\s*"\
           "(?P<measurement>\d+.\d+)"
 
 gmpbenchinst = TestInstaller(INSTALLSTEPS, deps=DEPS,
                                                 url=URL)
-gmpbenchrun = TestRunner(RUNSTEPS)
+gmpbenchrun = TestRunner(RUNSTEPS,default_options=DEFAULT_OPTIONS)
 gmpbenchparser = TestParser(PATTERN,
     appendall={'units':'operations/s', 'result':'pass'})
 testobj = Test(test_id="gmpbench", installer=gmpbenchinst,

=== modified file 'lava_test/test_definitions/gtkperf.py'
--- lava_test/test_definitions/gtkperf.py	2011-09-23 08:56:22 +0000
+++ lava_test/test_definitions/gtkperf.py	2011-10-20 12:16:24 +0000
@@ -22,9 +22,9 @@
 
 
 # Run tests automatically, 500 repetitions each
-gtkperf_options = "-a -c 500"
+DEFAULT_OPTIONS = "-a -c 500"
 
-RUNSTEPS = ["LANG=C gtkperf %s" % gtkperf_options]
+RUNSTEPS = ["LANG=C gtkperf  $(OPTIONS)"]
 
 class GtkTestParser(TestParser):
     def parse(self, artifacts):
@@ -58,7 +58,7 @@
 
 parse = GtkTestParser(appendall={'units':'seconds', 'result':'pass'})
 inst = TestInstaller(deps=["gtkperf"])
-run = TestRunner(RUNSTEPS)
+run = TestRunner(RUNSTEPS, default_options=DEFAULT_OPTIONS)
 
 testobj = Test(test_id="gtkperf", installer=inst,
                                   runner=run, parser=parse)

=== modified file 'lava_test/test_definitions/ltp.py'
--- lava_test/test_definitions/ltp.py	2011-09-12 09:19:10 +0000
+++ lava_test/test_definitions/ltp.py	2011-10-20 12:16:24 +0000
@@ -35,11 +35,12 @@
 make all
 SKIP_IDCHECK=1 make install
 """
+DEFAULT_OPTIONS = "-f syscalls -p -q"
 
 INSTALLSTEPS = ["echo '%s' > installltp.sh" % SCRIPT,
                 'chmod +x installltp.sh',
                 './installltp.sh']
-RUNSTEPS = ['cd build && sudo ./runltp -f syscalls -p -q']
+RUNSTEPS = ['cd build && sudo ./runltp $(OPTIONS)']
 PATTERN = "^(?P<test_case_id>\S+)    (?P<subid>\d+)  (?P<result>\w+)  :  (?P<message>.+)"
 FIXUPS = {"TBROK":"fail",
           "TCONF":"skip",
@@ -74,7 +75,7 @@
 
 ltpinst = TestInstaller(INSTALLSTEPS, deps=DEPS, url=URL,
                                            md5=MD5)
-ltprun = TestRunner(RUNSTEPS)
+ltprun = TestRunner(RUNSTEPS,default_options = DEFAULT_OPTIONS)
 ltpparser = LTPParser(PATTERN, fixupdict = FIXUPS)
 testobj = Test(test_id="ltp", test_version=VERSION,
                                   installer=ltpinst, runner=ltprun,

=== modified file 'lava_test/test_definitions/peacekeeper.py'
--- lava_test/test_definitions/peacekeeper.py	2011-09-12 09:19:10 +0000
+++ lava_test/test_definitions/peacekeeper.py	2011-10-20 12:16:24 +0000
@@ -30,12 +30,13 @@
 
 curdir = os.path.realpath(os.path.dirname(__file__))
 
+DEFAULT_OPTIONS = "firefox"
 INSTALLSTEPS = ['cp -rf %s/peacekeeper/* .'%curdir]
-RUNSTEPS = ['python peacekeeper_runner.py firefox']
+RUNSTEPS = ['python peacekeeper_runner.py $(OPTIONS)']
 DEPS = ['python-ldtp','firefox']
 
 my_installer = TestInstaller(INSTALLSTEPS, deps=DEPS)
-my_runner = TestRunner(RUNSTEPS)
+my_runner = TestRunner(RUNSTEPS, default_options=DEFAULT_OPTIONS)
 
 PATTERN = "^(?P<result>\w+): Score = (?P<measurement>\d+)"
 

=== modified file 'lava_test/test_definitions/peacekeeper/peacekeeper_runner.py'
--- lava_test/test_definitions/peacekeeper/peacekeeper_runner.py	2011-06-08 12:38:38 +0000
+++ lava_test/test_definitions/peacekeeper/peacekeeper_runner.py	2011-10-20 12:16:24 +0000
@@ -25,7 +25,6 @@
 }
 
 site = "http://service.futuremark.com/peacekeeper/run.action"
-#site = "http://service.futuremark.com/peacekeeper/results.action?key=5sdG"
 
 try:
     browser = browser_data[sys.argv[1]]
@@ -53,8 +52,13 @@
 
 closewindow(browser["title"])
 
-print result_url
-
+print "result_url = %s" %result_url
+
+# if the url not start with http ? append http to it.
+
+if result_url.find("http") != 0:
+    result_url = "http://" + result_url
+    
 if wait_loop > 0:
     fd = urlopen(result_url)
     data = fd.read()

=== modified file 'lava_test/test_definitions/posixtestsuite.py'
--- lava_test/test_definitions/posixtestsuite.py	2011-09-23 08:56:22 +0000
+++ lava_test/test_definitions/posixtestsuite.py	2011-10-20 12:16:24 +0000
@@ -34,9 +34,10 @@
 URL= "http://downloads.sourceforge.net/project/ltp/LTP Source/ltp-%s/"\
      "ltp-full-%s.bz2" % (VERSION, VERSION)
 MD5="6982c72429a62f3917c13b2d529ad1ce"
+DEFAULT_OPTIONS = ""
 INSTALLSTEPS = ['tar -xjf ltp-full-20100831.bz2']
 DEPS = ['gcc', 'bzip2']
-RUNSTEPS = ['cd ltp-full-20100831/testcases/open_posix_testsuite/ && make']
+RUNSTEPS = ['cd ltp-full-20100831/testcases/open_posix_testsuite/ && make $(OPTIONS)']
 
 PATTERN = "((?P<test_case_id>\A(\w+[/]+)+\w+[-]*\w*[-]*\w*) .*? (?P<result>\w+))"
 FIXUPS = {
@@ -69,7 +70,7 @@
 
 posix_inst = TestInstaller(INSTALLSTEPS, deps=DEPS,
     url=URL, md5=MD5)
-posix_run = TestRunner(RUNSTEPS)
+posix_run = TestRunner(RUNSTEPS, default_options=DEFAULT_OPTIONS)
 posixparser = PosixParser(PATTERN, fixupdict = FIXUPS)
 testobj = Test(test_id="posixtestsuite", test_version=VERSION,
                                   installer=posix_inst, runner=posix_run,

=== modified file 'lava_test/test_definitions/pwrmgmt.py'
--- lava_test/test_definitions/pwrmgmt.py	2011-10-05 22:10:55 +0000
+++ lava_test/test_definitions/pwrmgmt.py	2011-10-20 12:16:24 +0000
@@ -19,15 +19,15 @@
 from lava_test.core.runners import TestRunner
 from lava_test.core.tests import Test
 
-
+DEFAULT_OPTIONS = ""
 
 INSTALLSTEPS = ['git clone git://git.linaro.org/tools/pm-qa.git',
                 'cd pm-qa && make clean && make all']
-RUNSTEPS = ['cd pm-qa && make check']
+RUNSTEPS = ['cd pm-qa && make check $(OPTIONS)']
 DEPS = ['git-core', 'make', 'linux-libc-dev', 'util-linux']
 
 pwrmgmtinst = TestInstaller(INSTALLSTEPS, deps=DEPS)
-pwrmgmtrun = TestRunner(RUNSTEPS)
+pwrmgmtrun = TestRunner(RUNSTEPS, default_options=DEFAULT_OPTIONS)
 
 # test case name is before  ":" , the test log is between ":" and "...",
 # the result is after "..."

=== modified file 'lava_test/test_definitions/pybench.py'
--- lava_test/test_definitions/pybench.py	2011-09-12 09:19:10 +0000
+++ lava_test/test_definitions/pybench.py	2011-10-20 12:16:24 +0000
@@ -29,13 +29,13 @@
 
 VERSION='r27'
 URL="http://svn.python.org/projects/python/tags/%s/Tools/pybench/" %(VERSION)
-
+DEFAULT_OPTIONS = ""
 INSTALLSTEPS = ["svn export %s" %(URL)]
-RUNSTEPS = ['python pybench/pybench.py']
+RUNSTEPS = ['python pybench/pybench.py $(OPTIONS)']
 DEPS = ['subversion']
 
 my_installer = TestInstaller(INSTALLSTEPS, deps=DEPS)
-my_runner = TestRunner(RUNSTEPS)
+my_runner = TestRunner(RUNSTEPS,default_options=DEFAULT_OPTIONS)
 
 # test case name is first column and measurement is average column
 #
@@ -45,9 +45,7 @@
 
 PATTERN = "^\s+(?P<test_case_id>\w+):\s+(\d+)ms\s+(?P<measurement>\d+)ms"
 
-my_parser = TestParser(PATTERN,
-                                          appendall={'units':'ms',
-                                                     'result':'pass'})
+my_parser = TestParser(PATTERN, appendall={'units':'ms','result':'pass'})
 
 testobj = Test(test_id="pybench", installer=my_installer,
-                                  runner=my_runner, parser=my_parser)
+               runner=my_runner, parser=my_parser)

=== modified file 'lava_test/test_definitions/smem.py'
--- lava_test/test_definitions/smem.py	2011-09-23 08:10:25 +0000
+++ lava_test/test_definitions/smem.py	2011-10-20 12:16:24 +0000
@@ -18,13 +18,14 @@
 from lava_test.core.runners import TestRunner
 from lava_test.core.tests import Test
 
-
-RUNSTEPS = ['smem -w | tail -n 3']
+DEFAULT_OPTIONS = ""
+RUNSTEPS = ['smem -w $(OPTIONS) | tail -n 3']
 PATTERN = "^(?P<test_case_id>(\w+\s)+)\s\s+(?P<measurement>\d+)"
 DEPS = ['smem']
 
+
 smeminst = TestInstaller(deps=DEPS)
-smemrun = TestRunner(RUNSTEPS)
+smemrun = TestRunner(RUNSTEPS,default_options=DEFAULT_OPTIONS)
 smemparser = TestParser(PATTERN,
                appendall={'units':'KB', 'result':'pass'})
 testobj = Test(test_id="smem", installer=smeminst,

=== modified file 'lava_test/test_definitions/stream.py'
--- lava_test/test_definitions/stream.py	2011-09-12 09:19:10 +0000
+++ lava_test/test_definitions/stream.py	2011-10-20 12:16:24 +0000
@@ -21,11 +21,12 @@
 URL="http://www.cs.virginia.edu/stream/FTP/Code/stream.c"
 INSTALLSTEPS = ['cc stream.c -O2 -fopenmp -o stream']
 DEPS = ['gcc', 'build-essential']
-RUNSTEPS = ['./stream']
+DEFAULT_OPTIONS = ""
+RUNSTEPS = ['./stream $(OPTIONS)']
 PATTERN = "^(?P<test_case_id>\w+):\W+(?P<measurement>\d+\.\d+)"
 
 streaminst = TestInstaller(INSTALLSTEPS, deps=DEPS, url=URL)
-streamrun = TestRunner(RUNSTEPS)
+streamrun = TestRunner(RUNSTEPS,default_options=DEFAULT_OPTIONS)
 streamparser = TestParser(PATTERN,
                appendall={'units':'MB/s', 'result':'pass'})
 testobj = Test(test_id="stream", installer=streaminst,

=== modified file 'lava_test/test_definitions/tiobench.py'
--- lava_test/test_definitions/tiobench.py	2011-09-12 09:19:10 +0000
+++ lava_test/test_definitions/tiobench.py	2011-10-20 12:16:24 +0000
@@ -31,11 +31,11 @@
 VERSION="0.3.3"
 URL="http://prdownloads.sourceforge.net/tiobench/tiobench-%s.tar.gz" %(VERSION)
 MD5="bf485bf820e693c79e6bd2a38702a128"
+DEFAULT_OPTIONS = "--block=4096 --block=8192 --threads=2 --numruns=2"
 INSTALLSTEPS = ['tar -zxvf tiobench-%s.tar.gz' % VERSION,
                 'cd tiobench-%s && make' % VERSION]
 RUNSTEPS = ["cd tiobench-%s && "\
-            "./tiobench.pl --block=4096 --block=8192 --threads=2 "\
-            "--numruns=2" % (VERSION)]
+            "./tiobench.pl $(OPTIONS)" % (VERSION)]
 
 
 class TIObenchTestParser(TestParser):
@@ -70,7 +70,7 @@
 
 tiobench_inst = TestInstaller(INSTALLSTEPS, url=URL,
     md5=MD5)
-tiobench_run = TestRunner(RUNSTEPS)
+tiobench_run = TestRunner(RUNSTEPS,default_options=DEFAULT_OPTIONS)
 parse = TIObenchTestParser(appendall={'units':'MB/s', 'result':'pass'})
 testobj = Test(test_id="tiobench", test_version=VERSION,
     installer=tiobench_inst, runner=tiobench_run, parser=parse)

=== modified file 'lava_test/test_definitions/x11perf.py'
--- lava_test/test_definitions/x11perf.py	2011-09-12 09:19:10 +0000
+++ lava_test/test_definitions/x11perf.py	2011-10-20 12:16:24 +0000
@@ -46,11 +46,12 @@
     "-scroll500",
     ]
 
-RUNSTEPS = ["x11perf %s %s" % (x11perf_options,  " ".join(x11perf_tests))]
+DEFAULT_OPTIONS = "%s %s" % (x11perf_options,  " ".join(x11perf_tests))
+RUNSTEPS = ["x11perf $(OPTIONS)"]
 PATTERN = "trep @.*\(\W*(?P<measurement>\d+.\d+)/sec\):\W+(?P<test_case_id>.+)"
 
 inst = TestInstaller(deps=["x11-apps"])
-run = TestRunner(RUNSTEPS)
+run = TestRunner(RUNSTEPS,default_options=DEFAULT_OPTIONS)
 parse = TestParser(PATTERN,
                                       appendall={'units':'reps/s',
                                                  'result':'pass'})

=== modified file 'lava_test/test_definitions/xrestop.py'
--- lava_test/test_definitions/xrestop.py	2011-09-23 08:10:25 +0000
+++ lava_test/test_definitions/xrestop.py	2011-10-20 12:16:24 +0000
@@ -19,11 +19,12 @@
 from lava_test.core.tests import Test
 
 INSTALLSTEPS = ['bzr branch lp:~linaro-foundations/lava-test/xrestopscript']
-RUNSTEPS = ["./xrestopscript/xrestop.sh"]
+RUNSTEPS = ["./xrestopscript/xrestop.sh $(OPTIONS)"]
 PATTERN = "^(?P<test_case_id>\w+):\W+(?P<measurement>\d+)"
+DEFAULT_OPTIONS = ""
 
 xrestopinst = TestInstaller(INSTALLSTEPS, deps=["xrestop"])
-xrestoprun = TestRunner(RUNSTEPS)
+xrestoprun = TestRunner(RUNSTEPS,default_options=DEFAULT_OPTIONS)
 xrestopparser = TestParser(PATTERN,
                    appendall={'units':'KB', 'result':'pass'})
 testobj = Test(test_id="xrestop", installer=xrestopinst,