Merge lp:~asc/fluidity/testharnesscwd2 into lp:fluidity

[Adam Candy]

Added an option to testharness.py to search for test XMLs in folders of the current working directory. This is useful if you use one version of the script in multiple branches.

[Patrick Farrell (pefarrell)]

Hi Adam,

a) I'm a bit confused about what the use case of this can be. Can you tell the story of why you wanted this feature?

b) Couldn't this feature be much more simply implemented with a one-line change like

=== modified file 'tools/testharness.py'
--- tools/testharness.py 2011-08-05 11:19:06 +0000
+++ tools/testharness.py 2011-09-05 14:53:28 +0000
@@ -50,7 +50,7 @@
         for directory in testpaths:
           if os.path.exists(os.path.join(rootdir, directory)):
             dirnames.append(directory)
- testdirs = [ os.path.join( rootdir, x ) for x in dirnames ]
+ testdirs = [ os.path.join( rootdir, x ) for x in dirnames ] + [os.getcwd()]
         for directory in testdirs:
           subdirs = [ os.path.join(directory, x) for x in os.listdir(directory)]
           for subdir in subdirs:

? I don't really know why you didn't take this approach -- like I said, I don't know why you want it.

[Adam Candy (asc)]

Hi Patrick,

Thanks for the feedback. This simple fix was avoided because it could interfere with current usage. Adding the current working directory to the list of testdirs could result in additional tests being picked up unexpectedly, hence the additional structure was included to avoid changing the default behaviour.

[Adam Candy (asc)]

The motivation for the change is to:

 1. Use the script from a $PATH location,
 2. on a given test folder (which might be standalone),
 3. from an arbitrary location (already possible before the above, but more complicated in combination with 1 and 2).

Point 1 was because I have several modifications to the testharness.py and just wanted to use one instance of the script for multiple branches (e.g. place it in ~/bin) and point 2 because I have some tests in a standalone folder that I have been testing using multiple branches.

Of course there is an issue with 2, in that tests can have dependencies within the Fluidity source (in the mesh generation, for example). This could work if it was possible to provide the root Fluidity folder (for the binary and other test dependencies) together with the location of the test folder (the default being ../ and ../{examples,tests,longtests} respectively).

As it stands the changes need further thought, so I'll pull the merge request for now. It is possible to use with the limitations that the script itself, binary and associated tools must all be located in the Fluidity source tree, together with the folder of tests.

[Adam Candy (asc)]

review: Needs Fixing

Unmerged revisions

3569. By Adam Candy on 2011-09-05

Added an option to testharness.py to search for test XMLs in folders of the current working directory. This is useful if you use one version of the script in multiple branches.

=== modified file 'tools/testharness.py'
--- tools/testharness.py	2011-08-05 11:19:06 +0000
+++ tools/testharness.py	2011-09-05 10:31:28 +0000
@@ -15,7 +15,7 @@
 import elementtree.ElementTree as etree
 
 class TestHarness:
-    def __init__(self, length="any", parallel=False, exclude_tags=None, tags=None, file="", verbose=True, justtest=False,
+    def __init__(self, length="any", parallel=False, exclude_tags=None, tags=None, file="", cwd=False,  verbose=True, justtest=False,
         valgrind=False):
         self.tests = []
         self.verbose = verbose
@@ -44,13 +44,16 @@
         # step 1. form a list of all the xml files to be considered.
 
         xml_files = []
-        rootdir = os.path.abspath(os.path.join(os.path.dirname(sys.argv[0]), os.pardir))
-        dirnames = []
-        testpaths = ["examples", "tests", "longtests"]
-        for directory in testpaths:
-          if os.path.exists(os.path.join(rootdir, directory)):
-            dirnames.append(directory)
-        testdirs = [ os.path.join( rootdir, x ) for x in dirnames ]
+        if cwd:
+          testdirs = [ os.getcwd() ]
+        else:
+          rootdir = os.path.abspath(os.path.join(os.path.dirname(sys.argv[0]), os.pardir))
+          dirnames = []
+          testpaths = ["examples", "tests", "longtests"]
+          for directory in testpaths:
+            if os.path.exists(os.path.join(rootdir, directory)):
+              dirnames.append(directory)
+          testdirs = [ os.path.join( rootdir, x ) for x in dirnames ]
         for directory in testdirs:
           subdirs = [ os.path.join(directory, x) for x in os.listdir(directory)]
           for subdir in subdirs:
@@ -328,6 +331,7 @@
     parser.add_option("-n", "--threads", dest="thread_count", type="int",
                       help="number of tests to run at the same time", default=1)
     parser.add_option("-v", "--valgrind", action="store_true", dest="valgrind")
+    parser.add_option("-d", "--cwd", action="store_true", dest="cwd", help="look for test XMLs in folders of current working directory", default = False)
     parser.add_option("-c", "--clean", action="store_true", dest="clean", default = False)
     parser.add_option("--just-test", action="store_true", dest="justtest")
     parser.add_option("--just-list", action="store_true", dest="justlist")
@@ -364,7 +368,7 @@
     else:
       tags = options.tags
 
-    testharness = TestHarness(length=options.length, parallel=para, exclude_tags=exclude_tags, tags=tags, file=options.file, verbose=True,
+    testharness = TestHarness(length=options.length, parallel=para, exclude_tags=exclude_tags, tags=tags, file=options.file, cwd=options.cwd, verbose=True,
         justtest=options.justtest, valgrind=options.valgrind)
 
     if options.justlist: